Bazel For Originate-Source C/C++ Libraries Distribution

Within the past just a few days, I’ve been experimenting with Bazel as a library distribution mechanism for ccv.

I am gorgeous mindful of airtight design programs at this level. My main files comes from Buck relationship 8 years abet. At that time, it never took place to me such a design system would possibly per chance at final be a library distribution mechanism. At some level of the same 8 years, NPM has taken over the realm. Unique language-dependent equipment managers corresponding to Trek module, Cargo and Swift Equipment Supervisor popularized the idea of utilizing the public repositories (GitHub) as the dependency references. Languages outdated to this period, primarily C / C++ are transferring to this course, slowly.

ccv has a straightforward autoconf based totally mostly feature detection / configuration system. You need to ask the equipment to work when ./configure && make. Nonetheless, it never made any main strive to be too trim. My initial skills with monorepos at companies strongly influenced the selection to construct up a straightforward design system. I absolutely ask that main shoppers will vendor the library into their monorepo utilizing their very possess design programs.

This has been lawful for the past just a few years. But as I am winding up nnc and an increasing number of utilizing that for other closed-provide deepest tasks, affirming a closed-provide monorepo setup for my deepest tasks whereas upstreaming fixes in all fairness an nasty skills. On the opposite hand, nnc from the starting meant to be a low-degree implementation. I am anticipated to construct up excessive-degree language bindings at some level. On condition that I am doing extra utility-related fashion with nnc in closed-provide structure now, it feels adore the lawful time.

Even though there would possibly be nobody-lawful library distribution mechanism for C / C++, there are contenders. From the beautiful-extinct upright / rpm, to Conan, which has gained some mind-portion within the originate-provide world today.

The series of Bazel is now not accidental. I’ve been doing some Swift fashion with Bazel and the skills has been particular. Moreover, the series of excessive-degree binding language for nnc, I figured, would possibly per chance be Swift.

Configure

ccv’s design course of, as noteworthy as I’d pretty now not, is host-dependent. I employ autoconf to detect system-wide libraries corresponding to libjpeg and libpng, to configure upright compiler alternatives. Even though ccv will even be ragged with zero dependency, in that configuration, it’ll most incessantly be unhurried.

Coming from the monorepo background, Bazel doesn’t accumulate many utilities which would possibly per chance be as readily readily obtainable as in autoconf. You need to be in a series to write automated configurations in Starlark as repository rules, but there’ll not be any such thing as a gorgeous documentation on how to write sturdy ones.

I accomplished up letting whoever employ ccv to resolve how they are going to permit particular suggestions. For issues adore CUDA, such configuration is now not tenable. I accomplished up copying over TensorFlow’s CUDA rules.

Dependencies

Ravishing extinct C / C++ libraries are notoriously detached to libraries dependencies v.s. toolchains. Autoconf detects both toolchain configurations as effectively as readily obtainable libraries. These forms of host dependencies make deplorable-compilation a skill in itself.

Bazel is sweet for in-tree dependencies. For out-tree dependencies alternatively, there’ll not be any such thing as a established mechanism. The standard manner is to write a repository rules to load relevant dependencies.

This in actual fact works effectively for me. It is versatile ample to address cases that accumulate Bazel integrations and don’t accumulate any Bazel integrations.

Expend Bazel Dependencies

Consumption of the packaged Bazel dependencies then becomes as straightforward as adding git_repository to the WORKSPACE and contact upright _deps() repository rule.

After packaging ccv with Bazel, now Swift for nnc can use the packaged dependency.

Semantic Versioning Challenges

Whereas the Bazel-supplied library distribution mechanism works effectively for my case, it’s simplistic. For one, there would possibly be on occasion no beautiful manner to realize semantic versioning. It is understandable. Coming from a monorepo culture, it’s hard for anybody to dive into dependency hells of library versioning. A pretty assorted myth came about to Trek a whereas abet as effectively.

It is messy whereas you’ll capture to pin a explicit version of the library whereas your dependencies are now not agreeing with you. Here is going to be messy regardless in C / C++ world, except you prelink these extraordinarily fastidiously. Bazel’s philosophy from what I will stumble on, seems largely on conserving the trunk working side. It is working to this level, but one has to shock if this would possibly per chance scale if extra libraries adopted Bazel as the distribution mechanism.

Closing Phrases

The past just a few months skills with Bazel has been palatable. Whereas I’d continue to make employ of language explicit instruments (pip, Trek modules, Cargo, NPM) when doing fashion in that explicit language, Bazel is a succesful need for me when doing deplorable-language fashion. Concepts corresponding to workspace, git_repository, http_archive match effectively within the upper originate-provide ecosystem. And most surprisingly, it works for diverse-repo setup whereas you ever favor to.

Be taught More