Default compile takes 62 minutes, no knobs to exclude protobof-lite or support for unneeded languages from protoc?

[hartwork]

Hi!

It has come to my attention that a default compile (meaning plain cmake -S . -B build) of Protobuf (commit 24830cf of 2025-02-28 and GCC 13) takes 62 minutes on a Lenovo ThinkPad X220, and that much of that time could be saved if only the build system would allow to (1) compile protobuf compiler "protoc" with fewer languages included (e.g. when all I need is C++ and none of C#/Java/JKotlin/ObjectiveC/PHP/Python/Ruby/Rust) and (2) exclude protobuf-lite. 62 minutes!

This was in context of libprotobuf-mutator where switch -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON would download and compile an almost complete Protobuf originally e.g. because Protobuf in many mainline distros is so far behind Protobuf upstream, including Ubuntu (20.04/22.04/)24.04. I should note that currently 14 projects in OSS-Fuzz are doing that because they had no choice: compile almost all of Protobuf without genuine need to.

For a case where (nothing but) libprotobuf and a protoc compiler with C++ support are needed (as with libprotobuf-mutator), the only two build switches reducing compile time I found were -Dprotobuf_BUILD_TESTS:BOOL=OFF -Dprotobuf_BUILD_LIBUPB:BOOL=OFF but they break the build (see #20538) and are not cutting of much. I was demoing a quick-hack to reduce compile time in https://github.com/google/libprotobuf-mutator/pull/267/files earlier but it was not the right place, not in upstreamable form, not a true fix.

Is reducing the build to fewer unneeded parts at the source something that your are considering?

Best, Sebastian

CC @vitalybuka

[hartwork]

@protocolbuffers any thoughts?

[hartwork]

Closing for no reply…

[esrauchg]

@hartwork Can you provide more info about this? Over an hour seems very long for this build even on the hardware you're talking about.

Note that the cmake build doesn't build other language's runtimes, it only builds the code generators for each. The language generators really are pretty small and so is libupb, so likely there shouldn't be too much benefit from trimming them from your build.

In terms of 'lite', the Full C++Proto runtime deps onto the Lite C++Proto runtime, and protoc uses the Full C++ runtime, so work compiling LiteC++ is not duplicative. It may be theres duplicative linking or something there?

If you have any info on where your build times are being spent we may be able to help advise.

[mkruskal-google]

You might also try adding --parallel 8 or using ccache/sccache for incremental builds. Both of those can speed up cmake build dramatically.

[hartwork]

@hartwork Can you provide more info about this? Over an hour seems very long for this build even on the hardware you're talking about.

@esrauchg I am using the same hardware just now, but when trying to get you a build duration time for today, the build fails already at CMake stage, new issue #24205, both on Debian sid and Gentoo.

Note that the cmake build doesn't build other language's runtimes, it only builds the code generators for each. The language generators really are pretty small and so is libupb, so likely there shouldn't be too much benefit from trimming them from your build.

With my patching at google/libprotobuf-mutator#267 I was literally cutting build time in half back then.

In terms of 'lite', the Full C++Proto runtime deps onto the Lite C++Proto runtime, and protoc uses the Full C++ runtime, so work compiling LiteC++ is not duplicative. It may be theres duplicative linking or something there?

If you have any info on where your build times are being spent we may be able to help advise.

The 62 minutes are for a full build of the default build configuration. The patches there show what I was patching away. I'm not sure how you would call it.

You might also try adding --parallel 8 or using ccache/sccache for incremental builds. Both of those can speed up cmake build dramatically.

@mkruskal-google maybe, but that's only working around the core issue: that I'm building more than I need. I would give you a time with -j$(nproc) but then issue #24205 currently keeps me from successfully building anything for main or v33.

[hartwork]

Update: I learned from #22209 about -Dprotobuf_BUILD_TESTS=OFF now to workaround the build error and would like to offer these four data points from the reference hardware for the same original commit:

• 24830cf + -Dprotobuf_BUILD_TESTS=OFF + Debian sid + GCC 13 + Abseil 20240722.0 = 21m49.280s
• 24830cf + -Dprotobuf_BUILD_TESTS=OFF + Debian sid + GCC 15 + Abseil 20240722.0 = 23m40.501s
• 24830cf + -Dprotobuf_BUILD_TESTS=OFF + Gentoo + GCC 14 + Abseil 20250512.1 = 33m28.106s
• 24830cf + -Dprotobuf_BUILD_TESTS=OFF + Gentoo + GCC 15 + Abseil 20250512.1 = 30m41.150s

The version of Abseil is likely different to back in March, guaranteed different in the Gentoo case, the linker could be a different one also. I'm not sure what else could be different, how much build time went to the tests (if I built them back then) or why the build would be twice as fast as the original 62 minutes now. But then 33 minutes is still quite a bit.

[mkruskal-google]

#24205 is because we build tests by default and whatever version of Abseil you have installed didn't install test utils. I think disabling tests by default could be a reasonable change as they are extremely expensive to build (as you've pointed out with data). But it would be fairly dangerous on our end as we'd have to be extremely careful to explicitly enable them in all our test builds to continue testing with cmake in our CI

[hartwork]

#24205 is because we build tests by default and whatever version of Abseil you have installed didn't install test utils.

@mkruskal-google that's all fair, but a dedicated error message and detecting that case would be nice. Currently the user ends up in low level error soup where nothing but Google will be of much help.

I think disabling tests by default could be a reasonable change as they are extremely expensive to build (as you've pointed out with data). But it would be fairly dangerous on our end as we'd have to be extremely careful to explicitly enable them in all our test builds to continue testing with cmake in our CI.

I believe "make test" should fail right away if enable_testing has not been called, so that should make the change safe in the sense that it would fail make test if tests have been forgotten to enable before end thus would alert about need for adjustment in that CI. To demo that very case:

# cd "$(mktemp -d)"

# echo $'cmake_minimum_required(VERSION 3.10)\nproject(demo123)' | tee CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(demo123)

# cmake . && make test ; echo $?
-- The C compiler identification is GNU 14.3.1
-- [..]
-- Build files have been written to: /tmp/tmp.IjQWJsNGip
make: *** No rule to make target 'test'.  Stop.
2

[hartwork]

@hartwork Can you provide more info about this?

@esrauchg I now compiled with (simplified) /usr/bin/time gcc — GNU Time — instead of plain gcc to give you seconds runtimes for all the invocations of gcc (and g++) made. Here's how I did that:

cd "$(mktemp -d)"
git init
git fetch --depth 1 https://github.com/protocolbuffers/protobuf 24830cf07e559e912fa2622d3487a679ef0df9a9
git checkout FETCH_HEAD

echo $'#!/usr/bin/env bash\nexec /usr/bin/time -f "%e %C" -a -o "$(dirname "$0")"/time.log g++-14 "$@"' > time-g++.sh
echo $'#!/usr/bin/env bash\nexec /usr/bin/time -f "%e %C" -a -o "$(dirname "$0")"/time.log gcc-14 "$@"' > time-gcc.sh
chmod a+x time-gcc.sh time-g++.sh

time sh -c 'cmake -DCMAKE_C_COMPILER="${PWD}/time-gcc.sh" -DCMAKE_CXX_COMPILER="${PWD}/time-g++.sh" -Dprotobuf_BUILD_TESTS=OFF -S . -B build && make -C build VERBOSE=1' ; echo $?

The resulting time.log is this:

Download: time.log

The top 10 longest compile time files (for commit 24830cf) are:

# sort -g -r < time.log | head | sed "s|${PWD}|.|g" | awk '{print $1 " " $NF}'
20.48 ./src/google/protobuf/descriptor.cc
19.88 ./src/google/protobuf/descriptor.cc
19.87 ./src/google/protobuf/descriptor.cc
16.80 ./src/google/protobuf/compiler/cpp/message.cc
14.20 ./src/google/protobuf/compiler/cpp/file.cc
13.23 ./src/google/protobuf/text_format.cc
11.84 ./src/google/protobuf/compiler/cpp/helpers.cc
11.79 ./src/google/protobuf/compiler/command_line_interface.cc
10.23 ./src/google/protobuf/util/message_differencer.cc
9.98 ./src/google/protobuf/generated_message_tctable_lite.cc

PS: It seems like file src/google/protobuf/descriptor.cc was indeed compiled 3 times.

[mkruskal-google]

[@mkruskal-google](https://github.com/mkruskal-google) that's all fair, but a dedicated error message and detecting that case would be nice. Currently the user ends up in low level error soup where nothing but Google will be of much help.

I believe "make test" should fail right away if enable_testing has not been called, so that should make the change safe in the sense that it would fail make test if tests have been forgotten to enable before end thus would alert about need for adjustment in that CI. To demo that very case:

I think this is a valid point and worth a separate issue. I wasn't aware that make test did the sensible thing, so that does make me feel a lot better about it. I think we'd be open to a PR for this as well, as we usually have trouble prioritizing CMake improvements.

PS: It seems like file src/google/protobuf/descriptor.cc was indeed compiled 3 times.

Those results are extremely surprising. It looks like protoc-gen-upbdefs and protoc-gen-upb are rebuilding descriptor.cc from scratch. I would hope cmake reuses the results of previous builds and only actually runs the compiler over it once though. My expectation is that most of the duplicated work would come from linking not from compiling. Those plugins also both have dependencies on libprotobuf, so this seems completely incorrect and like it could lead to ODR violations...