Skip to content

lib: Fix parallel library build ordering#276

Open
luca-fancellu wants to merge 1 commit into
tytso:masterfrom
luca-fancellu:fix-build-race
Open

lib: Fix parallel library build ordering#276
luca-fancellu wants to merge 1 commit into
tytso:masterfrom
luca-fancellu:fix-build-race

Conversation

@luca-fancellu

Copy link
Copy Markdown

I'm currently building with a 56 core machine and hitting these issues with master branch, commit 43643a5.
Fixes and reasoning are in the patch.

Using these configure parameters:

[...]configure \
    --enable-elf-shlibs \
    --disable-libuuid \
    --disable-uuidd \
    --disable-libblkid \
    --enable-verbose-makecmds

And then make -j56:

make[2]: *** No rule to make target '../lib/libext2fs.so', needed by 'e2fsck'. Stop.
make[2]: *** Waiting for unfinished jobs....
gcc -c -I. -I../lib -I../../src/lib  -g -O2 -pthread  -DHAVE_CONFIG_H  ../../src/e2fsck/encrypted_files.c -o encrypted_files.o
make[2]: Leaving directory '/tmp/e2fsprogs-master-race.yfQYqM/build/e2fsck'
make[1]: *** [Makefile:445: all-progs-recursive] Error 1
make[1]: Leaving directory '/tmp/e2fsprogs-master-race.yfQYqM/build'
make: *** [Makefile:373: all] Error 2

Logs:
configure.log
make.log

When building with --enable-elf-shlibs, Makefile.elf-lib adds `all:: image`
to build the shared library image.
The static library stub also adds `all:: subdirs $(LIBRARY).a`.

The archive and ELF image are consumed through the top-level lib directory
by other subdirectories, so a library subdirectory must not return from
"make all" before both outputs have been created.

If libcom_err.so has not been created yet, libext2fs.so can fall back to
libcom_err.a. On x86-64 that can fail because lib/et/error_message.o contains
a thread-local object and the non-PIC static object has R_X86_64_TPOFF32
relocations that cannot be used when linking a shared object.

Make the static archive target depend on the ELF image when Makefile.elf-lib
is included, so the normal library build creates the shared object and
top-level `.so` symlinks before returning to the parent recursive make.

lib/ss has an additional ordering issue: its first local all:: rule only names
mk_cmds, while libss.a and ss.pc are added later. Put libss.a and ss.pc on
that first all:: target rule as well so lib/ss cannot complete after generating
only mk_cmds and leave debugfs without lib/ss/ss_err.h. Also make ss_err.h
depend on ss_err.c so parallel consumers do not run compile_et twice for the
same generated pair.

Finally, create the lib/ss install directories in the install recipe before
copying headers and data files. The ELF shared-library fragment also contributes
install and installdirs double-colon rules, so relying on the aggregate
installdirs target can still let the header copy run before $(includedir)/ss
exists during parallel install.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant