Bug #10

Parallel builds ignore generated files

Added by Andreas Fredriksson almost 11 years ago. Updated almost 10 years ago.

Status:ClosedStart date:01/18/2009
Priority:NormalDue date:
Assignee:Joshua Jensen% Done:

0%

Category:Building
Target version:0.4

Description

It turns out jamplus has an ancient bug from mainline Jam which was reported here: http://maillist.perforce.com/pipermail/jamming/2003-December/002252.html.

Here it is in Jamplus form. The key is that must be a circular dependency among the headers, and one of them includes a generated header:

rule SleepThenTouch
{
    Clean clean : $(<) ;
    DEPENDS all : $(<) ;
}

actions SleepThenTouch
{
    sleep 5
    touch $(<)
}

actions Compile
{
    test -f headerA.h
}

SleepThenTouch generated.h ;
Library foo : sourceA.c sourceB.c ;

This will fail 100% when run with -j2 but always succeed when running with -j1. With -j2, the foo compilation is started even through the header has not been generated. On win32, use coreutils from gnuwin32 to get sleep and touch. I've attached Matt Armstrong's repro case in Jamplus form.

repro.zip - Repro case in Jamplus form (1.3 KB) Andreas Fredriksson, 01/18/2009 09:53 AM

History

#1 Updated by Joshua Jensen almost 11 years ago

I'll look into this, but as I recall, this patch makes dependency checking very slow. If it is the patch I am thinking of, on one project, it took a 1-2 second dependency check up to 7 seconds.

#2 Updated by Joshua Jensen almost 11 years ago

  • Status changed from New to Closed
  • Target version set to 0.3

I noticed no discernable slowdown, so the patch is #ifdef'ed in Git and the latest nightly as OPT_CIRCULAR_GENERATED_HEADER_FIX. A test, similar to the one above, is in source:tests/generatedheader/circular.

#3 Updated by Andreas Fredriksson almost 11 years ago

I just ran the OPT_CIRCULAR_GENERATED_HEADER_FIX fix against the test case attached to this bug and it still doesn't compile with -j2. The issue remains.

I also tried the fix on a much larger code base and saw that Jam ran out of memory on a multi-level circular loop inside the loop the feature introduces.

#4 Updated by Andreas Fredriksson almost 11 years ago

A clarification; adding the <foo> grist to the generated.h filename makes my original case pass as well. The memory allocation issue remains however.

#5 Updated by Joshua Jensen almost 11 years ago

  • Status changed from Closed to Assigned
  • Assignee set to Joshua Jensen

I died with and without the <foo>generated.h. I had no problem with the -j2 after the OPT_CIRCULAR_GENERATED_HEADER_FIX. Parallelism is odd...

Nevertheless, loops concerned me with the patch, and it appears you hit them. The next message on the mailing list had more to the patch... a portion that could break loops. Let's see if the latest 'nightly' fixes this for you.

#6 Updated by Andreas Fredriksson over 10 years ago

Sort of, I can build the test case but in the huge codebase I still see the problem where source files are compiled before the headers have been generated. I know the dependencies are correct because if I build the single failing object file with a totally clean output directory Jam will correctly build all the generated files before compiling the object file.

Something is still rotten, but I suspect the bug is in action execution rather than in the dependency graph.

#7 Updated by Andreas Fredriksson over 10 years ago

Another followup on this.

I only see the bug in a huge codebase when batching is used. The issue disappears when BATCH_COMPILE_GROUP_SIZE is set to 1, thus disabling batching. Setting it to 8 or leaving it unset brings the bug back.

It would seem as if maybe not all .cpp files in the batch are scanned for dependencies correctly. I'll try to work out a reproduction case.

#8 Updated by Joshua Jensen over 10 years ago

I have put some additional patches in place that may take care of this issue. The latest code is in Git, but it is also in the nightly build as of JamPlus-090206-bin.zip.

Let me know if it helps.

Josh

#9 Updated by Andreas Fredriksson over 10 years ago

I just tried head from git and the it behaves exactly the same when I set BATCH_COMPILE_GROUP_SIZE to values greater than 1. The batching C++ actions are executed even through headers have not been generated yet despite the dependency graph being correct (as reported by -dg). Things also work just building the one failing object file with a clean output.

I'm also concerned that Jam crashed when I tried to limit batching to 1. It seemed to be writing a huge list of object files for a linker response file at the time.

In command.c:256 the recursive cmd_string() call is returning -1, and the code goes on and passes 0xfffffffe to tmp_write()'s count argument.

Unhandled exception at 0x0040dcb9 in jam.exe: 0xC0000005: Access violation reading location 0x0592a000.

jam.exe!_write_nolock(int fh=0x00000003, const void * buf=0x05924a28, unsigned int cnt=0xfffff000) Line 322 + 0xc bytes C

jam.exe!_write(int fh=0x00000003, const void * buf=0x05924a28, unsigned int cnt=0xfffff000) Line 75 + 0xe bytes C
jam.exe!_fwrite_nolock(const void * buffer=0x05924a28, unsigned int size=0x00000001, unsigned int num=0xfffffffe, iobuf * stream=0x0043e210) Line 165 + 0x11 bytes C
jam.exe!fwrite(const void * buffer=0x05924a28, unsigned int size=0x00000001, unsigned int count=0xfffffffe, _iobuf * stream=0x0043e210) Line 83 + 0x11 bytes C
jam.exe!tmp_write(_tmpfile * f=0x0335eab0, const char * p=0x05924a28, int count=0xfffffffe) Line 101 + 0x16 bytes C
jam.exe!cmd_string(_rule * rule=0x024fc330, const char * in=0x0035f805, _buffer * buff=0x0017779c, int outsize=0x00002800, _lol * lol=0x02dfca90, _tmplist * * response_files=0x02dfcab8, _cmd * cmd=0x02dfca80) Line 258 + 0x1a bytes C
jam.exe!cmd_new(_rule * rule=0x024fc330, _list * targets=0x0584ccb4, _list * sources=0x0584c648, _list * shell=0x00000000, int maxline=0x00002800) Line 76 + 0x2d bytes C
jam.exe!make1cmds(_target * t=0x02932f90, _actions * a0=0x028abe10) Line 1253 + 0x43 bytes C
jam.exe!make1b(_target * t=0x02932f90) Line 468 + 0x10 bytes C
jam.exe!make1c(_target * t=0x02932e60) Line 676 + 0xc bytes C
jam.exe!make1d(const char * outputname=0x056bfcc8, void * closure=0x02932e60, int status=0x00000000) Line 856 + 0x9 bytes C
jam.exe!execwait() Line 603 + 0x2c bytes C
jam.exe!execcmd(char * string=0x056bff08, void (const char , void *, int) func=0x00422280, void * closure=0x02151b30, _list * shell=0x00000000) Line 534 + 0x5 bytes C
jam.exe!make1c(_target * t=0x02151b30) Line 587 + 0x1c bytes C
jam.exe!make1b(_target * t=0x02151b30) Line 497 + 0x9 bytes C
jam.exe!make1c(_target * t=0x02150d88) Line 676 + 0xc bytes C
jam.exe!make1d(const char * outputname=0x056bfd58, void * closure=0x02150d88, int status=0x00000000) Line 856 + 0x9 bytes C
jam.exe!execwait() Line 603 + 0x2c bytes C
jam.exe!execcmd(char * string=0x056bfec0, void (const char , void *, int) func=0x00422280, void * closure=0x029f75c0, _list * shell=0x00000000) Line 534 + 0x5 bytes C
jam.exe!make1c(_target * t=0x029f75c0) Line 587 + 0x1c bytes C
jam.exe!make1b(_target * t=0x029f75c0) Line 497 + 0x9 bytes C
jam.exe!make1c(_target * t=0x029f7528) Line 668 + 0xc bytes C
jam.exe!make1d(const char * outputname=0x056bfd10, void * closure=0x029f7528, int status=0x00000000) Line 856 + 0x9 bytes C
jam.exe!execwait() Line 603 + 0x2c bytes C
jam.exe!execcmd(char * string=0x056bff08, void (const char , void *, int) func=0x00422280, void * closure=0x02150d88, _list * shell=0x00000000) Line 534 + 0x5 bytes C
jam.exe!make1c(_target * t=0x02150d88) Line 587 + 0x1c bytes C
jam.exe!make1b(_target * t=0x02150d88) Line 497 + 0x9 bytes C
jam.exe!make1c(_target * t=0x029e4950) Line 676 + 0xc bytes C
jam.exe!make1d(const char * outputname=0x056bfd10, void * closure=0x029e4950, int status=0x00000000) Line 856 + 0x9 bytes C
jam.exe!execwait() Line 603 + 0x2c bytes C
jam.exe!execcmd(char * string=0x056bfda0, void (const char , void *, int) func=0x00422280, void * closure=0x029e1e90, _list * shell=0x00000000) Line 534 + 0x5 bytes C
jam.exe!make1c(_target * t=0x029e1e90) Line 587 + 0x1c bytes C
jam.exe!make1b(_target * t=0x029e1e90) Line 497 + 0x9 bytes C
jam.exe!make1c(_target * t=0x0288a200) Line 676 + 0xc bytes C
jam.exe!make1d(const char * outputname=0x056bfc80, void * closure=0x0288a200, int status=0x00000000) Line 856 + 0x9 bytes C
jam.exe!execwait() Line 603 + 0x2c bytes C
jam.exe!make1(_target * t=0x024fda90) Line 156 + 0x5 bytes C
jam.exe!make(int n_targets=0x00000001, const char * * targets=0x0017f724, int anyhow=0x00000000) Line 314 + 0x18 bytes C
jam.exe!main(int argc=0x00000000, char * * argv=0x024f2364, char * * arg_environ=0x024f2378) Line 544 + 0x15 bytes C
jam.exe!
_tmainCRTStartup() Line 266 + 0x12 bytes C
kernel32.dll!@BaseThreadInitThunk@12() + 0xe bytes
ntdll.dll!___RtlUserThreadStart@8() + 0x23 bytes
ntdll.dll!__RtlUserThreadStart@8() + 0x1b bytes

#10 Updated by Joshua Jensen over 10 years ago

I ran into the child dependencies not updating before the parent issue last night, but it only reproduced once and did so in an entirely different scenario.

The patch causing this issue the parent dependencies updating first appears to be in make1.c under the #ifdef OPT_UPDATED_CHILD_FIX. Just turn off that block. Try it and let me know. Unfortunately, turning it off breaks source:tests/dependency/more_than_one_target.jam . Still, if we get closer to solving your issue, I can find a way to make the more_than_one_target.jam issue work.

I still want to build up a big generated source/header test case. I hope to do it soon, according to your descriptions above.

#11 Updated by Andreas Fredriksson over 10 years ago

Unfortunately turning off OPT_UPDATED_CHILD_FIX did not help.

#12 Updated by Joshua Jensen over 10 years ago

  • Target version changed from 0.3 to 0.4

#13 Updated by Joshua Jensen almost 10 years ago

  • Status changed from Assigned to Closed

I still can't repro this. I'm closing the bug. Let's reopen it if it is still an issue.

Also available in: Atom PDF