Possible bug with multiple passes and NoCare dependencies

Added by Chris Munn over 10 years ago

I came across this when I was trying to add a third pass to our build system. We have an action that runs in the second pass, and the source for this action has NoCare dependencies. This action worked fine when we only had 2 passes, but when I added a third pass it would no longer run. This only happens when all 3 passes are used, when I leave out either the first or third pass the action runs correctly.

Also, if I leave out the first pass, the action runs during the third pass - I assume this is by design, but I would prefer to have the action run in the second pass instead of the third. This is a much smaller problem than the first one.

I have attached a small example for reproducing this problem.

multipass_bug.zip - jamfile and .bat to reproduce (678 Bytes)


Replies (7)

RE: Possible bug with multiple passes and NoCare dependencies - Added by Joshua Jensen over 10 years ago

In the latest Git, I have committed a fix for this. I have also uploaded a new nightly binary. Give it a whirl. Let me know if it works.

Josh

RE: Possible bug with multiple passes and NoCare dependencies - Added by Chris Munn over 10 years ago

Everything works with the nightly. Thank you!

Chris

RE: Possible bug with multiple passes and NoCare dependencies - Added by Chris Munn over 10 years ago

I spoke too soon about everything working. The second pass does work now, but I came across these two things:

1. Files in the second pass with a NoCare dependency always build, regardless of whether or not they need to be rebuilt. This isn't a major issue for me, I can live with rebuilding a few files every time.

2. Nothing in the third pass is being built. I have tried several things to get around this, including splitting up each pass into its own jamfile and adding a fourth pass (to see if the problem is because it's the third pass or because it's the final pass).

I have attached another example so you can reproduce #2. Thank you again for all your help.

Chris

RE: Possible bug with multiple passes and NoCare dependencies - Added by Joshua Jensen over 10 years ago

Okay, that bug is fixed. It turns out that a call to make_fixprogress_init() was missing, even though the function was sitting there ready to be called. I don't know when the call disappeared. Either way, I reworked the code while I was in there.

Let me know how it goes.

RE: Possible bug with multiple passes and NoCare dependencies - Added by Joshua Jensen over 10 years ago

If you don't mind, can you tell us how you are utilizing three passes? The most I have ever used has been two passes. The first pass collected dependencies from large data files. The second pass acted upon the information from the first pass. I originally used the HDRPIPE facility to scan for dependencies, but since the Jam dependency graph scan is single threaded, the dependency scan took much longer. Putting the scan into its own pass dropped the scan time considerably on my multi-processor machine, at the expense of complicating the build with another pass.

RE: Possible bug with multiple passes and NoCare dependencies - Added by Joshua Jensen over 10 years ago

Oh, and above, I meant that you second point is fixed. I didn't have time to look at the NoCare building every time issue. I'll do so later on today.

RE: Possible bug with multiple passes and NoCare dependencies - Added by Chris Munn over 10 years ago

I still have NoCares building every time, so I decided to look at the source. I think the offending code is in make.c at line 905:

#ifdef OPT_MULTIPASS_EXT
    if( fate == T_FATE_MISSING && !t->actions && !t->depends && !queuedjamfiles )
#else
    if( fate == T_FATE_MISSING && !t->actions && !t->depends )
#endif
    {
        if( t->flags & T_FLAG_NOCARE )
        {
#ifdef OPT_GRAPH_DEBUG_EXT
        if( DEBUG_FATE )
        printf( "fate change  %s to STABLE from %s, " 
                "no actions, no dependents and don't care\n",
               t->name, target_fate[fate]);
#endif
        fate = T_FATE_STABLE;
        }
        else
        {
        printf( "don't know how to make %s\n", t->name );

        fate = T_FATE_CANTFIND;
        }
    }

The way I understand it, if there is another pass queued and a target has a missing dependency, jam will wait to build that target until the next pass. (correct me if I'm wrong about this)

However, if that dependency is NoCare, jam will build it in the current pass. The way this code is structured, if there is a jamfile queued it will never check if the dependency is NoCare, so fate will still be missing. I don't know why this causes the target to be built every time, but it does. I changed the code to this and the problem went away:

    if( fate == T_FATE_MISSING && !t->actions && !t->depends )
    {
        if( t->flags & T_FLAG_NOCARE )
        {
#ifdef OPT_GRAPH_DEBUG_EXT
        if( DEBUG_FATE )
        printf( "fate change  %s to STABLE from %s, " 
                "no actions, no dependents and don't care\n",
               t->name, target_fate[fate]);
#endif
        fate = T_FATE_STABLE;
        }
#ifdef OPT_MULTIPASS_EXT
        else if( !queuedjamfiles )
#else
       else
#endif
        {
        printf( "don't know how to make %s\n", t->name );

        fate = T_FATE_CANTFIND;
        }
    }

I don't know if this change will cause problems elsewhere, but hopefully it will be useful to you.

(1-7/7)