How can I run unit tests from my jamfile?
Added by Ben Hymers over 1 year ago
Hi,
What's the best way to add unit testing to a jamfile? By 'add' I mean run a built test executable and report failures. I saw the 'Shell' command - is this what I'm after? Is there some way to get the exit status from a shell-executed command?
Thanks,
Ben
Replies (9)
RE: How can I run unit tests from my jamfile?
-
Added by Joshua Jensen over 1 year ago
Pardon the late response.
I have pushed a new sample called unittest that illustrates this behavior. The basic idea is simple, though.
Assume you have an application called TestSuite, and you want the test suite to run whenever you build. We need the executable name. C.Application returns it to us.
TEST_SUITE_EXE = [ C.Application TestSuite : $(SRCS) ] ;
We need an action to run the test suite:
actions RunTestSuite
{
$(TEST_SUITE_EXE:TC)
}
We'd like the test suite to run all the time, so we'll attach it to the all target. The dependency chain will be all -> tests -> $(TEST_SUITE_EXE). We tell the tests target to run all the time. Without the Always rule, it will only run when the executable is built.
Depends tests : $(TEST_SUITE_EXE) ; Always tests ; NotFile tests ; RunTestSuite tests ; Depends all : tests ;
And that's it! :)
Josh
RE: How can I run unit tests from my jamfile?
-
Added by Ben Hymers over 1 year ago
That's much better than the way I was going to do it. I'll give that a go, I trust that it'll work since there's now a test for it! Thanks very much Josh, it's a pleasure to use such a well-maintained piece of software :)
RE: How can I run unit tests from my jamfile?
-
Added by Ben Hymers over 1 year ago
Ok, here's a question that might be stupid or might not be: my unit test executable outputs a a results file, and I want to put this in another directory so it's simple for the build server to pick it up. But the directory I want to put the file into doesn't exist, so the file doesn't get created. The way I see it there are two options; make the test executable create its output directory, or add the directory as a dependency of the tests in the Jamfile. The second is the most appealing, but I'm not entirely sure how to do it - I could use MkDir, Depends and FGristDirectories as recommended in the docs but I don't understand how jam will know that the directory needs to be created before the executable is run - both those steps are just dependencies of 'the tests'. How can I ensure it's created first? Sorry if this seems like a stupid question; I'm quite a stupid person :)
RE: How can I run unit tests from my jamfile?
-
Added by Joshua Jensen over 1 year ago
Try this:
RESULTS_LOG = <tests>results.log ; # Properly gristed Jam name also gives easy access from the command line. MakeLocate $(RESULTS_LOG) : c:/the/output/directory ; RunTestSuite tests $(RESULTS_LOG) ;
I didn't test this first, but it should be pretty close.
RE: How can I run unit tests from my jamfile?
-
Added by Ben Hymers over 1 year ago
I tried something very similar to that but it just doesn't create the directory. 'test' is the target for building the test executable, 'runtest' is... obvious :)
...
TEST_EXECUTABLE = [ C.Application test : $(TEST_SRCS) ] ;
TEST_OUTPUT_FILE = $(TOP)/../../results/$(CONFIG)/results.xml ;
actions RunTestSuite
{
$(TEST_EXECUTABLE:TC) $(TEST_OUTPUT_FILE:TC)
}
Depends runtest : $(TEST_EXECUTABLE) ;
Always runtest ;
NotFile runtest ;
RunTestSuite runtest ;
MakeLocate $(TEST_OUTPUT_FILE) : $(TEST_OUTPUT_FILE:D) ;
The directory isn't created. If I create it myself, results.xml gets put there, so the paths are all fine, there's just something going on that I don't understand... Do I need to do some gristing (I really need to learn when it's needed)? Is the ordering of these statements important?
I thought of another option; let the test executable create results.xml in its own directory, and copy that to where it's needed. The thing that confuses me about that is how to let jam know that running that particular executable produces a file with that particular name, that things can depend on.
RE: How can I run unit tests from my jamfile?
-
Added by Joshua Jensen over 1 year ago
The problem is that Jam doesn't know anything about TEST_OUTPUT_FILE in relation to runtest. You have to do:
RunTestSuite runtest $(TEST_OUTPUT_FILE) ;
That way, Jam knows TEST_OUTPUT_FILE is to be created during that action, and the dependency tree for TEST_OUTPUT_FILE should be run, creating the directory in the process.
Josh
RE: How can I run unit tests from my jamfile?
-
Added by Ben Hymers over 1 year ago
I realise I'm making myself look stupider with every post, but I'm intrigued; what does putting things after a rule name mean? 'RunTestSuite' doesn't take any arguments, and they're not being given separated by a colon, so how does it know that 'runtest' (arbitrarily the first item in a list) wants it to run, and that $(TEST_OUTPUT_FILE) (the second item in a list) is produced? Or is it just that providing an extra list argument (beyond what is expected, i.e. 0 arguments in this case) has some special meaning?
Back to the point, it's still not working :( I've got LOCATE_TARGET set somewhere before this in the jamfile, would that interfere? If so, how do I unset/reset it?
RE: How can I run unit tests from my jamfile?
-
Added by Joshua Jensen over 1 year ago
I updated the samples/unittest code to handle an output log file. I did it in a slightly different way than above. Let me kind of recap the approach here:
TEST_LOG = <tests>test.log ; MakeLocate $(TEST_LOG) : output ; Clean clean : $(TEST_LOG) ; Depends tests : $(TEST_LOG) : $(ADD_EXE) ;
I have made the output log file dependent on the executable. If the executable updates, then Jam will attempt to build the output log file. Then, as you can see, I made the tests target dependent on the output log file.
Works like a charm.
RE: How can I run unit tests from my jamfile?
-
Added by Ben Hymers over 1 year ago
You're the king of programming, Josh. Thanks :)
(1-9/9)