Workspace generation - relative paths, Lua, and include paths

Added by Ben Hymers over 8 years ago

Carrying on from the other thread, I have a few questions about workspace generation and generated workspaces (specifically for vs2010).

First of all, it looks like all the paths referenced by anything that's been generated are absolute. I've been trying to keep everything relative so users can check out my code to wherever they like, and I've also been trying to check in many generated and built files to make it very easy to get started. With absolute paths I can't check in the generated files which would mean an extra step before users can get started, but with relative paths I could check in the project files. Would it be a lot of work to make this change? Would it even be possible?

Secondly, this is probably a stupid question, but is Lua required by the generated workspaces? I see when I generate a workspace for vs2010 there are some Lua files there. Again this would be another dependency users of my repository would have to set up and I'd rather it was either not needed. It's not so bad though since I can check in Lua binaries too if needed.

Lastly, it would be nice if the generator could fill out the include paths in the generated projects - although they're not used for the build, tools like Visual Studio's Intellisense read them.


Replies (5)

RE: Workspace generation - relative paths, Lua, and include paths - Added by Joshua Jensen over 8 years ago

Ben Hymers wrote:

First of all, it looks like all the paths referenced by anything that's been generated are absolute. I've been trying to keep everything relative so users can check out my code to wherever they like, and I've also been trying to check in many generated and built files to make it very easy to get started. With absolute paths I can't check in the generated files which would mean an extra step before users can get started, but with relative paths I could check in the project files. Would it be a lot of work to make this change? Would it even be possible?

The workspace generator creates a completely out of source build structure that can even reside on another drive. That is one of the reasons why absolute paths are used. Another reason is that it is entirely possible for Jamfile projects to exist on other drives; there is, of course, no way to reference another drive with a relative path.

Workspaces are intended to be created on each user's machine. All intermediate files, output files, IDE project files, the local dependency cache with MD5 command line and content information, and other miscellaneous Jam related files are user/machine specific.

As your users edit Jamfiles, they can right click Build the !UpdateWorkspace project, and the generated solution/projects are updated according to the contents of the Jamfiles.

In a nutshell, the out of source build structure is not intended to be checked in. It is a difficult problem to resolve to everyone's satisfaction. If you'd like to read further information on the pros/cons of such a system, a Google search for the subject involving CMake should turn up items of interest.

If you or someone else would like to submit a patch to handle project generation with relative paths, I would be more than happy to consider its inclusion. For me, the absolute paths work fantastically, as long as I remember that I can't copy the out of source build directory between drives.

Secondly, this is probably a stupid question, but is Lua required by the generated workspaces? I see when I generate a workspace for vs2010 there are some Lua files there. Again this would be another dependency users of my repository would have to set up and I'd rather it was either not needed. It's not so bad though since I can check in Lua binaries too if needed.

If you are providing JamPlus to your users, it works best to check in the jamplus/bin/ directory as a whole. That grabs the Jam executable, Jambase, C module, C platform/compiler support modules, and the workspace generator. The jamplus/bin/ directory is architected to enable all platform binaries (Windows/Linux/Mac OS X) to sit side by side and checked into the same source control.

A miniature LuaPlus distribution (approximately 735k... and looking at it, the 150k ziparchive module is for the test suite only) is provided to make the workspace generator work. It is not required for builds, although you can take advantage of Lua scripting functionality in the middle of the Jamfiles should you desire.

So, you only need the small set of Lua binaries if you are using the workspace generator.

Lastly, it would be nice if the generator could fill out the include paths in the generated projects - although they're not used for the build, tools like Visual Studio's Intellisense read them.

This was a bug. I'm not sure when it became broken, because it used to work fine. I have checked in a fix. A new nightly build hasn't been posted for it, though, so you'll have to grab the fix from the Git repository.

Josh

RE: Workspace generation - relative paths, Lua, and include paths - Added by Ben Hymers over 8 years ago

Joshua Jensen wrote:

The workspace generator creates a completely out of source build structure that can even reside on another drive. That is one of the reasons why absolute paths are used. Another reason is that it is entirely possible for Jamfile projects to exist on other drives; there is, of course, no way to reference another drive with a relative path.

Workspaces are intended to be created on each user's machine. All intermediate files, output files, IDE project files, the local dependency cache with MD5 command line and content information, and other miscellaneous Jam related files are user/machine specific.

As your users edit Jamfiles, they can right click Build the !UpdateWorkspace project, and the generated solution/projects are updated according to the contents of the Jamfiles.

In a nutshell, the out of source build structure is not intended to be checked in. It is a difficult problem to resolve to everyone's satisfaction. If you'd like to read further information on the pros/cons of such a system, a Google search for the subject involving CMake should turn up items of interest.

If you or someone else would like to submit a patch to handle project generation with relative paths, I would be more than happy to consider its inclusion. For me, the absolute paths work fantastically, as long as I remember that I can't copy the out of source build directory between drives.

Ok, that makes complete sense. I appreciate it's a hard problem to solve and that this is a good solution. For now I'm happy to check in a batch file to run the generator to put project files inside the repository (but ignored by git), and to check in the LuaPlus binaries along with the JamPlus binaries.

If you are providing JamPlus to your users, it works best to check in the jamplus/bin/ directory as a whole. That grabs the Jam executable, Jambase, C module, C platform/compiler support modules, and the workspace generator. The jamplus/bin/ directory is architected to enable all platform binaries (Windows/Linux/Mac OS X) to sit side by side and checked into the same source control.

A miniature LuaPlus distribution (approximately 735k... and looking at it, the 150k ziparchive module is for the test suite only) is provided to make the workspace generator work. It is not required for builds, although you can take advantage of Lua scripting functionality in the middle of the Jamfiles should you desire.

So, you only need the small set of Lua binaries if you are using the workspace generator.

Yep, I'm actually already doing this, I really like the way the bin directory is structured :) I've got a clone of your repository on my own server with the .gitignores altered to allow binary files, which I'm including in my other projects using git-subtree, so users don't have to get/build/set up JamPlus and LuaPlus. Works like a charm.

This was a bug. I'm not sure when it became broken, because it used to work fine. I have checked in a fix. A new nightly build hasn't been posted for it, though, so you'll have to grab the fix from the Git repository.

Brilliant, thanks for looking at this!

RE: Workspace generation - relative paths, Lua, and include paths - Added by Ben Hymers over 8 years ago

The include path stuff is working properly now; thanks!

I have two more questions about workspace generation though:

Firstly, would it be possible to put headers in the workspaces? I can understand why that would be a pain, since there's no way for Jam to distinguish between includes from e.g. boost (which you wouldn't want to include in a workspace), and includes from your own project (which you would want in a workspace). Perhaps some jambase rules for manipulating workspace generation would be useful? Like some way to specify a set of files you definitely do want in workspaces, and which target/project you want to associate them with.

Secondly, would it be possible to organise the files in some way, possibly mirroring their directory layout? Currently all my sources are in one flat list. This is probably the sort of thing you've done already though; perhaps it's just broken for vs2010? That does things a bit differently to previous Visual Studio versions; the structure of the project is defined in a '.vcxproj.filters' file rather than the project file itself.

If these are low on your priority list, I wouldn't mind taking a look if you were to point me in the right direction; my current project at work is going to be winding down soon(ish) so I may finally have some free(ish) time on my hands ;)

RE: Workspace generation - relative paths, Lua, and include paths - Added by Joshua Jensen over 8 years ago

Ben Hymers wrote:

Firstly, would it be possible to put headers in the workspaces? I can understand why that would be a pain, since there's no way for Jam to distinguish between includes from e.g. boost (which you wouldn't want to include in a workspace), and includes from your own project (which you would want in a workspace). Perhaps some jambase rules for manipulating workspace generation would be useful? Like some way to specify a set of files you definitely do want in workspaces, and which target/project you want to associate them with.

Is there a reason you don't just add the headers to the SRCS lists for calls to C.Library and C.Application? I do so. They aren't necessary for the build, but they properly populate the workspace.

As far as Boost headers, you could always create an extra project for them:

Project Boost : c:/boost/**.h ;

Solution YourSolution : Boost ;

Secondly, would it be possible to organise the files in some way, possibly mirroring their directory layout? Currently all my sources are in one flat list. This is probably the sort of thing you've done already though; perhaps it's just broken for vs2010? That does things a bit differently to previous Visual Studio versions; the structure of the project is defined in a '.vcxproj.filters' file rather than the project file itself.

AutoSourceGroup YourProject : $(SRCS) ;

If you prefer to generate your own source groups, you use:

SourceGroup YourProject : My\\Deep\\Folder : $(SRCS) ;

Take care!

Josh

RE: Workspace generation - relative paths, Lua, and include paths - Added by Ben Hymers over 8 years ago

Ah, I didn't see those rules, I was looking in the modules section of the documentation! Yep, those look like they'll do exactly what I want. Thanks!

As for putting headers in SRCS, I guess that just wasn't intuitive to me; if it won't cause any problems then there's no reason for me not to do that. Thanks again :)

(1-5/5)