| Author |
 Topic  |
|
|
jay.carlton
Ketchup Master
USA
65 Posts |
 Posted - Apr 05 2013 : 09:50:45 AM
|
Hi,
So we have hundreds of project files for our application, whose source files share common and overlapping dependencies, with a total line count in the tens of millions. We don't currently configure these under a single solution, so while VA X is extremely fast at finding the sources of symbols (looking down the dependency graph), we don't have a way to look Up from a low-level symbol to find all references in all possible projects.
Is there a way to provide a list of projects to parse, and have VA X do a background parse of all of them? If so, is there a way to launch a large parsing job from the command line? If we could do that, could we potentially push a pre-parsed VA X database to all interested developers when we deploy a new build of our application? I don't know how long it will take to parse everything, but I would tend to think it would be faster if we could centralize that overnight than for everyone to parse everything with each build. Alternatively, it possible to trigger the parsing of files when they're checked into source control, and to check in the VA X database along with the source files?
If that's not possible, I'm thinking I can build a solution file to hold all the projects, but it would be difficult to have it open all at once. Can VA X be configured to parse projects that are Unloaded from Visual Studio, or does each project have to be loaded to be parsed?
Thanks again,
Jay |
|
|
accord
Whole Tomato Software
United Kingdom
3287 Posts |
Posted - Apr 05 2013 : 2:18:16 PM
|
Quite a big project. Like you said, VA is quite fast, even its parsing. I have never tried committing the database, it may introduce its own problem and wouldn't really be supported, officially. Maybe it will not work at all.
One option, maybe the best, would be to use one large solution and see how it works, and what time does it take for VA to parse everything. One thing you should know about this is that VA works a little bit differently than the compiler, so will only parse changed headers, and not the ones that are dependent on the change ones so it may be feasible to work with one solution. What do you think?
Another option is to enable the following setting:
VA Options -> Projects -> File handling -> ...if solution is not empty
As the tooltip says it will make VA parse all other files in a directory when you open a file from that dir. This is not a perfect solution, though, since not all files will be parsed, but more and more will as you work.
Future option: we are considering to add ability for user to define additional directories to parse into solution:
case=21519 |
 |
|
|
jay.carlton
Ketchup Master
USA
65 Posts |
Posted - Apr 05 2013 : 3:00:30 PM
|
Thanks for the quick response. That option sounds promising. I've already got the "if solution is not empty" box checked.
I tried briefly Unloading a project from a solution, and it appears VA X "forgets" the symbols in that project when I do that. I think if I had to have everything open and loaded to get everything parsed, I'm not going to be able to scale all the way up yet. Also, if I'd lose the database as soon as I closed the massive umbrella solution, and wouldn't be able to share it, I'm not sure that's going to be a scalable approach.
In general, the problem of "sister projects" is interesting to me. When I'm debugging and step into code from another dll and/or project, it can be tricky to find references to symbols from there, even if that project has been parsed separately (in another visual studio session). While it's very fast looking downwards, sideways and up are more difficult. The use case I have in mind is in refactoring low-level functions. Currently we are scared to do that because it's too difficult to find all the references to it.
Are there any plans for a hosted/distributed symbol database tied into source control, so that a team of developers could avoid individually parsing identical files?
Thanks again. |
|
|
|
feline
Whole Tomato Software
United Kingdom
18750 Posts |
Posted - Apr 05 2013 : 8:11:18 PM
|
One thought here. With a dedicated machine, running overnight, if you make sure you have:
VA Options -> Performance -> Parse all files when opening a project
turned on, and the same stable include directories as everyone else setup, you could launch the IDE, passing the full solution as a command line argument. The IDE will then load the solution, prompting VA to parse the entire solution.
Turning off the IDE intellisense parser will help, since then you only have one parser running, how do to this is described here:
http://docs.wholetomato.com?W133
The tricky bit is working out when VA has finished parsing, but if you time this you could just rely on giving VA plenty of time. Then the generated symbol databases could be copied to each of the machines that need them, completely replacing their current VA symbol databases. You can control where VA stores its symbol database, which might make this a little easier, as described here:
http://docs.wholetomato.com?W332 |
zen is the art of being at one with the two'ness |
|
|
|
jay.carlton
Ketchup Master
USA
65 Posts |
Posted - Apr 08 2013 : 08:45:28 AM
|
Thanks. That's very interesting. That would help with the syndication.
On the scalability side, I think the limiting factor will be the IDE choking when trying to open such a massive solution. If I open the projects one at a time, is there any way to have VA X append the results of parsing those to a single database? (Is that possible interactively, if I have two disjoint projects (that build different dll's, say) that I've opened in separate instances of Visual Studio, and I'm debugging from one but step into a function defined in the other one, is that a re-parse situation, or can it use the symbol definitions from the second project?)
That would seem to me to be stabler, as I fear I won't be able to get Visual Studio to more than a few dozen projects at a time. I'd also still want to be able to access the master symbol database when working in individual projects. That is, have it look for a pre-parsed symbol set on the server, and only do a local parse of that file/project if the one on the server is out of date. Ideally, we'd only have to do the whole system once, since we know which files have changed since the last build from source control. I realize all of this is much easier said than done.
Anyway, keep up the great work. All of my coworkers are raving about your product.
|
|
|
|
feline
Whole Tomato Software
United Kingdom
18750 Posts |
Posted - Apr 09 2013 : 09:23:10 AM
|
I am not sure what would happen with loading just one project at a time. You would need to use a single instance of the IDE for this, since if you have two instances of the IDE open at the same time, VA creates and uses two separate symbol databases, one per IDE instance. This is to make sure we don't get any problems caused by the two IDE instances "clashing" when using the same symbol database for different solutions.
Assuming you don't to anything to trigger a rebuild of VA's symbol database then each time you open a new project VA should be appending this knowledge to the existing symbol database. But this knowledge will come from a separate solution, so it might be tagged separately, rather than as part of the whole. I have never tried something like this, so I don't know what to expect.
Are some of your projects fairly stable? So their code won't change for several weeks at a time, perhaps several months? If so then if you could have a "working solution" that does not include these projects, just references them for linking, and add the stable projects to VA's stable include directories list, then VA will know these only have to be parsed the once. Once they have been parsed when initially building VA's symbol database they will simply be looked up and remembered, and not reparsed.
You cannot add these projects one by one to the stable include directories list, since changing the stable include directories list requires rebuilding VA's symbol database, but there is no need to actually open these projects in the IDE, so long as you point VA at the correct directories so it can find the code to parse on its own.
http://www.wholetomato.com/products/features/directories.asp
Does this sound like it might help? |
zen is the art of being at one with the two'ness |
|
|
|
jay.carlton
Ketchup Master
USA
65 Posts |
Posted - Apr 09 2013 : 11:17:08 AM
|
Thanks. That helps me understand better how things work.
There are some stable projects, but we don't currently treat them differently from a configuration standpoint; everything is on the same build and deployment schedule today. Also, the use cases I had in mind for a whole-application symbol set involve identifying impacts to changes to low-level functions, that is, thus "destabilizing" code other projects might consider stable.
|
|
|
|
feline
Whole Tomato Software
United Kingdom
18750 Posts |
Posted - Apr 10 2013 : 5:01:37 PM
|
Depending on the ease of setting up and using a "normal editing" solution, along side the full build solution, moving the relatively static projects into the VA stable include directories might well help, especially if you combine it with a machine that is producing a reference VA symbol database that gets imported onto the development machines.
If you do this, ideally you should remove the stable include directories / projects from the solution file, since VA does not expect your solution to overlap your stable include directories.
Hopefully this helps a bit, and you can work out a useful approach here. If you have any more questions then please do ask. |
zen is the art of being at one with the two'ness |
|
|
| |
Topic |
|
|
|
Parsing Unopened Projects in the Background?
