Whole Tomato Software Forums
Whole Tomato Software Forums
Main Site | Profile | Register | Active Topics | Members | Search | FAQ
User name:
Password:
Save Password
Forgot your password?

 All Forums
 Visual Assist
 Technical Support
 VS2019 CMake+VAX issues
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

kornman00
Junior Member

USA
23 Posts

Posted - Jan 29 2021 :  5:57:14 PM  Show Profile  Reply with Quote
A project at work switched to CMake last year. Ever since then, when I open the project's folder in VS2019, it will eventually crash soon after when I have VAX enabled. If I disable VAX, VS2019 does not crash.

With VAX disabled, VS2019 sits at around 800MB of used memory with this project open and idle.

With VAX enabled, I can watch VS2019 continue to grow in Task Manager until it ends up crashing. Usually around 2785MB or so (since VS2019 is 32-bit, this suggests to me a possible OOM)

I enabled logging in VAX and ended up with a 21.7MB va.log file. Inspecting this file, there's an enormous amount of CollectFolderProjs logs (231013 instances), including the last line itself.

I suspect the problem relates to CollectFolderProjs not respecting the "ExcludedItems" array in VSWorkspaceSettings.json. This is what we change to make VS not populate folders or file patterns in the solution explorer.

Our project's root folder has a CMakeList.txt, and so the process is to open the root. However, there are many sub-projects (in sub-folders) which are *not* part of the CMake setup. So their root sub-folders are in "ExcludedItems".

These sub-projects include not only code (still on sln/vcxproj) but GBs of tools and source content in thousands of folders (they're individual games).

I suspect that since CollectFolderProjs isn't ignoring the sub-projects/folders we list in "ExcludedItems" that it ends up not only wasting cycles enumerating those folders, but is allocating memory from those scan results and the crash itself may be an OOM. For most other crashes in VS2019, I get a .dmp file in AppData\Local\CrashDumps. However, with this crash with VAX enabled there's no dump generated.

Another thing I tried was first disabling VAX, opening the project folder, then re-enabling VAX with the folder still opened. This doesn't result in a crash, because it doesn't automatically enumerate the project to gather intellisense data upon re-enabling.

Given all of this, whenever I want to work in the CMake project, I currently have to disable VAX altogether. Which is annoying to (remember) to do, but also means I'm left without my VAX-based Intellisense/code highlighting safety blanket.

feline
Whole Tomato Software

United Kingdom
17186 Posts

Posted - Feb 01 2021 :  05:49:22 AM  Show Profile  Reply with Quote
This does indeed sound like an out of memory problem. Which version of VA are you using? VA 2393 introduced several changes to try and reduce our memory usage with large solutions, so if you are using an older version it is worth trying the current version to see if any of these changes help.

http://www.wholetomato.com/downloads/default.asp

Are you able to load this directory / CMake project with VA enabled long enough to use VA's Open File in Solution dialog to get a count of the number of files VA thinks are in the "solution"? I have a large test solution, an actual .SLN file, that holds 94,000 files, but since this uses a SLN file, and it probably does not have as complex a directory structure, I am not sure how well it compares to your situation.

I am also wondering if this is anywhere near the size of your directory tree.

VA's Open File in Solution dialog not respecting the ExcludedItems setting defined in VSWorkspaceSettings.json is a known bug:

case=141644

so it makes sense that VA is doing some parsing of all of these files and directories, exactly as the log suggests.

zen is the art of being at one with the two'ness
Go to Top of Page

kornman00
Junior Member

USA
23 Posts

Posted - Feb 03 2021 :  12:11:48 AM  Show Profile  Reply with Quote
Thanks, feline! I was on VA 2393. I'm noticing now that 2399 is out so I'm in the process of getting that and will attempt to repro the issue again. Even though I had "Automatically check for new version" enabled, 2393 wasn't notifying me there was a new version for some reason.

On 2393, Open File in Solution was reporting 0 out of 0 files up to the point of crashing (I was opening it then closing periodically after VAX's 'Reading solution projects...' message box came up then went away).

As for how large my directory tree is, this is what I have mapped down from Perforce (this is going to include build artifacts and such, plus a chunk of game-ready data for a couple of games):
2,829,261 Files, 321,181 Folders

I see that VSWorkspaceSettings.json was removed from docs in this PR from June 2019: https://github.com/MicrosoftDocs/visualstudio-docs/pull/3499 . "The file was never intended to be user-editable"

I'm guessing the direction they intend to go is with CppProperties.json? https://docs.microsoft.com/en-us/cpp/build/open-folder-projects-cpp?view=msvc-160

Although, VSWorkspaceSettings.json is called out in the Workspace settings docs too (which I'm guessing is how VAX would try and read "ExcludedItems", if that's the fix) https://docs.microsoft.com/en-us/visualstudio/extensibility/workspaces?view=vs-2019

I also need to see if setting a "files.exclude" in a .vs\settings.json file also gets picked up (that may also be the direction they're wanting to go, to align with VS Code?) https://code.visualstudio.com/docs/getstarted/settings.

Okay, I got 2399 setup and running while typing this up. It took a lot longer to crash (5-6mins, instead of <2mins maybe?). In Task Manager before it crashed (while I was writing this), I was seeing the overall VS2019 memory sitting around 3200MB (and counting) while the Visual Studio subnode itself is 2980MB (and counting). It was also heavily exercising disk IO that entire time, probably trying to process/parse nearly 3mil files (of course, not all of them source or even text).

Edited by - kornman00 on Feb 03 2021 12:29:54 AM
Go to Top of Page

feline
Whole Tomato Software

United Kingdom
17186 Posts

Posted - Feb 03 2021 :  08:20:16 AM  Show Profile  Reply with Quote
It is "good" to see our efforts to limit our memory usage are working, but clearly they are not enough here. I am going to set up a much bigger stress test here, this really should not be happening.

zen is the art of being at one with the two'ness
Go to Top of Page

feline
Whole Tomato Software

United Kingdom
17186 Posts

Posted - Feb 04 2021 :  08:02:21 AM  Show Profile  Reply with Quote
First, this may be a "silly" question, but if you are using CMake along side Visual Studio, have you considered using CMake to generate a Visual Studio solution, containing just the parts of the directory tree that you want to work with?

In the "VSWorkspaceSettings.json" file what syntax are you using to exclude a directory? I have a fairly large test case here, but so far trying to exclude directories from showing up in Solution Explorer isn't working for me. Given that all documentation for this file has been removed, it does suggest that this isn't a solid long term approach, but it does work for now.

As a temporary work around, would making a "fake" root folder be an option?

Assuming you have something like this:

C:\all_codeC:\all_code\old_game_oneC:\all_code\old_game_oneC:\all_code\live_code_oneC:\all_code\utility_code
and only want VA to pick up the live and utility code, create a new "fake" root directory:

C:\all_code\va_fake_root
and then use Windows file system junction points to add just the directory trees you want into the fake root directory. The junction point makes the directory appear to be in two different places at the same time, similar to a shortcut, and no extra storage is used. Unlike a shortcut, when you move into the junction directory, the directory really does appear to exist in this new location.

In this example, open a command prompt in the directory "C:\all_code\va_fake_root\" and run the commands:

mklink /J live_code_one "C:\all_code\live_code_one"
mklink /J utility_code "C:\all_code\utility_code"

this will cause the directory structure:

C:\all_code\va_fake_root\live_code_oneC:\all_code\va_fake_root\utility_code
to appear, along with all of the sub-directories.

zen is the art of being at one with the two'ness
Go to Top of Page

kornman00
Junior Member

USA
23 Posts

Posted - Feb 04 2021 :  6:08:24 PM  Show Profile  Reply with Quote
It's a fair question. I personally haven't tried to generate VS solutions from the CMake configs. For additional context, we use CMake+Ninja+UBT(UE4) for building the project. CMake+Ninja build some shared non-UE4 specific code and libraries, then CMake invokes UBT to build the actual UE4 project. I think there's also desire to setup the UE4 project so it is also fully CMake+Ninja and instead only invokes UHT beforehand, but I'm not sure of the if/when's of that.

For an example VSWorkspaceSettings.json based on what is in use, check this out: https://gist.github.com/kornman00/494971efa7916969c9a9bb10f1a74eb8

I've been informed that VS itself also supports the .vscode/settings.json file, which can exclude things via "files.exclude" https://code.visualstudio.com/docs/getstarted/settings. The VSWorkspaceSettings.json ExcludeItems setup is still a supported method in Visual Studio proper (seems things have changed since 2019), but "files.exclude" may be the more obvious solution to target instead (if not targeting both).

Thanks for the junction points workaround suggestion! I'll look at setting aside some time next week to give it a try
Go to Top of Page

feline
Whole Tomato Software

United Kingdom
17186 Posts

Posted - Feb 05 2021 :  09:38:56 AM  Show Profile  Reply with Quote
Knowing that you are using Unreal Engine adds in another factor. Visual Assist does have built in support for Unreal Engine, but so far I have only ever used this when working with a Visual Studio SLN file. To try and help this along, one of the new features in our latest build is:

Added support for reading configuration information from a new Visual Assist specific section of Unreal Engine solution files. (case=143077)

which isn't going to work if you are not using UE solution files, even if you are using a version of UE that adds this information to its solution files.

Part of our detection for the presence of Unreal Engine is a project inside your solution called "UE4", again solution and project based thinking.

zen is the art of being at one with the two'ness
Go to Top of Page

feline
Whole Tomato Software

United Kingdom
17186 Posts

Posted - Feb 05 2021 :  1:05:37 PM  Show Profile  Reply with Quote
Thank you for pointing me at the Visual Studio Code exclude files settings, that does indeed exclude files from both Visual Studio 2019 and Visual Studio 2017, so I have put in a feature request for VA to watch for, and respect this setting if found:

case=144389

zen is the art of being at one with the two'ness
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
© 2019 Whole Tomato Software, LLC Go To Top Of Page
Snitz Forums 2000