| T O P I C R E V I E W |
| MikeGameDev |
Posted - Jul 03 2018 : 11:05:10 AM
Hi,
I've got some very large (>300k lines) auto generated header files in my solution. VA attempts to parse these files which results in very long initialization times. Is there any way I can specify to VA to exclude these files form parsing?
Thanks |
| 30 L A T E S T R E P L I E S (Newest First) |
| feline |
Posted - Sep 04 2023 : 09:22:16 AM
I have the settings file, and have replied:
case=149977 |
| Wolfram |
Posted - Sep 04 2023 : 08:05:00 AM
I sent you an email |
| feline |
Posted - Sep 04 2023 : 07:24:13 AM
Are you opening a solution or a directory? Currently this setting only works if you are opening a directory, but we are looking to enable this when opening a solution as well.
The line:
"**/vcpkg": true,
should be working correctly, I have just double checked this here. Assuming you are opening a directory, not a solution, would you be able to send me your complete "settings.json" file for me to try testing here? I am wondering if something else in the file could be causing us problems, and thus causing this feature to fail.
Assuming this is possible, please send me the files via email:
[email protected]
including this thread ID or URL in the description, so we can match it up. |
| Wolfram |
Posted - Sep 04 2023 : 05:52:58 AM
Hi,
I think the feature is not working properly.
Here is a part from my used settings.json inside of .vscode folder:
{
... options set before ...
"files.exclude": {
"build": true,
"database": true,
"external": true,
"service/my-service": true,
"out": true,
"TestResults": true,
"tools": true,
"web": true,
"**/vcpkg": true,
"**/buildtrees": true
},
... more stuff comes her ...
}
I want to ignore multiple folders, but when starting Visual Studio you can see in the status bar that these folders are parsed. Since there is a boost installed here inside vcpkg, this takes a long time.
Folder 'external' is excluded, also 'vcpkg' is excluded. But VA is parsing files inside these folders.
I also have set the check mark in the VA options (and restarted of course)
VA_X64.dll file version 10.9.2500.0 built 2023.08.24
DevEnv.exe version 17.7.34024.191 Enterprise
msenv.dll version 17.0.34018.315
Comctl32.dll version 6.10.19041.1110
Windows 10 10.0 22H2 Build 19045.3324
16 processors (x86-64) |
| feline |
Posted - Jul 11 2023 : 07:48:45 AM
Very strange, I would like to understand why this is failing for you, since based on all of my tests this really should be working correctly. At least you have a workaround that is helping. |
| me76 |
Posted - Jul 10 2023 : 1:09:00 PM
At least I found workaround by removing the submodule. If I figure out the root cause, I'll post here. Thanks for your time. |
| feline |
Posted - Jul 10 2023 : 10:53:02 AM
No sign of the problem here. The settings file:
{
"files.exclude": {
"boost": true
}
}
is excluding the boost directory, and the files it contains, when the boost directory is in the base directory I am opening as a folder just fine for me. So I don't know why it isn't working for you. I can send you my test case for this if it helps, but the fact that you are seeing a different effect suggests there is some other factor at work here. |
| me76 |
Posted - Jul 10 2023 : 09:32:08 AM
When I moved boost into tool subdirectory (<project dir>/tools/boost), and excluded tools, it worked, as if there was some special handling for boost name. |
| me76 |
Posted - Jul 10 2023 : 09:03:21 AM
I am trying to exclude boost directory, not main.cpp.
We have many projects in single repository, and some of them use boost, so boost has been added as git submodule to main repo. The projects I am working on don't use boost, so I don't need VA to parse it. That is why I added boost to the test project without using it.
P.S. After I wrote this reply, I realized that I could remove the unused modules from my working directory with git submodule deinit. But I'm still curious why VA ignores exclusion settings for boost. |
| feline |
Posted - Jul 10 2023 : 08:24:52 AM
There is a bug in VA's exclusion handling in that it doesn't support excluding single files. It was designed to exclude directories, which it does handle correctly. So excluding just the "main.cpp" file won't work:
case=149167
But there should be no problems excluding the boost directory. But if you are not using boost, then why add it? Or did you only add it to text excluding a directory? |
| me76 |
Posted - Jul 08 2023 : 9:31:48 PM
It appears that it works for simple my_dir, but doesn't work for boost.
I created cmake project with empty main() function in root directory and empty foo() function in my_dir/foo.cpp. I played with file exclusion, and it worked as expected (foo.cpp was shown or hidden according to exclusion setting in settings.json). Then I added boost library as a subdirectory, and VA parsed it regardless of exclusion setting (I didn't use boost in my code at all).
I also tried .sln solution instead of cmake, I didn't create settings.json file at all, and VA didn't parse boost, I guess because boost was not used in my project.
Can you please test with cmake project with boost subdirectory? My CMakeLists.txt:
project(test)
add_executable(${PROJECT_NAME}
main.cpp
)
|
| feline |
Posted - Jul 07 2023 : 12:17:44 PM
It should not be. Testing using VS2022 and VA 2491.0, I have the following test case. I am telling VS2022 to open the folder:
C:\src_2022\test_shallow_exclude
this folder contains no SLN or project files. It does contain the file:
C:\src_2022\test_shallow_exclude\.vscode\settings.json
which has the content:
{
"files.exclude": {
"unreal": true
}
}
the folder:
C:\src_2022\test_shallow_exclude\unreal
is being filtered out from VA. With the setting:
VA Options -> Performance -> Do not parse files excluded by .vscode\settings.json (requires restart)
turned On VA's Open File in Solution dialog is reporting 36 files in this makefile solution. With this setting turned Off, VA is reporting 44 files in this solution.
I can even send you the test case as an email if it helps, it is rather simple and isn't doing anything clever, but it shows this feature is working quite happily for me, so I don't know why it isn't working for you. Have you tried the simplified, only one exclude rule version of the file? |
| me76 |
Posted - Jul 05 2023 : 3:21:42 PM
Yes, the file is in .vscode directory. The setting in VA Options is on, and I have restarted Studio.
Our project is a cmake project and doesn't have *.sln file (*.sln and *.vcxproj files are generated by cmake in separate directory) - could it be the reason? |
| feline |
Posted - Jul 05 2023 : 12:22:08 PM
The first exclude line should be working, so just using the file:
{
"files.exclude": {
"my_dir": true
}
}
should work as expected, based on all of my tests.
At the risk of asking a stupid question, you have made sure to place the file into a ".vscode" directory, so you end up with the file:
.vscode\settings.json
as described in the setting name:
VA Options -> Performance -> Do not parse files excluded by .vscode\settings.json (requires restart)
and you have turned this setting on and restarted the IDE? |
| me76 |
Posted - Jun 22 2023 : 09:50:39 AM
quote:
Originally posted by feline
We are looking to add support for excluding files and directories when opening a folder, based on the file "settings.json" which the IDE already supports:
case=144472
The feature was officially announced last year, but never worked for me. I have checked the "Do not parse files excluded by settings.json" box in VA settings, and have tried all possible forms to exclude my_dir directory:
{
"files.exclude": {
"my_dir": true,
"my_dir/**": true,
"my_dir\**": true,
"my_dir\\**": true
}
}
with no luck. my_dir is located at top level in the project, at same level as .vscode directory. Why could it not work? |
| feline |
Posted - Mar 10 2022 : 06:22:45 AM
Thank you for explaining, that makes quite a bit of sense. We are looking to add support for excluding files and directories when opening a folder, based on the file "settings.json" which the IDE already supports:
case=144472
I don't currently have an estimate for this, but we are hoping to add this soon, since opening directories is becoming more common. |
| rogerio.ramos |
Posted - Mar 09 2022 : 3:26:03 PM
The files I want to skip are part of the solution because it is VS solution based on a directory. Under that directory, they are many projects. Some of them are test projects not actively being worked on. They duplicate code, sometimes with a stub implementation just to make them compile or with a minimalist implementation for test purposes. VA ends up showing duplicate definitions for many artifacts. It also include artifacts from those test projects I'm not interested in see. So I want to exclude full directories from the search, even though they show up in the solution because it is a directory based solution. |
| feline |
Posted - Feb 28 2022 : 05:55:03 AM
Are the items you want VA to skip part of your solution, or part of your include directories?
If it is part of your include directories then changing:
VA Options -> C/C++ Directories -> Platform
to Custom, and then editing the include directories to include the ones you want, and exclude the ones you don't want VA to see should do a better job.
If you want VA to ignore parts of your solution to speed up parsing then setting:
VA Options -> Performance -> Parse all files when opening a project = OFF
may well do what you are looking for. |
| rogerio.ramos |
Posted - Feb 25 2022 : 2:49:05 PM
Is this still the better way to force VA to ignore code? I'm looking for ways to make VA to skip specific files and directories. |
| feline |
Posted - Jul 16 2018 : 10:31:20 AM
We are considering changes to help VA ignore files or code that is not currently being compiled, but given the out of memory crash, instead of waiting for a change in VA to make this easier, I would like to get file ignoring working now, somehow.
Currently VA is parsing code that has been excluded from the build, which is by design, so that VA can help you with DEBUG or Linux code, for example, while working under Windows. But this means that we cannot use this method here to help VA ignore your code.
Do you know much about why the asm change effected your build process? In theory, since it is surrounded by the #if block, it should have no effect on a C or C++ compiler, but obviously that was not enough to make it safe. I have a variation of this that, in theory, will be safe and will work.
Can you please place the following lines at the top of your generated header files:
#define VA_IGNORE_REST_OF_FILE
VA_IGNORE_REST_OF_FILE
it doesn't matter to much what the #define evaluates to, so long as it won't effect your build process, so perhaps change the #define to a comment?
Now create a new text file called "va_stdafx.h", and make sure that it is in the same directory as your .SLN file. Do not add this file to your solution, VA looks for this file in this location automatically, and if we find it, it is parsed before anything else is parsed, so we can use it to help VA understand confusing code, or in this case, help to hide code from VA.
Now edit the file so it contains the line:
#define VA_IGNORE_REST_OF_FILE _asm {
and make sure that the file ends with a blank line. In the IDE now press the button:
VA Options -> Performance -> Rebuild symbol databases
and restart your IDE. For me, this is causing VA to ignore all code in the header files that start with the #define.
I am assuming that your entire solution directory does not get re-build, thus removing the "va_stdafx.h" file, is this a safe assumption?
This does still leave you having to modify the generated header files before VA see's them, which isn't ideal. |
| MikeGameDev |
Posted - Jul 14 2018 : 8:32:11 PM
What did you have in mind? I can make changes to my local copies of the file, but I can't change the generation process. I have tried the "asm" suggestion earlier, but that had an impact on the build process.
The "pre-defining file pre-processor" suggestion I gave earlier seems to be working well with the default VS Intellisense, I guess they don't parse inactive blocks.
I never thought I'd be in a situation where VA worked too well  |
| feline |
Posted - Jul 14 2018 : 06:48:35 AM
Ouch! I had not realised just how massive these auto-generated files were!
Is editing the process that generates them an option? We have a couple of methods for getting VA to ignore the files, IF we can have them edited slightly before VA see's them.
The problem is that VA is designed to try rather hard to find all of the include files, since missing these tends to cause all sorts of problems. But here we have the opposite situation. |
| MikeGameDev |
Posted - Jul 13 2018 : 5:17:24 PM
When VS memory usage is at 2.8 GB the number of symbols is 2,048,000.
Note that VA is still parsing. It's those auto generated files that are killing us.
EDIT: The number fluctuates, the highest I've seen was 4,096,000... |
| feline |
Posted - Jul 13 2018 : 10:19:39 AM
This is a very small number of files for this much memory usage, and the parsing times that it seems you are running into.
Can you look in VA's Find Symbol dialog please, and see what the second number is when you set:
Find Symbol dialog -> Show only symbols defined in current solution = On
Find Symbol dialog -> Only classes, structs & namespaces = Off |
| MikeGameDev |
Posted - Jul 12 2018 : 2:51:38 PM
The second number is 2353. The files I want to ignore are not in this list, which I believe is a good thing. |
| feline |
Posted - Jul 12 2018 : 2:40:23 PM
This page explains where VA's symbol database is stored, since this varies with the version of the IDE you are using, its not as simple as one given path:
https://wholetomato.fogbugz.com/f/page?W332
Deleting entries manually from the symbol database files is definitely not recommended. I don't actually know what effect this would have, but I would expect problems, since manually deleting sections of any structured data files tends to be a bad idea, unless you can get all of the references, and just the references you want removed.
How many files do you have in your solution?
If you open VA's Open File in Solution dialog (Alt-Shift-O) the title bar contains two numbers. The first number is the number of files currently listed, which changes as you filter the list. The second number is the total number of files in the list, which is normally the number of files in your solution. What is this second number? |
| MikeGameDev |
Posted - Jul 11 2018 : 4:47:36 PM
I've been experiencing intermittent crashes, but I just thought it was a VS quirk, I never considered that it could have been VA. Memory usage with VS + dormant VA is still in the 1.2 GB area after a few hours, which is great. No crashes either. Once I enable VA it jumps up to 2.5+ GB after a few seconds. Stock Intellisense is actually holding up pretty well which is surprising.
Are the VA_M.idx and VA_D.idx files in the AppData folder the VA database? Looks like they are, as I can find symbols from the auto-generated header files in them. Currently mine are 130 MB and 90 MB in size, could I manually edit these file to remove some symbols I don't care about?
|
| feline |
Posted - Jul 11 2018 : 3:52:49 PM
This does sound like it could be an out of memory crash. The IDE is a 32bit process, so it is capped at a maximum of 4gig of memory, which is more like 3 in reality.
Is this a new problem? You have not mentioned this before.
Doing a rebuild of your VA symbol database MIGHT help here. We do try to keep VA's memory usage to a minimum, so this should not happen, but clearly something is going wrong here. |
| MikeGameDev |
Posted - Jul 11 2018 : 12:14:26 PM
Hmm, for now it looks like I need to disable VA as it seems to crash my VS. Is it possible that VS is running out of memory? Once VS memory usage hits the 3 GB range things become unstable, VS (the IDE itself) starts notifying me of exceptions. With VA disabled memory usage stays in the 1.2 GB range. |
| feline |
Posted - Jul 10 2018 : 5:30:42 PM
Without a good test case here to try out the idea, this is just an educated guess.
First try and make sure that all of the generated header files are in a single directory tree, since this is the structure we are expecting.
Now add a new dll project to your solution, and make sure it is called "UE4". Add all of the generated header files to this project. You will probably want to exclude the project from the build, just to make sure it does not effect your build process.
Make sure you have the VA options:
VA Options -> Unreal Engine -> Enable support for Unreal Engine 4 = On
When a solution contains a project named UE4 = True
and restart the IDE.
IF this works VA will still have to parse the generated header files once, but after this, should not be so keen to re-parse them. However, if the files are still changing all of the time, I am not sure if this will work as desired. |
|
|
