| T O P I C R E V I E W |
| Pascal Mueller |
Posted - Jan 24 2019 : 03:57:02 AM
Hi,
I am experiencing a very bad performance of visual studio, because of visual assist parsing the whole project, quite often. Because it includes the Unreal Engine source code, parsing the solution takes 12-15 minutes. This happens almost every second day when I start my project for the first time. In these two days only a handful of files have changed, so a whole parsing of the project is not necessary. In general I am very happy with visual assist, but at the moment I am loosing 30-60 min every week because of this. Is there an option to not reparse the whole solution?? What can be done to solve this?
Cheers Pascal |
| 30 L A T E S T R E P L I E S (Newest First) |
| feline |
Posted - Dec 12 2020 : 08:00:15 AM
grahfoster, back to your questions.
The "if Solution is empty" settings are for situations when you are just using the IDE as an editor, or are using an external compiler, so there is no solution to use to get a list of all of the files and directories you are working with. In this situation you can tell VA to parse other files in the same directory, so VA will pick up as you open / reference files.
If you are working with a solution, then VA monitors the solution, and projects, and detects when new files are added, and parses them as it goes along.
A new unreal project generates a solution, so all of the files we should parse are known when the IDE loads the solution, so everything should "just work".
The status bar message proves VA is parsing, as expected. But I don't understand why this only happens some of the time.
Have you used the setting:
VA Options -> Performance -> Use alternative directory for symbol database (requires restart)
to change the location of your symbol database? This should not matter at all, but I am just wondering if some other process could be interacting with VA's symbol database, for example anti-virus software?
You might want to try turning Off the setting:
VA Options -> Performance -> Parse all files when opening a project
to see if this helps much. This isn't normally used, but is useful when you have a very large solution, and don't want VA to check for changes everywhere, and just focus on the files you are editing and directly including. |
| feline |
Posted - Dec 12 2020 : 07:52:57 AM
Apologies, I have not explained the issues here very clearly. There are two separate effects at work here, but they overlap.
By default, VA does not consider directory trees that are part of your solution to be stable include directories. So if you open two different solutions that both point at Unreal Engine, then both solutions will want to parse everything from scratch, or at least check all of the files and see if they need to be parsed. This is based on the, generally reasonable, idea that different solutions don't have that much overlap.
Unreal Engine, being both a stable external library, and part of the solution at the same time, breaks these assumptions.
VA should not parse everything in unreal engine all of the time, regardless, if you are just opening the same solution again. VA will check to see if the files are different than last time, but that should be all that happens. So if we have excess parsing then this is a bug, regardless.
The junction points are used to make sure that VA's "you are working with unreal engine" checks work correctly, which enables certain unreal engine specific features. This also helps with parsing speed, but only when moving between different solutions, since we know that the entire unreal engine project, and all of its files, don't need to be checked when parsing the solution, since they are a stable external library.
I hope this makes a bit more sense. |
| amoldeshpande |
Posted - Dec 11 2020 : 12:12:26 PM
I don't understand what's hard about "Have I indexed a file in a location beginning with c:\something\unrealengine\4.25" before ? It should have nothing to do with parsing project files or trying to figure out install locations.
It doesn't have to be your ideal or final solution to do it this way. Just an interim solution.
It is incredibly frustrating that you guys seem to be waiting (for almost 2 years now) to figure out the most fancy ideal solution, which will probably never materialize.
As far as CPU goes, I don't want to hijack this thread, I'll open a different one.
Edit: I don't view junction points as an acceptable workaround. Too much Windows software gets confused with links and I don't want to deal with that headache. |
| feline |
Posted - Dec 11 2020 : 11:26:24 AM
amoldeshpande, better support for Unreal Engine location is a high priority bug report, and it is something we would like to get working fairly soon.
There are definitely clues to the location of unreal engine in the project files, the problem is that there are various special cases we have to deal with when working with unreal engine. It seems simple at first, but the more you look, the more odd situations you tend to encounter.
For now, if you are able to use the file system junction point then this seems to be a solid work around, at least in all of the tests I have run so far. If you are having problems with this I am more than happy to look at this with you.
For VA parsing impact, have you tried any of the settings on VA's Performance page? Specifically:
VA Options -> Performance -> Enable multithreaded parsing
VA Options -> Performance -> Limit the maximum number of concurrent threads to X (requires restart)
VA Options -> Performance -> Thread priority = Normal / Below Normal / Lowest (requires restart)
I have changed the number of concurrent threads to 3 out of 8 (8 core system) and triggered a VA symbol database rebuild, and loaded an Unreal Engine solution, for VA to parse.
Using the build in Windows 10 Resource Monitor reports that devenv.exe is now using 38% of the CPU time on my system, in line with this setting, so these settings should be working for you, to limit the CPU impact of VA parsing everything. |
| amoldeshpande |
Posted - Dec 10 2020 : 4:21:03 PM
quote:
Originally posted by feline
This sounds interesting. Unfortunately I have just opened 5 different copies of "UE4.vcxproj" from 5 different Unreal Engine game projects I have on one of my test machines.
Only 3 of these files have the BuiltWithUnrealBuildTool section. Where the section is present it does seem really rather helpful.
However the other 2 files have no such section. These are two different game projects created from the git source code downloaded and compiled version of Unreal Engine. I have even done a find in files across the project directory tree for the string "BuiltWithUnrealBuildTool", just to be sure, and no references.
In part the problem stems from the fact that Unreal Engine breaks one of our basic assumptions about solution structure, which is that your solution doesn't actually contain stable include directories as one of the projects. Normally this is a reasonable assumption, but not here.
There is nothing in the two project files that points to which version of Unreal Engine they are using ? That seems strange. What do their OutDir and Intdir properties look like ?
In any case, can you please at least optimize the more general case where we are not building UE4 from source ? Every time I create a new UE4 project VA wastes 30 minutes parsing the already parsed version of UE that I am using.
Even just matching the path prefix from the registry entries should be sufficient for the general case.
I'm getting really tired of my computer being unusable because VA doesn't respect the multiprocessing limits. If the UE4 indexing was fixed at least that annoyance would be smaller. |
| grahfoster |
Posted - Dec 10 2020 : 4:14:59 PM
Thanks for responding! I currently have "if Solution is empty" checked. I also have "Parse all files when opening a project" checked, which seems to be the default option. I didn't know what behavior to expect if I unchecked this (specifically, I worried about new files not being parsed). Also, if that box is unchecked, what happens when I generate a new solution for a new Unreal project?
It does definitely say "VA: Parsing." I have disabled Intellisense and even encountered this problem while the database was disabled, though I later restored it for convenience when it didn't improve anything.
As a side note, VA loaded my solution just fine today in about 30 seconds without having to parse. |
| feline |
Posted - Dec 10 2020 : 06:41:22 AM
Do you see "all" of your Unreal Engine sub-directories in the stable include directories list? Checking they are all there is a pain, so checking if you have a long list of them is ample.
What do you have the following set to:
VA Options -> Projects and Files -> Upon opening a file, parse all other unparsed files in the same directory:
if Solution is empty
if Solution is not empty
VA Options -> Performance -> Parse all files when opening a project
Are you seeing the status bar say "VA: parsing xxx"? I am just double checking that VA is actively parsing everything, and its not the IDE intellisense parser running instead.
This should not be happening, but clearly something is going on here. |
| grahfoster |
Posted - Dec 09 2020 : 2:30:27 PM
Just wanted to chime in that I'm having this problem as well. I'm on the trial version of VA and plan to purchase the full version when that runs out, regardless of whether this problem is fixed, but it would be great to get it ironed out eventually if possible.
Here is my system info copied from the VA options menu:
License: trial
VA_X.dll file version 10.9.2393.0 built 2020.10.28
DevEnv.exe version 16.8.30717.126 Community
msenv.dll version 16.0.30717.112
Comctl32.dll version 6.10.18362.1198
Windows 10 10.0 1909 Build 18363.1198
8 processors (x86-64, WOW64)
Language info: 1252, 0x409
Platform: Project defined
Stable Includes:
// whole lotta stuff
I have verified that I have Unreal Engine support enabled and that my stable includes are all there. Generally what happens is that every two days, or even within 24 hours, VA will have to re-parse all the files in my solution, which usually takes around 15 to 20 minutes. The last time this happened, I had literally done nothing to the project in the intervening time (though I'm sure some automatic updates could have happened to affect something somewhere). I'm currently running Unreal 4.26 but this problem was present on 4.25 as well. Let me know if there's any more info I can provide to help you find a fix for this. I haven't had any recent Visual Studio or Unreal crashes, so I know it isn't related to that. Just curious what could be provoking the re-parse.
Forgot to mention that during the reparse, VA can still recognize Unreal's symbols. It's just the autocomplete and a few other features that I end up having to wait around for.
|
| feline |
Posted - Dec 02 2020 : 12:19:32 PM
This sounds interesting. Unfortunately I have just opened 5 different copies of "UE4.vcxproj" from 5 different Unreal Engine game projects I have on one of my test machines.
Only 3 of these files have the BuiltWithUnrealBuildTool section. Where the section is present it does seem really rather helpful.
However the other 2 files have no such section. These are two different game projects created from the git source code downloaded and compiled version of Unreal Engine. I have even done a find in files across the project directory tree for the string "BuiltWithUnrealBuildTool", just to be sure, and no references.
In part the problem stems from the fact that Unreal Engine breaks one of our basic assumptions about solution structure, which is that your solution doesn't actually contain stable include directories as one of the projects. Normally this is a reasonable assumption, but not here. |
| amoldeshpande |
Posted - Nov 26 2020 : 1:08:03 PM
Well, I don't know the ins and outs of VaX, but every UE4.vcxproj seems to have a BuiltWithUnrealBuildTool configuration (or you could just go by project file name) to detect that this is a UE4 project. Within the project file, there seem to be OutDir and IntDir properties which point to the engine version in use by that project.
If all we're interested in is finding the UE4 source tree for a given project.
Rooting around in project files may seem weird, but it's likely to be a pretty stable approach since Epic doesn't usually go nuts changing things in the toolchain. You could probably ask on their licensee forums (or one of the users here who has a license could) if they have another, recommended way to detect the source tree.
|
| feline |
Posted - Nov 26 2020 : 12:28:35 PM
Ah, I knew it had to be somewhere 
Defining the location of the Unreal Engine tree sounds really simple, and in simple cases it is. Unfortunately once you get into complex cases, I have yet to pin down a good, simple rule that works well. Some of our users are also working with customised versions of Unreal Engine, so simply relying on features of the "original" Unreal Engine tree breaks down in these cases.
If you have a good plan I would certainly be interested! |
| amoldeshpande |
Posted - Nov 26 2020 : 12:21:22 PM
quote:
Checking on this machine, the registry key lists 5 different install locations, for 5 different versions of Unreal Engine, but there is no mention at all of the version built from the downloaded source code.
Well, obviously that's under HKCU\Software\Epic Games\Unreal Engine\Builds\someguid  |
| feline |
Posted - Nov 26 2020 : 11:22:53 AM
Unfortunately it's not that simple. I have one machine that has 2 or 3 different versions of Unreal Engine installed via the Epic Games launcher, it also has a separate version of the Unreal Engine source code downloaded from Github and compiled.
Checking on this machine, the registry key lists 5 different install locations, for 5 different versions of Unreal Engine, but there is no mention at all of the version built from the downloaded source code. |
| amoldeshpande |
Posted - Nov 26 2020 : 09:45:24 AM
quote:
Originally posted by feline
Thank you for the update, I am glad that the junction is helping. This is a work around for now.
The basic problem is that Unreal Engine is complex / hard for VA to handle correctly. You can easily have several different versions of the Engine installed at the same time, in different locations. Since the engine is listed as part of the solution you are opening, its really not obvious that this is actually a stable external library and should be treated differently to the rest of your solution.
The more I look into this problem, the more strange edge cases I seem to encounter.
Not trying to be a jerk here, but have you considered looking at HKLM\Software\EpicGames\UnrealEngine\4.xx for the install location and then parse the project file to find the version being used in the project to match ?
The toolchain architecture that Epic makes tends to be stable for years of an Engine version's life and major changes would roll out very slowly. Enough time to revisit if needed. |
| feline |
Posted - Nov 17 2020 : 08:57:16 AM
With one of your Unreal Engine solutions open, can you please look at:
VA Options -> C/C++ Directories
and check the list of stable include directories. Does this include all of your Unreal Engine sub-directories? You should should see every single sub-directory under the base directory for Unreal Engine in this list.
Even if the source code was downloaded to C drive, you can still run into the problem where VA does not realise that this is the unreal engine source code, depending on which directory the source code is based under. So this test should tell us if VA thinks that this is a stable set of directories for Unreal Engine or not. |
| scottarius |
Posted - Nov 16 2020 : 2:08:07 PM
Is there any updates on this? I just recently bought VA for Unreal Dev and the constant re-parsing is killing me.
I'm using a source build of 4.25. All the ue4 features seem to work fine otherwise. Everything is on my C drive, so I don't think this junction fix will apply. The .uproject certainly is using the correct path for my compiled editor. I normally open the .sln file directly, not via the UE4 Editor. Sometimes it will re-parse everything, sometimes not. I can't for the life of me figure out what triggers it.
I do have other versions of UE4 installed for different projects, and they all seem to have this reparsing issue, but everything is pretty separated. |
| feline |
Posted - Aug 31 2020 : 08:54:18 AM
Thank you for the update, I am glad that the junction is helping. This is a work around for now.
The basic problem is that Unreal Engine is complex / hard for VA to handle correctly. You can easily have several different versions of the Engine installed at the same time, in different locations. Since the engine is listed as part of the solution you are opening, its really not obvious that this is actually a stable external library and should be treated differently to the rest of your solution.
The more I look into this problem, the more strange edge cases I seem to encounter. |
| Spanky |
Posted - Aug 30 2020 : 09:18:12 AM
And to be clear, both the Unreal install and my projects still exist on my D drive. I can open my projects from their D drive path (once they are converted obviously). I only need to launch the Unreal Editor through the junction path (C drive) and not it's actual path (D drive). |
| Spanky |
Posted - Aug 30 2020 : 09:14:47 AM
I was also having this same problem and I can confirm that creating the junction does fix the issue for me.
I also do not have the launcher/engine installed on my C drive. I have it installed in "D:\Code\Epic Games".
I created a new project here "D:\Code\Unreal Projects\MPTesting". When I open this, VAX will end up parsing all of Unreal again which takes a while (I have created multiple projects at this point). Also checking my 'System include directories' in VAX didn't list the Unreal directories.
I created the junction and opened the vcxproj and vcxproj.filters files (on my D drive) and updataed the directories in both of those to point to the path through the C drive junction. I cleared the VAX database and reopened the project in Visual Studio. Unfortunately this does not work as the Engine project (the Unreal engine code) within my project solution is still referencing the files on my D drive. I did however notice that the 'System include directories' now does list all of the Unreal directories properly.
I then launched the UE4Editor.exe from the junctioned path (C:\Program Files\Epic Games\UE_4.25_source\Engine\Binaries\Win64\UE4Editor.exe) instead of from my D drive (D:\Code\Unreal Engine\UE_4.25\Engine\Binaries\Win64\UE4Editor.exe). I told Unreal to open the same project (D:\Code\Unreal Projects\MPTesting) and this time it informed me that the project was generated with a different version of the engine. I guess it matches the path somehow. I ended up converting the project in place (people should create a backup somehow or do the conversion to copy option here as this process may fail and render your project broken).
After a bit, the project opened in Unreal properly. I then clicked on File->Open Visual Studio from within Unreal. It then began parsing all the system includes and I could see it scanning the Unreal files on the C drive now. The Engine project is also referncing files on the C drive rather than the D drive. The Unreal symbols are now in italics
I then created another project (still launching Unreal from the junction on C) and opened it in Visual Studio. After a noticably longer 'Loading system symbols' process, it quickly parsed my project files (1 or 2 seconds) and all highlighting was then working properly with Unreal symbols now in italics. It did not parse any of the Unreal files again. As long as I launch everything from the junction path rather than where the files are actually stored, things work properly and it does not reparse Unreal every time. I would have to convert over any existing projects to use the new path which may or may not be viable for some people. |
| feline |
Posted - Jun 25 2020 : 05:19:37 AM
I am basically doing the same thing here... so why the difference?
If you just load your solution, as shown in the screen shot above, and then wait a few moments before running the UE editor, does VA start doing a full parse of Unreal Engine? Or does VA notice that nothing has changed, so there is no need to do a reparse?
I am trying to work out if something seems to be triggering the extra parsing.
Do you work with more than one instance of Visual Studio open at once, or just the one instance? |
| Jorge24 |
Posted - Jun 24 2020 : 6:42:12 PM
quote:
Originally posted by feline
When you load your solution in Visual Studio, how are you doing so?
Do you first load Visual Studio, then open the Unreal solution you are working on?
Do you launch the Unreal Editor first and open the solution from here?
I launch Visual Studio and then I click on one of my pinned solutions. Then I click the green arrow to run the UE editor.
|
| Jorge24 |
Posted - Jun 24 2020 : 6:38:44 PM
Thanks for continuing to test this. I'm having a hard time understanding the process you followed on your end. Did you say that the entire source-compiled UE must be parsed at least once for each project that references it? or that dosnt happen to you?
So, if you create 10 new UE blank project from the source compiled UE editor, will it parse the entire engine again for each project? and if so, does that only happen with the source compiled one for you or also with the Epic Launcher version? |
| feline |
Posted - Jun 24 2020 : 12:31:21 PM
I have found a situation where VA re-parses the entire Unreal Engine directory tree for the github version of Unreal Engine, but for me this is only happening after using the github source code compiled version of the Unreal Editor to create a new game.
When I do this, instead of loading the Unreal Editor with the game project, as expected, instead I am launched directly into VS2019 with the C++ solution source code loaded. At this point VA does a full parse of the Unreal Engine tree.
However if I then close the IDE, and Unreal Editor, and either load a new instance of Visual Studio, or of Unreal Editor, and then load one of these github source editor produced game solutions, there is no extra parsing at all.
I am only working with one instance of the IDE open at a time, since VA is designed to maintain a separate symbol database for each separate instance of Visual Studio that is used, so this would require a full reparsing of Unreal Engine for each new instance and symbol database that has never before parsed Unreal Engine.
How does this compare with your experience? Obviously you are seeing something different, so how should I change my tests to try and reproduce what you are doing and seeing? |
| feline |
Posted - Jun 24 2020 : 11:49:48 AM
When you load your solution in Visual Studio, how are you doing so?
Do you first load Visual Studio, then open the Unreal solution you are working on?
Do you launch the Unreal Editor first and open the solution from here?
It is starting to look like how the IDE is loaded is somehow a factor here. |
| feline |
Posted - Jun 22 2020 : 05:40:37 AM
Thank you for the information. I have the correct IDE settings, but I do have different debugger settings. I am building with VS2017, as you say, I probably should have remembered to mention that
I found the "UE4Editor.exe" file myself, and just ran this directly, which does load. But I was caught out by it building all of the shaders while loading, which looked exactly like it had hung and was not working. I will start running tests on what is happening here soon. |
| Jorge24 |
Posted - Jun 19 2020 : 11:38:12 PM
quote:
Originally posted by feline
OK, this might be a silly question, but how do you download and compile the Unreal Engine source code from GitHub?
Hmm, my first question would be: What version of Visual Studio are you running? I have read posts before where updating visual studio fixes things. I'm using VS 2019 v16.6. Note that updating VS will probably mean you will have to recompile the entire engine again so try updating VS if all the stuff below fails.
Also, which branch did you check out from the UE GitHub repo? Are you on the release branch that I linked in the previous post which has UE 4.25.1?
These are my UE4 project properties:
Is your build configuration set like this too?
Lastly, if it did build successfully but you just can't run the UE editor through Visual Studio, you can just run the editor's EXE manually by going to:
"\UnrealEngine\Engine\Binaries\Win64\UE4Editor.exe"
|
| Jorge24 |
Posted - Jun 19 2020 : 11:20:56 PM
Hi, thanks again for replying and sorry it's been a few days.
quote:
Originally posted by feline
For your two solutions using the source code compiled from GitHub, are you using the same source tree and location for both? Have you updated the Unreal Engine source code in the last couple of days?
Yes, I have only had 1 repo of the UE at a time. The first time I tried using VA, I had the release branch of UE which at that time was at 4.25.0. Now I have the release branch which is at 4.25.1. I have tried creating new projects while I had 4.25.0 and also with 4.25.1 but both lead to the same re-parsing.
The version I have now is:
https://github.com/EpicGames/UnrealEngine/tree/4.25.1-release |
| feline |
Posted - Jun 17 2020 : 11:46:16 AM
OK, this might be a silly question, but how do you download and compile the Unreal Engine source code from GitHub?
Following the steps on this page:
https://docs.unrealengine.com/en-US/GettingStarted/DownloadingUnrealEngine/index.html
I have the GitHub download done, all nice and straight forward.
Then I have followed the steps on this page:
https://docs.unrealengine.com/en-US/GettingStarted/DownloadingUnrealEngine/index.html
first running "Setup.bat" and then "GenerateProjectFiles.bat"
Now load VS2017, open the UE4 solution, and build the UE4 project node. It compiled without any errors. I have set the start up project, several times over.
When I try to run the solution, I get the error message:
Unable to start debugging. Check your debugger settings by
opening project properties and navigating to 'Configuration
Properties-->Debugging'
which isn't all that helpful. I have tried several searches, unsuccessfully, for help on this. The nearest solution was to set the startup project, which I have already done.
Better pictures here, so I am sure I am doing this right:
https://docs.unrealengine.com/en-US/Programming/Development/BuildingUnrealEngine/index.html
When I check the debugging settings, there are only two settings obviously set:
Command = $(TargetPath)
Amp Default Accelerator = WARP software accelerator
the rest are blank. Nothing obviously wrong or invalid here.
And yes, I have even tried a reboot, just in case |
| feline |
Posted - Jun 15 2020 : 08:34:30 AM
At least the behaviour with the Epic Launcher version is what I would expect.
For your two solutions using the source code compiled from GitHub, are you using the same source tree and location for both? Have you updated the Unreal Engine source code in the last couple of days?
I will set up a similar test here, and see what I see.
If you want to try telling VA that your source code version of Unreal Engine lives in a different directory, without actually moving any of your files, you can try the following. Open a command prompt in the directory:
C:\Program Files\Epic Games
and a command like:
mklink /J UE_4.25_source "C:\repos\UnrealEngine"
this will create a junction point at the file system level, making the directory tree appear to be in both locations at the same time. Now go into the director that holds your source code based solution. From here you should find the following files:
Intermediate\ProjectFiles\UE4.vcxproj
Intermediate\ProjectFiles\UE4.vcxproj.filters
Open both files in a text editor, and do a search and replace for the old Unreal Engine path to the new Unreal Engine path, so in this case search
"C:\repos\UnrealEngine"
And turn all instances into:
"C:\Program Files\Epic Games\UE_4.25_source\"
Save the changes and now reload the IDE and your solution. |
| Jorge24 |
Posted - Jun 14 2020 : 01:51:44 AM
I just created 2 new blank projects. The project I created with the UE from the Epic Launcher didn't have to re-parse. It just loaded the symbols and that was it. However, the new project I created with the UE I compiled from source, spent like 21 minutes parsing the entire UE source code. I have a very good desktop CPU (it's an Intel i7-9700K) and it was running at 100% throughout those 21 minutes. Prior to this test, VA had already parsed the entire engine for both versions, the Epic Launcher one, and the one I compiled from source. I only had 1 instance of Visual Studio open at a time during the tests. Both engines are version 4.25.1.
It seems like all the problems I've had only happen in the version I compiled from source. |
|
|
