| T O P I C R E V I E W |
| MichaelTraega |
Posted - Jul 10 2018 : 5:01:49 PM
I'm working in a massive project (UE4). If I Find References on functions named Update or Get, it starts returning functions of the same name but irrelevant contexts, not fitting the command's description: "The command ignores similarly named symbols of different contexts, and provides much more efficient access than Find in Files." It seems to return the function calls and definitions for auto and templates. This on its own is annoying, but what's even more annoying is how long it takes to finish when doing this. Finding references on a less generically named function takes less than 30 seconds for the entire project. Finding references for Update or Get takes at least 5 minutes every time. The VA output says something like "Creating instance of template" over and over.
I've basically always had these issues with VA.
- Why is it so slow if its going to return basically every irrelevant result?
- Why does it not only return the relevant results?
- Does it do anything smart like return the most relevant results first and then start evaluating the templates?
- Why doesn't it cache these results?
Basically there are some functions (and variables) I'd like to find references on but the tool is basically useless for them so I have to find all instances of the class and then look for where the function is called, which is not fun, or look through the results after they take forever to finish and if there's a lot of them try to figure out which results are relevant given the file name.
Currently on version 10.9.2270.0 in VS 2017. Disabled Intellisense. |
| 30 L A T E S T R E P L I E S (Newest First) |
| feline |
Posted - Jan 07 2019 : 09:33:08 AM
Longer names being faster makes sense, if only because longer names tend to be more unique, so there is less chance of seeing something that has to be parsed in depth. Also less used in the deeper, template heavy parts of the engine code that take more parsing.
We are aware of the problem of VA spending a lot of time creating instances of templates when running a Find References. I have put a note onto the case about this to see if we can cache the results, at least for situations like Unreal Engine, where the template instances are inside stable code:
case=25958 |
| MichaelTraega |
Posted - Jan 04 2019 : 4:01:05 PM
Yes Find References tends to run slow the first time. All of my finds are not slow. Longer named finds seem faster.
Yes I'm aware I can run it on the current project. Typically though for generic functions like Init I tend to be looking at the engine and trying to understand or find where that gets called. |
| feline |
Posted - Jan 04 2019 : 10:51:37 AM
Are you always seeing a Find References run this slowly? Testing this in a small, default C++ Unreal project, the first time I run Find References on the code:
UGameEngine::Init();
the find takes nearly 10 minutes to finish. The second time I run the same Find References search the find is a LOT faster. So things are being cached, but not everything that is worked out is saved when you restart the IDE.
Is this similar to what you are seeing? Or are all of your finds this slow?
Are you aware you can tell VA to only run Find References across the current project, so the game engine code won't be searched, if you are only looking for references inside your own game code? This should also help to make the searches a lot faster. |
| Zeblote |
Posted - Jan 02 2019 : 11:17:54 PM
Are you planning any more optimizations for this? For example, searching for references to UGameEngine::Init takes 5 minutes (!) and returns 11 results. Most of the time seems to be spent on "creating instance of template ..."
Maybe VA could cache template instances it has seen before to speed this up, if it isn't already? |
| sean |
Posted - Dec 03 2018 : 6:37:15 PM
case=108835 and case=117768 are fixed in build 2301 |
| feline |
Posted - Nov 07 2018 : 2:24:12 PM
Normally the thread is updated, it looks like we missed this thread on this occasion.
If all goes well you will be able to delete your solution specific va_stdafx.h file with the next build of VA, but for now, the current version of the file that I have worked out is posted below. Can you please try this in combination with VA 2291.5, and let me know if this works correctly for you, or if you encounter any more odd results in Unreal?
// va_stdafx.h Feline Unreal Engine fixes
// Version 2.2
// fixed dot converted to -> on iterator over TMap of pointer type
// helps ranged for loop iterate across TMap
template<class KeyType, class ValueType>
class TTuple
{
public:
KeyType Key;
ValueType Value;
};
// so that for ranged-loop works correctly when TTuple key is a pointer
// the iterator is still returning the TTuple, not the pointer key
template<typename ContainerType, typename ElementType, typename IndexType>
class TIndexedContainerIterator
{
ElementType operator* () const;
};
template<class ElementType, class _A>
class TArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> const_iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> reverse_iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> const_reverse_iterator;
// iterator access
iterator begin();
iterator end();
const_iterator cbegin() const;
const_iterator cend() const;
reverse_iterator rbegin();
reverse_iterator rend();
const_reverse_iterator crbegin() const;
const_reverse_iterator crend() const;
public:
// data access
ElementType& operator[](int32 Index);
};
template<class ElementType, class _A>
class TArrayView
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<class ElementType, class _A>
class TChunkedArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template <class ElementType>
class TDoubleLinkedList
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<typename ElementType, typename Allocator>
class TIndirectArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
public:
// data access
ElementType& operator[](int32 Index);
};
template<typename ElementType>
class TLinkedList
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<class KeyType, class ValueType, class _A, class _S>
class TMap
{
public:
// required since the member TIterator class has a different pair API
// uses functions not exposed members, unlike the STL style iterator below
// BUT also overloads -> to access Key and Value as members
template<class KeyType, class ValueType>
class TIterator
{
KeyType Key();
ValueType Value();
TTuple<KeyType, ValueType> operator->() const;
}
TMap::TIterator<KeyType, ValueType> CreateIterator();
TMap::TIterator<KeyType, ValueType> CreateConstIterator() const;
TMap::TIterator<KeyType, ValueType> CreateKeyIterator(KeyType InKey);
TMap::TIterator<KeyType, ValueType> CreateConstKeyIterator(KeyType InKey) const;
public:
// STL style iterator movement - ranged for loop
typedef TBaseIterator<TTuple<KeyType, ValueType> > iterator;
iterator begin();
iterator end();
};
template<typename ElementType, typename KeyFuncs, typename Allocator>
class TSet
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<class KeyType, class ValueType, class _A, class _S>
class TSortedMap
{
public:
// required since the member TIterator class has a different pair API
// uses functions not exposed members, unlike the STL style iterator below
template<class KeyType, class ValueType>
class TIterator
{
KeyType Key();
ValueType Value();
void RemoveCurrent();
}
TSortedMap::TIterator<KeyType, ValueType> CreateIterator();
TSortedMap::TIterator<KeyType, ValueType> CreateConstIterator() const;
TSortedMap::TIterator<KeyType, ValueType> CreateKeyIterator(KeyType InKey);
TSortedMap::TIterator<KeyType, ValueType> CreateConstKeyIterator(KeyType InKey) const;
public:
// STL style iterator movement - ranged for loop
typedef TIndexedContainerIterator<TTuple<KeyType, ValueType>, TTuple<KeyType, ValueType> > iterator;
iterator begin();
iterator end();
};
|
| MichaelTraega |
Posted - Nov 07 2018 : 10:17:15 AM
Oh I thought you guys normally responded to a thread when a case was fixed. I will have to update and just work with it for a bit to see how it feels.
How would I get the latest va_stdfx.h? Does it just come with the next VA update and I need to remove the one I added next to my .sln? |
| feline |
Posted - Nov 07 2018 : 08:23:29 AM
Which version of VA are you currently using?
Can you tell me what problems you are seeing, and where you are seeing them?
The problem with pointers to smart pointers (case=117758) has been fixed, so if you have the latest version of VA this should not be causing any problems any more.
Since my Unreal va_stdafx.h file was being discussed in a few different places I started version numbering it, for ease  I am currently at version 2.2, but there weren't many changes from the version above. Do you have a version with a version number at the top?
You may have found another edge case that I am not aware of yet, which still needs looking at. |
| MichaelTraega |
Posted - Nov 06 2018 : 2:51:26 PM
Any update on the fix for autos and smart pointers? I've been working with the va_stdafx for some time now and its working pretty well otherwise. |
| feline |
Posted - Aug 25 2018 : 06:01:28 AM
Having found another case that is not yet fixed, adding the following block helps:
// so that for ranged-loop works correctly when TTuple key is a pointer
// the iterator is still returning the TTuple, not the pointer key
template<typename ContainerType, typename ElementType, typename IndexType>
class TIndexedContainerIterator
{
ElementType operator* () const;
};
|
| feline |
Posted - Aug 23 2018 : 12:53:26 PM
Here we go, this is my current set of fixes, which are all working well for me, testing with Unreal Engine 4.20.1
// helps ranged for loop iterate across TMap
template<class KeyType, class ValueType>
class TTuple
{
public:
KeyType Key;
ValueType Value;
};
template<class ElementType, class _A>
class TArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> const_iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> reverse_iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> const_reverse_iterator;
// iterator access
iterator begin();
iterator end();
const_iterator cbegin() const;
const_iterator cend() const;
reverse_iterator rbegin();
reverse_iterator rend();
const_reverse_iterator crbegin() const;
const_reverse_iterator crend() const;
public:
// data access
ElementType& operator[](int32 Index);
};
template<class ElementType, class _A>
class TArrayView
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<class ElementType, class _A>
class TChunkedArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template <class ElementType>
class TDoubleLinkedList
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<typename ElementType, typename Allocator>
class TIndirectArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
public:
// data access
ElementType& operator[](int32 Index);
};
template<typename ElementType>
class TLinkedList
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<class KeyType, class ValueType, class _A, class _S>
class TMap
{
public:
// required since the member TIterator class has a different pair API
// uses functions not exposed members, unlike the STL style iterator below
template<class KeyType, class ValueType>
class TIterator
{
KeyType Key();
ValueType Value();
}
TMap::TIterator<KeyType, ValueType> CreateIterator();
TMap::TIterator<KeyType, ValueType> CreateConstIterator() const;
TMap::TIterator<KeyType, ValueType> CreateKeyIterator(KeyType InKey);
TMap::TIterator<KeyType, ValueType> CreateConstKeyIterator(KeyType InKey) const;
public:
// STL style iterator movement - ranged for loop
typedef TIndexedContainerIterator<TTuple<KeyType, ValueType>, TTuple<KeyType, ValueType> > iterator;
iterator begin();
iterator end();
};
template<typename ElementType, typename KeyFuncs, typename Allocator>
class TSet
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
iterator begin();
iterator end();
};
template<class KeyType, class ValueType, class _A, class _S>
class TSortedMap
{
public:
// required since the member TIterator class has a different pair API
// uses functions not exposed members, unlike the STL style iterator below
template<class KeyType, class ValueType>
class TIterator
{
KeyType Key();
ValueType Value();
void RemoveCurrent();
}
TSortedMap::TIterator<KeyType, ValueType> CreateIterator();
TSortedMap::TIterator<KeyType, ValueType> CreateConstIterator() const;
TSortedMap::TIterator<KeyType, ValueType> CreateKeyIterator(KeyType InKey);
TSortedMap::TIterator<KeyType, ValueType> CreateConstKeyIterator(KeyType InKey) const;
public:
// STL style iterator movement - ranged for loop
typedef TIndexedContainerIterator<TTuple<KeyType, ValueType>, TTuple<KeyType, ValueType> > iterator;
iterator begin();
iterator end();
};
|
| feline |
Posted - Aug 15 2018 : 2:14:48 PM
Excellent news, thank you. I am working on fixing some of the other problems I have found along the way, so will hopefully have another update for you to test soon. |
| MichaelTraega |
Posted - Aug 14 2018 : 5:13:39 PM
This seems to have fixed it. The only irrelevant updates I'm getting now are autos, smart pointers, and macros. I'm going to test Get again. |
| feline |
Posted - Aug 10 2018 : 1:34:16 PM
OK... there are two different iterators for TSortedMap, with slightly different members. In theory, and in my simple test case, replacing your current TSortedMap in the va_stdafx.h file with this version fixes the problem:
template<class KeyType, class ValueType, class _A, class _S>
class TSortedMap
{
public:
// required since the member TIterator class has a different pair API
// functions not exposed members, unlike the begin / end iterator
template<class KeyType, class ValueType>
class TIterator
{
KeyType Key();
ValueType Value();
void RemoveCurrent();
}
public:
// iterator movement
typedef TIndexedContainerIterator<TTuple<KeyType, ValueType>, TTuple<KeyType, ValueType> > iterator;
iterator begin();
iterator end();
};
but I am not sure this actually fixes the problem you are seeing here. So I am still looking at this, but you might want to try this version for now, and see if it helps.
I am also getting the feeling there are going to be quite a few of these "random" problems, but hopefully most of them can be fixed as they are found. |
| feline |
Posted - Aug 09 2018 : 3:40:37 PM
Ah, here I am carefully trying to fix for loops with auto, but this problem is back to my "friend" CreateIterator(). Working on this now. |
| MichaelTraega |
Posted - Aug 09 2018 : 10:11:19 AM
Still getting the same issue. Also now when I alt+g this line, it shows me a whole bunch of Update functions that aren't related from all over the codebase, third party included. I think at least before when I alt+g it would actually only show TSortedMap results. |
| feline |
Posted - Aug 08 2018 : 4:19:52 PM
To fix TSortedMap, add the following to your va_stdafx.h file:
template<class KeyType, class ValueType, class _A, class _S>
class TSortedMap
{
public:
// iterator movement
typedef TIndexedContainerIterator<TTuple<KeyType, ValueType>, TTuple<KeyType, ValueType> > iterator;
iterator begin();
iterator end();
};
this is working here for me, so hopefully this will fix it for you as well. |
| feline |
Posted - Aug 08 2018 : 10:26:19 AM
I am having a look at TSortedMap now. I am also working through all of the collection template classes here, looking for other problems in advance:
http://api.unrealengine.com/INT/API/Runtime/Core/Containers/index.html |
| MichaelTraega |
Posted - Aug 07 2018 : 11:23:32 AM
I am searching FStartFTestOnMap::Update and getting the following (ignoring autos, macros, and smart pointers):
// PlayerController.cpp ln 4063
for (TSortedMap<uint64, FDynamicForceFeedbackAction>::TIterator It(DynamicForceFeedbacks.CreateIterator()); It; ++It)
{
if (!It.Value().Update(DeltaTime, ForceFeedbackValues))
{
It.RemoveCurrent();
}
}
I haven't yet tested looked for the others specifically, but it seems at least TSortedMap isn't working. However, if I alt+g on the CreateIterator shown above, it lists SortedMap.h and va_stdafx.h, suggesting it is working to a degree? |
| MichaelTraega |
Posted - Aug 06 2018 : 5:51:38 PM
Alright everything is reparsing right now. Will try to test this tomorrow. |
| feline |
Posted - Aug 06 2018 : 5:41:12 PM
I have TArray fixed, at least in the simple test cases that I have checked so far. Can you please add the following to your va_stdafx.h file, and if you already have an entry for TArray, replace it with this new entry:
template<class ElementType, class _A>
class TArray
{
public:
// iterator movement
typedef TIndexedContainerIterator<ElementType, ElementType> iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> const_iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> reverse_iterator;
typedef TIndexedContainerIterator<ElementType, ElementType> const_reverse_iterator;
iterator begin();
iterator end();
const_iterator cbegin() const;
const_iterator cend() const;
reverse_iterator rbegin();
reverse_iterator rend();
const_reverse_iterator crbegin() const;
const_reverse_iterator crend() const;
public:
// data access
ElementType& operator[](int32 Index)
};
as before, you will need to press:
VA Options -> Performance -> Rebuild symbol databases
and restart your IDE for this to take effect. Hopefully this will work well for you. My next step is to see if I can see similar problems with other Unreal Engine collection types, and see if the same form of fix will work for them as well. |
| feline |
Posted - Aug 04 2018 : 10:44:34 AM
I have made some useful progress, on the "CreateIterator()" functions that several of the Unreal Engine container classes use. VA handles some of these functions, but not others. Adding this to your va_stdafx.h file fixes this for these classes:
template<typename InElementType, typename KeyFuncs = DefaultKeyFuncs<InElementType>, typename Allocator = FDefaultSetAllocator>
class TSet
{
typedef TIndexedContainerIterator<TSet, InElementType, int32> TIterator;
TIndexedContainerIterator<TSet, InElementType> CreateIterator();
};
template <typename KeyType, typename ValueType, typename ArrayAllocator, typename SortPredicate>
class TSortedMap
{
TIndexedContainerIterator<TSortedMap, ValueType> CreateIterator();
};
template<typename InElementType, typename Allocator>
class TSparseArray
{
TIndexedContainerIterator<TSparseArray, InElementType> CreateIterator();
};
now to see if I can generalise this to help with more of the iterator problems. |
| MichaelTraega |
Posted - Jul 23 2018 : 3:52:21 PM
I've been working in unreal for over 5 years now and I'm wondering when I'm going to learn it. It's so massive you're never going to cover every edge case. I'm just happy to see VAX handling it better. I was first introduced to VAX and Unreal (3) at the same time. Still learning what VAX can do as well. |
| feline |
Posted - Jul 23 2018 : 3:41:42 PM
Instead of trying to learn Unreal, I will just learn the bit's I need to solve the problems that come up... Just wondering how long before this plan turns into learning Unreal |
| MichaelTraega |
Posted - Jul 23 2018 : 3:29:29 PM
Namespaces are mostly used for oldschool typed enums. But doing a quick find text search, I found there are places in engine where they're used for encapsulating blocks of code. I use them as well. But they are never really used at a project scale unless it's a third party, like Box2D or physx.
Couple of examples:
MacTargetSettingsDetailsConstants.cpp
ConvexHull2d.h |
| feline |
Posted - Jul 23 2018 : 3:17:29 PM
I am currently searching through the Engine 4.18 code, looking for all calls to "CreateIterator()". The initial results are mixed. Sometimes the iterator is a clear type. Sometimes its a typedef inside the class scope. Sometimes its a typedef at file / global scope.
Have you ever seen any sign that Unreal Engine uses namespaces to separate out blocks of code? I am not seeing any signs yet, so suspect the answer is no.
Thank you for your answer, hopefully this will all make a bit more sense when I have finished listing these examples and checking them out. |
| MichaelTraega |
Posted - Jul 23 2018 : 1:23:37 PM
So I guess a workaround is that if the iterator is being created for a TArray or TIndirectArray, you assume its of type TIndexedContainerArray. Otherwise you assume it's definition is in whatever class its nested in, but it will always inherit from TBaseIterator |
| MichaelTraega |
Posted - Jul 23 2018 : 1:17:37 PM
Just alt+g'ing through these and I don't think all of these share the same base class.
Array.h has TIndexedContainerIterator which is typedef'd to TIterator in only 3 places (Array, IndirectArray, and FPreviewAssetAttachComponent). All of these are typedef'd.
Most others seem to be of type TBaseIterator. These are not typdef'd. They're nested classes. |
| feline |
Posted - Jul 23 2018 : 12:18:28 PM
Apologies, I did not explain my thinking at all clearly here. For point 5, I wanted to wait until I had the code in front of me before thinking about it to much, since sometimes its more about the types that are being used than it is about the loop structure.
It turns out that this is one of those cases. For some very strange reason VA is struggling with the line:
auto autoDeviceIterator = Devices.CreateIterator();
but is perfectly happy with the line:
TIterator directDeviceIterator = Devices.CreateIterator();
so we are failing at the type of the iterator in the outer loop...
URG and now I know why, but not what to do about this, or how Unreal avoids running into problems with doing this.
If I asked you what a TIterator was, what would you say?
In the file "UnrealString.h" I have a TIterator CreateIterator() function, and the typedef:
typedef TArray<TCHAR>::TIterator TIterator;
while in the file "" I also have a TIterator CreateIterator() function, and this time I am finding the class:
class TIterator : public TBaseIterator<false>
Hopefully all iterators have a base class, but what is the data type that is being returned when you ask for the iterator content? Urg. |
| MichaelTraega |
Posted - Jul 20 2018 : 11:12:48 AM
I rarely open visual studios from the editor, but I just tested it and it worked fine.
Point 5, the code looks the same after I updated to the release version of the engine. What I pasted above wasn't a range-based loop to begin with. |
|
|
