| Author |
 Topic  |
|
thruska
Ketchup Master
71 Posts |
 Posted - Mar 08 2007 : 02:25:31 AM
|
When VAX is enabled, it becomes fairly sluggish to make changes to my base coding library. The .cpp file is 1.4MB in size and the .h file is 200K. These two files get used in all my projects.
Running VAX 1548, but I've noticed it being pretty slow before.
I realize that the .cpp file is huge. It takes 5 minutes for VC++ to build a release mode build of just this one .cpp file. I put everything common'ish in one file because it is just easier to work with. But the VAX slowdown, while still relatively usable, is only going to get worse as I expand this file even more. I can't stand multiple files - it gets too messy trying to remember where each class is located.
My guess is that it has to do with syntax highlighting. Crimson Editor has a similar slowdown problem on large files. Maybe you could offer an option to just syntax highlight anything in the current class and draw black text otherwise in large files (anything over 250K is probably a large file). That would significantly reduce processing overhead. |
Thomas Hruska
CubicleSoft President
http://www.cubiclesoft.com/ |
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - Mar 08 2007 : 07:57:39 AM
|
I have seen a couple of other reports of this, which I have investigated, but so far I have been unable to reproduce the problem.
Is this a new problem in the last couple of builds, or have you been seeing this problem for a while?
Which operations are slow? Just scrolling? Editing? Everything?
Which IDE are you using? |
zen is the art of being at one with the two'ness |
 |
|
|
thruska
Ketchup Master
71 Posts |
Posted - Mar 08 2007 : 11:26:20 AM
|
I realized that I forgot to mention the IDE a couple of minutes ago. VS .NET 2003 (Visual C++).
It is slow editing - can't keep up with my typing (I have to force myself to type slower). Scrolling isn't a huge problem except every once in a while it gets stuck - but the IDE just has issues with scrolling anyway, so I don't expect VAX to fix them. I've seen the problem for a while but haven't really mentioned it because I don't usually makes lots of changes to the main .cpp file.
The easiest way to test this would probably be to dump 100 or so unique classes and templates into a .cpp and .h file - actual code. However, I should note that much of my code is interdependent on itself in a hierarchy sort of fashion (some classes call functions or depend on other classes - some derived, most just use variables). So the classes that get put in should have some level of interdependency on each other.
|
Thomas Hruska
CubicleSoft President
http://www.cubiclesoft.com/ |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - Mar 08 2007 : 11:56:43 AM
|
Can you run a CPU meter and try a quick test please?
Wait until the CPU usage has returned to normal, presumably very low, and then type into this file fairly quickly. VA should not start parsing the file until you pause typing, so if it is our parser causing this problem (it is an obvious place to start) then there should be no problems until you pause.
When you pause typing you should see a CPU spike, but hopefully only a short one. Once this spike has passed the file has been re-parsed and VA will not reparse the file until you next pause in your typing.
Does this make sense? The fact you have to slow down your typing bothers me, since it suggests something is causing problems when you are typing. Either re-parsing the file is taking a *long* time (many seconds, or even minutes) or something else is going on.
As for splitting up this file into smaller files, and finding classes, our FSIW dialog can be ideal for finding classes, but I do understand that how people organise their code is a very personal decision. |
zen is the art of being at one with the two'ness |
|
|
|
thruska
Ketchup Master
71 Posts |
Posted - Mar 08 2007 : 12:41:05 PM
|
| Did a quick test. As I type, the CPU spikes anywhere from 50% to 100%. I've got a 3GHz hyperthreaded CPU (Intel). Even after I stop typing, it takes about half a second for the usage to drop to 0%. Then a short bit after the CPU drops, it spikes again for about a half second. But I was only testing simple changes. I'm pretty much done editing the file for today. If you can come up with something I should have in a common coding library that I don't already have, I'll do a quick test of how I might develop it. I probably will do a screen recording as well so you can visually see what is going on (it'll probably be a huge file). |
Thomas Hruska
CubicleSoft President
http://www.cubiclesoft.com/ |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - Mar 08 2007 : 1:03:22 PM
|
Can you try creating a zero length read only NCB file in the solution base directory, replacing the IDE's current NCB file please?
What you are describing does not really sound like VA's parser on its own. At a guess VA is causing the second CPU usage spike, when you pause, but I am not sure where the rest of the load is coming from. If this zero length NCB file helps then this indicates part of the problem is the IDE's intellisense parser. |
zen is the art of being at one with the two'ness |
|
|
|
thruska
Ketchup Master
71 Posts |
Posted - Mar 08 2007 : 1:33:28 PM
|
| Nope. That didn't help. (And I said 'No' when Visual Studio asked to make the .ncb file writeable). |
Thomas Hruska
CubicleSoft President
http://www.cubiclesoft.com/ |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - Mar 11 2007 : 4:04:52 PM
|
I believe I have reproduced the problem. Certainly I have terribly slow typing in a large file.
case=5444
For me typing speed is reasonable in VA 1541, so you may want to consider rolling back to that build for now and see if this helps. |
zen is the art of being at one with the two'ness |
|
|
|
support
Whole Tomato Software
5566 Posts |
Posted - Apr 14 2007 : 12:12:42 AM
|
| Fixed in build 1553. |
|
|
|
thruska
Ketchup Master
71 Posts |
Posted - Apr 20 2007 : 12:52:40 PM
|
Installed 1553. Still have the same problem. Or at least it is still extremely sluggish. A lot of problems (close to 50 VAX-related bugs that I haven't reported - mostly because I can't replicate them outside of my core library) have cropped up in the past few months. I've got a massive code base and prefer keeping tight control over it, but also realize it would be really nice for everyone here if the bugs I'm experiencing got fixed - other people probably experience them as well. Any chance you could fly me and my source code down to Florida for a few days and we could hammer out a ton of VAX bugs on-site?
Stuff I've seen: Really sluggish performance (that no one seems to be able to replicate), red underlines (VAX says it is invalid code that is perfectly valid and compiles and runs just fine), incorrect tooltips, incorrect structure/class/function selection, incorrect . to -> conversions, incorrect order of operations evaluations, etc. (Don't even get me started on macro processing issues...I've got those en masse). Without VAX, I'd be completely sunk. With VAX, I'm slightly functional.
|
Thomas Hruska
CubicleSoft President
http://www.cubiclesoft.com/ |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - Apr 20 2007 : 1:43:21 PM
|
I tested typing speed in my large file here, and it is greatly improved. Typing is still slow, but nothing like as bad as it was in 1548.
Flying you down to Florida requires the executive maintenance plan 
Can we ignore the speed issue for the moment, and try and work on something else? If you are having this many problems it suggests that VA is confused by your code base. If it is confused then this is not going to help the speed. Extreme cases of confusing template code can cause the whole machine to just "hang" at 100% CPU usage for several minutes when scrolling through a file.
Underlining as mistyped symbol and dot to -> conversion are the two that that catch my attention first, since they might be slightly easier to tackle.
Can you reproduce dot to -> problems on demand in your code?
If so what are you typing dot after? Is it a local variable, a function call (so acting on its return value) or something else?
Are you making heavy use of macro's in your code? If so some tricks with VA's stdafx.h file might help to hide problem macro's.
http://docs.wholetomato.com?W302 |
zen is the art of being at one with the two'ness |
|
|
|
thruska
Ketchup Master
71 Posts |
Posted - Apr 20 2007 : 5:12:36 PM
|
As to typing speed, it is still roughly the same for me as the previous builds.
"Flying you down to Florida requires the executive maintenance plan." I don't what that is, but it sounds expensive. :)
Okay. Let's ignore speed issues for a bit.
VAX is definitely confused. I'll grant you that. I make _heavy_ use of templates. You've seen Block. That's the main template I use the most. Haven't experienced hangs but it is very slow to edit my main .cpp library file. So, let's take Block for instance. If I do something like:
size x, y;
Block<Process::ProcessList> TempList;
Process::GetProcessList(TempList);
y = TempList.GetSize();
for (x = 0; x < y; x++)
{
y2 = TempList[x].MxModules.GetSize();
for (x2 = 0; x2 < y2; x2++) printf("%s\\n", *TempList[x].MxModules[x2].MxName);
}
VAX gets confused around the second dot...especially if the class in Block contains another class in a Block that is a public subclass of a parent class.
Let's see, . to -> conversion...had that happen to me a couple days ago. It happened when I went to access a member of Block in a class dereferencing a SmartPtr in a ListNode.
y = Node->Value->IDs.GetSize();
Node is ListNode<class> *
Value is SmartPtr<class>
-> of Value accesses a member (operator overloaded ->) of SmartPtr<class> to return and dereference a pointer.
IDs is Block<size_t> and is a member of the 'class' class.
GetSize() is a member of Block<>
The . was (incorrectly) converting to ->
"Are you making heavy use of macro's in your code?"
You're kidding, right? The first 1200 lines of my main .cpp file is pure macro invocation (okay, 1153 lines). These macros form the basis of calling Win32 APIs within my code. Kind of pointless to disable them - a little bit of information is better than no information. I would rather see these ultimately resolve to the underlying APIs. |
Thomas Hruska
CubicleSoft President
http://www.cubiclesoft.com/ |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - Apr 23 2007 : 2:11:42 PM
|
Block? I don't remember Block. As for templates, I was not joking.
Uniwares has over 1,000 posts, and a good portion of our entire bug database  - I sort of remember his details and don't need to ask every time. Everyone else... it's a good idea to assume I have long since forgotten our last conversation and start at the beginning
A bit of forum searching has produced:
http://forum.wholetomato.com/forum/topic.asp?TOPIC_ID=5181
but unfortunately I cannot find any sign of the origional "Block", only the simplified test case in the bug report.Would you be able to re-send it, via this form:
http://www.wholetomato.com/support/contact.asp
including this thread ID or URL in the description, so we can match it up, along with anything else that might help me to reproduce some of these problems? I understand that most code cannot be shared, so if this is no longer available we will try something else.
1200 lines of macro's... suddenly VA's stdafx.h file sounds like a tempting way forward |
zen is the art of being at one with the two'ness |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 11 2007 : 11:27:51 AM
|
Hello,
I keep reverting to build 1549 after trying out newer builds (last was 1555) because the newer builds are almost unusable for me.
If I work on large files that take several seconds even to reparse every time I stop typing VA immediately reparses, and during this time VAG??s intellisense shuts off or stutters.
What happens is the following: I start typing a variable name and VA comes up with G?junkG?. If I then stop typing and wait for VA to finish parsing and hit ctrl-space the correct suggestion comes up.
Same e.g. after typing ->. First nothing or some (global) symbols but unrelated to the object the pointer refers to. So I wait until VA finishes parsing and hit ctrl-space againG?you get the picture.
In smaller files this is less annoying but even there it happens sometimes.
Could you please add an option or a registry switch to revert back to the G?oldG? reparsing style. I rather hit reparse from the toolbar than being unable to use VAG??s intellisense.
I am using VC6 Sp6 German edition, my machine is a AMD dual core with 2 Gigs of RAM, operating system is Windows XP SP2.
Apart from this I also noticed that some code is not recognized anymore by builds after build 1549. I also have the feeling that the parser is somewhat slower, but no hard evidence on that.
I would really like to use the newer builds (especially the cool new outline feature), so I hope you have some good news for me.
|
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - May 11 2007 : 1:58:58 PM
|
How big are the files? Approximate number of lines?
Some improvements were made for large files in the last couple of builds, and other people are finding that parsing is generally faster than it used to be.
Depending on the size of the files it is possible the problem is something other than the file size. |
zen is the art of being at one with the two'ness |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 12 2007 : 03:00:35 AM
|
The Files are between 80 and 300KB anything between 3000 and 12000 lines. The file I have the most problems with is 180KB, 7800 lines (of which about 1000 are includes).
To make it clear again: the last build I tested was 1555 and my problems started when the delay to start parsing was reduced.
Just on the side: every time I installed a build after 1549 it created a new VA toolbar, is that correct? |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - May 14 2007 : 08:07:06 AM
|
7,800 lines does not sound to terrible, but 1,000 include statements! That is a lot of include statements!
Where does the statement about the delay to start parsing being reduced come from? I don't remember hearing about any such change, and I cannot see any sign of it in the changes after 1549 listed here:
http://www.wholetomato.com/support/history.asp
Using a 22,000 line file I have just done some tests here with 1549 and 1555 and I am not seeing any obvious difference in the delay before VA starts parsing. I am looking at the status bar in VC6 and watching for the message that VA is parsing the file to decide how long it has waited.
Clearly though something is different, and it is causing you problems. I have asked about increasing the delay in very large files:
case=6562 |
zen is the art of being at one with the two'ness |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 14 2007 : 12:58:01 PM
|
Thank you for your quick response, I have looked into it a little further and found the following:
Build 1549 starts parsing approximately 2.5-3 seconds after I have stopped typing whereas in 1555 it starts almost immediately. The parsing itself takes 7-8 seconds.
In both builds intellisense is disrupted during the parsing process: if I type an identifier and wait until the parsing starts and then type dot the upcoming information is G?junkG? or incomplete at best.
As an example I have made two screenshots, the first made during parsing and the second below during G?normalG? operation and have uploaded it to PicTiger.com.
http://server7.pictiger.com/img/98988/other/vax1549_1.jpg
Free Image Hosting: http://www.pictiger.com.
|
|
|
|
sean
Whole Tomato Software
USA
2817 Posts |
Posted - May 14 2007 : 2:31:11 PM
|
1000 include statements...
Could you try something just to see if it makes a difference?
Put the 1000 includes into a new header file.
In the source file, replace the 1000 includes with the new single include.
Does this make a difference while editing that one source file?
|
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 15 2007 : 05:49:59 AM
|
Hello,
I did as you suggested and put he majority of the includes into a separate include file. Doing so reduced the reparse time from about 11 seconds to 6 seconds.
Besides, I was wrong concerning the number of includes. There are only a mere 391 includes.
The problem with the G?irritatedG?/not working intellisense during the parsing still remains.
|
|
|
|
sean
Whole Tomato Software
USA
2817 Posts |
Posted - May 15 2007 : 10:19:38 PM
|
Will you please send in logs per the instructions at http://docs.wholetomato.com?W305 ?
Reference topic 5987 when you submit them.
Does the broken intellisense problem exist in build 1549 while parsing is underway?
Is the duration of the parse in 1549 comparable to 1555 with the includes change (6 seconds)?
|
|
|
|
Rain Dog
Ketchup Master
88 Posts |
Posted - May 16 2007 : 03:51:05 AM
|
| Man, since VA can't say it, I will. Having a 3MB library file of source code is retarded. You break things up into files so you remember "Oh, Foo is in 'Foo.h'", not "Oh, Foo is in line 4,573 today, and it was 4689 yesterday". I sure hope nobody actually employs you. |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 16 2007 : 05:27:11 AM
|
@sean
I sent the logging as requested.
Yes, the problem exists in both builds.
Sorry that I forgot to mention that I did the timings in build 1549 because I did reverted to it in order to be able to use VA.
@Rain Dog
Please read the posts carefully before jumping to conclusions. I think you missed that we are currently talking about a file of 160KB and I just used the present thread since it seemed related to my problem. Besides, it is not a library file but the mainframe of an application that is merging several hundreds of classes that in turn are in a library. But to do so it has to include the headers and probably has a lot of message handling in it...
And please donG??t get the impression that the problem is only in this file. The problem exists, to a minor degree, in a lot of files, many of them much smaller.
Besides, even if you talked about the starting posts, I have to say that without background info you should be careful to judge somebody. It could be you posting this, e.g. if you were employed to work on some legacy app.
|
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - May 16 2007 : 08:41:49 AM
|
JCo the fact this problem occurs in much smaller files bothers me, but it also suggests that file size is not the sole issue here. Are you using a lot of templates in your code?
Rain Dog lets try to keep things polite here. From experience, when someone is doing something that makes no sense there is something you don't know. People come here to report problems, so they report the information we need to know, but often do not explain the background that explains why something came to be this way.
I have ended up with large files and functions myself on occasions, when something cannot sensibly be subdivided. At my old job I had a class that converted a data file to a report, and I ended up with a several thousand line file, all one class. There were good reasons for this, but without a description of why you might not understand that this was sensible. |
zen is the art of being at one with the two'ness |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 16 2007 : 12:17:54 PM
|
Hello feline,
yes, we use templates but I am not sure what a lot would be.
But you are right, there was something fishy going on hereG?
It seems that I added by accident (ages ago) two header files to the project which obviously confused either VA or MSDEV a lot.
After I removed these two headers the parsing was so fast that I could not measure it. Less than half a second maybe and about one second with the include files as before (means 10 times faster).
The intellisense problem is still here but the timeframe to hit it has decreased so it is not as much of a bother as before. Still, I think it would be nice to fix it since it could account for some G?strangeG? behaviour.
I have just upgraded to build 1555 and am looking forward to see how the new outline feature is working.
Thanks for your patience and keep up the good work, I really love VA (most of the time ).
|
|
|
|
Rain Dog
Ketchup Master
88 Posts |
Posted - May 16 2007 : 7:48:00 PM
|
| JCo, try the new feature as described on the beta board. It's pretty slick |
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - May 17 2007 : 2:07:29 PM
|
JCo see this thread for initial details on VA Outline:
http://forum.wholetomato.com/forum/topic.asp?TOPIC_ID=6149
These problem header files, is there any chance of getting a copy of these files for testing purposes?
As for templates, just using STL templates is normally no issue, but if you dig deeply into Boost, or write your own complex template classes this can cause our parser problems. Most users never see this, but it is something I watch for, since it causes a fair number of bug reports. |
zen is the art of being at one with the two'ness |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 18 2007 : 08:31:24 AM
|
Hello Feline,
I made some tests regarding the include files, and it seems that any present include will increase the parsing time.
I think the problem lies in the workspace itself.
I added several projects to it so that VAX G?seesG? all files, even the files that are only part of a library that my application links to. I did this so that alt-g works for my application as a whole, including .cpp files only present in the library.
In total the projects in my workspace have some 1600 files in them, but the active project only has a few (mainframe etc.). There are no project dependencies whatsoever.
The funny thing is: as soon as I have any include file in my active project, VAX takes 10 times longer to parse, even with only the simple one below.
#ifndef _VAXTESTCASE_
#define _VAXTESTCASE_
class VaxTestCase
{
public:
VaxTestCase();
~VaxTestCase();
protected:
int someInteger;
};
#endif // _VAXTESTCASE_
|
|
|
|
feline
Whole Tomato Software
United Kingdom
18751 Posts |
Posted - May 18 2007 : 10:56:37 AM
|
*puzzled* a header file like that should not make any difference.
Are all of these files on the same hard drive?
Is this a local or network hard drive?
Have you used subst or some other method to map a drive letter to a directory?
These things should not matter, but something is causing problems.
What anti virus are you using?
Perhaps something is getting in the way when VA tries to access and read the header files... |
zen is the art of being at one with the two'ness |
|
|
|
JCo
Senior Member
Germany
40 Posts |
Posted - May 18 2007 : 12:30:09 PM
|
All files are on a local hard drive, the drive letter however is substG??ed.
I am using TrendMicro OfficeScan here, but the directories from VAX (user data part) as well as the source code is on the exclusion list (I have verified that just now). And last time I tested TredMicro did not ignore this setting.
Using filemon I could spot a huge difference: with the header present there is a huge amount (approximately 100000) of unsuccessful lookups of header files logged.
When I looked up the va.log log file I came across the following line(s), I shorted it somewhat:
FindFile ERROR file not found rc/deficon.rch, 1
SYS C:\\Programme\\Microsoft Visual Studio\\VC98\\INCLUDE;C:\\Programme\\Microsoft Visual Studio\\VC98\\MFC\\INCLUDE;C:\\Programme\\Microsoft Visual Studio\\VC98\\ATL\\INCLUDE;W:\\include;W:\\;
ADDL
PROJ W:\\include\\;W:\\...really long list of directories here...:#Include not found: rc/deficon.rch in W:\\main\\...
EdMM
The interesting part: I had a long list configured in previous versions (pre VA.X) in a custom target definition as additional source directories (for alt-gG? you get the picture).
But I can not find any reference to this now, neither in the registry nor in any *.dsp, *.dsw or *.opt file.
Any idea where VAX is getting this PROJ list from?
|
|
|
|
sean
Whole Tomato Software
USA
2817 Posts |
Posted - May 19 2007 : 01:02:32 AM
|
VA builds up the proj list based on all of the files in your workspace; it is completely workspace dependent.
Do the rc/*.rch files actually exist? Any idea where VA is finding the #includes for those files?
|
|
|
|
Topic |
|
Sluggish on large files.
