Re: Preprocessing and scintilla_22
Posted: Wed Nov 10, 2010 11:05 pm
Can really rememberjfouche wrote:Why did you put an empty file name in MACROS table ?
Sure, go a head and change it
Eran
Sure, go a head and change it
Eran
Page 5 of 6
Sure, go a head and change it
Eran
Re: Preprocessing and scintilla_22
Posted: Wed Nov 10, 2010 11:41 pm
Well, I know : because there is no way to retrieve it currently 
I need to change the PPToken token class in order to add this member. No problem, I'll do it.
I need to change the PPToken token class in order to add this member. No problem, I'll do it.
Re: Preprocessing and scintilla_22
Posted: Sat Nov 13, 2010 2:40 pm
hello Eran
Here are some news :
That works
... but I encounter some bad behaviors :
1 - The current file crawler just returns the include files from the parser include files, not from the workspace. So how can I PPScan a workspace file which is included by the editing one :
2 - As the file formating is not the same in the database, and in the string representing those files in CL, I did not manage the file name in the query. The pb is that if a macro is defined out of the included files, it will be seen as defined.
3 - The problems from this post need huge design changes in order to make it works correctly. but we will talk about this later.
I'll post a patch this afternoon.
Here are some news :
That works
... but I encounter some bad behaviors :1 - The current file crawler just returns the include files from the parser include files, not from the workspace. So how can I PPScan a workspace file which is included by the editing one :
Code: Select all
#include "test.h" // not parsed, because not found by file crawler
#include <winnt.h> // OK3 - The problems from this post need huge design changes in order to make it works correctly. but we will talk about this later.
I'll post a patch this afternoon.
Re: Preprocessing and scintilla_22
Posted: Sat Nov 13, 2010 6:58 pm
Here it is. Your feedback is welcome. It's not finished yet, but this step seems good to commit.
Re: Preprocessing and scintilla_22
Posted: Sat Nov 13, 2010 8:31 pm
Some comments:
- I think you should return the output as a class and place it in the SetClientData of the command event (e.g. dont place the file name in the GetString() method of the wxCommandEvent)
- As part of the parser request you should also pass the workspace files list
have a look at Manager::RetagWorkspace():
- in ParseThread::ProcessInterrestingMacros(ParseRequest *req), the call to "macros.RemoveLast()" can cause to segault, if "macros" is empty
Eran
- I think you should return the output as a class and place it in the SetClientData of the command event (e.g. dont place the file name in the GetString() method of the wxCommandEvent)
- As part of the parser request you should also pass the workspace files list
have a look at Manager::RetagWorkspace():
Code: Select all
wxArrayString projects;
GetProjectList ( projects );
std::vector<wxFileName> projectFiles;
for ( size_t i=0; i<projects.GetCount(); i++ ) {
ProjectPtr proj = GetProject ( projects.Item ( i ) );
if ( proj ) {
proj->GetFiles( projectFiles, true );
}
}
// Create a parsing request
ParseRequest *parsingRequest = new ParseRequest();
for (size_t i=0; i<projectFiles.size(); i++) {
// filter any non valid coding file
if(!TagsManagerST::Get()->IsValidCtagsFile(projectFiles.at(i)))
continue;
parsingRequest->_workspaceFiles.push_back( projectFiles.at(i).GetFullPath().mb_str(wxConvUTF8).data() );
}
Eran
Re: Preprocessing and scintilla_22
Posted: Sun Nov 14, 2010 10:40 pm
Here it is. Once again, it's not perfect, due to the fact we look to all macros, even of the current file (so it's all grayed in a header file, when there is guard block).
I really need a way to find in specific files in the MACROS database.
I really need a way to find in specific files in the MACROS database.
Re: Preprocessing and scintilla_22
Posted: Fri Nov 19, 2010 11:01 pm
Hi Jeremie,
I finally got the time to apply this patch, and here are my comments:
- When I first ran the patached codelite it did not work, I then realized that I need to perform a full retag of codelite so the "Macros" table will be populated. This should be done automatically (codelite has a mechanism for schema changes) - this can be achieved by changing the value of "gTagsDatabaseVersion" global variable in tags_storage_sqlite3.h, when modified codelite will retag the workspace and will update the schema version
- The method 'Manager::UpdatePreprocessorFile' is called twice once from the event 'OnPageChanged' and once from ReloadFile() from cl_editor.cpp
- But the biggest problem is that 'Manager::UpdatePreprocessorFile' is *very* slow. I tried it on codelite workspace and it took several seconds for a file to load, but since we also call it from 'OnPageChanged' switching tabs also become laggy (2-3 seconds to swtich a tab...)
The cause of the laggy behvior is the code that collects the 'workspace files' and filter them - I will see how this code can be optimized
- And last problem is the one you mentioned: block guards cause the entire file to gray out
Improvements:
- We need to provide means in the UI to disable this option
- I am going to add 'C/C++ Parser options' to the project level. Once added, I am going to add two options:
** Automatically detect macros
** User macros - this way the user can set a macros in case the automatic process fails to detect it correctly
In addition, I will add an option to set 'search paths' per build configuration
Eran
I finally got the time to apply this patch, and here are my comments:
- When I first ran the patached codelite it did not work, I then realized that I need to perform a full retag of codelite so the "Macros" table will be populated. This should be done automatically (codelite has a mechanism for schema changes) - this can be achieved by changing the value of "gTagsDatabaseVersion" global variable in tags_storage_sqlite3.h, when modified codelite will retag the workspace and will update the schema version
- The method 'Manager::UpdatePreprocessorFile' is called twice once from the event 'OnPageChanged' and once from ReloadFile() from cl_editor.cpp
- But the biggest problem is that 'Manager::UpdatePreprocessorFile' is *very* slow. I tried it on codelite workspace and it took several seconds for a file to load, but since we also call it from 'OnPageChanged' switching tabs also become laggy (2-3 seconds to swtich a tab...)
The cause of the laggy behvior is the code that collects the 'workspace files' and filter them - I will see how this code can be optimized
- And last problem is the one you mentioned: block guards cause the entire file to gray out
Improvements:
- We need to provide means in the UI to disable this option
- I am going to add 'C/C++ Parser options' to the project level. Once added, I am going to add two options:
** Automatically detect macros
** User macros - this way the user can set a macros in case the automatic process fails to detect it correctly
In addition, I will add an option to set 'search paths' per build configuration
Eran
Re: Preprocessing and scintilla_22
Posted: Fri Nov 19, 2010 11:35 pm
Update:
In Manager::UpdatePreprocessorFile:
wxFileName == wxFileName is *very* slow. Changing this to comparison between the full paths of the two reduced 99% of the slowdown.
Still, we need to optimize the gathering the workspace files code.
Also, I removed the call to Manager::UpdatePreprocessorFile() from LEditor::ReloadFile() (it seems that handling it from Mainbook::OnPageChanged is enough)
I will commit the changes once I will add a GUI option to disable this functionality
Eran
I managed to reduce the overhead of loading the file, the main slow down was this line:eranif wrote:- But the biggest problem is that 'Manager::UpdatePreprocessorFile' is *very* slow. I tried it on codelite workspace and it took several seconds for a file to load, but since we also call it from 'OnPageChanged' switching tabs also become laggy (2-3 seconds to swtich a tab...)
In Manager::UpdatePreprocessorFile:
Code: Select all
if (projFileName == filename) {Still, we need to optimize the gathering the workspace files code.
Also, I removed the call to Manager::UpdatePreprocessorFile() from LEditor::ReloadFile() (it seems that handling it from Mainbook::OnPageChanged is enough)
I will commit the changes once I will add a GUI option to disable this functionality
Eran
Re: Preprocessing and scintilla_22
Posted: Sat Nov 20, 2010 1:27 am
Hi Eran
Glad to see your review about my patch. It's very interesting.
A pause was also good for me to think about this improvement. Here are my thought :
The main question I was asking myself is the following : What are really the macros value used by a file ?
The answer is not so easy : We need to work as a compiler : include the headers in the real order, and manage the macro retrieving / storing during this job. And of course, it depends on the current workspace configuration (think about the DEBUG macro for example). Retrieving headers as a std::set change the order (moreover, IIRC, CL doesn't parse directly an header when it find one).
The second point is to store the macros value even for simple macros for #if lines :
Well, I suppose it needs lot's of changes...
Maybe clang could help us for this task (when you'll integrate to CL )
Glad to see your review about my patch. It's very interesting.
A pause was also good for me to think about this improvement. Here are my thought :
The main question I was asking myself is the following : What are really the macros value used by a file ?
The answer is not so easy : We need to work as a compiler : include the headers in the real order, and manage the macro retrieving / storing during this job. And of course, it depends on the current workspace configuration (think about the DEBUG macro for example). Retrieving headers as a std::set change the order (moreover, IIRC, CL doesn't parse directly an header when it find one).
The second point is to store the macros value even for simple macros for #if lines :
Code: Select all
#if MACRO == 7Maybe clang could help us for this task (when you'll integrate to CL
)Re: Preprocessing and scintilla_22
Posted: Mon Nov 22, 2010 6:20 pm
Hi Eran
Could you tell me why the file crawler returns files with '/' instead of '\' under Windows ? May I change this ? and How ?
Now, I got something cool : It's faster and better (even for guard block in headers).
Could you tell me why the file crawler returns files with '/' instead of '\' under Windows ? May I change this ? and How ?
Now, I got something cool : It's faster and better (even for guard block in headers).