Maintenance is complete- We've moved from the domain to click to read more
As part of our journey to complete our separation work and our future integration with Micro Focus, we've now updated our domain for the community. This is an interim step, which will be followed by a series of future update / improvements: - Piloting Idea boards - Refreshing the entire UI. (more to come later) - and more

Crawls 50 times More than Audits

New Member.

Crawls 50 times More than Audits

 During a Web Application Vulnerability Assesment using HP-WebInspect, has any one come aross a state, where the number of Crawls is much more ( 50 times ) than the number of Audits ? If so, what might be the reason for this ?


According to HP's recommendation, the number of Audits should be 3"x" times the number of Crawls on an average. The same was happening earlier. Now We see a remarkable drift in the way the scan progress. Also the amount of time it takes to complete a scan is too much ( close to 10 days when compared to 2 days earlier ). We are not using a huge web server though.


Does anyone have a suggestion/recommendation on how to optimize the usage of HP-WebInspect ?

Acclaimed Contributor.

Re: Crawls 50 times More than Audits

I believe I am the one who commented on the Audit counts being 3x the Crawl counts.  That is typical.  What you are describing is very different and not expected.  I will assume that you are reporting the final counts once the scan has completed, not at some point when the scan was Paused or halted.


The first things I would investigate are the Session Exclusions.  Perhaps you defined a Session Exclusion that permits Crawling but no Auditing of the specified URI/page?


Next, the Content Analyzers settings for Javascript permit WebInspect to fetch script includes from off-site hosts.  If your target site incorporate many, many such links, the Crawl will be permitted to fetch and execute those to continue the Crawl, and yet because those links are not on the target Host they would automatically be excluded from any Auditing.  You do NOT want to add those hosts to the Allowed Hosts settings, this is the Best Practice behavior.


Finally, I must say that if your scan is running 10 days, then you are doing it wrong.  ;-)  Unless it is dealing with a truly large system, WebInspect scans should generally not even run overnight.  My suggestion is that you contact Fortify Support and review the scan results and the Scan Settings together.  I am sure there are features that could be enabled or configured to make your scan much more efficient.  Key settings that come to mind might be Redundant Page Detection, Session Exclusions, deep-linked Start URL field, et al.

-- Habeas Data
Micro Focus Fortify Customers-Only Forums –