The way to the success of this tool crafted by We are Segment’s experts, Htcap scanner, has been paved by the article written by Mr.Zorz, the editor in chief of Help Net Security and (IN)SECURE Magazine, describing it as “a free web application scanner that can crawl single page applications in a recursive manner by intercepting Ajax calls and DOM changes.”

Htcap is a web application scanner able to crawl single page application (SPA) in a recursive manner by intercepting ajax calls and DOM changes.

It is focused on the crawling process and uses external tools to discover vulnerabilities. It is designed to be a tool for both manual and automated penetration test of modern web applications. Here the success of this product lies.

htcap segment

HTCAP WORKING PROCESS

The scan process is divided in two parts, first, Htcap crawls the target and collects as many requests as possible (URLs, forms, ajax ecc..) and saves them to a sql-lite database. When the crawling is done it is possible to launch several security scanners against the saved requests and save the scan results to the same database.

Once all data is collected (at least the crawling one), it is possible to analyze it with ready-available tools such as sqlite3 or DBEaver or to export the results in various formats using the built-in utilities.

The tool also supports three scan modes: passive, active and aggressive. When in passive mode, the app does not interact with the page and only follows the links. In active mode, instead, it triggers all discovered events. This simulates a user interacting with the page without filling input values.

The aggressive mode makes htcap to also fill input values and post forms, simulating a user who performs as many actions as possible on the page.

AJAX CRAWLING

HTCAP features an algorithm able to perform recursive ajax-based pages crawling.

The algorithm works by capturing ajax calls, mapping DOM changes and repeating the process recursively against the newly added elements.

When the tool detects an ajax call, it waits until it is completed and calls the relative callback. If, after that, the DOM is modified, HTCAP runs the same algorithm against the added elements and repeats it until all the ajax calls have been deleted.

The creation of such an algorithm represented the biggest challenge for Segment hackers.

“The algorithm was written in JavaScript so that the concept of ‘waiting for something’ is limited to the asynchronous nature of the language. For example, if you want to detect when an Ajax call is completed, you have to pool its status with a non-synchronous loop. Since the algorithm is recursive, Htcap must wait both for Ajax calls and child call of the recursive function, in an asynchronous environment,” Cavallarin explained

SETUP

Requirements:

  1. Python 2.7
  2. PhantomJS v2
  3. Sqlmap (for sqlmap scanner module)
  4. Arachni (for arachni scanner module)

Download and Run:

$ git clone https://github.com/segment-srl/htcap.git htcap
$ htcap/htcap.py

——–

We are Segment is part of the group Interlogica offering numerous service, from consultancy to advanced technical training in the matters of the cybersecurity sector.