As you may have noticed, our red teamers regularly create useful tools (BFScan, UnUnicode, Collab2, osmo-nidc, various tools for Mythic). Today, we’ll talk about how we improved our work with the Hive module for collaboration on the Hexway Pentest Suite platform.
The Hive application allows you to gather scan results, notes, screenshots, and other information about services in one place. We also use Hive to track the work plan: marking what has already been checked, what needs attention, and which team member is currently reviewing which service. This prevents duplicate checks (or more, if the project team has many pentesters and everyone wants to run, for example, ffuf with the raft-large-directories-lowercase.txt dictionary) and allows filtering services by status.
The web interface of the application provides the ability to upload files with the results of various tools, parse CSV files, or add data manually. But this turned out to be insufficient for us; we wanted to use the API to upload some data.
Hexway has a Python library for working with the Hive REST API, but it hasn’t been updated in a while. However, the documentation describes how to obtain the OpenAPI specification.
As a result, we wrote the scan2hive utility, which parses the results of various tools and uploads them to Hive. Here’s what its modules do:
HTTPX. Parses the result of httpx in JSON format, adds a tag and a note to the ports in the following format:
httpx result:
url: {url}
title: {title}
webserver : {webserver}
tech: {tech}
final_url: {final_url}It’s convenient to scroll through the list of ports and immediately understand what service it is. You can use the filter by note for searching, for example, note == '%nginx%'.
Nmap. A module for importing nmap and masscan results (XML format). The Hive web interface has the ability to import files with the results of these tools, but we added a few features:
— If a host has more than 300 ports (you can specify the number with the -m parameter), the ports are not imported (used to filter out false positives), and a note is added to the IP address:
if len(host.ports) > self._max_ports:
host.notes.append(HiveLibrary.Note(text=f"{len(host.ports)} ports. No port will be imported"))
logger.info(f"Host {host.ip} has {len(host.ports)} open ports. Skip ports.")
host.ports = list()— A tag is added to each port.
— Script results can be added to notes or ignored (the --script-parsing parameter). By default, data is added to Records, as is the case when importing from the web interface.
Nuclei. Takes the template_id, severity (you can specify the --min-severity parameter), description, extracted_results, matched-at, and matcher-name from the JSON or JSONL format and adds a note to the port (one note per severity). You can ignore some templates.
Gowitness. Can import screenshots (by default, without screenshots; you can choose all or only for 200 OK responses) and add a note with data from the database:
gowitness result:
url: {url}
response_code: {response_code}
title: {title}
webserver : {webserver}
final_url: {final_url}
tech: {tech}
cookies: {cookies}Poseidon. Parses the JSON with the result of a task in the Poseidon C2 agent for Mythic.
In the tag, we usually indicate the IP from which we accessed the hosts, which helps to create a network map in an internal pentest or, for example, understand that some services are only accessible from a specific country. To check how a file will be parsed, you can use the --dry-run mode, and to upload to Hive, use the --upload mode.
And in /tests, we added generated scan results for nmap and masscan so that we don’t have to test on customer data.
