Skip to content

Agents

Easier integrations with Faraday Agents

Integrating systems is an elusive but mandatory job in any software product's life. Developers have to deal with languages they don't know, undocumented APIs or new paradigms. This leads to the fact that many product teams decide not to open the possibility to integrate to them.

In Faraday’s case, we are aware that integrations with other security tools are a critical part of our product. However, we’ve realized that our existing Plugin system wasn’t as easy as we expected to develop some integrations: it required some level of interactivity (either running a command from the console or importing a report), so it was hard to use on a periodic basis. It also forced integration developers to use our Python API, even when the tool to integrate with wasn’t programmed in Python, making it harder for the developer.

To solve this problem, we decided to implement the Agents feature in the already released Faraday v3.9. Since we expect this to be a core feature of our product, it will be available both in Community and commercial versions!.

Architecture

Overview of the Agents Feature

An Agent represents a process running continuously in a machine (not necessary the same running the Faraday Server). When a user decides to run an Agent (typically done through the Faraday Web UI), it will execute a code and send data back to a Workspace.

In the image above, you can see a workspace with many custom agents. They're in charge of finding hosts in the network with nmap, finding subdomains with sublist3r, and provision data from our Heroku-hosted services.

In addition to running the agents manually with their "Run" button, you can also schedule them to run periodically. This can be done from the web if you have our Corporate version:

Now you can Run an Agent for a single use!

or with simple cronjobs if you are using the Community or Paid version.

Agents technical details

To make integrations with Faraday as easy as possible, we developed a project we called Faraday Agent Dispatcher that is in charge of handling the communication between the Faraday Server and your own agents.

The only thing you’ll need to build your own integrations is a script (we call it an Executor) that prints to standard output all the data you want to send to Faraday (hosts info, vulnerabilities, etc.) encoded in JSON. There is no need to use complex APIs or communication methods, as all of this is abstracted by the Dispatcher. You just need to print JSON to standard out, and the Dispatcher will handle the rest! If you want to know a little bit about the idea behind build this project, one of our developers explain a little about it.

This also means that you could use virtually any programming language you want to build your integrations, as long as they support printing data to standard output. We've already made agents in Python, Bash, and even a Brainfuckone!

We have some official executors, which are ready to go with minimum configuration. If you want to develop and run your own executor, you can configure a custom one!

How to start using agents

Lets try with an example with Nmap to check the Faraday Agents’ capabilities both as a custom executor, and run an official one. First of all you have to get your Agent’s token. On your Web UI go to Settings > Agents.

Your token is: ImalA8Cg1L6Z5Qbx2u9CFAsob

Now you must install the Faraday Dispatcher inside the server you want to run the Agent on. You can do so running one of the following commands:

$ git clone https://github.com/infobyte/faraday_agent_dispatcher

$ git clone https://github.com/infobyte/faraday_agent_parameter__types

$ pip3 install faraday_agent_dispatcher

After this, you need to run the configuration wizard:

$ faraday-dispatcher config-wizard

Do you want to edit the [A]gent or the [E]xecutors? Do you want to [Q]uit? (A, E, Q) [Q]: A
Section: server
host [127.0.0.1]: server.faradaysec.com
ssl [True]:
ssl_port [443]:
ssl_cert []:
Trying to save with empty value
workspace [workspace]:
Section: tokens
registration [ACorrectTokenHas25CharLen]: ImalA8Cg1L6Z5Qbx2u9CFAsob
Section: agent
agent_name [agent]: nmap
Do you want to edit the [A]gent or the [E]xecutors? Do you want to [Q]uit? (A, E, Q) [Q]: E
The actual configured executors are: []
Do you want to [A]dd, [M]odify or [D]elete an executor? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Name: nmap
Is a custom executor? (Y, N) [N]: Y
Command to execute [exit 1]: python3 /home/faraday/faraday_agent_dispatcher/static/executors/official/nmap.py
Max data sent to server [65536]:
The actual nmap executor's environment variables are: []
Do you want to [A]dd, [M]odify or [D]elete an environment variable? Do you want to [Q]uit? (A, M, D, Q) [Q]:
The actual nmap executor's arguments are: []
Do you want to [A]dd, [M]odify or [D]elete an argument? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Argument name: port_list
Is mandatory? (Y, N): Y
The actual nmap executor's arguments are: ['port_list']
Do you want to [A]dd, [M]odify or [D]elete an argument? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Argument name: target
Is mandatory? (Y, N): Y
The actual nmap executor's arguments are: ['port_list', 'target']
Do you want to [A]dd, [M]odify or [D]elete an argument? Do you want to [Q]uit? (A, M, D, Q) [Q]: Q
The actual configured executors are: ['nmap']
Do you want to [A]dd, [M]odify or [D]elete an executor? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Name: nmapOfficial
Is a custom executor? (Y, N) [N]:
The executors are:
1: wpscan.py
2: sublist3r.sh
3: nmap.py
4: nikto2.py
5: nessus.py
+: Next page
Q: Don't choose
Choose one: 3
Max data sent to server [65536]:
The actual configured executors are: ['nmap', 'nmapOfficial']
Do you want to [A]dd, [M]odify or [D]elete an executor? Do you want to [Q]uit? (A, M, D, Q) [Q]:
Do you want to edit the [A]gent or the [E]xecutors? Do you want to [Q]uit? (A, E, Q) [Q]:

All the information you added with the configuration wizard should appear on a new dispatcher.ini file at your /.faraday/config/ directory:

$ cat  $HOME/.faraday/config/dispatcher.ini
[server]
host = server.faradaysec.com
ssl = True
api_port = 443
websocket_port = 443
ssl_cert =
workspace = agent_workspaces
[tokens]
registration = ImalA8Cg1L6Z5Qbx2u9CFAsob
[agent]
agent_name = nmap
executors = nmap,nmapOfficial
[nmap]
cmd = python3 /Users/famato/dev/faraday_agent_dispatcher/contrib/nmap.py
max_size = 65536
[nmap_params]
port_list = True
target = True
[nmap_varenvs]
[nmapOfficial]
max_size = 65536
repo_executor = nmap.py
[nmapOfficial_params]
port_list = True
target = True
[nmapOfficial_varenvs]

Now you have to run the dispatcher to register the Agent on your Faraday Server:

$ faraday-dispatcher run

2020-05-11 21:14:01,061 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:91 - register()]  token_registration_url: https://server.faradaysec.com:443/_api/v2/ws/agent_workspaces/agent_registration/
2020-05-11 21:14:01,977 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:115 - register()]  Registered successfully
2020-05-11 21:14:02,639 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:150 - connect()]  Connection to Faraday server succeeded

After this, you’ll see your new Agent on the Web UI:

The last step is that you just run the nmap Agent:

And the Dispatcher log will give you details about this action: 

2020-05-11 21:16:33,691 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:260 - run_once()]  Running nmap executor
{"hosts": [{"ip": "127.0.0.1", "os": "unknown", "hostnames": ["localhost"], "description": "", "mac": "00:00:00:00:00:00", "credentials": [], "services": [{"name": "http", "protocol": "tcp", "port": 80, "status": "open", "version": "", "description": "http", "credentials": [], "vulnerabilities": []}, {"name": "https", "protocol": "tcp", "port": 443, "status": "open", "version": "", "description": "https", "credentials": [], "vulnerabilities": []}], "vulnerabilities": []}], "command": {"tool": "Nmap", "command": "Nmap", "params": "", "user": "", "hostname": "", "start_date": "2020-05-11T21:16:33.867017", "duration": 1029, "import_source": "report"}}
stderr sent empty data,
2020-05-11 21:16:34,596 - faraday_agent_dispatcher - INFO {MainThread} [executor_helper.py:97 - processing()]  Data sent to bulk create
stdout sent empty data,
2020-05-11 21:16:34,596 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:281 - run_once()]  Executor nmap
finished successfully

Right after the Agent executes these actions, return to your Workspace and you can review all the information that was found. Now you are ready to deploy your own Faraday Agents through your network! The Dispatcher README file has more detailed documentation to run and build your Agents. Also, we recommend you to check our examples and official executors to get an idea of what a custom integration looks like.

What's Next

We will continue to improve Agents’ features, usability, and adding new executors into the Dispatcher repository. In the next few releases we would also like to give some Agents read access to their Workspace, so they can benefit from the existing data in order to find more valuable information.

We hope you enjoy this feature and find it useful!