Agents
Easier integrations with Faraday Agents¶
Integrating systems is a challenging yet essential task in the lifecycle of any software product. Developers often encounter unfamiliar languages, undocumented APIs, or new paradigms when attempting to integrate with external tools. Consequently, many product teams opt not to offer integration capabilities.
In the case of Faraday, we recognize that seamless integrations with other security tools are crucial for our product. However, we realized that our existing Plugin system didn't provide the expected ease of use for developing integrations. It required a certain level of interactivity, such as executing commands from the console or importing a report, which made it cumbersome for periodic usage. Additionally, the reliance on our Python API posed difficulties for developers when integrating with tools that weren't programmed in Python.
To address this challenge, we made the decision to introduce the Agents feature in the latest release of Faraday (v3.9). Recognizing its significance as a core feature, we are thrilled to offer it in both the Community and commercial versions of our product.
Architecture¶
Overview of the Agents Feature¶
An Agent represents a process running continuously in a machine (not necessary the same running the Faraday Server). When a user decides to run an Agent (typically done through the Faraday Web UI), it will execute a code and send data back to a Workspace.
The image displayed showcases a workspace featuring several customized agents. These agents perform various tasks, such as utilizing nmap to discover hosts within the network, employing sublist3r to identify subdomains, and retrieving data from our services hosted on Heroku.
Beyond manually executing the agents using their respective "Run" buttons, it is also possible to schedule them for periodic execution. However, please note that this scheduling capability is exclusively available through the web interface for users of our Corporate version.
Now you can Run an Agent for a single use!
or with simple cronjobs if you are using the Community or Paid version.
Agents technical details¶
To simplify integrations with Faraday, we have developed the Faraday Agent Dispatcher. This project acts as a communication middleware between the Faraday Server and your custom agents.
To create integrations, you can build an Executor script that outputs data in JSON format to the standard output. This script handles tasks like retrieving host information, vulnerabilities, and more. The Dispatcher abstracts the complexities of APIs and communication protocols, allowing you to focus on generating the necessary JSON data. By leveraging the standard output for data transfer, the Dispatcher seamlessly handles the integration process. For further insight into the project's concept, you can refer to the explanation provided by one of our developers here.
This flexible approach enables you to employ a wide range of programming languages for your integrations. As long as the language supports standard output printing, you can develop agents in languages like Python, Bash, or even unconventional choices like Brainfuck.
We offer official executors that are pre-configured and require minimal setup. Alternatively, you have the option to create and execute custom executors tailored to your specific requirements. For detailed instructions on developing and running custom executors, refer to our custom executor documentation!
How to start using agents¶
Lets try with an example with Nmap to check the Faraday Agents capabilities both as a custom executor, and run an official one.
First of all you must install the Faraday Dispatcher inside the server you want to run the Agent on. You can do so running one of the following commands:
$ git clone https://github.com/infobyte/faraday_agent_dispatcher
$ git clone https://github.com/infobyte/faraday_agent_parameter__types
$ pip3 install faraday_agent_dispatcher
Once you choose your desired configuration, you'll be able to select the tools that need to be run
After selecting the tools, you'll be able to download the dispatcher.yaml
Please follow the instruccions depending on what you are working with, in this case we are using Python, so in order to make this work the dispatcher.yaml must be placed in the following folder
$HOME/.faraday/config
Now you would be able to run the agent in your server
If you have done this correctly, the new agent will be displayed in the Agent view
Or you can just do it in the manual way running the configuration wizard:
$ faraday-dispatcher config-wizard
Do you want to edit the [A]gent or the [E]xecutors? Do you want to [Q]uit? (A, E, Q) [Q]: A
Section: server
host [127.0.0.1]: server.faradaysec.com
ssl [True]:
ssl_port [443]:
ssl_cert []:
Trying to save with empty value
workspace [workspace]:
Section: tokens
registration [ACorrectTokenHas25CharLen]: ImalA8Cg1L6Z5Qbx2u9CFAsob
Section: agent
agent_name [agent]: nmap
Do you want to edit the [A]gent or the [E]xecutors? Do you want to [Q]uit? (A, E, Q) [Q]: E
The actual configured executors are: []
Do you want to [A]dd, [M]odify or [D]elete an executor? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Name: nmap
Is a custom executor? (Y, N) [N]: Y
Command to execute [exit 1]: python3 /home/faraday/faraday_agent_dispatcher/static/executors/official/nmap.py
Max data sent to server [65536]:
The actual nmap executor's environment variables are: []
Do you want to [A]dd, [M]odify or [D]elete an environment variable? Do you want to [Q]uit? (A, M, D, Q) [Q]:
The actual nmap executor's arguments are: []
Do you want to [A]dd, [M]odify or [D]elete an argument? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Argument name: port_list
Is mandatory? (Y, N): Y
The actual nmap executor's arguments are: ['port_list']
Do you want to [A]dd, [M]odify or [D]elete an argument? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Argument name: target
Is mandatory? (Y, N): Y
The actual nmap executor's arguments are: ['port_list', 'target']
Do you want to [A]dd, [M]odify or [D]elete an argument? Do you want to [Q]uit? (A, M, D, Q) [Q]: Q
The actual configured executors are: ['nmap']
Do you want to [A]dd, [M]odify or [D]elete an executor? Do you want to [Q]uit? (A, M, D, Q) [Q]: A
Name: nmapOfficial
Is a custom executor? (Y, N) [N]:
The executors are:
1: wpscan.py
2: sublist3r.sh
3: nmap.py
4: nikto2.py
5: nessus.py
+: Next page
Q: Don't choose
Choose one: 3
Max data sent to server [65536]:
The actual configured executors are: ['nmap', 'nmapOfficial']
Do you want to [A]dd, [M]odify or [D]elete an executor? Do you want to [Q]uit? (A, M, D, Q) [Q]:
Do you want to edit the [A]gent or the [E]xecutors? Do you want to [Q]uit? (A, E, Q) [Q]:
All the information you added with the configuration wizard should appear on a new dispatcher.ini file at your /.faraday/config/ directory:
$ cat $HOME/.faraday/config/dispatcher.yaml
[server]
host = server.faradaysec.com
ssl = True
api_port = 443
websocket_port = 443
ssl_cert =
workspace = agent_workspaces
[tokens]
registration = ImalA8Cg1L6Z5Qbx2u9CFAsob
[agent]
agent_name = nmap
executors = nmap,nmapOfficial
[nmap]
cmd = python3 /Users/famato/dev/faraday_agent_dispatcher/contrib/nmap.py
max_size = 65536
[nmap_params]
port_list = True
target = True
[nmap_varenvs]
[nmapOfficial]
max_size = 65536
repo_executor = nmap.py
[nmapOfficial_params]
port_list = True
target = True
[nmapOfficial_varenvs]
Now you have to run the dispatcher to register the Agent on your Faraday Server:
$ faraday-dispatcher run
2020-05-11 21:14:01,061 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:91 - register()] token_registration_url: https://server.faradaysec.com:443/_api/v2/ws/agent_workspaces/agent_registration/
2020-05-11 21:14:01,977 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:115 - register()] Registered successfully
2020-05-11 21:14:02,639 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:150 - connect()] Connection to Faraday server succeeded
After this, you’ll see your new Agent on the Web UI. The last step is that you just run the nmap Agent:
And the Dispatcher log will give you details about this action:
2022-02-03 15:59:29,676 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:153 - register()] token_registration_url: https://pro-staging.app3.faradaysec.com:443/_api/v3/agent_registration
2022-02-03 15:59:30,479 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:192 - register()] Registered successfully
2022-02-03 15:59:31,434 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:235 - connect()] Connection to Faraday server succeeded
2022-02-03 16:10:11,385 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:256 - run_once()] Parsing data: {"execution_id": 9, "agent_id": 3, "workspace": "my_workspace", "action": "RUN", "executor": "nmap", "args": {"TARGET": "127.0.0.1"}}
2022-02-03 16:10:11,419 - faraday_agent_dispatcher - INFO {MainThread} [metadata_utils.py:59 - check_commands()] Dependency check ended. Ready to go
2022-02-03 16:10:11,419 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:415 - run_once()] Running nmap executor
{"hosts": [{"ip": "127.0.0.1", "os": "unknown", "hostnames": ["localhost"], "description": "", "mac": "00:00:00:00:00:00", "credentials": [], "services": [], "vulnerabilities": [], "tags": []}], "command": {"tool": "Nmap", "command": "Nmap", "params": "", "user": "", "hostname": "", "start_date": "2022-02-03T19:10:11.665787", "duration": 46861, "import_source": "report"}}
stderr sent empty data,
2022-02-03 16:10:13,246 - faraday_agent_dispatcher - INFO {MainThread} [executor_helper.py:118 - processing()] Data sent to bulk create
2022-02-03 16:10:14,021 - faraday_agent_dispatcher - INFO {MainThread} [executor_helper.py:148 - end_f()] Data sent to bulk create
stdout sent empty data,
2022-02-03 16:10:14,021 - faraday_agent_dispatcher - INFO {MainThread} [dispatcher.py:456 - run_once()] Executor nmap finished successfully
Right after the Agent executes these actions, return to your Workspace and you can review all the information that was found. Now you are ready to deploy your own Faraday Agents through your network! The Dispatcher README file has more detailed documentation to run and build your Agents. Also, we recommend you to check our examples and official executors to get an idea of what a custom integration looks like.
What's Next¶
Recently we added Appscan, Qualys and Tenable.io compatibility! try them out!
We are committed to enhancing the features and usability of Agents while continuously adding new executors to the Dispatcher repository. In upcoming releases, our focus will also be on providing select Agents with read access to their Workspace. This access will enable them to leverage existing data for uncovering more valuable information.
We hope you enjoy this feature and find it useful!