Skip to content

Deduplicate Assets

Use this feature to clean up workspaces by merging duplicate assets into a new workspace. This helps organize data when multiple scans or imports result in duplicate assets.

In dynamic IT environments, the same physical or virtual asset is often discovered and reported multiple times through:

  • Different vulnerability scanners
  • Network discovery tools
  • Manual asset imports
  • Integrated CMDB synchronization
  • Agent-based and agentless assessments

This frequently leads to the same IP address being registered with inconsistent casing (e.g., SERVER-01, server-01, Server-01), creating what the platform interprets as distinct assets due to case-sensitive matching.

Overview

The command generates a new workspace with deduplicated assets based on a source workspace. The original workspace remains intact but is deactivated to avoid confusion.

Usage

faraday-manage deduplicate-assets [OPTIONS]

Options

  • -w, --workspace-name TEXT: (Required) Source workspace name.
  • -n, --new-workspace-name TEXT: (Optional) Name for the new workspace. Default: {workspace_name}_deduplicated.
  • -h, --help: Show help message and exit.

How it Works

1. Workspace Creation

A new workspace is created inheriting properties (Customer, Dates, etc.) from the source. The workspace flag force_lowercase_assets is enabled in the new workspace, this option forces only lowercase assets to be created in any other future asset creation.

Two or more assets are identified as duplicates and merged when they share the same normalized IP address field (ASSET column of field in the user interface). The normalization process converts all IP address values to lowercase before comparison.

2. Identification & Canonical Asset Creation

The system scans all assets in the source workspace and groups those with case-insensitive matching IP addresses.

For each group of duplicates:

  • A new master asset is created with the IP address in lowercase format.
  • Hostnames from all duplicates are aggregated into the master asset.
  • Descriptions: different descriptions identified and merged. The phrase “Merged from N Assets” is appended.
  • Attributes (OS, MAC, others) are taken from the oldest asset (older creation date).
  • Services from all duplicates are aggregated:
  • If the same port & protocol exist multiple times, one service is created using attributes from the oldest service (service name, port, protocol).
  • Only Banner and Description fields from duplicates are combined.
  • “Merged from N services” is appended.
  • If any duplicated asset is marked as Owned, the master asset is also marked as Owned.

3. Vulnerability Consolidation

All vulnerabilities from duplicate assets are merged into the new canonical asset.

For duplicate vulnerabilities (same name, type, description, associated service):

  • The oldest vulnerability is chosen.
  • If its service name no longer exists, a service with the same port and protocol is selected.

4. Final Step

The source workspace is marked inactive but not deleted.