index vault

Beta feature

This feature is currently available as a beta release. Beta functionality is stable but possibly incomplete and subject to change. We strongly discourage using beta features in production.

Note

You must have version 0.17.0 or higher of the Vault Radar CLI installed.

To check the current version of your CLI, use the version command.

The index vault command is used for creating an index of secrets from Vault.

Authentication and authorization

vault-radar requires the VAULT_ADDR and VAULT_TOKEN environment variables in order to connect to your Vault cluster. The index vault command will traverse the full namespace hierarchy. Within each namespace, an attempt to scan every AppRole and KVv2 mount will be made. Access can be limited via the policies attached to the token provided. vault-radar will attempt to use the following endpoints:

Mount	Resource	Method	Endpoint
System	List namespaces	`LIST`	sys/namespaces
System	List auth methods	`GET`	sys/auth
System	List secret mounts	`GET`	sys/mounts
AppRole	List roles	`LIST`	auth/:mount-path/role
AppRole	List secret ID accessors	`LIST`	auth/:mount-path/role/:role_name/secret-id
AppRole	Lookup secret ID accessors	`POST`	auth/:mount-path/role/:role_name/secret-id-accessor/lookup
KVv2	Read KV engine config	`GET`	:mount-path/config
Kvv2	List secrets	`LIST`	:mount-path/metadata/:path
KVv2	Read secret metadata	`GET`	/:mount-path/metadata/:path
KVv2	Read secret version	`GET`	/:mount-path/data/:path?version=:version-number

Index generation

You can generate an index of KVv2 secrets from Vault using the index vault command:

$ vault-radar index vault --outfile </path/to/outfile.jsonl>

The index output will be JSONL-formatted. There will be an entry per secret sub-key, per version, per KVv2 secret. For example, there will be 6 entries for a secret with 3 versions that contains an AWS access key ID and secret key.

Each index entry will contain the following fields:

value_hash - The hashed version of the secret value
secret_key - The underlying sub-key within the Vault secret (e.g. aws_secret_key)
secret_type - The of type of secret determined by its key and/or underlying value (e.g.GitHub personal access token, AWS secret key)
secret_age_days - The time elapsed in days since creation
location - A full URL that can be used to retrieve the secret (e.g. vault://127.0.0.1:8205/v1/eng/team1/app-foo/data/aws?version=1)

The index can then be used to compare against in other scans. For example, the following command can be used to run a Confluence scan using a generated Vault index file:

$ vault-radar scan confluence --outfile=confluence.csv \
    --url="http://localhost:8090" \
    --space-key=VRD --index-file=vault.idx

An in-memory index keyed off of secret hashes will be generated prior to scanning the source. This index will be used to annotate whether a risk exists in Vault.

Generate index file using HCP provided salt

Index files are generated in an "online mode", meaning that the secret hash produced is using a salt that is provided from HCP. This requires the Project Service Principals to be configured for your system as outlined by the HCP upload page.

$ vault-radar index vault -r <REGION CODE> -o <PATH TO OUTPUT>.jsonl

HCP Vault Dedicated considerations

In a HCP Vault Dedicated cluster, the root namespace is restricted and the users will only have access to admin namespace and all the child namespaces within it. For vault-radar index vault to work against a HCP Vault Dedicated cluster, it is mandatory to set VAULT_PARENT_NAMESPACE environment variable to the namespace that needs to be scanned.

$ export VAULT_PARENT_NAMESPACE=admin
$ vault-radar index vault --outfile index-results.jsonl

The above command will scan all the namespaces within the admin, including the admin namespace.

Note

The VAULT_PARENT_NAMESPACE will also work on Vault Enterprise but it is not mandatory to set.