What is secret scanning

7min
|
HCP
Vault
Video

HCP Vault Radar automates the detection and identification of unmanaged secrets in your code so that teams can take appropriate actions to remediate issues.

Leaked or exposed secrets can lead to unauthorized access to systems and other critical data.

Secrets scanning is a process that allows you to find and identify secrets and other sensitive data hidden in source code or other locations such as documentation. With the correct tools, secret scanning functionality helps deliver secure code without compromising speed or innovation.

Having the ability to scan for secrets and other sensitive data will help protect your customers, limit the potential for breaches due to leaked credentials, and enhance the company's reputation as one that prioritizes security.

Vault Radar provides a Software-as-a-Service (SaaS) solution for scanning source code for secrets and sensitive data. Radar scans for the following types of information:

Secrets
Personally identifiable information (PII)
Non-inclusive language (NIL)

Once the scanning completes, Vault Radar displays the detected risks by categories and risks.

In this series of tutorials, you will learn about Vault Radar through the lens of HashiCups as their engineering team attempts to remove sensitive data in their source code.

Scenario introduction

HashiCups produces and sells its coffee cups at both retail locations and through its online store. They support both a web application and a mobile application. The team at HashiCups has concerns about leaking secrets such as usernames and passwords, and API keys in their source code.

The CTO and CISO have presented the following business and technical requirements to the engineering teams:

All source code must be free of sensitive data
Teams get notifications any time the solution detects sensitive data
Scans for leaked secrets must occur at various stages of the software development life cycle
Any potential solution cannot store HashiCups-owned source code

The team has several groups who will work together on the review of, and implementation of the selected solution(s).

Click on each tab to learn more about the teams and their responsibilities.

Alice leads the engineering architect team. The architect team:

Understands system, resource, and connectivity requirements for all users and applications.
Identifies supported services within the solution that other users, and systems will use to authenticate.
Compares and contrasts features and functions available in any proposed solution.
Designs the implementation process, including support for high availability, disaster recovery, observability, and support runbooks.
Creates as-built documentation to hand off to other teams.

Oliver manages the operations team. The operations team ensures:

All systems are implemented according to the architect team's design.
Performs system and application deployments, updates, and upgrades.
Ensures source code, systems, and applications have appropriate security scanning in place.
Tunes systems and applications to ensure adequate performance.
Provides support to other teams when accessing applications and systems.
Manages access to applications and systems.

Danielle is the lead engineer and manager for the development team. The development team:

Develops HashiCups internal and customer-facing applications.
Works with the architecture team to identify new platform services the team can integrate into its applications, such as single-sign-on.
Provides support for incidents that may require temporary access to systems.
Implements vendor SDKs to support integrating third-party applications.

HashiCups has brought in HashiCorp to see how they can help achieve the goals set by the CISO and CTO.

HCP Vault Radar concepts

Before diving into how Vault Radar works, there are several key concepts that the teams at HashiCups would like to understand.

Data sources

Danielle, who leads the development team, has asked how and when Vault Radar scans their source code.

Through the HCP Portal you can connect Vault Radar to GitHub, GitLab, Bitbucket, and Azure DevOps.

Vault Radar can scan both cloud-based and on-premises data sources. Public data sources connect to the HCP cloud scanner. The Vault Radar agent scans on-premises data sources, or you can expose on-premises data sources to the cloud scanner.

The Vault Radar CLI supports other data sources such as local system files and directories, Docker, Amazon S3, and Terraform Enterprise.

Danielle has asked how they can achieve one of the goals set by the CISO and CTO of being able to scan for sensitive data throughout the SDLC.

Vault Radar can also scan for sensitive data throughout the SDLC. You can use pre-commit hooks during local development, when a developer pushes a branch to a source code repository, or when someone opens a pull request.

Types of sensitive data

Steve from the SRE team would like to understand what types of sensitive data Vault Radar scans for - is it just passwords and keys?

Vault Radar can natively scan for different formats of sensitive data, including:

Secrets such as usernames, passwords, and keys.
Personally identifiable information (PII) such as social security, or credit card numbers.
Non-inclusive language(NIL) such as race or gender attributes.

Beyond the supported patterns that Vault Radar can scan for, HashiCups can also create their own custom regular expressions (regex) to scan for sensitive data. The custom regex scans can be for any type of data, such as product model numbers, financial information, or other sensitive information.

Integrations

Oliver points out that scanning for sensitive data is only one of the requirements. They would like to know how Vault Radar notifies the operations and SecOps teams so they can triage alerts.

Vault Radar supports the alert and triage requirements set by the HashiCups CISO and CTO.

HashiCups can configure alerts for sensitive data found by Vault Radar using native integrations for PagerDuty, Slack, and Splunk.

You can configure different alert integrations to match your existing processes. For example, you can enable the Microsoft Teams or Slack integration for real-time notifications. You can also enable the PagerDuty integration to follow your defined escalations until the team responsible resolves the alert.

HashiCups can also use the ticketing integrations to open a ticket in Jira or ServiceNow, allowing teams to track the incident through to the incident's conclusion.

How HCP Vault Radar works

The team sees the possibilities in Vault Radar, however, Alice from the architecture team would like more detail on what happens with HashiCups source code when Vault Radar detects secrets.

The first step to set up HCP Vault Radar is to connect a supported source code management (SCM) system. Once set up, the HCP Vault Radar scanning engine reviews the selected repositories, including available branches for sensitive data.

Vault Radar does not send or store source code or sensitive data. Instead, Vault Radar performs a two-phase hash or peppering so Vault Radar can identify if the sensitive data exists in different data sources. This hash is then tokenized and returns a universally unique identifier (UUID) which Vault Radar stores in the HashiCorp Cloud Platform.

The generated UUID, the commit hash, and the line number where Vault Radar finds the sensitive data is available in the HCP Portal.

Next steps

In the next tutorial, the engineering teams at HashiCups will work together to implement a proof-of-concept deployment of HCP Vault Radar.

The POC will include the following steps:

Scan the organization's GitHub repositories to detect leaked and unmanaged secrets.
Integrate with PagerDuty to receive security incidents from Vault Radar.
Set up ticket integration using Jira to triage and track incidents.

Collection Overview

HCP Vault Radar

Scan a repository for secrets