Centralize packages and dependencies

Applications often rely on other software, known as dependencies, to perform their intended function. Dependencies allow developers to include specific functionality in their application without having to create the code to do so. They can decrease development time and complexity and they allow developers to leverage specialized knowledge that they may not have themselves. Some common dependencies include those that format output, collect application metrics, export logging data, create calls to other applications over an API, communicate with databases, and test code.

In order to function correctly, a piece of software must be packaged with any dependencies it uses into a build artifact, which is the output of the software build process. During this build process, dependencies are retrieved, source code is compiled, and tests are run. The dependency retrieval step can become complicated as different services may be queried to get the required packages. When multiple software projects are being developed simultaneously, each requiring dependency retrieval and management, the process can become even more complex.

Use an artifact and package manager

A package manager is responsible for managing the packages required by a software project. Often the project defines a list of dependencies in a file that the package manager reads and uses to retrieve dependencies during the build process.

The main purpose of an artifact manager is to store artifacts from the build process but it can also manage packages from package managers and act as a proxy for software being built in a developer or CI workflow. Sonatype Nexus Repository and JFrog Artifactory are examples of artifact managers that can be used as a proxy for common package managers including npm, Go, Maven, and NuGet.

Manually configuring artifact managers across environments can lead to configuration drift and inconsistencies. You can use Terraform to provision artifact managers with infrastructure as code and ensure consistent deployment across development, testing, and production environments.

Streamline and simplify the build process

Developers and CI builds can access one repository for all of their dependencies. The artifact manager provides convenience as a single source for dependencies and streamlines the build process as common dependencies can be downloaded once and cached for future builds, which decreases the total build time. Caching dependencies also creates a locally redundant repository in the event that the original source goes offline, increasing availability during build time.

Facilitate environment promotions

Proper artifact versioning and tracking are key to artifact management. Making artifacts immutable ensures the contents of the artifact match exactly what the software bill of materials (SBOM) and metadata state are in the artifact. Using immutable artifacts in development, testing, and production environments ensures that any issues are limited to differences between each environment and not the application. A centralized artifact repository with artifact metadata facilitates a consistent rollback process with previously known working artifacts in the event of a failed deployment.

HCP Packer manages machine image artifacts (like AMIs, Docker images, or VM templates) with channels and metadata tracking, including SBOM storage. By defining channels like "development", "staging", and "production", you can create gates that allow only tested and approved images through. Terraform can automatically retrieve the latest artifacts from these channels with HCP Packer data blocks and do not require you to manually update the configuration code

Enhance security and governance

A centralized artifact manager acts as a single source of truth for all artifacts in an organization. Governance and audit become easier with one entry-point from which to conduct checks and remediation. Both internal and external artifacts can be scanned for Common Vulnerabilities and Exposures (CVEs) before being promoted or included in builds.

Access credentials to your artifact repository should be securely stored in a secrets management tool like Vault and write-access should be limited to CI tools and build processes. Vault can generate dynamic, short-lived credentials for your artifact repository and CI tools to significantly reduce the risk of credential compromise.

Vault's audit logging capabilities track access to artifact repository credentials, providing a comprehensive audit trail for compliance and security investigations. You can implement fine-grained access control policies that determine which teams, services, or individuals can publish or retrieve specific artifacts. Additionally, Vault can encrypt artifacts at rest and manage the encryption keys, ensuring your intellectual property and dependencies remain protected.

HashiCorp resources:

External resources:

Next steps

In this section of Define your process, you learned high-level process automation and how the different components work together. Centralize packages and dependencies is part of the Define and automate processes pillar.

Try out HCP Packer by visiting the HashiCorp Cloud Platform.