Evolve your manual deployments

Manually deploying and configuring cloud infrastructure is a natural starting point for cloud practitioners. However, as applications mature, maintaining manually deployed infrastructure is unsustainable and is an industry anti-pattern.

In this guide, you will learn how to identify manual cloud infrastructure deployments and progress from semi-automated to fully-automated deployments.

In the following video, HashiCorp co-founder Armon Dadgar explains what infrastructure as code is and why it's important.

Evaluate current deployments

You should evaluate where your organization currently is in the automation maturity model. The automation maturity model describes the criteria from manual deployments to automotive deployments and what you need to do to achieve each level of automation.

• Manual deployments: Common methods include using the cloud UI dashboard or connecting to a remote server and running commands through CLI or RDP. Manual deployments introduce risks such as a lack of version control, human error, consistent repeatability, hard-to-audit changes, and difficulty scaling.

• Semi-automation: Instead of running deployment and configuration commands, you run scripts, such as bash, python, and cloud-init or user-data. Semi-automation gives you repeatability and basic version control. However, sometimes, the script you run drifts from the version of the script in source control.

• Fully-automated deployments: You use version controlled scripts to deploy your infrastructure in a repeatable and controlled manner. Automated deployments use automation tools such as Ansible or Terraform, and run on CI/CD systems.

HashiCorp resources:

• Evaluating Your Current Provisioning Practices

Identify manual processes

You should review your infrastructure deployments to identify manual processes. You can then determine which processes you should focus on automating.

The following are ways you can identify manual deployments in your cloud environment:

• Start with the basics of your infrastructure. If your application runs on a virtual server and you deploy it through the cloud UI, automating the creation of that server with IaC is a good starting point.
• Do you SSH or RDP into your servers to run commands? Common manual configuration includes installing dependencies, setting up filesystems, and installing security patches.
• Review ticketing processes like JIRA or other help-desk software to find recurring infrastructure requests, such as increasing server size, giving users permissions, or creating new cloud infrastructure. Your organization might be manually resolving these requests, and they are perfect examples of manual processes to automate.
• Ask your team what infrastructure processes they commonly work on. Specifically, ask if any work is done through the cloud UI, as you should automate that work.

It is important to note that connecting to a system to run manual commands is sometimes unavoidable. In this case, you should use a secure connection method like HashiCorp Boundary. Boundary provides access to applications and critical systems with fine-grained authorizations without managing credentials or exposing your network.

While it might seem overwhelming to find all of the things to automate, you can narrow the scope and focus on the following:

• High-value processes such as database backups
• Repeatable tasks like creating development and test environments
• Error-prone operations such as security configurations
• Time-consuming activities like large-scale system updates, performance testing, and audits.

HashiCorp resources:

• Get started with HCP Boundary

Implement semi-automated deployments

Once you identify the manual deployments in your organization, you can start automating them. Going from manual to fully automated deployments can be challenging and overwhelming. It is common to migrate to semi-automated deployments before evolving into fully-automated ones.

You should adopt the following semi-automated best practices:

Use version control systems (VCS)

You should store your automation scripts and IaC in version control systems like GitHub or GitLab. Version control systems increase repeatability and reliability when deploying and configuring infrastructure. Each time you run a script, you should pull it down from your version control system to ensure it is up-to-date since the last time you ran it. You can also version scripts for environments. You can use tags in VSC for each environment, one for dev, staging, and production.

Use scripts instead of manually running commands

Infrastructure automation through scripting is a fundamental shift that can increase the reliability and security of your infrastructure and application. When you script your infrastructure, you reduce human error, enable scaling, increase consistency, and create a foundation for further automation.

You can use infrastructure as code tools, such as Terraform and Ansible, to deploy and configure your infrastructure. Infrastructure as code lets you define your infrastructure using code and configuration files instead of manually configuring servers, networks, and other resources. This approach allows you to version, test, and deploy infrastructure changes just like you would with application code, making it easier to maintain consistency, automate deployments, and quickly recover from failures. By treating infrastructure as code, you can ensure your environments are reproducible, scalable, and maintainable.

Create application images with code

You should automate building images that your application runs on. These images contain dependencies, security patches, and the application. Manual configuration can lead to installing incorrect dependencies, missing security patches, and inconsistent application installation and configuration.

Packer and Dockfiles create images using IaC and automation, provisioning with scripts and tools like Ansible. Packer is a community tool for creating identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs on every major operating system, and is highly performant, creating machine images for multiple platforms in parallel.

You can use Packer to build a golden image pipeline that creates a reusable base image called a golden image. A golden image is an image on top of which developers can build applications, letting them focus on the application itself instead of system dependencies and patches. A typical golden image includes common system, logging, and monitoring tools, recent security patches, and application dependencies.

You can use this golden image as the base to install your application on. Once Packer installs and configures your application, you can use Terraform to deploy it. Terraform will tear down the old server and replace it with a new one which runs the new version of your Packer generated image.

HCP Packer further simplifies image creation and deployment by providing a central artifact registry for operations and development teams. HCP Packer stores metadata about the artifacts you build, including when you create the artifact, the associated platform, and which Git commit is associated with your build.

Create immutable infrastructure

Immutable infrastructure is infrastructure that, once deployed, is never modified, only replaced. For example, in a mutable server, you update the server either by connecting to the server and running commands or by using a script. With immutable infrastructure, you would fully replace the server with a new one.

Terraform, Packer, and Nomad help facilitate immutable infrastructure.

Terraform allows you to define and provision infrastructure environments, such as servers, virtual networks, and IAM roles and policies. When you change your IaC, Terraform will update your infrastructure components by updating the configuration or destroying and recreating it. To see what changes Terraform will apply, you can run terraform plan before running terraform apply.

Packer creates and configures machine images that are ready to be run. You can use Terraform to deploy these images and user_data or other start-up scripts to ensure services start. With Packer, you can create new images when you need to update your application rather than update the existing infrastructure.

Nomad orchestrates application workloads across clusters, treating application containers and instances as immutable. When you need to update your application, Nomad can replace running instances with new versions rather than modifying existing ones. HashiCorp co-founder and CTO Armon Dadgar explains the differences and trade-offs between mutable and immutable infrastructure.

Audit your cloud logs

Once you start automating your infrastructure, it is important to make sure that infrastructure isn't manually changed or deployed. By using tools like AWS CloudTrail, Azure Activity Log, or Google Cloud Audit Logs, you can uncover who's making manual changes through the console, when these changes occur, and what resources are being modified outside of automated processes.

HashiCorp resources:

• Codify your entire infrastructure with Terraform
• Best practices to automate infrastructure
• Manage infrastructure and service monitoring
• Introduction to Terraform, Packer, and Nomad
• Build Immutable Infrastructure with Packer in CI/CD
• Learn to build a golden image pipeline with HCP Packer

External resources:

• Ansible community documentation
• Cloud logs documentation for GCP, AWS, and Azure

Implement fully-automated deployments

A fully automated system is a holistic approach to managing the IaC lifecycle. A fully automated infrastructure deployment system will be composed of Git-driven infrastructure changes, implemented through a CI/CD system, with automated tests and monitoring.

Fully automated systems build on the best practices of semi-automated systems (VSC, automation tools, audit). The following are additional best practices when you implement fully automated infrastructure.

Run Git-driven deployments through CI/CD

When you manage your IaC through Git-driven development, you combine your IaC management with interactions to your Git VCS and CI/CD. When you want to modify your infrastructure, such as standing up or tearing down a server, you must commit a change to your VCS, which will kick off a CI/CD job to change your infrastructure. This process ensures that every infrastructure modification goes through proper testing, security scanning, and validation before reaching production.

Using Git-driven development, you gain the following benefits:

• Ties code to infrastructure which gives you documentation as code.
• Audit trail of changes. The commit history provides a complete audit trail, showing who made what changes and when.
• Allows you to scale infrastructure efficiently
• Automated infrastructure tests

With HCP Terraform, you can use the built-in VCS workflow to automatically trigger runs based on changes to your VCS repositories. The CLI-driven workflow allows you to quickly iterate on your configuration and work locally, while the VCS-driven workflow enables collaboration within teams by establishing your shared repositories as the source of truth for infrastructure configuration.

You can manage your image creation with Git and CI/CD, similar to how you manage your other infrastructure. Once you commit a change to your Packer file, your CI/CD should trigger a Packer build. Upon completion, your CI/CD system should tag and upload your image to an image repository. You can use HCP Packer to store metadata about the images you build, including when you create the artifact, the associated platform, and which Git commit is associated with your build. HCP Packer allows your downstream processes, like Terraform, to consume these images efficiently.

Deploy self-service infrastructure

Application developers can utilize IaC and temporary infrastructure without writing IaC. You increase application development velocity by creating a process for developers to quickly and reliably build the infrastructure they need to run their application code.

HCP Waypoint is an internal developer platform (IDP) that allows platform teams to define golden patterns and workflows that enable a self-service experience for developers. Developers understand the requirements of their application, such as dependencies like MySQL and Redis, but they shouldn't have to create and maintain the infrastructure on which their application runs.

Platform teams define golden workflows for actions such as building an application, deploying to production, performing a rollback, and other workflows. Developers can execute these workflows with a simple UX while being abstracted from the details. HCP Waypoint integrates with your existing CI/CD systems while providing a consistent abstraction layer.

Once developers deploy the templates to create infrastructure, they must maintain their application. HCP Waypoints actions allow developers to perform day 2 operations to maintain their application, such as rotating secrets, rebuilding a database index, or flushing an application's cache.

Use infrastructure monitoring and testing

Infrastructure monitoring and testing are important to help prevent outages, security breaches, and performance issues before they impact your business. By continuously monitoring your infrastructure, you gain real-time visibility into system health, resource utilization, and performance metrics, allowing you to detect and address issues proactively rather than reactively.

Testing complements monitoring by validating that your infrastructure works as intended before promoting it to production. Through comprehensive testing, including load testing, security scanning, and disaster recovery drills, you can verify that your infrastructure is not just running, but running correctly and securely.

You can use Terraform and Packer to install and deploy monitoring agents into your application images. By automating the installation of the monitoring agents and deploying the application image, you can ensure you will have visibility over the infrastructure your application runs on.

You can also use Terraform to configure cloud-native tools to monitor your cloud infrastructure stacks. You can create dashboards and alarms and automate responses to alerts.

HashiCorp resources:

• Learn how to use VCS-driven workflow with HCP Terraform
• Learn how toautomate Packer with GitHub Actions
• What is HCP Packer?
• What is HCP Waypoint?
• Get started with HCP Waypoint
• Manage infrastructure and service monitoring

External resources:

• What is GitOps?

Next steps

In this document, you learned how to identify infrastructure that you should automate. You also learned the differences between semi and fully-automated infrastructure deployments. You now have the tools to go from manual to automated deployments.

You can learn best practices on writing Terraform with our Terraform code style guide.