Create immutable containers

Immutable containers package your application and its dependencies into unchangeable container images that you build once and deploy across all environments. You create container images with Docker or Packer, deploy them with orchestrators like Kubernetes or Nomad, and never modify running containers. When you need to make changes, you rebuild the image with your updates and deploy new containers to replace the old ones. The following sections explain why you should use immutable containers, how to build container images, and how to deploy them with orchestrators.

Why use immutable containers

Immutable containers address the following operational challenges:

• Remove environment inconsistencies: Traditional container workflows allow developers to modify running containers with docker exec, creating containers that differ from their source images. Immutable containers prevent modifications to running containers, ensuring every container across development, staging, and production matches its image exactly.

• Prevent configuration drift: In orchestrated environments, containers are regularly recreated during updates, scaling, or node failures, causing manual changes to disappear and creating inconsistent behavior. Immutable containers force all changes through the image build process, making configurations permanent and version-controlled.

• Simplify rollbacks: When container updates cause issues, rolling back requires identifying what changed and how to reverse it. Immutable containers enable rollbacks by redeploying the previous image version, eliminating troubleshooting under pressure.

Build immutable container images

Building immutable container images involves defining your application, dependencies, and runtime configuration as code, then using that code to produce a container image that you push to a container registry. The image contains everything your application needs to run, including the base operating system layer, runtime dependencies, your application code, and startup configuration. Once built and pushed to a registry, the image is ready for deployment across all environments.

You can build container images with Docker using Dockerfiles or with Packer using HCL templates. Docker provides the fastest build and iteration cycles for local development, while Packer offers consistent tooling across both containers and virtual machines. Both tools produce identical immutable container images that you push to a registry and deploy with orchestrators.

Learn how to build container images in Package applications with containers and machine images.

Deploy immutable containers with orchestrators

Container orchestrators manage the deployment, scaling, and lifecycle of your containers. Kubernetes and Nomad deploy containers from images, monitor their health, and replace failed or outdated containers automatically.

Deploy containers with Kubernetes

Kubernetes deploys containers using Deployment resources that define which image to run, how many replicas to create, and how to handle updates. Kubernetes pulls images from your container registry and manages the container lifecycle.

The following Kubernetes Deployment deploys the Python application container:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: python-app
  labels:
    app: python-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: python-app
  template:
    metadata:
      labels:
        app: python-app
    spec:
      containers:
      - name: python-app
        image: your-registry.com/team1/python-app:v1
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "500m"
          limits:
            memory: "512Mi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: python-app
spec:
  selector:
    app: python-app
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: LoadBalancer

The Kubernetes Deployment creates three replicas of your container, defines resource limits, configures health checks, and exposes the application through a LoadBalancer Service. When you update your application, you build a new container image with a new tag, update the Deployment to reference the new image tag, and apply the changes with kubectl apply. Kubernetes performs a rolling update, gradually replacing old containers with new ones while maintaining availability.

You can manage Kubernetes resources with Terraform using the Kubernetes provider, enabling infrastructure as code for your entire deployment.

Deploy containers with Nomad

Nomad deploys containers using job specifications that define which image to run, how many instances to create, and how to handle updates. Nomad integrates with Consul for service discovery and health checking.

The following Nomad job specification deploys the Python application container:

job "python-app" {
  datacenters = ["dc1"]
  type        = "service"

  group "app" {
    count = 3

    network {
      port "http" {
        to = 8080
      }
    }

    task "python-app" {
      driver = "docker"

      config {
        image = "your-registry.com/team1/python-app:v1"
        ports = ["http"]
      }

      resources {
        cpu    = 500
        memory = 256
      }

      service {
        name = "python-app"
        port = "http"

        check {
          type     = "http"
          path     = "/health"
          interval = "10s"
          timeout  = "2s"
        }
      }
    }

    update {
      max_parallel     = 1
      min_healthy_time = "10s"
      healthy_deadline = "3m"
      auto_revert      = true
    }
  }
}

The Nomad job specification creates three instances of your container, configures resource limits, defines health checks through Consul, and specifies rolling update behavior. When you update your application, you build a new container image, update the job specification with the new image tag, and run nomad job run python-app.nomad.hcl. Nomad performs a rolling deployment, replacing containers one at a time while verifying health checks. The auto_revert setting automatically rolls back to the previous version if health checks fail.

You can manage Nomad jobs with Terraform using the Nomad provider, enabling infrastructure as code for your application deployments.

Immutable container workflow

The complete workflow for immutable containers includes the following steps:

1. Build the container image: Create a Dockerfile or Packer template defining your application and dependencies. Build the image with Docker or Packer and push it to your container registry with a version tag.

2. Create the deployment configuration: Write a Kubernetes Deployment or Nomad job specification referencing your container image. Define replica counts, resource limits, and health checks.

3. Deploy the containers: Submit your deployment configuration to Kubernetes with kubectl apply or to Nomad with nomad job run. The orchestrator pulls the image and creates containers.

4. Update and redeploy: When you need to update your application, modify your Dockerfile or Packer template, rebuild the image with a new version tag, update your deployment configuration to reference the new tag, and redeploy. The orchestrator performs a rolling update, replacing old containers with new ones.

5. Rollback if needed: If the new version causes issues, update your deployment configuration to reference the previous image tag and redeploy. The orchestrator immediately replaces failing containers with the previous working version.

You can improve the workflow with the following:

• Automate image builds: Use CI/CD systems to automatically build and push container images when you commit code changes.
• Automate deployments: Use HCP Terraform or GitOps tools to automatically deploy updated containers when new images are available.
• Track image metadata: Use HCP Packer to store metadata about your container images, tracking which Git commit built each image and which environments are running each version.

HashiCorp resources:

• Learn about immutable infrastructure concepts and benefits
• Create immutable virtual machines with Packer
• Learn how to package applications with Packer for containers
• Learn how to deploy applications with Kubernetes and Nomad
• Implement automated testing for containerized applications
• Implement semi-automated deployments and fully-automated deployments with container CI/CD

Packer for containers:

• Get started with the Packer documentation for core concepts
• Follow hands-on Packer tutorials for image building
• Build a Docker image with Packer
• Learn about the Docker builder in Packer
• Use Docker post-processors for tagging images
• Track image metadata with HCP Packer

Nomad deployment resources:

• Read the Nomad documentation for orchestration features
• Follow Nomad tutorials for container deployment examples
• Manage Nomad jobs with the Nomad Terraform provider
• Create a Nomad cluster on AWS, GCP, and Azure
• Learn about Nomad job specifications for container workloads

Kubernetes deployment resources:

• Manage Kubernetes resources with the Kubernetes Terraform provider
• Learn to deploy applications to Kubernetes
• Read about Kubernetes Deployments with Terraform
• Use the Helm Terraform provider for Helm charts

External resources:

• Dockerfile reference documentation
• Kubernetes Deployments

Next steps

In this section of Define your processes, you learned why immutable containers increase reliability, how to build container images with Docker and Packer, and how to deploy them with Kubernetes and Nomad. Create immutable containers is part of the Define and automate processes pillar.

To learn more about immutable infrastructure, continue to the following topics:

• Create immutable infrastructure - Overview of immutable infrastructure concepts
• Create immutable virtual machines - Build and deploy immutable VM images