Using Sentinel policy as code for Nomad

Sentinel is HashiCorp's policy-as-code framework and language, and is part of Nomad Enterprise. It allows you to enforce fine-grained policies using business, security or compliance logic across your Nomad clusters.

By leveraging Sentinel, you can ensure compliance, security, and operational best practices are consistently applied to all workloads.

When to Use Sentinel and when to use access control lists

While both Sentinel and access control lists (ACLs) contribute to having a complete security and governance posture, they serve different purposes.

• Use Sentinel for complex, dynamic policies that require logic-based decision making, for example, enforcing resource quotas based on namespace or job name, allowlisting approved image registries, enforcing sensible defaults, or restricting when deployments can happen.
• Use ACLs for basic access control, such as determining which users or teams can submit jobs, or exec into them to specific namespaces.

Sentinel and ACLs are complementary. Use them in tandem for a comprehensive security and compliance strategy. ACLs provide the foundational access control, while Sentinel adds an additional layer of fine-grained policy enforcement.

Best practices for authoring Sentinel policies

When crafting Sentinel policies for Nomad Enterprise consider these recommended practices.

1. Modularity: Write focused policies that address specific concerns. This approach enhances readability and maintainability.
2. Performance: Be aware of the performance impact of Sentinel policies - all of them run, sequentially, on every job submission. Keep policies efficient, avoid unnecessary complexity and duplication of logic, and having too many small policies. As an example, condense two separate policies checking CPU and memory limits into a single one checking for both.
3. Clear naming: Use descriptive names for your policies and functions. For example, enforce_resource_limits is more informative than check_resources.
4. Thorough testing: Always test your policies against various scenarios to ensure they behave as expected. The Sentinel CLI and the Sentinel playground are invaluable tools for this purpose. To get mock data for realistic testing, you can use the Nomad Enterprise API to get the JSON definition an existing job, requiring conversion to snake case (example with jq) and put into a job object.
5. Progressively Roll out policies: When introducing new policies, consider rolling them out progressively to avoid sudden workflow disruptions. Start with advisory mode to give users the time to prepare and adjust, upgrade to soft-mandatory, which requires an extra ACL and flag/parameter to bypass, to enforce the policy but allow users to override, and then switch to hard-mandatory to enforce the policy without exceptions. Note that as a Nomad operator, there is no way of tracking failed advisory-level policies. Soft-mandatory and hard-mandatory failures get reported in the audit logs.
6. Treat Sentinel policies as code: Version and track changes to your Sentinel policies in a git repository to facilitate collaboration, allowing rollbacks if needed. Implement a peer review process for policy changes to help catch potential issues and ensure policies align with organizational goals. Manage these using the Terraform provider for Nomad's nomad_sentinel_policy resource.

The new Sentinel UI in Nomad

Since Nomad Enterprise 1.8, there is now a dedicated Sentinel UI in the Administration tab, which enables you to manage and visualize your Sentinel policies directly within the Nomad UI. It has a policy overview, a policy writing UI with syntax highlighting and validation, and a template library to use.

Example Sentinel policies

Use the Sentinel UI in the Administration tab of the Nomad Enterprise UI for practical example policies. Also refer to the nomad-guides GitHub repository.