ACL system overview

The ACL system is designed to be intuitive, high-performance, and to provide administrative insight. At the highest level, there are three major components to the ACL system: tokens, policies, and capabilities. The components are illustrated in the diagram below.

Policies

Policies consist of a set of rules defining the capabilities or actions to be granted. For example, a readonly policy might only grant the ability to list and inspect running jobs, but not to submit new ones. No permissions are granted by default, making Nomad a default-deny system.

Each policy must have a unique name, an optional description, and a rule set. The rules define the capabilities of a Nomad ACL token for accessing the objects in a Nomad cluster, objects like namespaces, node, agent, operator, quota, etc. A client ACL token can be associated with multiple policies; a request is allowed if any of the associated policies grant the capability. Management tokens cannot be associated with policies because they are granted all capabilities.

The special anonymous policy can be defined to grant capabilities to anonymous requests. An anonymous request is a request made to Nomad without the X-Nomad-Token header specified. This can be used to allow anonymous users to list jobs and view their status, while requiring authenticated requests to submit new jobs or modify existing jobs. By default, there is no anonymous policy set meaning all anonymous requests are denied.

Roles

Roles group one or more ACL policies into a container which can then be used to generate ACL tokens for authorisation. This abstraction allows easier control and updating of ACL permissions, particularly in larger, more diverse clusters.

Capabilities

Capabilities are the set of actions that can be performed. This includes listing jobs, submitting jobs, querying nodes, etc. A management token is granted all capabilities, while client tokens are granted specific capabilities via ACL policies. The full set of capabilities is discussed in the rule specifications.

Binding rules

Policies are comprised of one or more rules. The rules define the capabilities of a Nomad ACL token for accessing the objects in a Nomad cluster, objects like namespaces, node, agent, operator, quota, etc.

Binding rules provide a mapping between a Nomad user's SSO authorisation claims and internal Nomad objects such as ACL Roles and ACL Policies. A binding rule is directly related to a single auth method, and therefore only evaluated by login attempts using that method. All binding rules mapped to an auth method are evaluated during each login attempt.

A successful selector match between an SSO provider claim and a binding rule will result in the generated ACL token having the identified ACL role or policy assigned to it. If the BindType parameter is management, the ACL token generated will be a management token, rather than a client token. This matcher supersedes role or policy assignments, and therefore should be used with caution.

Rules and scope

The following table summarizes the rules that are available for constructing ACL policies:

Constructing policies from these rules is covered in detail in the Nomad ACL Policy Concepts guide.

Tokens

ACL tokens are used to authenticate requests and determine if the caller is authorized to perform an action. Each ACL token has a public Accessor ID which is used to name a token and a Secret ID which is used to make requests to Nomad. The Secret ID is provided using a request header (X-Nomad-Token) and is used to authenticate the caller. Tokens are either management or client types. The management tokens are effectively "root" in the system and can perform any operation. The client tokens are associated with one or more ACL policies or roles which grant specific capabilities.

When ACL tokens are created, they can be optionally marked as Global. This causes them to be created in the authoritative region and replicated to all other regions. Otherwise, tokens are created locally in the region the request was made and not replicated. Local tokens cannot be used for cross-region requests since they are not replicated between regions.

Workload Identity

Nomad allocations can receive workload identities in the form of a JSON Web Token (JWT). The Workload Identity concept page has more information on this topic.

Authentication methods

Authentication methods dictate how Nomad should talk to SSO providers when a user requests to authenticate using one. Currently, Nomad supports the OpenID Connect (OIDC) SSO workflow which allows users to log in to Nomad via applications such as Auth0, Okta, and [Vault][vault], and non-interactive login via externally-issued JSON Web Tokens (JWT).

Since both the oidc and jwt auth methods ultimately operate on JWTs as bearer tokens, use the following to determine which method fits your use case:

• JWT

  • Ideal for machine-oriented, headless login where an operator may have already arranged for a valid JWT to be dropped on a VM or provided to a container.
  • User or application performing the Nomad login must have a valid JWT to begin login.
  • Does not require browser interaction.

• OIDC

  • Ideal for human-oriented, interactive login where an operator or administrator may have deployed SSO widely and doesn't want to distribute Nomad ACL tokens to every authorized user.
  • User performing the Nomad login does not need a JWT.
  • Requires browser interaction.

Multi-Region configuration

Nomad supports multi-datacenter and multi-region configurations. A single region is able to service multiple datacenters, and all servers in a region replicate their state between each other. In a multi-region configuration, there is a set of servers per region. Each region operates independently and is loosely coupled to allow jobs to be scheduled in any region and requests to flow transparently to the correct region.

When ACLs are enabled, Nomad depends on an "authoritative region" to act as a single source of truth for ACL policies and global ACL tokens. The authoritative region is configured in the server stanza of agents, and all regions must share a single authoritative source. Any ACL policies or global ACL tokens are created in the authoritative region first. All other regions replicate ACL policies and global ACL tokens to act as local mirrors. This allows policies to be administered centrally, and for enforcement to be local to each region for low latency.

Global ACL tokens are used to allow cross-region requests. Standard ACL tokens are created in a single target region and not replicated. This means if a request takes place between regions, global tokens must be used so that both regions will have the token registered.

Replication

Multi-region federated clusters run replication process to replicate ACL objects from the authoritative region. The replication processes run on each federated leader and replicate ACL policies, roles, auth methods, binding rules, and token marked as Global.