Nomad
ACL system overview
The ACL system is designed to be intuitive, high-performance, and to provide administrative insight. At the highest level, there are three major components to the ACL system: tokens, policies, and capabilities. The components are illustrated in the diagram below.
Policies
Policies consist of a set of rules defining the capabilities or actions to be
granted. For example, a readonly
policy might only grant the ability to list
and inspect running jobs, but not to submit new ones. No permissions are
granted by default, making Nomad a default-deny system.
Each policy must have a unique name, an optional description, and a rule set. The rules define the capabilities of a Nomad ACL token for accessing the objects in a Nomad cluster, objects like namespaces, node, agent, operator, quota, etc. A client ACL token can be associated with multiple policies; a request is allowed if any of the associated policies grant the capability. Management tokens cannot be associated with policies because they are granted all capabilities.
The special anonymous
policy can be defined to grant capabilities to anonymous
requests. An anonymous request is a request made to Nomad without the
X-Nomad-Token
header specified. This can be used to allow anonymous users to
list jobs and view their status, while requiring authenticated requests to
submit new jobs or modify existing jobs. By default, there is no anonymous
policy set meaning all anonymous requests are denied.
Roles
Roles group one or more ACL policies into a container which can then be used to generate ACL tokens for authorisation. This abstraction allows easier control and updating of ACL permissions, particularly in larger, more diverse clusters.
Capabilities
Capabilities are the set of actions that can be performed. This includes listing jobs, submitting jobs, querying nodes, etc. A management token is granted all capabilities, while client tokens are granted specific capabilities via ACL policies. The full set of capabilities is discussed in the rule specifications.
Binding rules
Policies are comprised of one or more rules. The rules define the capabilities of a Nomad ACL token for accessing the objects in a Nomad cluster, objects like namespaces, node, agent, operator, quota, etc.
Binding rules provide a mapping between a Nomad user's SSO authorisation claims and internal Nomad objects such as ACL Roles and ACL Policies. A binding rule is directly related to a single auth method, and therefore only evaluated by login attempts using that method. All binding rules mapped to an auth method are evaluated during each login attempt.
Note
Binding rules are evaluated in no specific order, and should there be an overlap in their selectors or scope, a "sum" of all the binding rules will be applied, thus the least granular binding rules will always override the more granular ones, as long as they apply to the same auth method and identity.A successful selector match between an SSO provider claim and a binding rule
will result in the generated ACL token having the identified ACL role or policy
assigned to it. If the BindType
parameter is management
, the ACL token
generated will be a management
token, rather than a client token. This
matcher supersedes role or policy assignments, and therefore should be used
with caution.
Rules and scope
The following table summarizes the rules that are available for constructing ACL policies:
Rule | Scope |
---|---|
agent | Utility operations in the Agent API |
host_volume | Host Volume related operations |
namespace | Job related operations by namespace |
node | Node-level catalog operations |
operator | Cluster-level operations in the Operator API |
plugin | CSI Plugin related operations |
quota | Quota specification related operations |
Constructing policies from these rules is covered in detail in the Nomad ACL Policy Concepts guide.
Tokens
ACL tokens are used to authenticate requests and determine if the caller is
authorized to perform an action. Each ACL token has a
public Accessor ID which is used to name a token and a Secret ID which is used
to make requests to Nomad. The Secret ID is provided using a request header
(X-Nomad-Token
) and is used to authenticate the caller. Tokens are either
management or client types. The management
tokens are effectively "root" in
the system and can perform any operation. The client
tokens are associated
with one or more ACL policies or roles which grant specific capabilities.
When ACL tokens are created, they can be optionally marked as Global
. This
causes them to be created in the authoritative region and replicated to all
other regions. Otherwise, tokens are created locally in the region the request
was made and not replicated. Local tokens cannot be used for cross-region
requests since they are not replicated between regions.
Workload Identity
Nomad allocations can receive workload identities in the form of a JSON Web Token (JWT). The Workload Identity concept page has more information on this topic.
Authentication methods
Authentication methods dictate how Nomad should talk to SSO providers when a user requests to authenticate using one. Currently, Nomad supports the OpenID Connect (OIDC) SSO workflow which allows users to log in to Nomad via applications such as Auth0, Okta, and [Vault][vault], and non-interactive login via externally-issued JSON Web Tokens (JWT).
Since both the oidc
and jwt
auth methods ultimately operate on JWTs as
bearer tokens, use the following to determine which method fits your use case:
JWT
- Ideal for machine-oriented, headless login where an operator may have already arranged for a valid JWT to be dropped on a VM or provided to a container.
- User or application performing the Nomad login must have a valid JWT to begin login.
- Does not require browser interaction.
OIDC
- Ideal for human-oriented, interactive login where an operator or administrator may have deployed SSO widely and doesn't want to distribute Nomad ACL tokens to every authorized user.
- User performing the Nomad login does not need a JWT.
- Requires browser interaction.
Multi-Region configuration
Nomad supports multi-datacenter and multi-region configurations. A single region is able to service multiple datacenters, and all servers in a region replicate their state between each other. In a multi-region configuration, there is a set of servers per region. Each region operates independently and is loosely coupled to allow jobs to be scheduled in any region and requests to flow transparently to the correct region.
When ACLs are enabled, Nomad depends on an "authoritative region" to act as a
single source of truth for ACL policies and global ACL tokens. The authoritative
region is configured in the server
stanza of agents, and all regions must
share a single authoritative source. Any ACL policies or global ACL tokens are
created in the authoritative region first. All other regions replicate ACL
policies and global ACL tokens to act as local mirrors. This allows policies to
be administered centrally, and for enforcement to be local to each region for
low latency.
Global ACL tokens are used to allow cross-region requests. Standard ACL tokens are created in a single target region and not replicated. This means if a request takes place between regions, global tokens must be used so that both regions will have the token registered.
Replication
Multi-region federated clusters run replication process to replicate ACL
objects from the authoritative region. The replication processes run on
each federated leader and replicate ACL policies, roles, auth methods, binding
rules, and token marked as Global
.