Vault
Evaluate Sentinel policies on HTTP requests
Enterprise Only
Sentinel requires Vault Enterprise.
Sentinel is a language framework for policy built to embed in Vault Enterprise, and enable fine-grained, logic-based policy decisions which cannot be fully handled by the ACL policies.
If you are not yet familiar with using Sentinel policies in Vault, review Sentinel Policies.
Challenge
Sentinel Endpoint Governing Policies (EGP) are flexible and offer rich capabilities and rules to enable uses like Multi Factor Authentication requirements or per path policy delegation. What if your use case requires policy rules which get information from the output of an external HTTP API?
Solution
The Sentinel http import enables the use of HTTP-accessible data from outside the runtime in Sentinel policy rules. At a basic level, the import uses an HTTP GET request and by default, any request response that is not a successful "200 OK" HTTP response causes a policy error. This behavior can be further customized to your particular use case.
In this tutorial you will write a Sentinel EGP that uses the HTTP import to query the Vault server API for information about enabled secrets engines. The Sentinel EGP will either allow or deny certain operations based on the specific criteria defined in the policy.
Personas
Sentinel policies provide a declarative way to grant or forbid access to certain paths and operations in Vault. Therefore, the steps described in this tutorial are performed by Vault admins or security operations.
Prerequisites
To perform the tasks described in this tutorial, you need to have:
- Vault Enterprise binary and license key.
- jq binary installed in your system PATH.
- Git binary installed in your system PATH.
You should have familiarity with Vault and Sentinel, and be comfortable authoring and deploying a Sentinel EGP for Vault to follow the example in this tutorial.
Policy requirements
This tutorial requires two Vault tokens - one for the Vault admin who will be creating and testing the Sentinel EGP, and one to use to for the EGP.
You need to authenticate to Vault with a token that has the required capabilities to follow the example workflow.
Warning
Do not use the initial root token for any purpose other than creating the required tokens. A root token is not subject to Sentinel policy enforcement, and will cause issues with policy development and testing.
Vault admin token policy
Review the Vault ACL policy which the admin user's token needs to create the EGP.
# To list policies
path "sys/policies/*"
{
  capabilities = ["list"]
}
# To manage ACLs
path "sys/policies/acl/*"
{
  capabilities = ["create", "read", "update", "delete", "list"]
}
# To manage EGPs
path "sys/policies/egp/*"
{
  capabilities = ["create", "read", "update", "delete", "list"]
}
# To manage secrets engines
path "sys/mounts/*"
{
  capabilities = ["create", "read", "update", "delete", "list"]
}
Going forward, the tutorial refers to this token as the admin token.
Sentinel EGP token policy
The Sentinel EGP example in this tutorial makes a call to the /sys/mounts API to learn about the enabled secrets engines.
This is an authenticated API endpoint, so the EGP requires a Vault token with the minimum required ACL policy capability
to read from /sys/mounts.
That policy is as follows.
path "/sys/mounts" {
  capabilities = ["read"]
}
Going forward, the tutorial refers to this token as the EGP token.
Lab setup
- Before you can use the Sentinel HTTP import module, you need to configure Vault and define additional enabled modules. You can do this with the additional_enabled_modules parameter. - For this tutorial, you will use a Vault server in dev mode and pass just the Sentinel specific configuration to it at runtime from the configuration file, - vault-sentinel-configuration.hcl.- Write the configuration file to enable the HTTP module. - $ cat > vault-sentinel-configuration.hcl << EOF sentinel { additional_enabled_modules = ["http"] } EOF
- Export an environment variable for the Vault Enterprise license . - $ export VAULT_LICENSE=actual-license-key
- In the same terminal session where you created the configuration file and - VAULT_LICENSEenvironment variable, start the Vault dev mode server and pass the configuration file to the- -configflag.- $ vault server -dev \ -dev-root-token-id root \ -config vault-sentinel-configuration.hcl- The Vault dev server defaults to running at - 127.0.0.1:8200. The server is automatically initialized, and unsealed.- Insecure operation - Do not run a Vault dev server in production. This tutorial uses a dev mode server to simplify the unsealing process for the hands on lab. 
- Open a new terminal and export an environment variable for the - vaultCLI to address the Vault server.- $ export VAULT_ADDR=http://127.0.0.1:8200
- Export an environment variable for the - vaultCLI to authenticate with the Vault server.- $ export VAULT_TOKEN=root- Note - For these tasks, you can use Vault's root token. Keep in mind though, that the best practice is to use root tokens just for initial setup or in emergencies. - The Vault server is ready. 
Write ACL policies
Create two ACL policies and generate the admin token and EGP token.
- Create the admin ACL policy. - $ vault policy write admin - << EOF # To list policies path "sys/policies/*" { capabilities = ["list"] } # To manage ACLs path "sys/policies/acl/*" { capabilities = ["create", "read", "update", "delete", "list"] } # To manage EGPs path "sys/policies/egp/*" { capabilities = ["create", "read", "update", "delete", "list"] } # To manage secrets engines path "sys/mounts/*" { capabilities = ["create", "read", "update", "delete", "list"] } EOF- Example output: - Success! Uploaded policy: admin
- Create the EGP ACL policy. - $ vault policy write sentinel-egp-token - << EOF path "/sys/mounts" { capabilities = ["read"] } EOF- Example output: - Success! Uploaded policy: sentinel-egp-token
- Capture the admin user token value to the - VAULT_ADMIN_TOKENenvironment variable.- $ export VAULT_ADMIN_TOKEN="$(vault token create -policy=admin -field=token)"
- Review the - VAULT_ADMIN_TOKENtoken.- $ echo $VAULT_ADMIN_TOKEN
- Capture the EGP token value to the - VAULT_EGP_TOKENenvironment variable.- $ export VAULT_EGP_TOKEN="$(vault token create -policy=sentinel-egp-token -field=token)"
- Review the - VAULT_EGP_TOKENtoken.- $ echo $VAULT_EGP_TOKEN
- Unset the - VAULT_TOKENenvironment variable.- $ unset VAULT_TOKEN
- Login with the admin token. - $ vault login -no-print $VAULT_ADMIN_TOKEN
Review Sentinel EGP
With the initial Vault configuration in place, you will now create the Sentinel EGP.
In this example, you will limit the number of enabled Transit secrets engines instances to just a single one.
You do so by leveraging Sentinel with the HTTP import to query the Vault API, and check the number of enabled Transit secrets engines.
The Sentinel policy can then further choose to allow or deny enabling of new secrets engines based on the current count of enabled transit secrets engines.
Example policy
Review the details in the Sentinel policy.
# Validate that a transit secrets engine is not already enabled before
# allowing the enabling of a transit secrets engine.
import "http"
import "json"
# Set parameters for Vault address and EGP token values
param vault_addr default "http://127.0.0.1:8200"
param vault_token default "$VAULT_EGP_TOKEN"
# Print some information about the request.
# Note that these messages are printed only when the policy is violated.
print("Request path:", request.path)
print("Request data:", request.data)
print("Request operation:", request.operation)
validate_transit_not_present = func() {
  # Start with validated set to false
  validated = false
  # Make request to the Vault API
  req = http.request(vault_addr + "/v1/sys/mounts").
                      with_header("X-Vault-Token", vault_token)
  resp = http.accept_status_codes([200, 404]).get(req)
  body = json.unmarshal(resp.body)
  print("Body Keys:", keys(body))
  # if transit type secrets engine is found, return validated with false value
  if "data" in keys(body) {
    for values(body.data) as secrets_engine {
      if secrets_engine.type is "transit" {
        validated = false
        break
      } else {
        validated = true
      }
    }
  }
  return validated
}
# Main rule
main = rule when request.path matches "sys/mounts/*" and
                                 request.data.type in ["transit"] and
                                 request.operation in ["update"] {
  validate_transit_not_present()
}
- Lines 3-4 declare the http and json imports needed for making HTTP requests and processing JSON.
- Lines 7-8 define parameter values for the Vault address and Vault token for the EGP itself. You should replace http://127.0.0.1:8200 with the address of your Vault server in the same URL format with port (ex. http://localhost:8200) and replace \$VAULT_EGP_TOKEN with the actual token value for your EGP token that you created in step 3.
- Lines 12-14 print statements which are useful for debugging. Vault prints this information whenever a request violates the policy and fails.
- Lines 16-37 includes the EGP logic in a single function, validate_transit_not_present(). It makes the HTTP request to the Vault /sys/mounts API endpoint to query the state of enabled secrets engines by examining each enabled engine. If one is the type "transit", then the validated variable value returns as false, causing the main rule evaluation to fail; permission to enable a transit secrets engine for a non-root token users gets denied.
- Lines 40-44 includes the main rule and sets conditions for its evaluation based on request parameters; the request must be for the /sys/mounts path, it must be an update operation, and the data type for the operation needs to be transit. If the request matches these conditions, then validate_transit_not_presentchecks for an enabled transit secrets engine.
Create and test Sentinel EGP
- Write the - transit-check.sentinelSentinel EGP.- $ cat > transit-check.sentinel << EOF # Validate that a transit secrets engine is not already enabled before # allowing the enabling of a transit secrets engine. import "http" import "json" # Set parameters for Vault address and EGP token values param vault_addr default "$VAULT_ADDR" param vault_token default "$VAULT_EGP_TOKEN" # Print some information about the request. # Note that these messages are printed only when the policy is violated. print("Request path:", request.path) print("Request data:", request.data) print("Request operation:", request.operation) validate_transit_not_present = func() { # Start with validated set to false validated = false # Make request to the Vault API req = http.request(vault_addr + "/v1/sys/mounts"). with_header("X-Vault-Token", vault_token) resp = http.accept_status_codes([200, 404]).get(req) body = json.unmarshal(resp.body) print("Body Keys:", keys(body)) # if transit type secrets engine is found, return validated with false value if "data" in keys(body) { for values(body.data) as secrets_engine { if secrets_engine.type is "transit" { validated = false break } else { validated = true } } } return validated } # Main rule main = rule when request.path matches "sys/mounts/*" and request.data.type in ["transit"] and request.operation in ["update"] { validate_transit_not_present() } EOF- Warning - If you deploy the Sentinel EGP with the HTTP API or Web UI, you need to replace the - $VAULT_EGP_TOKENvalue with the actual EGP token value before deploying the policy.
- Test the Sentinel policy before deployment to check syntax and to document expected behavior by downloading the Sentinel simulator. - $ curl https://releases.hashicorp.com/sentinel/0.19.5/sentinel_0.19.5_darwin_amd64.zip --output sentinel_0.19.5_darwin_amd64.zip
- Unzip the downloaded file. - $ unzip sentinel_0.19.5_darwin_amd64.zip -d /usr/local/bin
- Create the - test/transit-checkdirectory structure.- $ mkdir -p test/transit-check- Create the tests under the - test/<policy_name>folder.
- Write a passing test case for the - transit-checkpolicy,- test/transit-check/success.json.- $ cat > test/transit-check/success.json << EOF { "global": { "request": { "operation": "update", "path": "sys/mounts/transit", "data": { "type": "transit" } }, "body": { "sys/": { "accessor": "system_ab0f1127", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0, "passthrough_request_headers": [ "Accept" ] }, "description": "system endpoints used for control, policy and debugging", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "system", "uuid": "5e4101dd-ea91-5d26-9f09-85b675a72a95" }, "identity/": { "accessor": "identity_9228cd88", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "identity store", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "identity", "uuid": "c584e265-8417-4671-7fec-225393fcf307" }, "cubbyhole/": { "accessor": "cubbyhole_da4d2e11", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "per-token private secret storage", "external_entropy_access": false, "local": true, "options": null, "seal_wrap": false, "type": "cubbyhole", "uuid": "59b388df-c992-e9b9-43ec-7830d9eed35d" }, "secret/": { "accessor": "kv_086ea056", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "key/value secret storage", "external_entropy_access": false, "local": false, "options": { "version": "2" }, "seal_wrap": false, "type": "kv", "uuid": "25bbe652-5f01-664d-a4ac-64a1bb45c478" }, "request_id": "aac465cc-3101-cfa8-67d0-cc2bce8c38b1", "lease_id": "", "renewable": false, "lease_duration": 0, "data": { "cubbyhole/": { "accessor": "cubbyhole_da4d2e11", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "per-token private secret storage", "external_entropy_access": false, "local": true, "options": null, "seal_wrap": false, "type": "cubbyhole", "uuid": "59b388df-c992-e9b9-43ec-7830d9eed35d" }, "identity/": { "accessor": "identity_9228cd88", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "identity store", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "identity", "uuid": "c584e265-8417-4671-7fec-225393fcf307" }, "secret/": { "accessor": "kv_086ea056", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "key/value secret storage", "external_entropy_access": false, "local": false, "options": { "version": "2" }, "seal_wrap": false, "type": "kv", "uuid": "25bbe652-5f01-664d-a4ac-64a1bb45c478" }, "sys/": { "accessor": "system_ab0f1127", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0, "passthrough_request_headers": [ "Accept" ] }, "description": "system endpoints used for control, policy and debugging", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "system", "uuid": "5e4101dd-ea91-5d26-9f09-85b675a72a95" } }, "wrap_info": null, "warnings": null, "auth": null }, "test": { "main": true, "validate_transit_not_present": true } } } EOF- This test will pass because it does not include an enabled transit secrets engine. - The optional - testdefinition adds more context to why the test should pass. The expected behavior is that the test passes because- validate_transit_not_presentreturns- trueand- mainreturns- true.
- Write a failing test for the - transit-checkpolicy,- test/transit-check/fail.json.- $ cat > test/transit-check/fail.json << EOF { "global": { "request": { "operation": "update", "path": "sys/mounts/transit", "data": { "type": "transit" } }, "body": { "sys/": { "accessor": "system_ab0f1127", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0, "passthrough_request_headers": [ "Accept" ] }, "description": "system endpoints used for control, policy and debugging", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "system", "uuid": "5e4101dd-ea91-5d26-9f09-85b675a72a95" }, "identity/": { "accessor": "identity_9228cd88", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "identity store", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "identity", "uuid": "c584e265-8417-4671-7fec-225393fcf307" }, "cubbyhole/": { "accessor": "cubbyhole_da4d2e11", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "per-token private secret storage", "external_entropy_access": false, "local": true, "options": null, "seal_wrap": false, "type": "cubbyhole", "uuid": "59b388df-c992-e9b9-43ec-7830d9eed35d" }, "secret/": { "accessor": "kv_086ea056", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "key/value secret storage", "external_entropy_access": false, "local": false, "options": { "version": "2" }, "seal_wrap": false, "type": "kv", "uuid": "25bbe652-5f01-664d-a4ac-64a1bb45c478" }, "transit/": { "accessor": "transit_1f715ad9", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "transit", "uuid": "dcb11046-caec-5b9b-e902-9f1281c520de" }, "request_id": "d73a6a7a-a537-dd97-79a9-9e1b3654cc73", "lease_id": "", "renewable": false, "lease_duration": 0, "data": { "cubbyhole/": { "accessor": "cubbyhole_da4d2e11", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "per-token private secret storage", "external_entropy_access": false, "local": true, "options": null, "seal_wrap": false, "type": "cubbyhole", "uuid": "59b388df-c992-e9b9-43ec-7830d9eed35d" }, "identity/": { "accessor": "identity_9228cd88", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "identity store", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "identity", "uuid": "c584e265-8417-4671-7fec-225393fcf307" }, "secret/": { "accessor": "kv_086ea056", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "key/value secret storage", "external_entropy_access": false, "local": false, "options": { "version": "2" }, "seal_wrap": false, "type": "kv", "uuid": "25bbe652-5f01-664d-a4ac-64a1bb45c478" }, "sys/": { "accessor": "system_ab0f1127", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0, "passthrough_request_headers": [ "Accept" ] }, "description": "system endpoints used for control, policy and debugging", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "system", "uuid": "5e4101dd-ea91-5d26-9f09-85b675a72a95" }, "transit/": { "accessor": "transit_1f715ad9", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "transit", "uuid": "dcb11046-caec-5b9b-e902-9f1281c520de" } }, "wrap_info": null, "warnings": null, "auth": null }, "test": { "main": false, "validate_transit_not_present": false } } } EOF- This test will fail because it includes an enabled transit secrets engine. - The optional - testdefinition adds more context to why the test should fail. The expected behavior is that the test fails because- validate_transit_not_presentreturns- falseand- mainreturns- false.
- Verify success and failure tests creation. - ├── transit-check.sentinel └── test │ └── transit-check ├── fail.json └── success.json
- Execute the Sentinel test. - $ sentinel test- Example output: - PASS - transit-check.sentinel PASS - test/transit-check/fail.json PASS - test/transit-check/success.json- Note - If you want to see the tracing and log output for tests, run the command with a - -verboseflag.
Deploy Sentinel EGP
Sentinel policies have three enforcement levels:
| Level | Description | 
|---|---|
| advisory | The policy permitted to fail. Useful as a tool to educate new users. | 
| soft-mandatory | The policy must pass unless user specifies override. | 
| hard-mandatory | The policy must pass no matter what! | 
Deploy the Sentinel policy with an enforcement level of hard-mandatory.
- Login with the admin token. - $ vault login -no-print $VAULT_ADMIN_TOKEN
- Store the Base64 encoded - transit-check.sentinelpolicy in an environment variable named- POLICY.- $ POLICY=$(base64 -i transit-check.sentinel)
- Create a policy - transit-checkwith enforcement level of hard-mandatory if there is more than one transit secrets engine enabled.- $ vault write sys/policies/egp/transit-check \ policy="${POLICY}" \ paths="sys/mounts/*" \ enforcement_level="hard-mandatory"- Example output: - Success! Data written to: sys/policies/egp/transit-check
- Read the - transit-checkpolicy.- $ vault read sys/policies/egp/transit-check- Example output: - Key Value --- ----- enforcement_level hard-mandatory name transit-check paths [sys/mounts/*] policy # Validate that a transit secrets engine is not already enabled before # allowing the enabling of a transit secrets engine. import "http" import "json" # Set parameters for Vault address and EGP token values param vault_addr default "http://127.0.0.1:8200" param vault_token default "s.7j0ybDH9pklYXctdPbhAYoNI" # Print some information about the request. # Note that these messages are printed only when the policy is violated. print("Request path:", request.path) print("Request data:", request.data) print("Request operation:", request.operation) validate_transit_not_present = func() { # Start with validated set to false validated = false # Make request to the Vault API req = http.request(vault_addr + "/v1/sys/mounts"). with_header("X-Vault-Token", vault_token) resp = http.accept_status_codes([200, 404]).get(req) body = json.unmarshal(resp.body) print("Body Keys:", keys(body)) # if transit type secrets engine is found, return validated with false value if "data" in keys(body) { for values(body.data) as secrets_engine { if secrets_engine.type is "transit" { validated = false break } else { validated = true } } } return validated } # Main rule main = rule when request.path matches "sys/mounts/*" and request.data.type in ["transit"] and request.operation in ["update"] { validate_transit_not_present() }
Verification
Once you've deployed the policy, Vault will deny permission for all operations which fail the policy rule assertions when the server learns there is already a transit secrets engine enabled.
- Login with the admin token. - $ vault login -no-print $VAULT_ADMIN_TOKEN
- Enable one instance of the transit secrets engine. - $ vault secrets enable transit Success! Enabled the transit secrets engine at: transit/
- Try to enable another instance of the transit secrets engine at the path another-transit-secrets-engine. - $ vault secrets enable -path=another-transit-secrets-engine transit- Example output: - Error enabling: Error making API request. URL: POST http://127.0.0.1:8200/v1/sys/mounts/another-transit-secrets-engine Code: 403. Errors: * 2 errors occurred: * egp standard policy "root/transit-check" evaluation resulted in denial. The specific error was: <nil> A trace of the execution for policy "root/transit-check" is available: Result: false Description: Main rule print() output: Request path: sys/mounts/another-transit-secrets-engine Request data: {"config": {"default_lease_ttl": "0s", "force_no_cache": false, "max_lease_ttl": "0s", "options": null}, "description": "", "external_entropy_access": false, "local": false, "options": null, "seal_wrap": false, "type": "transit"} Request operation: update Body Keys: ["sys/" "request_id" "renewable" "lease_duration" "secret/" "warnings" "identity/" "transit/" "auth" "lease_id" "cubbyhole/" "wrap_info" "data"] Rule "main" (byte offset 1256) = false * permission denied- A policy violation results in the inclusion of debug - print()statement output.- Warning - As with ACL policies, - roottokens are NOT subject to Sentinel policy checks. Be sure to use a non-root token for the verification test.
Going even further with Vault Enterprise Namespaces
If you want to try a more advanced example involving the Vault Enterprise Namespaces feature, you can follow the guidance in Restrict Members of a Group with Sentinel. You will learn how to restrict the subgroups and entities of a group by requiring all subgroups and entities of a group to belong to the same namespace.
Cleanup
To clean up the example files and stop your Vault dev server follow these steps.
- Remove the files. - $ rm -f transit-check-payload.json \ transit-check.sentinel\ vault-sentinel-configuration.hcl
- Remove the Sentinel tests. - $ rm -rf test
- In the terminal session where you started the Vault dev server, press CTRL+C to stop Vault. 
Troubleshooting
If you have an error such as this example while testing the EGP.
Error enabling: Error making API request.
URL: POST http://127.0.0.1:8200/v1/sys/mounts/example
Code: 403. Errors:
* 2 errors occurred:
  * egp standard policy "root/transit-example" evaluation resulted in denial.
The specific error was:
root/transit-example:4:1: Import "http" is not available
A trace of the execution for policy "root/transit-example" is available:
Result: false
Error: false
Description: Validate that a Transit secrets engine is not already enabled
when an operation to enable a Transit secrets engine is attempted
  * permission denied
Note the Import "http" is not available part of the error. This indicates that Vault is not configured to allow the HTTP import. Check the instructions in the Lab setup section and make sure you have configured and restarted your Vault.