infra/tickets

Fork 1

AWS Permissions Request: KMS Key for HashiCorp Vault Auto-Unseal #13224

New issue

Open

opened 2026-03-23 19:34:20 +00:00 by mvadkert · 1 comment

mvadkert commented

2026-03-23 19:34:20 +00:00

Description of request

Context

The Testing Farm team (AWS account 125523088429) is deploying HashiCorp Vault for centralized secrets management, replacing our current ansible-vault setup. Vault will run on a dedicated EKS cluster in us-east-1.

Vault requires an AWS KMS key for its auto-unseal mechanism. Without KMS auto-unseal, Vault requires manual operator intervention (entering unseal keys) every time a Vault pod restarts, which is operationally unsustainable in a Kubernetes environment.

What We Need

Option A: Grant us permission to create a KMS key (preferred)

Grant the following IAM permissions to our existing roles, scoped to a single key alias:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VaultKMSKeyManagement",
      "Effect": "Allow",
      "Action": [
        "kms:CreateKey",
        "kms:CreateAlias",
        "kms:UpdateAlias",
        "kms:DeleteAlias",
        "kms:DescribeKey",
        "kms:GetKeyPolicy",
        "kms:GetKeyRotationStatus",
        "kms:ListAliases",
        "kms:ListResourceTags",
        "kms:TagResource",
        "kms:UntagResource",
        "kms:EnableKeyRotation",
        "kms:ScheduleKeyDeletion"
      ],
      "Resource": "*",
      "Condition": {
        "StringLike": {
          "kms:ResourceAliases": "alias/vault-unseal*"
        }
      }
    }
  ]
}

Additionally, the Vault pods running on EKS need runtime access to use the key for encrypt/decrypt operations. This would be granted via an IRSA (IAM Role for Service Accounts) role. The following permissions are needed for that role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VaultKMSUnseal",
      "Effect": "Allow",
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "<KMS_KEY_ARN>"
    }
  ]
}

Note: For the IRSA role, we also need enable_irsa = true on the EKS cluster (OIDC provider creation), which requires:

{
  "Sid": "EKSIRSAOIDCProvider",
  "Effect": "Allow",
  "Action": [
    "iam:CreateOpenIDConnectProvider",
    "iam:DeleteOpenIDConnectProvider",
    "iam:GetOpenIDConnectProvider",
    "iam:TagOpenIDConnectProvider",
    "iam:ListOpenIDConnectProviders"
  ],
  "Resource": "arn:aws:iam::125523088429:oidc-provider/oidc.eks.us-east-1.amazonaws.com/*"
}

Option B: Create the KMS key for us

If granting key creation permissions is not possible, please create a symmetric KMS key with the following configuration:

Key alias: alias/vault-unseal
Region: us-east-1
Key spec: SYMMETRIC_DEFAULT
Key usage: ENCRYPT_DECRYPT
Key rotation: Enabled (annual)
Description: HashiCorp Vault auto-unseal key for Testing Farm
Tags:
- FedoraGroup: ci
- ServiceOwner: TFT
- ServiceName: TestingFarm
- ServiceComponent: Vault

Then grant the following principal access to use the key:

The EKS node role arn:aws:iam::125523088429:role/aws-fedora-ci (until we set up IRSA)
Or an IRSA role we will create for the Vault service account

Required key policy for the Vault principal:

{
  "Sid": "AllowVaultUnseal",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::125523088429:role/aws-fedora-ci"
  },
  "Action": [
    "kms:Encrypt",
    "kms:Decrypt",
    "kms:DescribeKey"
  ],
  "Resource": "*"
}

Security Notes

The KMS key is used only for encrypting/decrypting Vault's master key — it does not encrypt individual secrets (Vault handles that internally)
Automatic key rotation is enabled for compliance
Access is restricted to the Vault service account only (via IRSA when available, or node role as fallback)
The key alias is scoped to vault-unseal* to prevent use for other purposes

References

### Description of request ## Context The Testing Farm team (AWS account `125523088429`) is deploying HashiCorp Vault for centralized secrets management, replacing our current ansible-vault setup. Vault will run on a dedicated EKS cluster in `us-east-1`. Vault requires an **AWS KMS key** for its [auto-unseal mechanism](https://developer.hashicorp.com/vault/docs/configuration/seal/awskms). Without KMS auto-unseal, Vault requires manual operator intervention (entering unseal keys) every time a Vault pod restarts, which is operationally unsustainable in a Kubernetes environment. ## What We Need ### Option A: Grant us permission to create a KMS key (preferred) Grant the following IAM permissions to our existing roles, scoped to a single key alias: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "VaultKMSKeyManagement", "Effect": "Allow", "Action": [ "kms:CreateKey", "kms:CreateAlias", "kms:UpdateAlias", "kms:DeleteAlias", "kms:DescribeKey", "kms:GetKeyPolicy", "kms:GetKeyRotationStatus", "kms:ListAliases", "kms:ListResourceTags", "kms:TagResource", "kms:UntagResource", "kms:EnableKeyRotation", "kms:ScheduleKeyDeletion" ], "Resource": "*", "Condition": { "StringLike": { "kms:ResourceAliases": "alias/vault-unseal*" } } } ] } ``` Additionally, the Vault pods running on EKS need **runtime access** to use the key for encrypt/decrypt operations. This would be granted via an IRSA (IAM Role for Service Accounts) role. The following permissions are needed for that role: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "VaultKMSUnseal", "Effect": "Allow", "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:DescribeKey" ], "Resource": "<KMS_KEY_ARN>" } ] } ``` **Note:** For the IRSA role, we also need `enable_irsa = true` on the EKS cluster (OIDC provider creation), which requires: ```json { "Sid": "EKSIRSAOIDCProvider", "Effect": "Allow", "Action": [ "iam:CreateOpenIDConnectProvider", "iam:DeleteOpenIDConnectProvider", "iam:GetOpenIDConnectProvider", "iam:TagOpenIDConnectProvider", "iam:ListOpenIDConnectProviders" ], "Resource": "arn:aws:iam::125523088429:oidc-provider/oidc.eks.us-east-1.amazonaws.com/*" } ``` ### Option B: Create the KMS key for us If granting key creation permissions is not possible, please create a **symmetric KMS key** with the following configuration: - **Key alias:** `alias/vault-unseal` - **Region:** `us-east-1` - **Key spec:** `SYMMETRIC_DEFAULT` - **Key usage:** `ENCRYPT_DECRYPT` - **Key rotation:** Enabled (annual) - **Description:** `HashiCorp Vault auto-unseal key for Testing Farm` - **Tags:** - `FedoraGroup`: `ci` - `ServiceOwner`: `TFT` - `ServiceName`: `TestingFarm` - `ServiceComponent`: `Vault` Then grant the following principal access to use the key: - The EKS node role `arn:aws:iam::125523088429:role/aws-fedora-ci` (until we set up IRSA) - Or an IRSA role we will create for the Vault service account Required key policy for the Vault principal: ```json { "Sid": "AllowVaultUnseal", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::125523088429:role/aws-fedora-ci" }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:DescribeKey" ], "Resource": "*" } ``` ## Security Notes - The KMS key is used **only** for encrypting/decrypting Vault's master key — it does not encrypt individual secrets (Vault handles that internally) - Automatic key rotation is enabled for compliance - Access is restricted to the Vault service account only (via IRSA when available, or node role as fallback) - The key alias is scoped to `vault-unseal*` to prevent use for other purposes ## References - [Vault AWS KMS Auto-Unseal documentation](https://developer.hashicorp.com/vault/docs/configuration/seal/awskms) - [EKS IRSA documentation](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)

zlopez added the

aws

label

2026-03-24 10:13:21 +00:00

zlopez added the

day-to-day

points

labels

2026-06-25 15:38:52 +00:00

kevin self-assigned this

2026-06-25 15:40:15 +00:00

kevin added this to the Sprint ( 2026-06-15 to 2026-06-28) project

2026-06-25 15:40:22 +00:00

kevin modified the project from Sprint ( 2026-06-15 to 2026-06-28) to Sprint ( 2026-06-29 to 2026-07-12)

2026-06-29 14:28:08 +00:00

kevin commented

2026-07-09 23:45:37 +00:00

Owner

I think we can do the first one... but I am not sure, do you already have a OIDC provider setup for the cluster, or you wanted me to do that also?

The more detailed you can be the easier it will be for me. :)

I think we can do the first one... but I am not sure, do you already have a OIDC provider setup for the cluster, or you wanted me to do that also? The more detailed you can be the easier it will be for me. :)

kevin modified the project from Sprint ( 2026-06-29 to 2026-07-12) to Sprint ( 2026-07-13 to 2026-07-26)

2026-07-13 14:57:44 +00:00

kevin removed this from the Sprint ( 2026-07-13 to 2026-07-26) project

2026-07-27 14:19:57 +00:00