How to rotate API secrets without downtime

Rotating an API secret shouldn't need both sides to deploy at the same minute. If it does, it won't happen, and the secret from three years ago stays in production.

The fix is an overlap: for a while, both the old and the new secret work. Callers switch on their own schedule, and the old secret stops working on yours.

The overlap, step by step

1. Issue a new secret, and give the old one an expiry. Don't revoke anything yet. Pick an expiry that fits the caller's release cycle: a day for a team that deploys hourly, two weeks for one that ships monthly.
2. Deliver the new secret through a secret manager. Not email, not chat. Put it where the caller's service already reads secrets from.
3. The caller deploys or reloads. Requests start arriving with the new secret. The old one still works, so nothing breaks mid-rollout.
4. Watch which secret each request uses. When the old one stops appearing, the caller has finished.
5. Expire the old secret. On the date, or earlier once it's unused.

Accepting two secrets on the server

Store a hash of each secret, never the plaintext. During the overlap, a request is valid if it matches either hash and the old one hasn't expired:

import hashlib
import hmac
from datetime import datetime, timezone

def hash_secret(secret: str, pepper: bytes) -> str:
    return hmac.new(pepper, secret.encode(), hashlib.sha256).hexdigest()

def matching_secret(credential, presented: str, pepper: bytes) -> str | None:
    digest = hash_secret(presented, pepper)
    if hmac.compare_digest(digest, credential.secret_hash):
        return "current"
    if (
        credential.previous_secret_hash
        and credential.previous_expires_at > datetime.now(timezone.utc)
        and hmac.compare_digest(digest, credential.previous_secret_hash)
    ):
        return "previous"
    return None

Log the return value with the client ID. That log line is step 4: a count of "previous" by client tells you who hasn't switched yet.

Pitfalls

• Tokens outlive secrets. If the secret is exchanged for an access token, tokens minted with the old secret keep working until they expire. Add the token lifetime to your overlap, or revoke tokens when the old secret expires.
• Environment variables need a restart. A service that reads its secret at boot won't pick up the new one until it restarts. Know which of your callers work this way before you set the expiry.
• Keep the client ID. Rotate the secret only. A new client ID changes the caller's configuration, their logs and your usage history.
• A leak is different. If a secret was exposed, expire the old one now. A short outage is better than an attacker's overlap window.
• Rotate on a schedule. Every 90 days, say. A process you run four times a year works when you need it in a hurry.

Where EndPointBlank fits

EndPointBlank gives each service its own client ID and secret, scoped to one environment. Rotating a secret issues a new one while the previous secret keeps working until an expiry you choose, and the client ID stays the same. Secrets are stored as digests and shown once, at creation.

See how machine-to-machine auth works or start free.