Skip to content
Cloudflare Docs

Web Bot Auth

Web Bot Auth is an authentication method that leverages cryptographic signatures in HTTP messages to verify that a request comes from an automated bot.

1. Generate a valid signing key

You need to generate a signing key which will be used to authenticate your bot's requests.

  1. Generate a unique Ed25519 ↗ private key to sign your requests. This example uses the OpenSSL ↗ genpkey command:

    Terminal window
    openssl genpkey -algorithm ed25519 -out private-key.pem

  2. Extract your public key.

    Terminal window
    openssl pkey -in private-key.pem -pubout -out public-key.pem

  3. Convert the public key to JSON Web Key (JWK) using a tool of your choice. This example uses jwker ↗ command line application.

    Terminal window
    go install github.com/jphastings/jwker/cmd/jwker@latest
    jwker public-key.pem public-key.jwk

By following these steps, you have generated a private key and a public key, then converted the public key to a JWK.

2. Host a key directory

You need to host a key directory which creates a way for Cloudflare to authenticate your bot's requests.

  1. Host a key directory at a well known message signatures directory. The key directory should serve a JSON Web Key Set (JWKS) including the public key derived from your signing key.

    An example directory would be:

    /.well-known/http-message-signatures-directory/

  2. Serve the web page over HTTPS (not HTTP).

  3. Calculate the base64 URL-encoded JWK thumbprint ↗ associated with your Ed25519 public key.

  4. Sign your HTTP response using the HTTP message signature specification by attaching one signature per key in your key directory. This ensures no one else can mirror your directory and attempt to register on your behalf. Your response must include the following headers:

    • Signature: Construct a Signature header ↗ over your chosen components.
    • Signature-Input: Construct a Signature-Input header ↗ over your chosen components. The header must meet the following requirements.
      Required component parameterRequirement
      tagThis should be equal to http-message-signatures-directory.
      algThis should be equal to ed25519.
      keyidJWK thumbprint of the corresponding key in your directory.
      createdThis should be equal to a Unix timestamp associated with when the message was sent by your application.
      expiresThis should be equal to a Unix timestamp associated with when Cloudflare should no longer attempt to verify the message.

    The following example shows the annotated request and response with required headers against https://example.com.

    GET /.well-known/http-message-signatures-directory HTTP/1.1
    Host: example.com
    Accept: application/http-message-signatures-directory+json
    

    HTTP/1.1 200 OK
    Content-Type: application/http-message-signatures-directory+json
    Signature: sig1=:TD5arhV1ved6xtx63cUIFCMONT248cpDeVUAljLgkdozbjMNpJGr/WAx4PzHj+WeG0xMHQF1BOdFLDsfjdjvBA==:
    Signature-Input: sig1=("@authority");alg="ed25519";keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U";nonce="ZO3/XMEZjrvSnLtAP9M7jK0WGQf3J+pbmQRUpKDhF9/jsNCWqUh2sq+TH4WTX3/GpNoSZUa8eNWMKqxWp2/c2g==";tag="http-message-signatures-directory";created=1750105829;expires=1750105839
    Cache-Control: max-age=86400
    {
      "keys": [{
        "kty": "OKP",
        "crv": "Ed25519",
        "x": "JrQLj5P_89iXES9-vFgrIy29clF9CC_oPPsw3c5D0bs", // Base64 URL-encoded public key, with no padding
      }]
    }

You can use the Cloudflare-developed http-signature-directory CLI tool ↗ to assist you in validating your directory.

3. Register your bot and key directory

You need to register your bot and its key directory to add your bot to the list of verified bots.

  1. Log in to the Cloudflare dashboard ↗, and select your account and domain.
  2. Go to Manage Account > Configurations.
  3. Go to the Verified Bots tab.
  4. For Verification Method: select Request Signature.
  5. For Validation Instructions: enter the URL of your key directory. You can additionally supply User Agents values (and their match patterns) that will be sent by your bot.
  6. Select Submit.

Cloudflare accepts all valid Ed25519 keys found in your key directory. In the event a key already exists in Cloudflare's registered database, Cloudflare will work with you to supply a new key, or rotate your existing key.

4. (After verification) Sign your requests

After your bot has been successfully verified, you need to sign your bot's requests.

4.1. Choose a set of components to sign

Choose a set of components to sign.

A component is either an HTTP header, or any derived components ↗ in the HTTP Message Signatures specification. Cloudflare recommends the following:

  • Choose at least the @authority derived component, which represents the domain you are sending requests to. For example, a request to https://example.com will be interpreted to have an @authority of example.com.
  • Use components that only contain ASCII values. HTTP Message Signature specification disallows non-ASCII characters, which will result in failure to validate your bot's requests.
  • Add a Content-Digest header if you wish to sign your message content ↗, then specify Content-Digest as a component to sign.

4.2. Calculate the JWK thumbprint

Calculate the base64 URL-encoded JWK thumbprint ↗ associated with your Ed25519 public key registered with Cloudflare.

4.3. Construct the required headers

Construct the three required headers for Web Bot Auth.

Signature-Input header

Construct a Signature-Input header ↗ over your chosen components. The header must meet the following requirements.

Required component parameterRequirement
tagThis should be equal to web-bot-auth.
algThis should be equal to ed25519.
keyidThis should be equal to the thumbprint computed in step 2.
createdThis should be equal to a Unix timestamp associated with when the message was sent by your application.
expiresThis should be equal to a Unix timestamp associated with when Cloudflare should no longer attempt to verify the message. A short expires reduces the likelihood of replay attacks, and Cloudflare recommends choosing suitable short-lived intervals.

Signature header

Construct a Signature header ↗ over your chosen components.

Signature-Agent header

Construct a Signature-Agent header ↗ that points to your key directory. Note that Cloudflare will fail to verify a message if:

  • The message includes a Signature-Agent header that is not an https://.
  • The message includes a valid URI but do not enclose it in double quotes.
  • The message has a valid Signature-Agent header, but does not include it in the component list in Signature-Input.

4.4. Add the headers to your bot's requests

Attach these three headers to your bot's requests.

An example request may look like this:

Signature-Agent: "https://signature-agent.test"
Signature-Input: sig2=("@authority" "signature-agent")
 ;created=1735689600
 ;keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U"
 ;alg="ed25519"
 ;expires=1735693200
 ;nonce="e8N7S2MFd/qrd6T2R3tdfAuuANngKI7LFtKYI/vowzk4lAZYadIX6wW25MwG7DCT9RUKAJ0qVkU0mEeLElW1qg=="
 ;tag="web-bot-auth"
Signature: sig2=:jdq0SqOwHdyHr9+r5jw3iYZH6aNGKijYp/EstF4RQTQdi5N5YYKrD+mCT1HA1nZDsi6nJKuHxUi/5Syp3rLWBA==:

Additional resources

You may wish to refer to the following resources.