Let's Encrypt: ACME Protocol Explained

The most surprising thing about Let’s Encrypt is that it’s not actually a certificate authority (CA) in the traditional sense; it’s a protocol that automates the process of obtaining and renewing certificates from a real CA.

Let’s see this in action. Imagine you’re setting up a new web server, example.com, and you need an SSL/TLS certificate. Instead of manually going to a CA website, filling out forms, and uploading files, you can use an ACME client.

Here’s a simplified certbot command that does this:

sudo certbot certonly --standalone -d example.com -d www.example.com

When you run this, certbot (the ACME client) doesn’t issue the certificate itself. It communicates with Let’s Encrypt’s servers (which act as a front-end to their actual CA, ISRG Root X1). The ACME protocol orchestrates a series of challenges and responses.

The core of the ACME protocol is the "challenge-response" mechanism. Let’s Encrypt needs to verify that you control the domain name for which you’re requesting a certificate. There are two primary ways it does this:

1. HTTP-01 Challenge: This is the most common method for web servers.

   • Client Action: Your ACME client (like certbot) tells the Let’s Encrypt server, "I want a certificate for example.com."
   • Let’s Encrypt Server: Responds with a unique token and a piece of content (a URL and a key authorization string). It tells your client, "Go ahead and serve this specific file at http://example.com/.well-known/acme-challenge/TOKEN."
   • Client Action: Your ACME client configures your web server (or starts a temporary one) to serve that exact file at the specified URL.
   • Let’s Encrypt Server: Makes an HTTP request to http://example.com/.well-known/acme-challenge/TOKEN. It expects to receive the specific content it provided earlier. If it gets the right content, it knows you control example.com because only someone with access to that server could have served that file.
   • Example certbot config for HTTP-01: If certbot is run with --standalone, it spins up its own temporary web server on port 80 to handle this. If you’re already running a web server (like Nginx or Apache) and have configured it to proxy requests to /.well-known/acme-challenge/ to certbot, you’d use --webroot.

2. DNS-01 Challenge: This method is used when you can’t easily modify your web server’s configuration (e.g., you don’t have direct access, or you’re behind a strict firewall) but can modify your domain’s DNS records.

   • Client Action: Your ACME client tells the Let’s Encrypt server, "I want a certificate for example.com."
   • Let’s Encrypt Server: Responds with a unique token and a key authorization string. It tells your client, "Go ahead and create a TXT record at _acme-challenge.example.com with the value KEY_AUTHORIZATION_STRING."
   • Client Action: Your ACME client (or a plugin for your DNS provider) creates this TXT record in your DNS zone.
   • Let’s Encrypt Server: Queries DNS for the TXT record at _acme-challenge.example.com. If it finds the correct value, it verifies your control.
   • Example certbot config for DNS-01: You’d typically use a plugin for your DNS provider, e.g., certbot certonly --dns-cloudflare --domain example.com.

Once the challenge is passed, the ACME client sends a "certificate request" to Let’s Encrypt. This request contains the domain name(s), the public key from a new key pair generated by the client, and proof of the successful challenge. Let’s Encrypt then uses its CA infrastructure to sign this public key with its intermediate CA certificate, creating your SSL/TLS certificate.

The ACME protocol also handles renewals. Certificates are valid for 90 days. Your ACME client is typically configured to run as a cron job or systemd timer, checking periodically if certificates are due for renewal (usually when they are less than 30 days from expiring). When it’s time, it re-initiates the challenge-response process for the existing certificate’s account. This is why it’s crucial for your ACME client to be able to pass the same challenge again without manual intervention.

The ACME protocol relies on JWK (JSON Web Key) or JWS (JSON Web Signature) for secure communication between the client and the server. Each ACME client registers an account with Let’s Encrypt, which is identified by a unique account URL. This account is used to associate the certificate requests and manage renewals. The client signs all its requests with the private key corresponding to the public key registered with its account, proving its identity to Let’s Encrypt.

What most people don’t realize is that the ACME protocol is designed to be extensible, allowing for various challenges and integrations. This is why you see clients supporting not just HTTP-01 and DNS-01, but also TLS-ALPN-01 (which uses the TLS handshake itself for verification) and integrations with numerous DNS providers and web servers. The protocol itself is a set of API endpoints and message formats defined by RFC 8555, enabling automation beyond just certificate issuance, including account management, revocation, and key rollover.

The next logical step after mastering certificate acquisition and renewal is understanding how to secure your web server to use these certificates effectively, which often involves diving into Nginx or Apache configuration for SSL/TLS.