Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Hardware Keys (PKCS#11)

By default the Local CA’s issuing key is a PEM file on disk, protected by nothing but its 0600 permissions. key_source = "pkcs11" moves that key into a hardware token — a YubiKey, a network HSM, or SoftHSM2 for development — where it is created once and can never be read back out. acme-proxy sends the token the bytes to be signed and receives a signature; the private key never enters this process’s memory.

PKCS#11 rather than a vendor-specific PIV library, so a YubiKey today and an enterprise HSM tomorrow are the same configuration with a different module_path.

What this protects, and what it does not

Everything else about the Local CA is unchanged: the same CSR sanitisation, the same leaf_validity_days clamping, the same CRL and revocations. Only where the signature comes from moves.

It protects the CA issuing key — the one that, if stolen, lets an attacker mint certificates your fleet trusts. It does not protect the ACME account keys, the TLS server key (server.tls.key_path), or the database; those stay on disk.

Requirements

  • A build with the hsm feature. It is off by default, so the stock binary does not have it and key_source = "pkcs11" on one is a startup error naming the feature:

    cargo build --release --features hsm
    
  • A PKCS#11 module (.so), loaded at runtime — nothing is linked at build time.

  • An existing CA certificate, for the reason below.

Two rules that differ from the software path

Both are startup errors, so you will meet them immediately rather than in production.

The CA is never generated

With key_source = "file", a missing cert_path/key_path means “generate a CA and write it here”. With key_source = "pkcs11" there is no such thing: the private key is created inside the token by its own tooling, and this server cannot produce one that a token would then hold. So cert_path must already exist, and key_path is neither read nor written.

Both walkthroughs below cover creating that certificate.

The key and the certificate are cross-checked

At startup, the token key’s SubjectPublicKeyInfo is compared against the one in cert_path. A mismatch — almost always a wrong key_label — stops the server.

This is stricter than the file path, where (as Local CA warns) nothing checks that key_path corresponds to cert_path, and a mismatched pair simply produces certificates that verify nowhere. Here a typo is caught before the first certificate is issued rather than discovered by a client days later.

Reference

Reaching this table at all takes signer.local_ca.key_source = "pkcs11", which is documented with the rest of [signer.local_ca] in Local CA. Everything below is [signer.local_ca.pkcs11], read only when that key is set.

pkcs11.module_path (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__MODULE_PATH

The PKCS#11 module to load. Required. See each walkthrough for the usual paths.

pkcs11.token_label (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__TOKEN_LABEL

Which token to use, by label. Preferred over slot_id: slot numbers are assigned dynamically and change across reboots and re-plugs on most drivers (SoftHSM2 will hand you something like 276468771).

pkcs11.slot_id (Integer) — Default: unset | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__SLOT_ID

Which slot to use, for tokens with no usable label. Consulted only when token_label is empty.

pkcs11.key_label (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__KEY_LABEL

The private key’s CKA_LABEL. Required. On a YubiKey the labels are fixed by the driver, so this is something you look up rather than choose — see below.

pkcs11.key_id (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__KEY_ID

The key’s CKA_ID as hex (01, or 01:ff), to disambiguate a token holding several keys under one label. Optional; two keys sharing a label and no key_id to separate them is a startup error rather than a coin flip.

pkcs11.pin_file (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__PIN_FILE

A file holding the user PIN. Trailing whitespace is trimmed, so a PIN written with echo works. The file is checked for permissions and warns if it is world-readable, exactly as ca.key does.

pkcs11.pin (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__PIN

SENSITIVE. The PIN directly. Prefer pin_file, or set this through the environment variable; a PIN in config.toml is a long-lived secret in a file that tends to get copied around. pin_file wins when both are set, and having neither is a startup error.

A PIN is not a password. Tokens block after a small number of wrong attempts — three on a YubiKey PIV applet, after which you need the PUK. This is why acme-proxy retries a failed signature at most once, and why the trailing newline in your PIN file is worth getting right.


Walkthrough A — SoftHSM2

SoftHSM2 is a software token: no hardware needed, and the same setup the project uses in CI. Use it to try the feature before committing to hardware.

# Debian/Ubuntu
sudo apt install softhsm2
# Arch
sudo pacman -S softhsm

Step 1 — Create a token

softhsm2-util --init-token --free --label acme-ca --so-pin 3737 --pin 1234

--free takes the first uninitialised slot. Note that the token is reassigned to a new slot number afterwards — which is exactly why token_label is the selector to use, not slot_id.

Step 2 — Create the CA key and certificate

The key must exist inside the token, and cert_path must hold a certificate for it. For SoftHSM2 the simplest route is to generate both locally, import the key, and destroy the local copy:

# The CA key and its self-signed certificate
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ca.key

openssl req -x509 -new -key ca.key -sha256 -days 3650 -out ca.pem \
  -subj "/CN=Example Corp Issuing CA/O=Example Corp" \
  -addext "basicConstraints=critical,CA:true,pathlen:0" \
  -addext "keyUsage=critical,keyCertSign,cRLSign"

# Move the key into the token, then remove it from disk
softhsm2-util --import ca.key --token acme-ca --label ca-key --id 01 --pin 1234
shred -u ca.key

For a real HSM, generate the key in the token instead so it never exists outside it — pkcs11-tool --module <module> --token-label acme-ca --login --keypairgen --key-type EC:prime256v1 --label ca-key --id 01 (from the opensc package), then certify that public key with your offline root. The import above is a development convenience, and the reason it is acceptable here is that a SoftHSM2 token is a directory of files anyway.

pathlen:0 matches what the Local CA generates for itself: it may issue leaves but no further CAs.

Step 3 — Configure

[signer]
backend = "local_ca"

[signer.local_ca]
cert_path  = "ca.pem"
crl_path   = "ca.crl"
key_source = "pkcs11"

[signer.local_ca.pkcs11]
module_path = "/usr/lib/softhsm/libsofthsm2.so"
token_label = "acme-ca"
key_label   = "ca-key"
pin_file    = "/etc/acme-proxy/hsm.pin"
printf '1234' > /etc/acme-proxy/hsm.pin
chmod 600 /etc/acme-proxy/hsm.pin

If SoftHSM2’s token store is not in its default location, SOFTHSM2_CONF must be set in the server’s environment — it is read by the module, not by acme-proxy.

Step 4 — Confirm it is really using the token

RUST_LOG=info acme-proxy serve
INFO acme_proxy::signer::local_ca::pkcs11: the local CA's issuing key is on a PKCS#11 token
  event="local_ca_pkcs11_opened" module=/usr/lib/softhsm/libsofthsm2.so
  slot=276468771 key_label=ca-key algorithm=PKCS_ECDSA_P256_SHA256
  mechanism=CKM_ECDSA_SHA256
INFO acme_proxy::signer::local_ca: event="local_ca_pkcs11_loaded" cert_path="ca.pem" key_label=ca-key

local_ca_pkcs11_opened is the line that proves it: it names the module, the slot the token actually landed in, the curve read off the key, and the mechanism chosen. If you see local_ca_loaded or local_ca_generated instead, the configuration is still on the file path.

Then issue something and check it chains:

openssl verify -CAfile ca.pem /path/to/issued/cert.pem
# cert.pem: OK

Walkthrough B — YubiKey (libykcs11)

A YubiKey 5 exposes its PIV applet through libykcs11, shipped with yubico-piv-tool.

# Debian/Ubuntu → /usr/lib/x86_64-linux-gnu/libykcs11.so
sudo apt install yubico-piv-tool
# Arch → /usr/lib/libykcs11.so
sudo pacman -S yubico-piv-tool

Both paths are in circulation; check which one you have before configuring module_path.

Step 1 — Generate the key and certificate on the device

Use slot 9c (Digital Signature). Its PIV policy requires the PIN for every private-key operation, which is the right posture for a CA key and the reason to prefer it over 9a.

# Generate the key inside the YubiKey — it never leaves
yubico-piv-tool -s 9c -a generate -A ECCP256 -o ca_pub.pem

# Self-sign a certificate for it, on the device
yubico-piv-tool -s 9c -a verify-pin -a selfsign-certificate \
  -S '/CN=Example Corp Issuing CA/O=Example Corp/' \
  --valid-days 3650 -i ca_pub.pem -o ca.pem

# Store the certificate in the slot as well (optional, but conventional)
yubico-piv-tool -s 9c -a import-certificate -i ca.pem

Copy ca.pem to wherever cert_path points.

Touch policy must be never for the CA slot. If the slot is provisioned to require a touch, every issuance blocks until somebody physically touches the key. That is correct for an offline root and catastrophic for an ACME server expected to issue unattended.

Step 2 — Find the key label

You do not choose the label on a YubiKey — libykcs11 assigns fixed ones per PIV slot. Read it off the device:

pkcs11-tool --module /usr/lib/libykcs11.so --list-objects --login

Slot 9c reports as Private key for Digital Signature; 9a as Private key for PIV Authentication. Use that string verbatim.

Step 3 — Configure

[signer.local_ca]
cert_path  = "ca.pem"
crl_path   = "ca.crl"
key_source = "pkcs11"

[signer.local_ca.pkcs11]
module_path = "/usr/lib/libykcs11.so"
token_label = "YubiKey PIV #12345678"
key_label   = "Private key for Digital Signature"
pin_file    = "/etc/acme-proxy/hsm.pin"

The PIN is the PIV PIN (factory default 123456), not the PIV management key and not the FIDO PIN.

Step 4 — Expect CKM_ECDSA

libykcs11 does not offer CKM_ECDSA_SHA256, so acme-proxy computes the SHA-256 digest itself and asks the token to sign that. The startup line reads:

mechanism=CKM_ECDSA+SHA256

This is normal and not a downgrade — the same signature, with the hashing done on this side of the USB cable.

Performance

A YubiKey signature takes roughly 50–300 ms, and signings are serialised by a mutex. That is comfortable for hundreds of certificates a day and is not a throughput solution; the signing call runs on the blocking thread pool, so it does not stall the rest of the server while it waits. For higher volumes use a networked HSM, or the Custom Script signer against a KMS.


Operations

Backup and disaster recovery

The key cannot be backed up. That is the point of the feature, and it makes recovery something to plan before you need it. Two workable approaches:

  • Two tokens, one offline root. Keep an offline root CA, use it to certify an intermediate held on each of two tokens, and hand acme-proxy one of them. A lost token is replaced by provisioning a new intermediate; clients trust the root and never notice. See Multi-Tier PKI.
  • Accept re-enrolment. For a small internal fleet, losing the CA and distributing a new one is survivable — just make sure it is a decision rather than a discovery.

Revocations live in the database like every other CA’s, so backing up the database backs them up; crl_path is only an export of the current CRL.

When the token disappears

If the session drops — the YubiKey is unplugged, a network HSM times out — acme-proxy reopens the session, logs back in and retries the signature once. The relevant log lines are local_ca_pkcs11_session_lost followed by either a successful issuance or local_ca_pkcs11_reconnect_failed.

If that fails, finalize requests return serverInternal (500) and clients retry, which is the right behaviour: the order stays valid and issuance resumes once the token is back. GET /crl keeps working throughout — the current CRL is read from the database and serving it signs nothing.

Sharing one token between profiles

Several profiles may use the same module, and even the same key. acme-proxy opens one PKCS#11 context per module for the whole process and shares it, so this works without special configuration. Two profiles naming the same token key with otherwise different signer settings is refused at startup, for the same reason two profiles sharing ca.key are: one key under two configurations would issue under two policies from one identity.

Troubleshooting

SymptomCause and fix
key_source = "pkcs11" … built withoutThe binary has no PKCS#11 support. Rebuild with --features hsm.
CKR_PIN_INCORRECT at startupUsually a stray character in pin_file. Trailing newlines are trimmed, but leading or embedded whitespace is not. Check with xxd. Do not retry blindly — see the PIN warning above.
CKR_PIN_LOCKEDToo many wrong attempts. A YubiKey PIV PIN is unblocked with the PUK (yubico-piv-tool -a unblock-pin).
is not the key certified by …The SPKI cross-check failed: key_label/key_id resolve to a different key than cert_path describes. List the token’s objects and compare.
no PKCS#11 token labelled …The label is wrong, or the token is not plugged in. The message lists the labels actually present.
N PKCS#11 private keys are labelled …Set key_id to pick one.
unsupported PKCS#11 curveOnly P-256 and P-384 are supported. The message prints the CKA_EC_PARAMS it found.
supports neither … nor CKM_ECDSAThe token cannot do ECDSA signing at all, or will not report its mechanisms.
Certificates that verify nowhereShould not happen — the SPKI cross-check catches the usual cause at startup. If it does, capture the local_ca_pkcs11_opened line and the failing certificate and open an issue.

See also Maintenance & Troubleshooting.