Hardware Keys (PKCS#11)
By default the Local CA’s issuing key is a PEM file on disk, protected by
nothing but its 0600 permissions. key_source = "pkcs11" moves that key into
a hardware token — a YubiKey, a network HSM, or SoftHSM2 for development — where
it is created once and can never be read back out. acme-proxy sends the token
the bytes to be signed and receives a signature; the private key never enters
this process’s memory.
PKCS#11 rather than a vendor-specific PIV library, so a YubiKey today and an
enterprise HSM tomorrow are the same configuration with a different
module_path.
What this protects, and what it does not
Everything else about the Local CA is unchanged: the same CSR
sanitisation, the same leaf_validity_days
clamping, the same CRL and revocations. Only where the signature comes
from moves.
It protects the CA issuing key — the one that, if stolen, lets an attacker
mint certificates your fleet trusts. It does not protect the ACME account
keys, the TLS server key (server.tls.key_path), or the database; those stay on
disk.
Requirements
-
A build with the
hsmfeature. It is off by default, so the stock binary does not have it andkey_source = "pkcs11"on one is a startup error naming the feature:cargo build --release --features hsm -
A PKCS#11 module (
.so), loaded at runtime — nothing is linked at build time. -
An existing CA certificate, for the reason below.
Two rules that differ from the software path
Both are startup errors, so you will meet them immediately rather than in production.
The CA is never generated
With key_source = "file", a missing cert_path/key_path means “generate a
CA and write it here”. With key_source = "pkcs11" there is no such thing: the
private key is created inside the token by its own tooling, and this server
cannot produce one that a token would then hold. So cert_path must already
exist, and key_path is neither read nor written.
Both walkthroughs below cover creating that certificate.
The key and the certificate are cross-checked
At startup, the token key’s SubjectPublicKeyInfo is compared against the one
in cert_path. A mismatch — almost always a wrong key_label — stops the
server.
This is stricter than the file path, where (as Local
CA warns) nothing checks
that key_path corresponds to cert_path, and a mismatched pair simply
produces certificates that verify nowhere. Here a typo is caught before the
first certificate is issued rather than discovered by a client days later.
Reference
Reaching this table at all takes signer.local_ca.key_source = "pkcs11", which
is documented with the rest of [signer.local_ca] in Local
CA. Everything below is [signer.local_ca.pkcs11], read
only when that key is set.
pkcs11.module_path (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__MODULE_PATH
The PKCS#11 module to load. Required. See each walkthrough for the usual paths.
pkcs11.token_label (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__TOKEN_LABEL
Which token to use, by label. Preferred over slot_id: slot numbers are
assigned dynamically and change across reboots and re-plugs on most drivers
(SoftHSM2 will hand you something like 276468771).
pkcs11.slot_id (Integer) — Default: unset | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__SLOT_ID
Which slot to use, for tokens with no usable label. Consulted only when
token_label is empty.
pkcs11.key_label (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__KEY_LABEL
The private key’s CKA_LABEL. Required. On a YubiKey the labels are fixed by
the driver, so this is something you look up rather than choose — see below.
pkcs11.key_id (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__KEY_ID
The key’s CKA_ID as hex (01, or 01:ff), to disambiguate a token holding
several keys under one label. Optional; two keys sharing a label and no key_id
to separate them is a startup error rather than a coin flip.
pkcs11.pin_file (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__PIN_FILE
A file holding the user PIN. Trailing whitespace is trimmed, so a PIN written
with echo works. The file is checked for permissions and warns if it is
world-readable, exactly as ca.key does.
pkcs11.pin (String) — Default: "" | Env: ACME_PROXY_SIGNER__LOCAL_CA__PKCS11__PIN
SENSITIVE. The PIN directly. Prefer pin_file, or set this through the
environment variable; a PIN in config.toml is a long-lived secret in a file
that tends to get copied around. pin_file wins when both are set, and having
neither is a startup error.
A PIN is not a password. Tokens block after a small number of wrong attempts — three on a YubiKey PIV applet, after which you need the PUK. This is why
acme-proxyretries a failed signature at most once, and why the trailing newline in your PIN file is worth getting right.
Walkthrough A — SoftHSM2
SoftHSM2 is a software token: no hardware needed, and the same setup the project uses in CI. Use it to try the feature before committing to hardware.
# Debian/Ubuntu
sudo apt install softhsm2
# Arch
sudo pacman -S softhsm
Step 1 — Create a token
softhsm2-util --init-token --free --label acme-ca --so-pin 3737 --pin 1234
--free takes the first uninitialised slot. Note that the token is reassigned
to a new slot number afterwards — which is exactly why token_label is the
selector to use, not slot_id.
Step 2 — Create the CA key and certificate
The key must exist inside the token, and cert_path must hold a certificate for
it. For SoftHSM2 the simplest route is to generate both locally, import the key,
and destroy the local copy:
# The CA key and its self-signed certificate
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ca.key
openssl req -x509 -new -key ca.key -sha256 -days 3650 -out ca.pem \
-subj "/CN=Example Corp Issuing CA/O=Example Corp" \
-addext "basicConstraints=critical,CA:true,pathlen:0" \
-addext "keyUsage=critical,keyCertSign,cRLSign"
# Move the key into the token, then remove it from disk
softhsm2-util --import ca.key --token acme-ca --label ca-key --id 01 --pin 1234
shred -u ca.key
For a real HSM, generate the key in the token instead so it never exists outside it —
pkcs11-tool --module <module> --token-label acme-ca --login --keypairgen --key-type EC:prime256v1 --label ca-key --id 01(from theopenscpackage), then certify that public key with your offline root. The import above is a development convenience, and the reason it is acceptable here is that a SoftHSM2 token is a directory of files anyway.
pathlen:0 matches what the Local CA generates for itself: it may issue leaves
but no further CAs.
Step 3 — Configure
[signer]
backend = "local_ca"
[signer.local_ca]
cert_path = "ca.pem"
crl_path = "ca.crl"
key_source = "pkcs11"
[signer.local_ca.pkcs11]
module_path = "/usr/lib/softhsm/libsofthsm2.so"
token_label = "acme-ca"
key_label = "ca-key"
pin_file = "/etc/acme-proxy/hsm.pin"
printf '1234' > /etc/acme-proxy/hsm.pin
chmod 600 /etc/acme-proxy/hsm.pin
If SoftHSM2’s token store is not in its default location, SOFTHSM2_CONF must
be set in the server’s environment — it is read by the module, not by
acme-proxy.
Step 4 — Confirm it is really using the token
RUST_LOG=info acme-proxy serve
INFO acme_proxy::signer::local_ca::pkcs11: the local CA's issuing key is on a PKCS#11 token
event="local_ca_pkcs11_opened" module=/usr/lib/softhsm/libsofthsm2.so
slot=276468771 key_label=ca-key algorithm=PKCS_ECDSA_P256_SHA256
mechanism=CKM_ECDSA_SHA256
INFO acme_proxy::signer::local_ca: event="local_ca_pkcs11_loaded" cert_path="ca.pem" key_label=ca-key
local_ca_pkcs11_opened is the line that proves it: it names the module, the
slot the token actually landed in, the curve read off the key, and the mechanism
chosen. If you see local_ca_loaded or local_ca_generated instead, the
configuration is still on the file path.
Then issue something and check it chains:
openssl verify -CAfile ca.pem /path/to/issued/cert.pem
# cert.pem: OK
Walkthrough B — YubiKey (libykcs11)
A YubiKey 5 exposes its PIV applet through libykcs11, shipped with
yubico-piv-tool.
# Debian/Ubuntu → /usr/lib/x86_64-linux-gnu/libykcs11.so
sudo apt install yubico-piv-tool
# Arch → /usr/lib/libykcs11.so
sudo pacman -S yubico-piv-tool
Both paths are in circulation; check which one you have before configuring
module_path.
Step 1 — Generate the key and certificate on the device
Use slot 9c (Digital Signature). Its PIV policy requires the PIN for every private-key operation, which is the right posture for a CA key and the reason to prefer it over 9a.
# Generate the key inside the YubiKey — it never leaves
yubico-piv-tool -s 9c -a generate -A ECCP256 -o ca_pub.pem
# Self-sign a certificate for it, on the device
yubico-piv-tool -s 9c -a verify-pin -a selfsign-certificate \
-S '/CN=Example Corp Issuing CA/O=Example Corp/' \
--valid-days 3650 -i ca_pub.pem -o ca.pem
# Store the certificate in the slot as well (optional, but conventional)
yubico-piv-tool -s 9c -a import-certificate -i ca.pem
Copy ca.pem to wherever cert_path points.
Touch policy must be
neverfor the CA slot. If the slot is provisioned to require a touch, every issuance blocks until somebody physically touches the key. That is correct for an offline root and catastrophic for an ACME server expected to issue unattended.
Step 2 — Find the key label
You do not choose the label on a YubiKey — libykcs11 assigns fixed ones per
PIV slot. Read it off the device:
pkcs11-tool --module /usr/lib/libykcs11.so --list-objects --login
Slot 9c reports as Private key for Digital Signature; 9a as Private key for PIV Authentication. Use that string verbatim.
Step 3 — Configure
[signer.local_ca]
cert_path = "ca.pem"
crl_path = "ca.crl"
key_source = "pkcs11"
[signer.local_ca.pkcs11]
module_path = "/usr/lib/libykcs11.so"
token_label = "YubiKey PIV #12345678"
key_label = "Private key for Digital Signature"
pin_file = "/etc/acme-proxy/hsm.pin"
The PIN is the PIV PIN (factory default 123456), not the PIV management
key and not the FIDO PIN.
Step 4 — Expect CKM_ECDSA
libykcs11 does not offer CKM_ECDSA_SHA256, so acme-proxy computes the
SHA-256 digest itself and asks the token to sign that. The startup line reads:
mechanism=CKM_ECDSA+SHA256
This is normal and not a downgrade — the same signature, with the hashing done on this side of the USB cable.
Performance
A YubiKey signature takes roughly 50–300 ms, and signings are serialised by a mutex. That is comfortable for hundreds of certificates a day and is not a throughput solution; the signing call runs on the blocking thread pool, so it does not stall the rest of the server while it waits. For higher volumes use a networked HSM, or the Custom Script signer against a KMS.
Operations
Backup and disaster recovery
The key cannot be backed up. That is the point of the feature, and it makes recovery something to plan before you need it. Two workable approaches:
- Two tokens, one offline root. Keep an offline root CA, use it to certify
an intermediate held on each of two tokens, and hand
acme-proxyone of them. A lost token is replaced by provisioning a new intermediate; clients trust the root and never notice. See Multi-Tier PKI. - Accept re-enrolment. For a small internal fleet, losing the CA and distributing a new one is survivable — just make sure it is a decision rather than a discovery.
Revocations live in the database like every other CA’s, so backing up the
database backs them up; crl_path is only an export of the current CRL.
When the token disappears
If the session drops — the YubiKey is unplugged, a network HSM times out —
acme-proxy reopens the session, logs back in and retries the signature
once. The relevant log lines are local_ca_pkcs11_session_lost followed by
either a successful issuance or local_ca_pkcs11_reconnect_failed.
If that fails, finalize requests return serverInternal (500) and clients
retry, which is the right behaviour: the order stays valid and issuance resumes
once the token is back. GET /crl keeps working throughout — the current CRL is
read from the database and serving it signs nothing.
Sharing one token between profiles
Several profiles may use the same module, and even the
same key. acme-proxy opens one PKCS#11 context per module for the whole
process and shares it, so this works without special configuration. Two profiles
naming the same token key with otherwise different signer settings is refused
at startup, for the same reason two profiles sharing ca.key are: one key under
two configurations would issue under two policies from one identity.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
key_source = "pkcs11" … built without | The binary has no PKCS#11 support. Rebuild with --features hsm. |
CKR_PIN_INCORRECT at startup | Usually a stray character in pin_file. Trailing newlines are trimmed, but leading or embedded whitespace is not. Check with xxd. Do not retry blindly — see the PIN warning above. |
CKR_PIN_LOCKED | Too many wrong attempts. A YubiKey PIV PIN is unblocked with the PUK (yubico-piv-tool -a unblock-pin). |
is not the key certified by … | The SPKI cross-check failed: key_label/key_id resolve to a different key than cert_path describes. List the token’s objects and compare. |
no PKCS#11 token labelled … | The label is wrong, or the token is not plugged in. The message lists the labels actually present. |
N PKCS#11 private keys are labelled … | Set key_id to pick one. |
unsupported PKCS#11 curve | Only P-256 and P-384 are supported. The message prints the CKA_EC_PARAMS it found. |
supports neither … nor CKM_ECDSA | The token cannot do ECDSA signing at all, or will not report its mechanisms. |
| Certificates that verify nowhere | Should not happen — the SPKI cross-check catches the usual cause at startup. If it does, capture the local_ca_pkcs11_opened line and the failing certificate and open an issue. |
See also Maintenance & Troubleshooting.