WIUNIX ACADEMY
HAProxy with Let’s Encrypt and validated certificate renewal
Page 3 of 4. Review the earlier steps and prerequisites before continuing.
YOUR SYSTEM. YOUR STEPS.
Where are you working?
Choose your OS and version. The prerequisites, commands and troubleshooting follow your environment.
USE YOUR OWN VALUES
Configure this tutorial
Set the example values for your environment; related explanations and copy panels update together. Never enter passwords, private keys or tokens. Values stay in this tab and are not sent to the server or share links. Check actual DNS, certificates and network policy before running commands.
Different IP addresses, domains or ports? Open the settings below, change the values and select “Apply to tutorial”, then copy the updated commands.
Set values for my environment7 editable values
LET’S BUILD
Validate, replace and gracefully reload
The first backup is the HTTP-only bootstrap. Validate the full TLS candidate before any replacement; a failed check leaves it untouched. Copy to a same-filesystem temporary name then rename. If reload fails, restore the bootstrap deliberately before investigating; the next recovery panel provides commands. A successful reload still requires the independent TLS probe.
Bash on dedicated edge VM; external checks noted in text
set -eu
sudo haproxy -c -f '/srv/academy-haproxy-tls/haproxy.tls.cfg'
sudo cp -a '/srv/academy-haproxy-tls/haproxy.cfg' '/srv/academy-haproxy-tls/haproxy.bootstrap.cfg'
sudo install -m 0644 '/srv/academy-haproxy-tls/haproxy.tls.cfg' '/srv/academy-haproxy-tls/haproxy.cfg.next'
sudo mv '/srv/academy-haproxy-tls/haproxy.cfg.next' '/srv/academy-haproxy-tls/haproxy.cfg'
sudo systemctl reload academy-haproxy.service
sudo systemctl --no-pager status academy-haproxy.service
HAProxy validates, the master gracefully reloads and the dedicated service remains active.
LET’S BUILD
Verify trust and backend selection independently
Run the first curl from outside the edge without --resolve and without -k. Its normal CA validation must succeed for app.example.com. The local resolve probe tests this specific listener using the same hostname. Send spoofed proxy headers and inspect only a private controlled application endpoint/log: the observed Forwarded must be absent and X-Forwarded-For must be the direct client, with https and port443. Never expose a debug header endpoint publicly.
Take one backend out of readiness during a controlled application maintenance window, wait at least two failed checks and confirm repeated requests reach the healthy node. Restore readiness and observe two successful checks. Do not kill an unrelated process or infer failover from a single200; readiness, session state and real response markers matter.
Bash on dedicated edge VM; external checks noted in text
set -eu
curl --fail --show-error 'https://app.example.com/'
curl --fail --show-error --resolve 'app.example.com:443:192.0.2.10' \
-H 'Forwarded: for=198.51.100.99;proto=http' \
-H 'X-Forwarded-For: 198.51.100.99' 'https://app.example.com/'
openssl s_client -connect '192.0.2.10:443' -servername 'app.example.com' \
-verify_hostname 'app.example.com' -verify_return_error -CApath /etc/ssl/certs </dev/null
Normal CA/hostname validation succeeds; controlled backend evidence shows the sanitized direct-client headers and healthy-node routing.
LET’S BUILD
Install a root-only renewal deploy hook
The next Python file is a deploy hook, not an ACME client. It checks the exact lineage, locks concurrent deployments, validates certificate/name/key and a private candidate config, then preserves the previous PEM and atomically swaps the candidate. Validation or issuance failure cannot replace the healthy PEM. A reload exception restores the previous PEM and attempts a validated reload, then exits with failure. A failure of that recovery remains visible for the operator; it cannot guarantee service recovery under every fault.
Bash on dedicated edge VM; external checks noted in text
set -eu
test ! -e '/srv/academy-haproxy-tls/deploy.py'
test ! -L '/srv/academy-haproxy-tls/deploy.py'
sudo install -m 0700 /dev/null '/srv/academy-haproxy-tls/deploy.py'
sudo nano '/srv/academy-haproxy-tls/deploy.py'
Only root can edit/execute the new hook; the complete file follows.
LET’S BUILD
Validated atomic deployment
Complete file. Capture failures in the dedicated service journal; never log key bytes. A deploy-hook failure occurs after Certbot may already have renewed its lineage: repair and rerun this hook manually with the reviewed lineage rather than forcing a new order.
File on the dedicated edge VM
#!/usr/bin/python3
"""Root-only deploy hook; never print private material."""
import fcntl
import os
from pathlib import Path
import subprocess
import tempfile
ROOT = Path("/srv/academy-haproxy-tls")
DOMAIN = "app.example.com"
LINEAGE = ROOT / "acme/config/live" / DOMAIN
PEM = ROOT / "certs" / (DOMAIN + ".pem")
CONFIG = ROOT / "haproxy.cfg"
CAFILE = "/etc/ssl/certs/ca-certificates.crt"
def run(*args):
return subprocess.run(args, check=True, capture_output=True).stdout
def main():
if os.geteuid() != 0:
raise SystemExit("root required")
if Path(os.environ.get("RENEWED_LINEAGE", "")).resolve() != LINEAGE.resolve():
raise SystemExit("unexpected lineage; no changes")
os.umask(0o077)
with (ROOT / "deploy.lock").open("a") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
cert, chain, key = [LINEAGE / n for n in ("cert.pem", "chain.pem", "privkey.pem")]
run("openssl", "x509", "-in", str(cert), "-checkend", "86400", "-noout")
run("openssl", "verify", "-CAfile", CAFILE, "-untrusted", str(chain),
"-purpose", "sslserver", "-verify_hostname", DOMAIN, str(cert))
public = run("openssl", "x509", "-in", str(cert), "-pubkey", "-noout")
if public != run("openssl", "pkey", "-in", str(key), "-pubout"):
raise SystemExit("certificate/key mismatch; no changes")
if not PEM.is_file() or PEM.is_symlink():
raise SystemExit("initial reviewed PEM must already exist")
old = PEM.read_bytes()
candidate = None
changed = False
try:
fd, name = tempfile.mkstemp(prefix=".candidate-", suffix=".pem", dir=PEM.parent)
candidate = Path(name)
with os.fdopen(fd, "wb") as output:
output.write((LINEAGE / "fullchain.pem").read_bytes() + b"\n" + key.read_bytes())
output.flush()
os.fsync(output.fileno())
# Replace only the one exact crt path in a private validation config.
config = CONFIG.read_text()
if config.count("crt " + str(PEM)) != 1:
raise SystemExit("unexpected crt binding; no changes")
with tempfile.TemporaryDirectory(prefix="validate-", dir=ROOT) as work:
check = Path(work) / "haproxy.cfg"
check.write_text(config.replace("crt " + str(PEM), "crt " + str(candidate)))
run("/usr/sbin/haproxy", "-c", "-f", str(check))
backup = PEM.with_suffix(".previous.pem")
fd, name = tempfile.mkstemp(prefix=".backup-", dir=PEM.parent)
with os.fdopen(fd, "wb") as output:
output.write(old)
output.flush()
os.fsync(output.fileno())
os.replace(name, backup)
os.replace(candidate, PEM)
changed = True
run("/usr/sbin/haproxy", "-c", "-f", str(CONFIG))
run("/usr/bin/systemctl", "reload", "academy-haproxy.service")
except BaseException:
if changed:
fd, name = tempfile.mkstemp(prefix=".rollback-", dir=PEM.parent)
with os.fdopen(fd, "wb") as output:
output.write(old)
output.flush()
os.fsync(output.fileno())
os.replace(name, PEM)
run("/usr/sbin/haproxy", "-c", "-f", str(CONFIG))
run("/usr/bin/systemctl", "reload", "academy-haproxy.service")
raise
finally:
if candidate is not None:
candidate.unlink(missing_ok=True)
print("validated PEM deployed; verify the public certificate separately")
if __name__ == "__main__":
main()
The file compiles; it accepts only the dedicated production lineage and retains a protected previous PEM.
LET’S BUILD
Exercise the deployment and separate dry-run behavior
First redeploy the current trusted certificate through the hook; this tests its real validation and reload path without asking for a new certificate. Then run Certbot renewal dry-run against staging. Deploy hooks are normally skipped by dry-run; --run-deploy-hooks explicitly runs them with the current active certificate, not the untrusted staging certificate. Do not claim an actual renewal from this simulation. Verify public TLS again after hook execution.
Bash on dedicated edge VM; external checks noted in text
set -eu
sudo python3 -m py_compile '/srv/academy-haproxy-tls/deploy.py'
sudo env RENEWED_LINEAGE='/srv/academy-haproxy-tls/acme/config/live/app.example.com' '/srv/academy-haproxy-tls/deploy.py'
sudo certbot renew --dry-run --run-deploy-hooks --cert-name 'app.example.com' \
--config-dir '/srv/academy-haproxy-tls/acme/config' --work-dir '/srv/academy-haproxy-tls/acme/work' \
--logs-dir '/srv/academy-haproxy-tls/acme/logs' --deploy-hook '/srv/academy-haproxy-tls/deploy.py'
curl --fail --show-error 'https://app.example.com/'
The current trusted PEM reloads; a successful CA dry run explicitly invokes the hook and the public certificate remains trusted.
LET’S BUILD
Use one dedicated renewal schedule
Use only one schedule for this lineage. The distribution timer was disabled on this otherwise empty lab VM; do not disable timers serving other certificates on a shared host. The named service uses the dedicated config/work/log paths and runs as root so no key permission workaround is needed.
Bash on dedicated edge VM; external checks noted in text
set -eu
test ! -e /etc/systemd/system/academy-certbot.service
test ! -L /etc/systemd/system/academy-certbot.service
sudo install -m 0644 /dev/null /etc/systemd/system/academy-certbot.service
test ! -e /etc/systemd/system/academy-certbot.timer
test ! -L /etc/systemd/system/academy-certbot.timer
sudo install -m 0644 /dev/null /etc/systemd/system/academy-certbot.timer
sudo nano /etc/systemd/system/academy-certbot.service /etc/systemd/system/academy-certbot.timer
Both new scheduler files are ready; the next two panels define the exact executable and schedule.
LET’S BUILD
Dedicated Certbot service
Save this complete unit file.
File on the dedicated edge VM
[Unit]
Description=Renew the Academy HAProxy certificate
After=network-online.target academy-acme-webroot.service
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --config-dir /srv/academy-haproxy-tls/acme/config --work-dir /srv/academy-haproxy-tls/acme/work --logs-dir /srv/academy-haproxy-tls/acme/logs --cert-name app.example.com --deploy-hook /srv/academy-haproxy-tls/deploy.py --quiet
The unit uses only the dedicated article lineage.
LET’S BUILD
Twice-daily timer with jitter
Save this complete unit file.
File on the dedicated edge VM
[Unit]
Description=Check the Academy certificate twice daily
[Timer]
OnCalendar=*-*-* 00,12:00:00
RandomizedDelaySec=1h
Persistent=true
[Install]
WantedBy=timers.target
The unit uses only the dedicated article lineage.