What are we doing here?
Let's set up HAProxy with some lovely free certs from Let's Encrypt via certbot for a couple of domains. Everything running in docker, and all tied together with docker-compose. We'll use docker user-defined networks, because that's the Right Thing To Do.
I'll use 'domain1.example.com' and 'domain2.example.com' as the domains involved. The domain1 site is served from a container called 'container1', and domain2 from a container called 'domain2' You might have more. Or less. Edit as appropriate.
This wiki is set up very similarly to the below - the running config is on my github.
This should be easy. Right?
Docker: easy. HAProxy: easy. Let's Encrypt: easy.
Docker and HAProxy and Let's Encrypt: pain in the arse.
There's a few things that make this a bit of a hassle:
- We want haproxy to be running on port 80/443, but those are the ports certbot needs to do validation
We'll have to do this in two stages. - haproxy with the default config won't start up if it can't resolve the container IPs for the backends.
Since certbot is just a command to be run in a container, it probably won't be running when haproxy starts up.
Some extra config is needed in haproxy. - certbot needs to be run once in one way to request the certs, and then every couple of days/weeks in another way to check and renew certs.
We'll need two different incantations for certbot. - When the certs are renewed, we'll need to tell haproxy to pick them up
Some docker-in-docker magic is required. - certbot doesn't know how to make haproxy-complicit cert pem files
We'll need to do a little scripting.
Stage 0 - setup
This all assumes that your soon-to-be certificated domains' A records are all pointing at the docker host (or port-forwarding router, or whatever), and you can reach the docker host on port 80 and 443 from the Interwebs. Your docker host should have docker and docker-compose installed, and docker running.
Stage 1 - get some certs
Since this is a greenfield setup, we can let certbot take care of the initial cert request on its own - HAProxy should be down for this.
Dockerfile
The Dockerfile for the letsencrypt image looks like:
FROM ubuntu:latest
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:certbot/certbot && \
apt-get update && \
apt-get install -y certbot docker.io
COPY deploy-hook /deploy-hook
RUN chmod +x /deploy-hookNote we're installing the docker.io package, and copying in a script. We'll need them later on.
deploy-hook
The deploy-hook script looks like:
#!/usr/bin/env bash
cat /etc/letsencrypt/live/domain1.example.com/fullchain.pem \
/etc/letsencrypt/live/domain1.example.com/privkey.pem \
> /etc/letsencrypt/haproxy.pem \
&& docker kill -s HUP haproxydocker-compose-stage1.yml
To run the container, we'll wrap it up in a docker-compose file called docker-compose-stage1.yml.
version: '3'
letsencrypt:
build: .
image: letsencrypt
container_name: letsencrypt
restart: no
volumes:
- letsencrypt_etc:/etc/letsencrypt
command: bash -c 'certbot certonly \
--standalone \
--preferred-challenges http-01 \
--http-01-port 8000 \
--agree-tos \
--non-interactive \
-m your.email@fastmail.com \
-d "domain1.example.com" \
-d "domain2.example.com"; \
cat /etc/letsencrypt/live/domain1.example.com/fullchain.pem \
/etc/letsencrypt/live/domain1.example.com/privkey.pem \
> /etc/letsencrypt/haproxy.pem'
ports:
- 80:8000
volumes:
letsencrypt_etc:Things of note:
- certbot listens on port 8000, which docker is mapping to port 80 and making available to the outside world for Let's Encrypt to talk to. We don't need port 443 mapped, because this is an initial request and Let's Encrypt should be fine with just port 80.
- We're attaching a Volume to /etc/letsencrypt - that's where the certs end up, and that's how we'll make them available to haproxy.
- The command concatenates the cert chain and private key into a format that haproxy understands, and dumps it out into the mounted /etc/letsencrypt volume.
- certbot names the certs for the first domain specified, so that ends up in all of the paths under /etc/letsencrypt. You might be able to change that, but see rule 1.
Go!
With all three files in your current directory, run: docker-compose -f docker-compose-stage1.yml up and you should hopefully see a message like the following after a couple of seconds:
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:Certs gotten, and stashed away in a docker volume. Happy days!
Stage 2 - haproxy
We've got ourselves some certs so it's time to fire up haproxy and enjoy all the HAing and Proxying. Don't know about you, but I am excited.
Dockerfile
We don't need one. Becuase we're using the official image. Because we're adhering to rule 1.
haproxy.cfg
Do an mkdir haproxy/bind.
Put something like the below in haproxy/bind/haproxy.fg:
global
maxconn 4096
daemon
log stdout format raw local0 debug
tune.ssl.default-dh-param 2048
ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
resolvers docker
nameserver docker1 127.0.0.11:53
defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
default-server init-addr none
frontend http_in
bind *:8080
bind *:8443 ssl crt /etc/letsencrypt/haproxy.pem
mode http
redirect scheme https code 301 if !{ ssl_fc }
capture request header Host len 256
capture request header User-Agent len 256
acl acme_pth path_beg -i /.well-known/acme-challenge
acl domain1_hdr hdr(host) -i domain1.example.com
acl domain2_hdr hdr(host) -i domain2.example.com
use_backend letsencrypt if acme_pth
use_backend domain1 if domain1_hdr
use_backend domain2 if domain2_hdr
backend letsencrypt
server letsencrypt letsencrypt:8000 resolvers docker check
backend domain1
server domain1-1 container1:5000 resolvers docker check
backend wiki
server domain2-1 container2:3000 resolvers docker checkWhat's going on here?
- The global section logs everything to stdout, because that's what you do with docker. rule 6 does not apply in dockerland.
- We're setting the Mozilla recommended ciphers and DH values. Check the current recommendations if you're foolish enough to go into production with this stuff.
- We're using 'resolvers' and 'default-server init-addr none' to get around problem of containers not being up at startup time. Docker with user-defined networks always puts a resolver at 127.0.0.11:53, and haproxy can use that to resolve container names at runtime instead of startup time.
- We're binding to port 8080 and 8443, and setting the cert to the Let's Encrypt cert we dumped out in the previous section. The ports will be mapped back to 80 and 443 by docker later on.
- Always redirect to https.
- All traffic that matches the certbot ACME challenge protocol is directed to our letsencrypt container (to be created later).
- Other traffic is matched by request hostname to their respective containers. Your routing will probably be more complicated than this, but it's a start.