Changes in 8c16f9c: Updated haproxy-letsencrypt-docker.md

				haproxy-letsencrypt-docker.md
			
          @@ -2,13 +2,14 @@

           <!-- SUBTITLE: This is 3 lines of bash when docker's not involved -->

           # What are we doing here?

          -Let's set up [HAProxy](https://www.haproxy.org/) with some lovely free certs from [Let's Encrypt](https://letsencrypt.org/) via [certbot](https://certbot.eff.org/) for a couple of domains.

          +Let's set up [HAProxy](https://www.haproxy.org/) with some lovely free certs from [Let's Encrypt](https://letsencrypt.org/) via [certbot](https://certbot.eff.org/) for a couple of domains, each domain fronted by a different container.

           Everything running in [docker](https://www.docker.com), and all tied together with [docker-compose](https://docs.docker.com/compose/).

          -We'll use docker [user-defined networks](https://docs.docker.com/v17.09/engine/userguide/networking/#user-defined-networks), because that's the Right Thing To Do.

          +No k8s, no swarm, just one woman/man/other and one host/VM/other. Old-ish skool.

          +We'll use [docker user-defined networks](https://docs.docker.com/v17.09/engine/userguide/networking/#user-defined-networks), because that's the Right Thing To Do here.

          -I'll use 'domain1.example.com' and 'domain2.example.com' as the domains involved. The domain1 site is served from a container called 'container1', and domain2 from a container called 'domain2' You might have more. Or less. Edit as appropriate.

          +I'll use 'domain1.example.com' and 'domain2.example.com' as the domains involved. The domain1 site is served from a container called 'container1', and domain2 from 'container2' You might have more. Or less. Edit as appropriate.

          -This wiki is set up very similarly to the below - the running config is on [my github](https://github.com/ilikejam/home-network).

          +This wiki you're reading is set up very similarly to the below - the running config is on [my github](https://github.com/ilikejam/home-network).

           # This should be easy. Right?

           Docker: easy.

          @@ -18,21 +19,24 @@ Let's Encrypt: easy.

           Docker and HAProxy and Let's Encrypt: pain in the arse.

           There's a few things that make this a bit of a hassle:

          -1. We want haproxy to be running on port 80/443, but those are the ports certbot needs to do validation<br/>We'll have to do this in two stages.

          -2. haproxy with the default config won't start up if it can't resolve the container IPs for the backends.<br/>Since certbot is just a command to be run in a container, it probably won't be running when haproxy starts up.<br/>Some extra config is needed in haproxy.

          -3. certbot needs to be run once in one way to request the certs, and then every couple of days/weeks in another way to check and renew certs.<br/>We'll need two different incantations for certbot.

          -4. When the certs are renewed, we'll need to tell haproxy to pick them up<br/>Some docker-in-docker magic is required.

          -5. certbot doesn't know how to make haproxy-complicit cert pem files<br/>We'll need to do a little scripting.

          +* We want haproxy to be running on port 80/443, but those are the ports certbot needs to do validation.<br/>We'll do this in two stages for minimum pain.

          +* haproxy with the default config won't start up if it can't resolve the container IPs for the backends.<br/>This is a general problem with haproxy and containers. We'll do some config to make it work.

          +* certbot needs to be run once in one way to request the certs, and then every couple of days/weeks in another way to check and renew certs.<br/>We'll need two different incantations for certbot.

          +* When the certs are renewed, we'll need to tell haproxy to pick them up<br/>Some docker-in-docker magic is required.

          +* certbot doesn't know how to make haproxy-complicit cert pem files<br/>We'll need to do a little scripting. Not much though. 3-lines, max.

          +

          +Let's do this thing.

           # Stage 0 - setup

           This all assumes that your soon-to-be certificated domains' A records are all pointing at the docker host (or port-forwarding router, or whatever), and you can reach the docker host on port 80 and 443 from the Interwebs.

          -Your docker host should have docker and docker-compose installed, and docker running.

          +Your docker host should have docker, docker-compose, and openssl installed (for testing), and the docker daemon should be running.

          -# Stage 1 - get some certs

          +# Stage 1 - get a cert

           Since this is a greenfield setup, we can let certbot take care of the initial cert request on its own - HAProxy should be down for this.

          +Create a 'letsencrypt' directory to stuff this in, in your current directory.

           ## Dockerfile

          -The Dockerfile for the letsencrypt image looks like:

          +The letsencrypt/Dockerfile file looks like:

           ```dockerfile

           FROM ubuntu:latest

          @@ -47,10 +51,10 @@ COPY deploy-hook /deploy-hook

           RUN chmod +x /deploy-hook

           ```

          -Note we're installing the docker.io package, and copying in a script. We'll need them later on.

          +Note we're installing the docker.io package, and copying in a script. We'll need them later on. We could probably use the official certbot image, but chances are you'll already have 'ubuntu:latest' in-cache, so we might as well use it.

           ## deploy-hook

          -The `deploy-hook` script looks like:

          +The letsencrypt/deploy-hook script looks like:

           ```sh

           #!/usr/bin/env bash

          @@ -62,12 +66,12 @@ cat /etc/letsencrypt/live/domain1.example.com/fullchain.pem \

           ```

           ## docker-compose-stage1.yml

          -To run the container, we'll wrap it up in a docker-compose file called `docker-compose-stage1.yml`.

          +To run the container, we'll wrap it up in a docker-compose file called `docker-compose-stage1.yml`. Put this in your current directory:

           ```yaml

           version: '3'

             letsencrypt:

          -    build: .

          +    build: ./letsencrypt

               image: letsencrypt

               container_name: letsencrypt

               restart: no

          @@ -94,12 +98,12 @@ volumes:

           Things of note:

           1. certbot listens on port 8000, which docker is mapping to port 80 and making available to the outside world for Let's Encrypt to talk to. We don't need port 443 mapped, because this is an initial request and Let's Encrypt should be fine with just port 80.

          -2. We're attaching a Volume to /etc/letsencrypt - that's where the certs end up, and that's how we'll make them available to haproxy.

          +2. We're attaching a docker volume to /etc/letsencrypt - that's where the certs end up, and that's how we'll make them available to haproxy.

           3. The command concatenates the cert chain and private key into a format that haproxy understands, and dumps it out into the mounted /etc/letsencrypt volume.

           4. certbot names the certs for the first domain specified, so that ends up in all of the paths under /etc/letsencrypt. You might be able to change that, but see [rule 1](/rules#love-thy-defaults).

           ## Go!

          -With all three files in your current directory, run: `docker-compose -f docker-compose-stage1.yml up` and you should hopefully see a message like the following after a couple of seconds:

          +Run: `docker-compose -f docker-compose-stage1.yml up` and you should hopefully see a message like the following after a couple of seconds:

           ```text

           IMPORTANT NOTES:

          @@ -112,11 +116,13 @@ Certs gotten, and stashed away in a docker volume. Happy days!

           We've got ourselves some certs so it's time to fire up haproxy and enjoy all the HAing and Proxying. Don't know about you, but I am excited.

           ## Dockerfile

          -We don't need one. Becuase we're using the official image. Because we're adhering to [rule 1](/rules#love-thy-defaults).

          +We don't need one.

          +Because we're using the official image.

          +Because we're adhering to [rule 1](/rules#love-thy-defaults).

           ## haproxy.cfg

          -Do an `mkdir haproxy/bind`.

          -Put something like the below in `haproxy/bind/haproxy.fg`:

          +Do an `mkdir -p haproxy/bind`.

          +Put something like the below in `haproxy/bind/haproxy.cfg`:

           ```text

           global

          @@ -168,10 +174,10 @@ backend wiki

               server domain2-1 container2:3000 resolvers docker check

           ```

          -What's going on here?

          +What's going on here then?

           1. The global section logs everything to stdout, because that's what you do with docker. [rule 6](/rules#thou-shalt-respect-the-sanctity-of-stdout) does not apply in dockerland.

           2. We're setting the Mozilla recommended ciphers and DH values. Check the [current recommendations](https://mozilla.github.io/server-side-tls/ssl-config-generator/) if you're foolish enough to go into production with this stuff.

          -3. We're using 'resolvers' and 'default-server init-addr none' to get around problem of containers not being up at startup time. Docker with user-defined networks always puts a resolver at 127.0.0.11:53, and haproxy can use that to resolve container names at runtime instead of startup time.

          +3. We're using 'resolvers' and 'default-server init-addr none' to get around the problem of containers not being up at haproxy startup time. Docker with user-defined networks always puts a resolver at 127.0.0.11:53, and haproxy can use that to resolve container names at runtime instead of startup time.

           4. We're binding to port 8080 and 8443, and setting the cert to the Let's Encrypt cert we dumped out in the previous section. The ports will be mapped back to 80 and 443 by docker later on.

           5. Always redirect to https.

           6. All traffic that matches the certbot [ACME](https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html) challenge protocol is directed to our letsencrypt container (to be created later).

...	...	@@ -2,13 +2,14 @@
2	2	<!-- SUBTITLE: This is 3 lines of bash when docker's not involved -->
3	3
4	4	# What are we doing here?
5		-Let's set up [HAProxy](https://www.haproxy.org/) with some lovely free certs from [Let's Encrypt](https://letsencrypt.org/) via [certbot](https://certbot.eff.org/) for a couple of domains.
	5	+Let's set up [HAProxy](https://www.haproxy.org/) with some lovely free certs from [Let's Encrypt](https://letsencrypt.org/) via [certbot](https://certbot.eff.org/) for a couple of domains, each domain fronted by a different container.
6	6	Everything running in [docker](https://www.docker.com), and all tied together with [docker-compose](https://docs.docker.com/compose/).
7		-We'll use docker [user-defined networks](https://docs.docker.com/v17.09/engine/userguide/networking/#user-defined-networks), because that's the Right Thing To Do.
	7	+No k8s, no swarm, just one woman/man/other and one host/VM/other. Old-ish skool.
	8	+We'll use [docker user-defined networks](https://docs.docker.com/v17.09/engine/userguide/networking/#user-defined-networks), because that's the Right Thing To Do here.
8	9
9		-I'll use 'domain1.example.com' and 'domain2.example.com' as the domains involved. The domain1 site is served from a container called 'container1', and domain2 from a container called 'domain2' You might have more. Or less. Edit as appropriate.
	10	+I'll use 'domain1.example.com' and 'domain2.example.com' as the domains involved. The domain1 site is served from a container called 'container1', and domain2 from 'container2' You might have more. Or less. Edit as appropriate.
10	11
11		-This wiki is set up very similarly to the below - the running config is on [my github](https://github.com/ilikejam/home-network).
	12	+This wiki you're reading is set up very similarly to the below - the running config is on [my github](https://github.com/ilikejam/home-network).
12	13
13	14	# This should be easy. Right?
14	15	Docker: easy.
...	...	@@ -18,21 +19,24 @@ Let's Encrypt: easy.
18	19	Docker and HAProxy and Let's Encrypt: pain in the arse.
19	20
20	21	There's a few things that make this a bit of a hassle:
21		-1. We want haproxy to be running on port 80/443, but those are the ports certbot needs to do validation<br/>We'll have to do this in two stages.
22		-2. haproxy with the default config won't start up if it can't resolve the container IPs for the backends.<br/>Since certbot is just a command to be run in a container, it probably won't be running when haproxy starts up.<br/>Some extra config is needed in haproxy.
23		-3. certbot needs to be run once in one way to request the certs, and then every couple of days/weeks in another way to check and renew certs.<br/>We'll need two different incantations for certbot.
24		-4. When the certs are renewed, we'll need to tell haproxy to pick them up<br/>Some docker-in-docker magic is required.
25		-5. certbot doesn't know how to make haproxy-complicit cert pem files<br/>We'll need to do a little scripting.
	22	+* We want haproxy to be running on port 80/443, but those are the ports certbot needs to do validation.<br/>We'll do this in two stages for minimum pain.
	23	+* haproxy with the default config won't start up if it can't resolve the container IPs for the backends.<br/>This is a general problem with haproxy and containers. We'll do some config to make it work.
	24	+* certbot needs to be run once in one way to request the certs, and then every couple of days/weeks in another way to check and renew certs.<br/>We'll need two different incantations for certbot.
	25	+* When the certs are renewed, we'll need to tell haproxy to pick them up<br/>Some docker-in-docker magic is required.
	26	+* certbot doesn't know how to make haproxy-complicit cert pem files<br/>We'll need to do a little scripting. Not much though. 3-lines, max.
	27	+
	28	+Let's do this thing.
26	29
27	30	# Stage 0 - setup
28	31	This all assumes that your soon-to-be certificated domains' A records are all pointing at the docker host (or port-forwarding router, or whatever), and you can reach the docker host on port 80 and 443 from the Interwebs.
29		-Your docker host should have docker and docker-compose installed, and docker running.
	32	+Your docker host should have docker, docker-compose, and openssl installed (for testing), and the docker daemon should be running.
30	33
31		-# Stage 1 - get some certs
	34	+# Stage 1 - get a cert
32	35	Since this is a greenfield setup, we can let certbot take care of the initial cert request on its own - HAProxy should be down for this.
	36	+Create a 'letsencrypt' directory to stuff this in, in your current directory.
33	37
34	38	## Dockerfile
35		-The Dockerfile for the letsencrypt image looks like:
	39	+The letsencrypt/Dockerfile file looks like:
36	40
37	41	```dockerfile
38	42	FROM ubuntu:latest
...	...	@@ -47,10 +51,10 @@ COPY deploy-hook /deploy-hook
47	51	RUN chmod +x /deploy-hook
48	52	```
49	53
50		-Note we're installing the docker.io package, and copying in a script. We'll need them later on.
	54	+Note we're installing the docker.io package, and copying in a script. We'll need them later on. We could probably use the official certbot image, but chances are you'll already have 'ubuntu:latest' in-cache, so we might as well use it.
51	55
52	56	## deploy-hook
53		-The `deploy-hook` script looks like:
	57	+The letsencrypt/deploy-hook script looks like:
54	58
55	59	```sh
56	60	#!/usr/bin/env bash
...	...	@@ -62,12 +66,12 @@ cat /etc/letsencrypt/live/domain1.example.com/fullchain.pem \
62	66	```
63	67
64	68	## docker-compose-stage1.yml
65		-To run the container, we'll wrap it up in a docker-compose file called `docker-compose-stage1.yml`.
	69	+To run the container, we'll wrap it up in a docker-compose file called `docker-compose-stage1.yml`. Put this in your current directory:
66	70
67	71	```yaml
68	72	version: '3'
69	73	letsencrypt:
70		- build: .
	74	+ build: ./letsencrypt
71	75	image: letsencrypt
72	76	container_name: letsencrypt
73	77	restart: no
...	...	@@ -94,12 +98,12 @@ volumes:
94	98
95	99	Things of note:
96	100	1. certbot listens on port 8000, which docker is mapping to port 80 and making available to the outside world for Let's Encrypt to talk to. We don't need port 443 mapped, because this is an initial request and Let's Encrypt should be fine with just port 80.
97		-2. We're attaching a Volume to /etc/letsencrypt - that's where the certs end up, and that's how we'll make them available to haproxy.
	101	+2. We're attaching a docker volume to /etc/letsencrypt - that's where the certs end up, and that's how we'll make them available to haproxy.
98	102	3. The command concatenates the cert chain and private key into a format that haproxy understands, and dumps it out into the mounted /etc/letsencrypt volume.
99	103	4. certbot names the certs for the first domain specified, so that ends up in all of the paths under /etc/letsencrypt. You might be able to change that, but see [rule 1](/rules#love-thy-defaults).
100	104
101	105	## Go!
102		-With all three files in your current directory, run: `docker-compose -f docker-compose-stage1.yml up` and you should hopefully see a message like the following after a couple of seconds:
	106	+Run: `docker-compose -f docker-compose-stage1.yml up` and you should hopefully see a message like the following after a couple of seconds:
103	107
104	108	```text
105	109	IMPORTANT NOTES:
...	...	@@ -112,11 +116,13 @@ Certs gotten, and stashed away in a docker volume. Happy days!
112	116	We've got ourselves some certs so it's time to fire up haproxy and enjoy all the HAing and Proxying. Don't know about you, but I am excited.
113	117
114	118	## Dockerfile
115		-We don't need one. Becuase we're using the official image. Because we're adhering to [rule 1](/rules#love-thy-defaults).
	119	+We don't need one.
	120	+Because we're using the official image.
	121	+Because we're adhering to [rule 1](/rules#love-thy-defaults).
116	122
117	123	## haproxy.cfg
118		-Do an `mkdir haproxy/bind`.
119		-Put something like the below in `haproxy/bind/haproxy.fg`:
	124	+Do an `mkdir -p haproxy/bind`.
	125	+Put something like the below in `haproxy/bind/haproxy.cfg`:
120	126
121	127	```text
122	128	global
...	...	@@ -168,10 +174,10 @@ backend wiki
168	174	server domain2-1 container2:3000 resolvers docker check
169	175	```
170	176
171		-What's going on here?
	177	+What's going on here then?
172	178	1. The global section logs everything to stdout, because that's what you do with docker. [rule 6](/rules#thou-shalt-respect-the-sanctity-of-stdout) does not apply in dockerland.
173	179	2. We're setting the Mozilla recommended ciphers and DH values. Check the [current recommendations](https://mozilla.github.io/server-side-tls/ssl-config-generator/) if you're foolish enough to go into production with this stuff.
174		-3. We're using 'resolvers' and 'default-server init-addr none' to get around problem of containers not being up at startup time. Docker with user-defined networks always puts a resolver at 127.0.0.11:53, and haproxy can use that to resolve container names at runtime instead of startup time.
	180	+3. We're using 'resolvers' and 'default-server init-addr none' to get around the problem of containers not being up at haproxy startup time. Docker with user-defined networks always puts a resolver at 127.0.0.11:53, and haproxy can use that to resolve container names at runtime instead of startup time.
175	181	4. We're binding to port 8080 and 8443, and setting the cert to the Let's Encrypt cert we dumped out in the previous section. The ports will be mapped back to 80 and 443 by docker later on.
176	182	5. Always redirect to https.
177	183	6. All traffic that matches the certbot [ACME](https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html) challenge protocol is directed to our letsencrypt container (to be created later).