Running certbot under podman

Follow docker instructions but remember your SELinux tagging

After making a new static site for an app of mine I wanted to add HTTPS to the site, a step which all sites should totally do these days. LetsEncrypt is a great source for certificates at an excellent price point (free) and trust across lots of browsers. The EFF even has a tool called certbot to make generating those certs easier.

I'm a fan of containerized applications where I can, and running Fedora I'm moving from docker to podman. Following the directions at https://certbot.eff.org/docs/install.html#running-with-docker I started off by making some directories on my laptop to hold the certificates that come out of certbot.

mkdir etc-output
mkdir var-output

Then, I ran the command telling certbot to go make some certs. Immediately, I saw an access denied error. Using --entrypoing /bin/sh to investigate in the container, it was clear that while the volumes got mounted, processes in the container didn't have any permission to access them, either reading or writing.

I spent a little while making sure that the user ID mapping was correct in podman. Any time that I'm using rootless invocations of podman, that's where I generally start. For this container, the user in the container is root (id=0) which maps cleanly to my unprivileged user outside of the container, so that wasn't it. The next thing I've had difficuly with in podman volumes is SELinux.

SELinux has a lot of feelings and restrictions around which binaries can do things on the filesystem, and reasonably so. Enforcing SELinux on your machine can make it to where your webserver doesn't surprise-edit a bunch of files outside of its webroot, eliminating an entire class of exploits. While tempting to just turn it off, SELinux does enough good that it's worth getting your application to work right with it. I'm not super sure why docker doesn't have quite as many fights with SELinux, but it really seems to affect podman more. Something something unprivileged filesystem access probably.

https://www.redhat.com/sysadmin/user-namespaces-selinux-rootless-containers was probably the best resource I found when learning about both the UID mapping parts of podman as well as the SELinux changes. From that, I learned about the magic :Z flag which makes SELinux problems go away. It tells SELinux that the directories you're attaching are intended to be used by a container.

Incorporating that into the command gave me this, which worked.

podman run --rm -it -v "$(pwd)/etc-output:/etc/letsencrypt:Z" -v "$(pwd)/var-output:/var/lib/letsencrypt:Z"  certbot/certbot certonly --manual -d somedomain.com -d www.somedomain.com

I was able to follow the directions in the app to generate the certificates, then upload them to my server. Looks great now. Here is the video of the stream where I fought with this discovery.