Automatic HTTPS Certs Using GoDaddy and Gitlab APIs

Jul 23, 2019
DeploymentGoDaddySSLHTTPS

Before I start, here is some technical information about my website -

  • Hosted on Gitlab and usage its CD for automatic deployment, which is triggered on a git commit.
  • The domain name registrar is GoDaddy.
  • Let's Encrypt as the TLS certificate provider for my domain name.

Let's Encrypt provides certificates for 90 days only, so I was forced to set up the certbotcertbot every three months, then generate the certificate and manually deploy the certificates to GitLab. It was frustrating because in three months, my shell probably will not remember the command I used last time, and I was too lazy to document the process somewhere.

Then I came across a project , which was trying to do a similar thing with other domain name registrar. I thought that this is an excellent setup to adapt and modify it to suit my needs. So I decided to write my module for automatic deployment of Let's Encrypt certificates to GitLab.

How does Let's Encrypt work?

Before Let's Encrypt can generate the certificate for the domain, it requires the user to prove domain ownership. Let's Encrypt provides two methods to do this task -

  • Updating the DNS records of the domain registrar (DNS-01 challenge)
  • Adding an HTTP resource under a well-known URI on the website (HTTP-01 challenge)

Using the second method requires me to add a file on my website, and I did not want to do that. Moreover, if in the future, I decide to host some other service, which does not have a website, then this method will fail.

The first method requires that the domain configuration on the domain registrar is modified. GoDaddy provides a robust API, and it is not difficult to utilize that API to automate specific tasks. The probability of changing the registrar is quite low, at least for the next few years. It will be easier to generate the certificates for any subdomains, as all the subdomains will also be hosted under the same domain registrar.

How does the tool work?

So here are the steps:

  1. Call certbotcertbot with all the domain names
  2. Invoke GoDaddy API to update the DNS records as indicated by Certbot
  3. Wait for 10 minutes for DNS changes to propagate
  4. Let certbotcertbot verify the DNS changes
  5. Use the GitLab API to deploy the generated certificates to GitLab pages

Invoking certbot

In step 1, we call certbotcertbot will the preferred method DNS. We also need to supply an email id (used by Let's Encrypt to notify domain expiration). The EMAIL_IDEMAIL_ID environment variable can be used to store the email id of the user.

Certbot runs in an interactive mode by default. It is not desired in a scripted environment. Certbot also provides mechanisms to deploy the certificates to a local server automatically, but as we are hosting our website on Gitlab, we do not want the automatic deployment facility. So we need to invoke the certbotcertbot command with --manual--manual and certonlycertonly modes.

certbot   --manual \
          --preferred-challenges dns \
          --agree-tos \
          --email "${EMAIL_ID}" \
          --no-eff-email \
          --expand \
          --renew-by-default \
          --manual-public-ip-logging-ok \
          --noninteractive \
          --redirect \
          --config-dir ${DIR}/generated/config \
          --work-dir ${DIR}/generated/work \
          --logs-dir ${DIR}/generated/logs \
          --manual-auth-hook ${DIR}/auth_hook.sh \
          -d yashagarwal.in \
          certonly
certbot   --manual \
          --preferred-challenges dns \
          --agree-tos \
          --email "${EMAIL_ID}" \
          --no-eff-email \
          --expand \
          --renew-by-default \
          --manual-public-ip-logging-ok \
          --noninteractive \
          --redirect \
          --config-dir ${DIR}/generated/config \
          --work-dir ${DIR}/generated/work \
          --logs-dir ${DIR}/generated/logs \
          --manual-auth-hook ${DIR}/auth_hook.sh \
          -d yashagarwal.in \
          certonly

The explanation for most of the flags used in the above command can be found by running the following command -

certbot --help
certbot --help

The --manual-auth-hook--manual-auth-hook flag is worth looking. This hook provides a mechanism to specify the executable, which can be used to facilitate domain ownership validation. In this case, the hook points to a script auth_hook.shauth_hook.sh, which then calls a Go client, which interacts with GoDaddy API.

Adding DNS entry to GoDaddy DNS manager

Certbot supplies two environment variables CERTBOT_DOMAINCERTBOT_DOMAIN, which contains the domain name to be verified and CERTBOT_VALIDATIONCERTBOT_VALIDATION, which includes a random string corresponding to _acme-challenge TXT_acme-challenge TXT entry. What this means is that, if I have

CERTBOT_DOMAIN=yashagarwal.in
CERTBOT_VALIDATION=6VNg5kDVI_BF1S9N5s74LTBHQnwDpQqKlblKRjIzBwM
CERTBOT_DOMAIN=yashagarwal.in
CERTBOT_VALIDATION=6VNg5kDVI_BF1S9N5s74LTBHQnwDpQqKlblKRjIzBwM

Then the DNS manager should contain a TXT entry _acme-challenge.yashagarwal.in_acme-challenge.yashagarwal.in with the value of 6VNg5kDVI_BF1S9N5s74LTBHQnwDpQqKlblKRjIzBwM6VNg5kDVI_BF1S9N5s74LTBHQnwDpQqKlblKRjIzBwM.

The auth_hook.shauth_hook.sh file calls the Go client with the abovementioned environment variables. The relevant code can be found here .

Once all the DNS entries are added, the auth_hook.shauth_hook.sh script will sleep for 10 minutes. It is to allow DNS changes to propagate throughout the Internet. It is a random duration as I could not find any GoDaddy support page mentioning the exact period used by them.

Generation of certificates

Once the auth_hook.shauth_hook.sh script returns successfully, certbotcertbot will verify the DNS records. If the verification is successful, certbotcertbot will generate the certificates in ./generated/config/live/{CERTBOT_DOMAIN}./generated/config/live/{CERTBOT_DOMAIN} directory.

Deploying the certificates to GitLab

I use the following command to deploy the certificates to Gitlab pages where my website is hosted -

curl  -vvv \
      --request PUT \
      --header "Private-Token:${GITLAB_TOKEN}" \
      --form "certificate=@${key_dir}/fullchain.pem" \
      --form "key=@${key_dir}/privkey.pem" \ "https://gitlab.com/api/v4/projects/yashhere%2Fyashhere.gitlab.io/pages/domains/yashagarwal.in"
curl  -vvv \
      --request PUT \
      --header "Private-Token:${GITLAB_TOKEN}" \
      --form "certificate=@${key_dir}/fullchain.pem" \
      --form "key=@${key_dir}/privkey.pem" \ "https://gitlab.com/api/v4/projects/yashhere%2Fyashhere.gitlab.io/pages/domains/yashagarwal.in"

where

key_dir="./generated/config/live/yashagarwal.in"
key_dir="./generated/config/live/yashagarwal.in"

Moreover, GITLAB_TOKENGITLAB_TOKEN is an environment variable that contains the API token generated from the Gitlab settings page.

Automatic Deployment using Travis CI

It is not automation if I have to run this script manually every three months. So I created a Travis CI job to automate this process. The job will run every month and deploy my certificates automatically. It has been four months, and I have not faced any issues with this setup.

The code for this post can be viewed at Github .

Thanks for reading. Cheers 😄

Share this article on Twitter