How to setup Trusted TLS Development Environments


by Filip Skokan / March 3rd 2018

In the ProSiebenSat.1 Media group a lot of software is written by different development teams, but all of our tech teams face very similar issues. A prime example of this is having to deal with local secure development environments and the changes to .dev domains in the Chrome 63 release. This happened during the development of our Single Sign On solution following the OpenID Connect standard – there has always been a limit to what a local, non-secure http setup can deliver. Running a development environment on http is only workable until you get a first Implicit(-flow) or Hybrid(-flow) Relying Party, at which point a certified solution cannot allow non-secure callback URIs. It's always been a thorn in our side but we learned to deal with it and just commented out the validations locally. However, the recent changes to Chrome have motivated us to actually improve our local setups and remove this limitation completely, ending up with a solution that allows our team members to run a local service with a valid trusted TLS certificate using a shared set of credentials.

There you are on the evening of December 5, 2017, working on your project. All of your services are running locally and you're accessing them through Google Chrome. Your environment is set up to use an address such as http://thenextbigthing.dev (using pow or via host entries). You save your work and call it a night. You'll finish tomorrow.

Little did you know that on December 6, 2017 Google Chrome 63 would roll out and break everything for you. You navigate to http://thenextbigthing.dev and get automatically redirected to https:// instead. This obviously has devastating consequences since you don't have a server listening on port 443 and even if you did, you don't have a valid certificate for the domain. What's going on?

Proposed as part of the gTLD expansion program, .dev, along with about 100 other domains, has belonged to Google for quite a while now and clearly they must have some plans for it. The culprit is this tiny commit that is included in Chrome 63 that forces all requests for domains ending with .dev to be redirected to HTTPS via a preloaded HTTP Strict Transport Security (HSTS) header. You can’t disable this behavior, there's no workaround. Well, you could not use Chrome for development, but really, what else is there? The easiest way out is to change your setup to use a different gTLD, preferably a protected one such as .localhost, .test, or .example

Or you could forget about this non-secure nonsense, embrace today's available tools and bring your local setup closer to production by furnishing your website and APIs with a valid trusted SSL certificate.

Here's what you'll need to do this:

  • Caddy web server
  • A domain managed through Route53, at least for the purposes of this post.
  • Access to AWS Console for an IAM user and role setup

Domain Registration

You can either register a domain through Route53 Registration or point an existing domain's DNS to Route53 if you have one available. While there, set the following three DNS Records for the domain:

enter image description here

After the registration or domain transfer is complete, take note of its Hosted Zone ID for the next steps.

IAM User & Role

We'll set up a specific IAM User with restricted privileges as the only one who can manage this domain's records. Do this from the AWS Console IAM Home. Add a user with a descriptive name such as dev-caddyserver and its Access type to Programmatic access, so that you get an AWS_ACCESS_KEY_ID and an AWS_SECRET_ACCESS_KEY at the end of the process.

Next up are user permissions. Choose 'Attach existing policies directly' and then 'Create policy'. Use this JSON policy definition and replace with the one you got earlier, give your policy a name, a description if you want, and create it.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "route53:ChangeResourceRecordSets",
      "Resource": "arn:aws:route53:::hostedzone/<Hosted Zone ID>"
    },
    {
      "Effect": "Allow",
      "Action": [

        "route53:GetChange",
        "route53:ListHostedZonesByName"
      ],
      "Resource": "*"
    }
  ]
}

This policy allows the user to manage DNS Records for your domain, query the state of the specific change it is making, and list all hosted zones by name; no harm is done to your other Route53-managed domains.

Attach the newly created policy to your IAM User and create it. Once created, you’ll get an AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Take note of those now too, you’ll never see them again!

Next up, you'll want to store your credentials in a place where all AWS SDKs look for them. This insures that the folder and files exist (without replacing existing ones), and appends your credentials to the specific profile we'll be using.

$ mkdir -p ~/.aws
$ touch ~/.aws/credentials
$ echo "
[dev-caddyserver]
aws_access_key_id=<AWS_ACCESS_KEY_ID>
aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>" >> ~/.aws/credentials

That's about it for the AWS part.

Installing Caddy

You'll need Go to install Caddy and you'll need to include a specific plugin for seamless certificate management. Unfortunately the available homebrew recipe does not allow you to specify the list of plugins you wish to include but there's a simple-to-use one-liner for installation provided by Caddy.

$ brew install go
$ curl https://getcaddy.com | bash -s personal tls.dns.route53

Setting up the first website

The following makes sure the Caddy folder structure is set up and that the first website configuration file is there. Replace p7s1.cool with the domain you've registered.

$ mkdir -p ~/.caddy/sites
$ cd ~/.caddy
$ echo "import sites/test.conf" >> Caddyfile
$ echo "https://test.p7s1.cool {
  tls {
    dns route53
  }

  proxy / localhost:9292 {
    transparent
  }
}" > "sites/test.conf"

Let's run a simple webserver on port 9292 with this one-liner (preferably in a new terminal tab).

$ node -e "require('http').createServer(function (req, res) { res.end('OK CADDY'); }).listen(9292);"

Start Caddy

sudo AWS_PROFILE=dev-caddyserver caddy --conf ~/.caddy/Caddyfile

The first time Caddy is started it will prompt you for your email address but you can just leave it empty. Caddy will now attempt to retrieve certificates from Let's Encrypt and will retain them for 60 days. It will also automatically renew the certificates when necessary.

Activating privacy features... done.
https://test.p7s1.cool
http://test.p7s1.cool

Open your broken .dev Chrome and navigate to http://test.p7s1.cool. You should be automatically redirected to the secure website and see a valid trusted certificate being presented to Chrome now.

How the ... ?

Caddy is using the tls.dns.route53 plugin to push DNS entries that Let's Encrypt is looking for when confirming the domain ownership. By default this would be done using a challenge exposed via http but your .dev host is not really accessible to them. Hence the DNS verification.

Other providers are available

There are alternative DNS providers and plugins to choose from, should the monthly 50¢ Route53 DNS management charge be an issue. I haven't tested them myself but I'm sure you will be able to figure them out if you follow their documentation. At the time of writing these are

  • tls.dns.azure
  • tls.dns.cloudflare
  • tls.dns.digitalocean
  • tls.dns.dnsimple
  • tls.dns.dnspod
  • tls.dns.dyn
  • tls.dns.exoscale
  • tls.dns.gandi
  • tls.dns.googlecloud
  • tls.dns.linode
  • tls.dns.namecheap
  • tls.dns.ovh
  • tls.dns.rackspace
  • tls.dns.rfc2136
  • tls.dns.route53
  • tls.dns.vultr

And you can read up and inspect the source for each here.

Trust your TLS offloading Caddy

Be sure to consult the documentation of your web application frameworks to ensure they take the proxy-added headers (e.g. X-Real-IP, X-Forwarded-For, X-Forwarded-Proto) into consideration when resolving current URLs and similar. This varies from framework to framework. A few examples for popular NodeJS frameworks

// Koa http://koajs.com
app.proxy = true;

// Express https://expressjs.com
app.enable('trust proxy');

MORE BLOG POSTS