Simple AWS API Gateway Setup

My new iOS-based smart mirror uses the OpenWeatherMap and MBTA APIs, just like the old Magic Mirror did. The initial version had you enter in the API keys directly into the app. I wanted to look into distributing the app (or derivatives of it) through the App Store, and I wasn’t sure this would make it though, plus it is an annoying burden to the user.

The question is where to put the key. I could hard code it into the app, possibly with some obfuscation, but it wouldn’t be hard to extract it or to sniff the key from the HTTP/HTTPS request. Since OpenWeatherMap charges based on use above a certain limit, it would be pretty bad if my key were stolen.

The only safe way I could find to store the keys was to put them on some kind of HTTP proxy, also known as an API gateway. This ensures that the keys are stored on a server under my control, and that no one can get direct access to them. This isn’t perfect, since my URL could still be sniffed and used by third parties directly, but at least the keys are safe. I could also add extra capabilities like rate-limit access, limiting excessive use of the API for all users, including malicious ones. I could go even further and set up authentication, but I didn’t want users to have to make an account for a simple dashboard like this.

Amazon has one a well-documented API Gateway with HTTP proxy support, so I decided to use that. I thought this would be easy, and it is — once you figure out all the arcane web backend terminology you may have never encountered before, which was the situation I was in.

This guide is rudimentary, but should explain how to set up the most basic gateway, especially if you’ve only started dabbling with creating and accessing REST APIs like me. This doesn’t 100% protect your API from abuse, but it does allow you to keep control of the API key, and provides the basis for adding extra layers of control to your API in the future.

Creating an AWS Account

The first step is to make an account. AWS charges for access, but it’s very cheap, like $1/month for 333,000 accesses. If I ever get enough traffic where AWS is a significant cost, I'll be doing very well indeed.

This process is pretty straight forward, so I won't cover it here. You’ll need to provide a credit card for payment, but you get a year of a number of services free, so you won’t be charged for anything for quite some time.

Setting up an API Gateway

This is the part that took me a while to figure out.

Here’s what I wanted to do: The input (from my app) would be to call a URL that takes the latitude and longitude for a location as query string arguments (something like “/weather?lat=12&lon=34”). This call would not include an API key, just the coordinates. The gateway would add the API key before passing the request on to the OpenWeatherMap API, also known as the “endpoint”. OpenWeatherMap would then return the JSON data to AWS, which would then be passed through to my app for parsing.

There’s a lot more you can do in an API gateway, including authentication (for user logins), lambda functions (parsing/processing the data both from the source app and as returned from the endpoint before sending it back to your app), access controls, etc. These all cost extra and are overkill for my simple task fo protecting my API keys.

First, we create the API itself:

• From the API Gateway part of the Amazon web console, go to API and click Create API.

• Choose REST as your API type (the one that is public, not the one that only works from VPCs).

• Select New API to create an API from scratch.

• Enter an API name, like “weather”

• Enter a description of your API for your own use.

• Choose an Endpoint Type. Regional seems to be what most people need.

• Click Create.

Your API is now created. It doesn’t do anything yet, but it exists.

You now need to add Resources, which are paths in your API. You can just use the root path of “/”, or you can have a sub-path like ”/weather”. Use the Actions button and Create Resource if you want to add a sub-path. You can do complicated things here like setting up proxies, which allows one method to handy a procedurally-defined hierarchy of paths, but that’s not something I needed here.

Setting up a Method

One you have a Resource (either the default one or one you created), you next need to add a Method to it from the Action menu. This is how you set up the actual work that you want the URL to do depending on the kind of HTTP request it received. Since we’re passing through to OpenWeatherMap which uses HTTP GET, I wanted a GET method too. After you’ve created a method choosing Create Method, you click the popup on the newly-created method entry, choose GET, and then click the checkmark button to actually add it.

The GET - Setup page lets you configuration for the new method:

• HTTP Proxy Integration isn’t the kind of proxy I was looking for (that’s for handling a bunch of child URLs from the same handler), so I left that unchecked.

• Endpoint URL is the target you’re eventually going to redirect to, which in my case is the one-call method for OpenWeatherMap, https://api.openweathermap.org/data/2.5/onecall.

• I left Content Handling on Passthrough, meaning whatever HTML body is provided will be passed to the API will go directly to the endpoint URL unmodified.

• I used the default timeout for the endpoint URL to respond.

Click Save, and you are presented with a kind of flowchart. We are interested in two things here, the Method Request and the Integration Request.

The Method Request is where you define the arguments to your API. These are arguments that you’ll pass to the API from your app. Defining them this way lets the gateway do some validation before passing it on to the endpoint. I needed URL Query String Parameters that would be passed as part of the URL, specifically, “lat” and “lon”.

To add these, just click Add Query string, type in the name and click the check to create it. I flagged both as Required so that the request would fail if they weren’t set. Caching might be useful in the future (this caches responses for the same URL with the same arguments to avoid hitting the endpoint too often), but that’s a paid service and isn’t something I need just yet. I left all the other options blank.

Next is the Integration Request, which you can get to from that flowchart page. This is where you take the data sent to our URL by tour app and modify it for the endpoint. The default options are already set and should be correct. The only thing you have to modify is the URL Query String Parameters You should already see the “lat” and “lon” parameters, which were automatically added from your Method Request definition.

We need to add OpenWeatherMap API key as well. For this, you click Add Query String link, type in “appid” for the name, and paste your API key in from the OpenWeatherMap site, wrapping it in single quotes. This last step is important, and took me a while to figure out, finally stumbling across a bit of docs that explained how to set a static string almost in passing. It seems the field is commonly resolved from an API call, such as the one that sets “lat” and “lon” from the Method Request state, but it is possible to use single quotes to pass in a predefined string.

And that’s it — we’re done with defining our method. We don’t care about the Method Response or Integration Response. These allow you to transform the response from the endpoint before sending it to the app, but we don’t need to do anything here and are just going to let the app handled the unmodified response directly.

You can go back up and click on the Test block to try the method out. Enter some query string arguments, like “lat=20&lon=40”, then click Test. if all goes well, you’ll see a JSON response from OpenWeatherMap.

Deployment and Staging

To use the API publicly, you have to deploy it and stage it. To deploy it, click the Actions popup, then click Deploy API.

You’ll be asked to select a stage. Stages are for things like development vs beta vs production, and can represent different versions of the API for different phases of development. You need at least one stage, so just create one here with whatever name you like and click Deploy.

Throttling is on by default in your new stage. Since OpenWeatherMap has a low threshold on the free plan, I set the throttling to a fairly low number, but you can probably leave it alone if you want. I didn’t change any other settings.

Notice the blue header at the top of the stage page that shows the Invoke URL. This is the URL that has your API, and what your app will call to use it. You can enter it into a web browser with the “lat” and “lon” arguments to test your new API out.

Associating with a Domain Name

I wanted to create a sub-domain from my website to get to the API. This should have been easy, but was slightly complicated.

I use Hover as my registrar. It was fairly simple to add a new CNAME that pointed to my Invoke URL. The thing is, it didn’t actually work, and would return “forbidden” when I tried to access it.

Before the CNAME will work, you have to authorize the domain from the AWS API Gateway, and that requires a certificate. Thankfully, Amazon provides a fairly simple way to create the certificate for you.

First go to the Custom Domain Names button at the left of the API Gateway console. Then hit the Create button on the Custom Domain Names page.

From here you can enter a domain for your API, such as “myapi.myWebSite.com”. You’ll need an existing domain for what I’m describing, with the CNAME for the subdomain already configured at your registrar.

I left the TLS settings alone, since they seemed find. You will need to do the endpoint configuration, though. . Here is where you provide a certificate from Amazon. Click the Create ACM Certificate link to make one. From the Certificates page, click Request a Certificate, and from the next page leave it at Request A Public Certificate and click the new Request a Certificate button.

On the new page, you once again enter the domain (“myapi.myWebSite.com”), then click “Next”.

Amazon will ask how you want to authenticate. I choose CNAME, which requires you to add another CNAME at your registrar for your site. This will ensure that you own the domain. You can also do email authentication, which I didn’t try. That uses the email address associated with your domain registration, which Amazon will look up from the registrar; you don’t provide an email address directly.

It will take half an hour or so for the CNAME to propagate and for Amazon to verify it and issue the certification. Once it’s ready, you can go back to the Create Domain Name page and choose your certificate from the popup. You an also delete the verification CNAME entries from your registrar once you have the certificate.

You’ll then choose a stage and base path for the API you want to associate with the domain. For example, you could choose “MyBasePath” as your base path, and then you’d access your API from “myapi.myWebSite.com/MyBasePath”, passing query string parameters immediately after. You don’t have to enter the name of the stage or API, just the base path. Or you can leave the base path blank and query directly from the subdomain’s URL directly, if you prefer.

And that’s it — you should now have a fully functional API gateway ready to use, with your keys safely protected from prying eyes, and with only a small cost associated with it. This first time took me a couple of days of looking through docs and trying things until I finally figured out what subset of things I needed to make it work. I can now do the whole thing in about ten minutes, now that I know what t do. Hopefully this will help anyone else trying to set up something similar