In this post we’re going to create a cheap, zero-admin API on Amazon AWS which accesses a Lambda function to check if an URL is valid and is made public available via an Amazon S3 hosted static webpage. Before we start, let's get talk a bit about the core concepts.
Core concepts within Amazon AWS
Amazon API Gateway
This July, Amazon released Amazon API Gateway, a managed service that allows developers to build easy scalable APIs. AWS enables APIs to be created that can act as a 'front door' to access data, business logic, or functionality from backend services, such as applications running on Amazon EC2 or code running on AWS Lambda. In order to successfully host an API backend there must be a supporting infrastructure, which provides security, manages traffic, implements monitoring and provides other essential foundational services. Amazon API Gateway provides this infrastructure, and handles the tasks involved in accepting and processing up to 'hundreds of thousands' of concurrent API calls, including traffic management, authorisation and access control, monitoring, and API version management.
Amazon Lambda
A bit earlier, in April, Amazon launched AWS Lambda as a zero-admin compute platform. You don’t have to run Amazon Elastic Compute Cloud(EC2) instances, think about scale, or worry about fault tolerance. You simply create a Lambda function (using Java or Node.js) and connect the function to your AWS resources in this case your Amazon API.
S3
Amazon Simple Storage Service is Amazons alternative for cloud storage. Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data. You can access S3 directly or via 1 of the other Amazon services.
How we need to tie them together, we’ll explain in the diagram below.
Which illustrates the following steps
1. First the client loads the webpage from the s3 bucket
2. Types an url to validate and clicks submit
3. The data is posted to the API Gateway
4. The API Gateway passes the information through to the Lambda function
5. The lambda functions does his magic and returns a result to the API Gateway
6. The API gateway passes the result back to the client
So let's get started!
These are our steps:
Create API Gateway API
Create a basic Lambda function
Connect the API to the Lambda Function
Deploy the API
Test the Basic API
Extend the API
Create, upload and test a URL validation Lambda Function in Node.js
Build a static front-end
Enable Cross-origin resource sharing (CORS)
1. Create API Gateway API
Login in to amazon API Gateway via the console. The first time you will get a welcome message, otherwise you’ll get the API console homepage
Create a new API by clicking the “Create API” button
a. Fill in a name and description for the API and click “Create API”. You will get the following screen:
b. We are going to create a new method, by clicking “Create Method”.
c. Select “Post” in the dropdown, since we are going to be posting url’s to this service from a static website. You will get the following screen:
d. Select Lambda Function for “Integration type” and a “Lambda region”, since we didn’t create any lambda functions before we will get the following alert:
We're ready to create our first Lambda function
2. Create a basic Lambda function
Click “Create a Lambda Function” in the alert
Click “Skip” in the “Select blueprint” screen
New function
a. Type a name, description and pick Node.js as the runtime
b. Select “edit code inline” for now and past the following code, this will return the text “hello world” if you call the Lambda function:
c. Keep index.handler as the handler
d. For Role we are going to create a new “* Basic execution role”, this will show the following screen, Leave the settings as they are for now and click “Allow”
e. Keep the advanced settings as they are, click next, review the settings and click “Create function”
Test your function, by clicking “Test”, leave template as is and click “submit”, the output under “Execution result” will show {“Hello “: “World”}
3. Connect the API to the Lambda Function
Return to API gateway
In Lambda Function start typing the name of the Lambda function you created, and select it in the dropdown and click “Save”.
Click “Ok” in the “Add Permission to Lambda Function” modal
Click the method “Post” under “/”, you’ll get the following screen:
Test the API by clicking “Test” above the client-box in the “Method execution” screen. Click “Test” again and the response body will show {“Hello “: “World”}
4. Deploy the API
In the resources screen, click “Deploy API”
For Deployment stage, choose New Stage and type a name.
Click Deploy.
Copy the url
5. Test the Basic API
Use a tool, I use Postman for chrome, to create a post request
https://[my-api-id].execute-api.[region-id].amazonaws.com/[api-name]
Set the header to Content-Type: application/json
Send the request, again the result should be { "Hello": "World"}
We created the skeleton of our Amazon API Gateway. Now we we’re ready to build our URL validator
5. Create a more advanced Lambda Function in Node.js
Create the node-js app in a directory on your computer, let’s say “urlvalidator”
We use valid-url (https://www.npmjs.com/package/valid-url) as a the module to validate if the supplied url is valid.Install by “npm install valid-url” in the directory
Create a file index.js
Copy or type the following text in index.js:
5. Package all the contents of the directory (index.js + node_modules) into a zip-file
6. Go back to Amazon Lambda
7. Select the previous created Lambda Function, you will get this screen:
8. Select “Upload a .ZIP file”, select the zip and click “Save”
9. In the dropdown “Actions” select “configure sample event”
10. Type { "url":”http://www.bluemango.nl “} as a sample and click “Submit”, the result should be "Looks like an URI"
11. Now change the sample event to { "url":”http://www.blueman|go.nl“} , the result should be “not a valid URI”
12. The Lambda function now validates if the url contains valid characters (Note: there’s a lot more room for improvement in the validation)
13. Go back to Amazon API Gateway
14. Select the previous created API and the post method
15. Click “Test” above client-box in the Method Execution Screen
16. Type {"url":"http://www.bluemango.nl" } in the Request Body, and click “Test”,the result in the Response Body should be "Looks like an URI"
17. You can test the API again with the tool to create a post request. (Note: the body should be raw, json and have the following format {"url":"http://www.bluemango.nl"})
18. Your API is now ready to use
6. Build a static front-end
Build a static page to serve as a input form to the API
1. Create an index.html with a form and a jquery function to post to the API
2. Code for the Post
3.Upload the page to S3
4. Go to https://console.aws.amazon.com/s3/.
5. Create a new bucket, I choose url-validator.
6. Go to the properties of the bucket.
7. Click Enable Static Website Hosting and check “Enable website hosting”.
8. Type “index.html” for the Index Document, click “Save”.
9. upload the index,html to to bucket.
7. Enable Cross-origin resource sharing (CORS)
We need to enable CORS before we can call the webservice from the static index.html page
1. Go to amazon web api, click the URL-Validator API
2. Click the Method Response-box, Click Create Method, select “Option” in the dropdown
3. Select “Lambda Function” and select the same Function as before, click Save
4. Open the HTTP Status 200
5. Add three headers, “Access-Control-Allow-Headers”, “Access-Control-Allow-Methods”, “Access-Control-Allow-Origin”.
6. Go back to “Method Execution” and click the “Integration Response” box
7. Open the Method response status 200 and open the Header Mappings
8. For Access-Control-Allow-Headers type “'Content-Type,X-Amz-Date,Authorization'”, for Access-Control-Allow-Methods type “'POST'”, for Access-Control-Allow-Origin type “' * '” (Note: be aware for the single qoutes)
9. Now we need to do the same for the Post method, so click the Method Response-box, go to the POST method and repeat step 3 through 8
10. Deploy the API again
11. Go to the url of your s3 bucket, fill in the form and click “Submit”
12. Success!
Conclusion
The Amazon API Gateway shows great promise, it does what it states, it's a gateway to their other services. The API is very scalable it can handle a million calls very smoothly. But for now just use it if you need a quick, easy API for a prototype. Don't immediately rewrite/rebuild your SaaS to use the Amazon API Gateway. It lacks workflow and testing functionality, development is only in Node or Java and setting up is still a lot of work (things like authorisation, headers, CORS is rather tedious).
Note: The running cost are very doable for. Running the example we build: it would cost us approx. $4.00 a month for a million calls