How to Build a Serverless Backend with AWS Lambda and Nodejs

2018-11-13

Serverless architecture is a cloud computing execution model where a cloud provider like AWS, Azure or Google Cloud is used to deploy backend or server-side code. In comparison to traditionally deployed web applications, in serverless architecture, the developer does not has to maintain the servers or the infrastructure. They only have to pay a subscription to the third party vendor whereas the vendor is responsible to handle the operation of the backend logic of a server along with scalability, reliability, and security.

There are two ways a serverless architecture can be implemented in order to deploy your server-side code. First one is Backend as a Service or BaaS. A good example of this is Firebase which you can often see in conjunction between a web or a mobile application to a database or providing user authentication.

What we are going to focus in this article is called Function as a Service or FaaS. With FaaS, the server code is run inside containers that are usually triggered by common events such as HTTP requests from the client, database operations, file uploads, scheduled events and so on. The code on the cloud provider that is deployed and getting executed is in the form of a function.

In FaaS, these functions are deployed in modular form. One function corresponds to each operation, thus eliminating the rest of the code and time spent on writing boilerplate code for setting up a server and data models. These modular functions can further be scaled automatically and independently. This way, more time can be spent on writing the logic of the application that a potential user is going to interact with. You do not have to scale for the entire application and pay for it. Common use cases of FaaS so far have been implemented are scheduled tasks (or cron jobs), automation, web applications, and chatbots.

Common FaaS service platform providers are:

In the following tutorial, we are going to create a demo to deploy on a serverless infrastructure provider such as AWS Lambda.

What is AWS Lambda?

In order to build and deploy a backend function to handle a certain operation, I am going to start with setting up the service provider you are going to use to follow this article. AWS Lambda supports different runtimes such as Node.js, Java, Python, .NET Core and Go for you to execute a function.

The function runs inside a container with a 64-bit Amazon Linux AMI. You might be thinking, ‘why I am telling you all of this?’ Well, using serverless for the first time can be a bit overwhelming and if you know what you are getting in return, that’s always good! More geeky stuff is listed below.

  • Memory: 128MB — 3008MB
  • Ephemeral disk space: 512MB
  • Max execution duration: 300 seconds
  • Compressed package size: 50MB
  • Uncompressed package size: 250MB

The execution duration here means that your Lambda function can only run a maximum of 5 minutes. This does mean that it is not meant for running longer processes. The disk space is the form of a temporary storage. The package size refers to the code necessary to trigger the server function. In case of Node.js, this does mean that any dependencies that are being imported into our server (for example, node_modules/ directory).

A typical lambda function in a Node.js server will look like below.

In the above syntax, handlerFunction is the name of our Lambda function. The event object contains information about the event that triggers the lambda function on execution. The context object contains information about the runtime. Rest of the code is written inside the Lambda function and at last a callback is invoked with an error object and result object. We will learn more about these objects later when are going to implement them.

Setting up AWS Lambda

In order to setup a Lambda function on AWS, we need to first register an account for the access keys. Use your credentials to login or signup a new account on console.amazon.com and once you are through the verification process you will be welcomed by the following screen.

To get the keys and permissions in order to deploy a function, we have to switch to Identity and Access Management (IAM). Then go to Users tab from the left hand sidebar and click on the button Add user. Fill in the details in the below form and do enable Access Type > Programmatic Access.

Then on the next page, select Attach Existing Policies Directly and then select a policy name AdministratorAccess.

Click Next: Review button and then click Create User button when displayed. Proceeding to the next step you will see the user was created. Now, and only now, will you have access to the users Access Key ID and Secret Access Key. This information is unique for every user you create.

Creating a Serverless Function

We are going to use install an npm dependency first to proceed and scaffold a new project. Open up your terminal and install the following.

1npm install -g serverless

Once installed, we can run the serverless framework in the terminal by running the command:

1serverless

Or use the shorthand sls for serverless. This command will display all the available commands that come with the serverless framework.

After installing the serverless dependency as a global package, you are ready to create your first function. To start, you will need to configure your AWS registered user credentials. AWS gives you a link to download access keys when creating a user.

You can also visit your username and visit Security Credentials like below.

Now let us configure AWS with the serverless package.

1sls config credentials --provider aws --key ACCESS_KEY --secret SECRET_KEY

If the above command runs successfully you will get a success message like below

The good thing about using serverless npm package is that it comes with pre-defined templates that you can create in your project using a command and also creates a basic configuration for us that is required to deploy our Lambda function. To get started, I am going to use aws-nodejs template inside a new directory.

1sls create -t aws-nodejs -p aws-serverless-demo && cd aws-serverless-demo

The -p flag will create a new directory with name aws-serverless-demo. The -t flag uses the pre-defined boilerplate. The result of this will create three new files in your project directory.

  • Usual .gitignore
  • handler.js where we will write our handle function
  • serverless.yml contains the configuration

The default handler file looks like below.

1'use strict'
2
3module.exports.hello = async (event, context) => {
4 return {
5 statusCode: 200,
6 body: JSON.stringify({
7 message: 'Go Serverless v1.0! Your function executed successfully!',
8 input: event
9 })
10 }
11
12 // Use this code if you don't use the http event with the LAMBDA-PROXY integration
13 // return { message: 'Go Serverless v1.0! Your function executed successfully!', event };
14}

In the above file, hello is the function that has two parameters: event, and context. module.exports is basic Nodes syntax as well as the rest of the code. You can clearly see it also supports ES6 features. An event is an object that contains all the necessary request data. The context object contains AWS-specific values. We have already discussed it before. Let us modify this function to our needs and add a third parameter called thecallback. Open handler.js file and edit the hello function.

1'use strict'
2
3module.exports.hello = (event, context, callback) => {
4 console.log('Hello World')
5 callback(null, 'Hello World')
6}

The callback function must be invoked with an error response as the first argument, in our case it is null right now or a valid response as the second argument which is currently sending a simple Hello World message. We can now deploy this handler function using the command below from your terminal window.

1sls deploy

It will take a few minutes to finish the process. Our serverless function gets packed into a .zip file. Take a notice at the Service Information below. It contains all the information what endpoints are available, what is our function, where it is deployed and so on.

You can try the invoke attribute like following to run the function and see the result.

1sls invoke --function hello

The output will look like below.

Take a look at the configuration in serverless.yml.

1# Welcome to Serverless!
2#
3# This file is the main config file for your service.
4# It's very minimal at this point and uses default values.
5# You can always add more config options for more control.
6# We've included some commented out config examples here.
7# Just uncomment any of them to get that config option.
8#
9# For full config options, check the docs:
10# docs.serverless.com
11#
12# Happy Coding!
13
14service: aws-nodejs # NOTE: update this with your service name
15
16# You can pin your service to only deploy with a specific Serverless version
17# Check out our docs for more details
18# frameworkVersion: "=X.X.X"
19
20provider:
21 name: aws
22 runtime: nodejs8.10
23
24# you can overwrite defaults here
25# stage: dev
26# region: us-east-1
27
28# you can add statements to the Lambda function's IAM Role here
29# iamRoleStatements:
30# - Effect: "Allow"
31# Action:
32# - "s3:ListBucket"
33# Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ] }
34# - Effect: "Allow"
35# Action:
36# - "s3:PutObject"
37# Resource:
38# Fn::Join:
39# - ""
40# - - "arn:aws:s3:::"
41# - "Ref" : "ServerlessDeploymentBucket"
42# - "/*"
43
44# you can define service wide environment variables here
45# environment:
46# variable1: value1
47
48# you can add packaging information here
49#package:
50# include:
51# - include-me.js
52# - include-me-dir/**
53# exclude:
54# - exclude-me.js
55# - exclude-me-dir/**
56
57functions:
58 hello:
59 handler: handler.hello
60# The following are a few example events you can configure
61# NOTE: Please make sure to change your handler code to work with those events
62# Check the event documentation for details
63# events:
64# - http:
65# path: users/create
66# method: get
67# - s3: ${env:BUCKET}
68# - schedule: rate(10 minutes)
69# - sns: greeter-topic
70# - stream: arn:aws:dynamodb:region:XXXXXX:table/foo/stream/1970-01-01T00:00:00.000
71# - alexaSkill: amzn1.ask.skill.xx-xx-xx-xx
72# - alexaSmartHome: amzn1.ask.skill.xx-xx-xx-xx
73# - iot:
74# sql: "SELECT * FROM 'some_topic'"
75# - cloudwatchEvent:
76# event:
77# source:
78# - "aws.ec2"
79# detail-type:
80# - "EC2 Instance State-change Notification"
81# detail:
82# state:
83# - pending
84# - cloudwatchLog: '/aws/lambda/hello'
85# - cognitoUserPool:
86# pool: MyUserPool
87# trigger: PreSignUp
88
89# Define function environment variables here
90# environment:
91# variable2: value2
92
93# you can add CloudFormation resource templates here
94#resources:
95# Resources:
96# NewResource:
97# Type: AWS::S3::Bucket
98# Properties:
99# BucketName: my-new-bucket
100# Outputs:
101# NewOutput:
102# Description: "Description for the output"
103# Value: "Some output value"

REST API with Serverless Stack

In this part of the tutorial, I will show you how to hook up a MongoDB database as a service to a Serverless REST API. We are going to need three things that will complete our tech stack. They are:

  • AWS Lambda
  • Node.js
  • MongoDB Atlas

We already have the first two, all we need is to setup a MongoDB cloud database called Atlas. MongoDB Atlas is a database as a service developed by the team behind the MongoDB itself. Along with providing a free/paid tier for storing your data on the cloud, MongoDB Atlas provides a lot of analytics that is essential to manage and monitor your application. MongoDB Atlas does provide a free tier that we will be using with our serverless stack.

Creating a database on MongoDB Atlas

We will start by creating a database on the MongoDB Atlas. Login to the site and create an account if you do not have it already. We just need a sandbox environment to get hands-on experience so we must opt for free tier. Once you have your account set up, open up your account page and add a new organization.

Now, after entering the name, proceed further and click on Create Organization.

You will be then prompted to the main screen where you can create a new project. Type in the name of your project and proceed further.

MongoDB Atlas is secured by default. You need to set permissions before we leverage its usage in our app. You can name the database at the pointed field below.

Now, we can add our free sandbox to this project. It is called a cluster.

After all that, just add an admin user for the cluster and give him a really strong password. As you can see the price for this cluster will be $0.00 forever. Your cluster will take a few minutes to deploy. While that is underway, let us finally start writing some code.

Building the API

Next, we install all the necessary dependencies in order to create the API.

1init -y
2npm i --save mongoose dotenv

After that, we configure the serverless.yml and add the other handler functions that we need to deploy.

1# Welcome to Serverless!
2#
3# This file is the main config file for your service.
4# It's very minimal at this point and uses default values.
5# You can always add more config options for more control.
6# We've included some commented out config examples here.
7# Just uncomment any of them to get that config option.
8#
9# For full config options, check the docs:
10# docs.serverless.com
11#
12# Happy Coding!
13
14service: aws-nodejs # NOTE: update this with your service name
15
16# You can pin your service to only deploy with a specific Serverless version
17# Check out our docs for more details
18# frameworkVersion: "=X.X.X"
19
20provider:
21 name: aws
22 runtime: nodejs8.10
23
24functions:
25 hello:
26 handler: handler.hello
27 create:
28 handler: handler.create # point to exported create function in handler.js
29 events:
30 - http:
31 path: notes # path will be domain.name.com/dev/notes
32 method: post
33 cors: true
34 getOne:
35 handler: handler.getOne
36 events:
37 - http:
38 path: notes/{id} # path will be domain.name.com/dev/notes/1
39 method: get
40 cors: true
41 getAll:
42 handler: handler.getAll # path will be domain.name.com/dev/notes
43 events:
44 - http:
45 path: notes
46 method: get
47 cors: true
48 update:
49 handler: handler.update # path will be domain.name.com/dev/notes/1
50 events:
51 - http:
52 path: notes/{id}
53 method: put
54 cors: true
55 delete:
56 handler: handler.delete
57 events:
58 - http:
59 path: notes/{id} # path will be domain.name.com/dev/notes/1
60 method: delete
61 cors: true

The CRUD operations that will handle the functionalities of the REST API are going to be in the file handler.js. Each event contains the event information of the current event that will be invoked from the handler.js. In the above configuration file, we have defined each CRUD operation along with an event and the name. Also notice, when defining the events in above file, we are associating an HTTP request with a path that is going to be the endpoint of the CRUD operation in the API, the HTTP method and lastly, cors option.

I am going to demonstrate a simple Note taking app through our REST API. These CRUD operations are going to be the core of it. Since our API is going to be hosted remotely, we have to enable Cross-Origin Resource Sharing. No need to install another dependency on that. Serverless configuration file has support for it. Just specify in the events section like cors: true. By default, it is false.

Defining the Handler Functions

If you are familiar with Node.js and Express framework you will notice there is little difference in creating a controller function that leads to the business logic of a route. The similar approach we are going to use to define in each handler function.

1'use strict'
2
3module.exports.hello = (event, context, callback) => {
4 console.log('Hello World')
5 callback(null, 'Hello World')
6}
7
8module.exports.create = (event, context, callback) => {
9 context.callbackWaitsForEmptyEventLoop = false
10
11 connectToDatabase().then(() => {
12 Note.create(JSON.parse(event.body))
13 .then(note =>
14 callback(null, {
15 statusCode: 200,
16 body: JSON.stringify(note)
17 })
18 )
19 .catch(err =>
20 callback(null, {
21 statusCode: err.statusCode || 500,
22 headers: { 'Content-Type': 'text/plain' },
23 body: 'Could not create the note.'
24 })
25 )
26 })
27}
28
29module.exports.getOne = (event, context, callback) => {
30 context.callbackWaitsForEmptyEventLoop = false
31
32 connectToDatabase().then(() => {
33 Note.findById(event.pathParameters.id)
34 .then(note =>
35 callback(null, {
36 statusCode: 200,
37 body: JSON.stringify(note)
38 })
39 )
40 .catch(err =>
41 callback(null, {
42 statusCode: err.statusCode || 500,
43 headers: { 'Content-Type': 'text/plain' },
44 body: 'Could not fetch the note.'
45 })
46 )
47 })
48}
49
50module.exports.getAll = (event, context, callback) => {
51 context.callbackWaitsForEmptyEventLoop = false
52
53 connectToDatabase().then(() => {
54 Note.find()
55 .then(notes =>
56 callback(null, {
57 statusCode: 200,
58 body: JSON.stringify(notes)
59 })
60 )
61 .catch(err =>
62 callback(null, {
63 statusCode: err.statusCode || 500,
64 headers: { 'Content-Type': 'text/plain' },
65 body: 'Could not fetch the notes.'
66 })
67 )
68 })
69}
70
71module.exports.update = (event, context, callback) => {
72 context.callbackWaitsForEmptyEventLoop = false
73
74 connectToDatabase().then(() => {
75 Note.findByIdAndUpdate(event.pathParameters.id, JSON.parse(event.body), {
76 new: true
77 })
78 .then(note =>
79 callback(null, {
80 statusCode: 200,
81 body: JSON.stringify(note)
82 })
83 )
84 .catch(err =>
85 callback(null, {
86 statusCode: err.statusCode || 500,
87 headers: { 'Content-Type': 'text/plain' },
88 body: 'Could not fetch the notes.'
89 })
90 )
91 })
92}
93
94module.exports.delete = (event, context, callback) => {
95 context.callbackWaitsForEmptyEventLoop = false
96
97 connectToDatabase().then(() => {
98 Note.findByIdAndRemove(event.pathParameters.id)
99 .then(note =>
100 callback(null, {
101 statusCode: 200,
102 body: JSON.stringify({
103 message: 'Removed note with id: ' + note._id,
104 note: note
105 })
106 })
107 )
108 .catch(err =>
109 callback(null, {
110 statusCode: err.statusCode || 500,
111 headers: { 'Content-Type': 'text/plain' },
112 body: 'Could not fetch the notes.'
113 })
114 )
115 })
116}

The context contains all the information about the handler function. How long it has been running, how much memory it is consuming among other things. In above, every function has the same value of context.callbackWaitsForEmptyEventLoop set to false and starts with connectToDatabase function call. The context object property callbackWaitsForEmptyEventLoop value is by default set to true. This property is used to modify the behavior of a callback.

By default, the callback will wait until the event loop is empty before freezing the process and returning the results to the invoked function. By setting this property’s value to false, it requests the AWS Lambda to freeze the process after the callback is called, even if there are events in the event loop. You can read more about this context property at the official Lambda Documentation.

Connecting MongoDB

We need to create a connection between the database and our serverless functions in order to consume the CRUD operations in real-time. Create a new file called db.js in the root and append it with following.

1const mongoose = require('mongoose')
2mongoose.Promise = global.Promise
3let isConnected
4
5module.exports = connectToDatabase = () => {
6 if (isConnected) {
7 console.log('=> using existing database connection')
8 return Promise.resolve()
9 }
10
11 console.log('=> using new database connection')
12 return mongoose.connect(process.env.DB).then(db => {
13 isConnected = db.connections[0].readyState
14 })
15}

The is common Mongoose connection that you might have seen in other Nodejs apps if using MongoDB as a database. The only difference here is that we are exporting connectToDatabase to import it inside handler.js for each CRUD operation. Modify handler.js file and import it at the top.

1'use strict'
2
3const connectToDatabase = require('./db')

Next step is to define the data model we need in order for things to work. Mongoose provides this functionality too. Serverless stack is unopinionated about which ODM or ORM you use in your application. Create a new file called notes.model.js and add the following.

1const mongoose = require('mongoose')
2const NoteSchema = new mongoose.Schema({
3 title: String,
4 description: String
5})
6module.exports = mongoose.model('Note', NoteSchema)

Now import this model inside handler.js for our callbacks at the top of the file.

1const Note = require('./notes.model.js')

Using Dotenv and Environment Variables

Protecting our keys and other essentials is the first step to a secured backend application. Create a new file called variables.env. In this file, we will add our MONGODB connection URL that we have already used in db.js as a process.env.DB. The good thing about environment variables is that they are global to the scope of the application.

To find out our MongoDB URL, we need to go back to the mongodb atlas, to out previously created cluster. Click the button Connect and then you will be prompted a page where you can choose how to access the application. Click Allow Access From Anywhere.

Copy the mongodb URL from above and paste it in the variables.env file.

1DB=mongodb://<user>:<password>@cluster0-shard-00-00-e9ai4.mongodb.net:27017,cluster0-shard-00-01-e9ai4.mongodb.net:27017,cluster0-shard-00-02-e9ai4.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin

Replace the user and password field with your credentials. Now to make it work, all we have to add the following line in our handler.js.

1require('dotenv').config({ path: './variables.env' })

Deployment

All you have to do is run the deploy command from the terminal.

1sls deploy

Since we have connected our Lambda function, this command will prompt us with a different endpoints. Each handler function is deployed as a separate REST endpoint.

You can test your API using CURL command from the terminal like below.

1curl -X POST https://7w3e8tfao0.execute-api.us-east-1.amazonaws.com/dev/notes --data '{"title": "My First Note", "description": "This is a note."}'

The complete code for the tutorial at this Github repository

Originally published at Crowdbotics

I'm Aman working as an independent fullstack developer with technologies such as Node.js, ReactJS, and React Native. I try to document and write tutorials to help JavaScript, Web and Mobile developers.