Try Epsagon
Search post

How to Setup AWS Lambda With SQS - Everything You Should Know!

0
how-to-setup-aws-lambda-with-sqs-1
AWS recently introduced SQS integration to Lambda. We have been waiting for this for a long time now. In the following post, I'm going to share a quick-and-simple tutorial on how to get started with message distribution between Lambda functions, using SQS. 

In addition, I will also compare between SQS and SNS (i.e. why should we choose one over the other), and present an in-depth performance analysis of using SQS as a message distributor.
 

Setup

In this tutorial, I will use the Serverless Framework, to send a message from one Lambda function to another Lambda function via SQS. 

Message sending using SQS
Message sending using SQS
 
Prerequisites:
  1. npm installed.
  2. Serverless Framework installed.
  3. Python - Feel free to use your favorite programming language.
  4. AWS account (duh).

Let's start by creating the serverless project: 

serverless create --template aws-python3 --path sqs-lambda-tutorial
cd sqs-lambda-tutorial
 

At this point, we can configure our serverless.yml and handler.py files. The code can be found in the following GitHub repo. 

serverless.yml

This yml file configures our functions, permissions and SQS resource. 

serverless.yml
 
serverless.yml
 
handler.py
This is where our code runs. It contains two functions. Each function represents a different Lambda function. One will be triggered manually, and the other one will be triggered by an SQS message. 
handler.py
handler.py
 
Deploy!
This is the magic of Lambda. Just run the following command while you are in the same folder as serverless.yml: 

sls deploy 

The output should look roughly like this: 

sls deploy output
 
sls deploy output

Now we can invoke the "start" function by running the following commands: 
sls invoke -f start-lambda
sls logs -f start-lambda
sls logs -f end-lambda
 

You will be able to see the CloudWatch Logs of the Lambda functions with the SQS message. 

We are done. Congrats! 🎉👏⚡️ 

Event structure

If you plan to use SQS in your application, you will need the event structure and data. You can find it in the following Gist.

SQS vs. SNS

When applied to Lambda only, this is a tough call. Up to this moment, only SNS supported triggering Lambda functions. Now both SNS and SQS can trigger. Let's compare the relevant parameters for Serverless applications: 

SQS performance analysis
SQS performance analysis 
 
The main advantage for SQS is the batch messaging, which can also lead to cost reduction.

Performance analysis

Distributing messages between services can be complicated. Understanding the performance impact on our applications is almost impossible. At Epsagon, we love to see the bigger picture for highly distributed applications. In a previous post, we compared several methods of distributing messages between functions (direct sync invocation, direct async invocation, SNS and Kinesis), and now, using our serverless performance monitoring tool, it's a great time to introduce the results for SQS: 
SQS performance analysis
SQS performance analysis
 
Compared to the results from the previous post - this is much better!

Conclusion

Lambda integrations are great. The more, the better. In this post, we learned how to deploy Lambda functions that distribute messages using SQS. 

We also saw an updated comparison between SQS and SNS for Serverless applications and understood the performance implications of distributing messages between functions. 

Considering features, cost and performance, SQS is the best choice for the task of message distribution between Lambda functions.
How to Handle AWS Lambda Errors Like a Pro
5 Ways to Gain Serverless Observability

About Author

Ran Ribenzaft
Ran Ribenzaft

I'm a passionate developer, with vast experience in network, infrastructure, and cyber-security. Constantly chasing new technologies - as the current one is Serverless. Love sharing open source tools to make everyone lives easier :) In my current role, I'm the co-founder and CTO at Epsagon - monitoring for serverless applications. I love swimming, traveling around the world, and taking breathtaking pictures.

Related Posts
The Most Popular Deployment Tools for Serverless
Best Practices for AWS Lambda Timeouts
Hitchhiker's Guide to AWS Step Functions

Comment

Subscribe To Blog

Subscribe to Email Updates