Serverless Framework Node SQS Producer Consumer on AWS

This template demonstrates how to develop and deploy a simple SQS-based producer-consumer service running on AWS Lambda using the traditional Serverless Framework.

Serverless Framework Node SQS Producer-Consumer on AWS

This template demonstrates how to develop and deploy a simple SQS-based producer-consumer service running on AWS Lambda using the Serverless Framework and the Lift plugin. It allows to accept messages, for which computation might be time or resource intensive, and offload their processing to an asynchronous background process for a faster and more resilient system.

Anatomy of the template

This template defines one function producer and one Lift construct - jobs. The producer function is triggered by http event type, accepts JSON payloads and sends it to a SQS queue for asynchronous processing. The SQS queue is created by the jobs queue construct of the Lift plugin. The queue is set up with a "dead-letter queue" (to receive failed messages) and a worker Lambda function that processes the SQS messages.

To learn more:

Deployment

Install dependencies with:

npm install

Then deploy:

serverless deploy

After running deploy, you should see output similar to:

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
........
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service aws-node-sqs-worker.zip file to S3 (21.45 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
................................................
Serverless: Stack update finished...
Service Information
service: aws-node-sqs-worker
stage: dev
region: us-east-1
stack: aws-node-sqs-worker-dev
resources: 17
api keys:
  None
endpoints:
  POST - https://xxxx.execute-api.us-east-1.amazonaws.com/produce
functions:
  producer: aws-node-sqs-worker-dev-producer
  jobsWorker: aws-node-sqs-worker-dev-jobsWorker
layers:
  None
jobs:
  queueUrl: https://sqs.us-east-1.amazonaws.com/xxxx/aws-node-sqs-worker-dev-jobs

Note: In current form, after deployment, your API is public and can be invoked by anyone. For production deployments, you might want to configure an authorizer. For details on how to do that, refer to http event docs.

Invocation

After successful deployment, you can now call the created API endpoint with POST request to invoke producer function:

curl --request POST 'https://xxxxxx.execute-api.us-east-1.amazonaws.com/produce' --header 'Content-Type: application/json' --data-raw '{"name": "John"}'

In response, you should see output similar to:

{"message": "Message accepted!"}