AWS Integration
Adding your AWS Account
Once you have created an org in Serverless Console you will be asked to add an AWS Account as a new Integration.
It takes several minutes for the integration to complete as it performs a number of steps:
- Creates a set of IAM Roles using CloudFormation. These roles are used to create additional infrastructure needed in your account.
- Creates an EventBridge Event Bus and CloudTrial Event Trail for handling Lambda deploy events. This is used by Serverless Console to ensure functions are instrumented upon deployment.
- Creates a CloudWatch Metric Stream for collection metrics.
- Creates two Kinesis Firehoses for sending logs and metrics to Serverless Consoles.
Note: this same process can be initiated from the CLI using Serverless Framework
Integration Status
Once you have initiated the creation CloudFormation stack the process will take a few minutes and the status of your integration will be one of the following.
Running - The integration has setup the initial infrastructure, and it is currently syncing the resources. In addition to initial syncing of resources, the integration may later appear as running when periodic syncing occurs.
Complete - A complete integration has all infrastructure in place and inventory is up to date.
Incomplete - An incomplete integration is missing infrastructure and may or may not have accurate inventory information. You will need to delete this integration or contact support.
Environment and Namespace Tags
During the initial sync Serverless Console will identify all Lambda functions and CloudFormation stacks to help determine a helpful Environment, and Namespace value.
The Environment is determined by the Lambda env var STAGE
, it represents
the app environment, like development
or production
. To set the Environment
go to Settings -> Integrations -> Edit Integration. Each function will have the
ability to select a pre-populated Environment tag, or to create a new one.
The Namespace is determined from the service
name specified in
CloudFormation, it represents a common business outcome, like shopping-cart
.
To set the Namespace tag go to Settings -> Integrations -> Edit Integration.
Each function will have the ability to select a pre-populated set of Namespaces
or add a new one. Only one namespace tag can be added per function.
AWS Account Infrastructure Updates and Data Collection
The following is an overview of the changes Serverless Console makes to your AWS Infrastructure, including IAM Roles, and the data it collects, when you add an AWS Integration.
IAM Roles
Serverless Console configures three IAM Roles when adding your AWS Account. Where possible the least privileges are applied and detailed comments are maintained about the permissions.
-
ServerlessRole
- The primary role used for the setting up general infrastructure. -
ServerlessEventSubscriptionExecutionRole
- Configures EventBridge for processing new CloudTrail events. -
ServerlessMonitoringRole
- Creates required Kinesis Firehose instances and CloudWatch Metric Stream.
CloudTrail Events
Serverless Console use CloudTrial for identify updates to resources and enabling instrumentation on those resources if necessary. An EventBridge rule is setup in each region where instrumented resources exist.
Metric Streams
Serverless Console collects metrics for all your Lambda functions using Cloudwatch Metric Streams.
Currently metric streams are limited to collect metrics from Lambda, and API Gateway. A unique metric stream and Kinesis Firehose is created in each region you have instrumented resources.
CloudWatch Log Subscriptions
When you enable instrumentation for a function, Serverless Console will setup a Cloudwatch Subscription to collect logs for that function. Cloudwatch Subscriptions are configured in the region your function is deployed.
Lambda Layer with Dev Mode Instrumentation Extension
When Dev Mode instrumentation is enabled, an AWS Lambda Layer with an external extension is added to the function. This extension is responsible for collecting the trace details, logs, events, and forwarding them to Serverless Console. This extension enables the real-time logging in DevMode by skipping CloudWatch Logs.
Currently Dev Mode is supported on the Node.js 12+ and Python 3.8+ runtimes on AWS Lambda only. Support for other runtimes is coming soon.
Lambda Layer with the Serverless SDK
When Dev or Prod mode instrumentation is enabled, an AWS Lambda Layer with the Serverless SDK is added to the function. The Serverless SDK is responsible for auto-instrumentation of traces and spans and collecting events.
The traces, spans, and events are binary encoded and logged in CloudWatch where Serverless Console can consume the events via CloudWatch log subscription groups.
Currently instrumentation is supported on the Node.js 12+ and Python 3.8+ runtimes on AWS Lambda only. Support for other runtimes is coming soon.
Use the Node.js and Python Serverless SDK to add custom instrumentation.
Automatic updating of AWS Lambda Layers
If instrumentation is enabled on a function and a new version of the lambda layers is released, the layers on the AWS Account will automatically be upgraded to the latest version. No manual intervention or redeployment is necessary.
Automatically removing an AWS Integration
It is best to use the Console UI to remove any AWS Accounts you have setup. This automates the process of removing all associated infrastructure in your account.
Manually removing an AWS Integration
It is recommended to use the automatic AWS Integration removal process; however, it is also possible to remove the integration manually.
-
Remove the IAM Roles - Go to your IAM Roles in AWS Console and delete the roles
ServerlessRole
,ServerlessEventSubscriptionExecutionRole
andServerlessMonitoringRole
. -
Remove the CloudFormation Stack - Go to your CloudFormation Stacks in AWS Console. Look for the stack named
Serverless-Inc-Role-Stack
and delete it. This stack is only created inus-east-1
. -
Remove the Kinesis Firehose - Go to Kinesis Delivery Streams in AWS Console and delete the delivery streams named
serverless_logs-firehose
andserverless_metrics-firehose
. You will need to repeat this for each region in which Lambda functions were instrumented with Serverless Console. -
Remove the Cloudwatch Metric Streams - Go to Cloudwatch MetricStreams in AWS Console and delete the metric stream name
serverless_metrics-stream
. You will need to repeat this for each region in which Lambda functions were instrumented with Serverless Console. -
Remove the CloudTrail Trail - Go to CloudTrail Trails in AWS Console and delete the rule
serverless_trail
. You will need to repeat this for each region in which Lambda functions were instrumented with Serverless Console. -
Remove the EventBridge rule - Go to the EventBridge rules in AWS Console and delete the rule
serverless_lambda_deploy_events
. You will need to repeat this for each region in which Lambda functions were instrumented with Serverless Console. -
Remove s3 bucket - Go to the S3 Buckets in AWS Console and delete the buckets named
serverless.logs-firehose-backup-GUID
andserverless.metrics-firehose-backup-GUID
. You will need to repeat this for each region in which Lambda functions were instrumented with Serverless Console. -
Remove the Cloudwatch Log Subscriptions - Go the your Cloudwatch Log Groups in the AWS Console. Open Log Group (the function name will appear in the Log Group path). Under the Log Group click on the
Subscription Filters
tab and remove the delete the filter nameserverless_logs-filter
. You will need to repeat this for each Lambda functions instrumented with Serverless Console. -
Remove the Layers and env vars - Go to your Lambda functions in AWS Console. Go to the Lambda function and go to the Layers section of the function and delete the layers with
sls-sdk-node
andsls-external-extension
. Under Configuration -> Environment Variables remove the environment variablesAWS_LAMBDA_EXEC_WRAPPER
,SLS_DEV_MODE_ORG_ID
andSLS_ORG_ID
. You will need to repeat this for each Lambda functions instrumented with Serverless Console.