[PYTHON] [AWS] Try tracing API Gateway + Lambda with X-Ray

What is X-Ray?

A service for analyzing and debugging distributed applications. Click here for details. AWS X-Ray

Premise

Again, I will try to build a sample with the following configuration.

For basic knowledge such as SAM, please refer to here.

-[AWS] Serverless Application Model (SAM) Basic Summary -[AWS] Create API Gateway + Lambda + DynamoDB sample with Serverless Application Model (SAM)

Let's make a sample

Project creation

As usual, we'll create a Hello World-based project in SAM.

$ sam init --runtime=python3.8
Which template source would you like to use?
	1 - AWS Quick Start Templates
	2 - Custom Template Location
Choice: 1

Project name [sam-app]:

Cloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git

AWS quick start application templates:
	1 - Hello World Example
	2 - EventBridge Hello World
	3 - EventBridge App from scratch (100+ Event Schemas)
	4 - Step Functions Sample App (Stock Trader)
	5 - Elastic File System Sample App
Template selection: 1

-----------------------
Generating application:
-----------------------
Name: sam-app
Runtime: python3.8
Dependency Manager: pip
Application Template: hello-world
Output Directory: .

Next steps can be found in the README file at ./sam-app/README.md

Addition of X-Ray

Add decorator to source code

First, add the X-Ray decorator to your code. Try the following, including refactoring the hello_world / app.py generated by default.

hello_world/app.py


import json
from aws_xray_sdk.core import xray_recorder

@xray_recorder.capture('hello world')
def lambda_handler(event, context):
    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "hello world",
        })
    }

Import SDK

To get the X-Ray SDK library into Lambda, modify hello_world / requirements.txt as follows.

hello_world/requirements.txt


aws-xray-sdk

Modify SAM template

Finally, in order to enable tracing of Lambda function and API Gateway, modify the Globals part as follows.

template.before yml modification


Globals:
  Function:
    Timeout: 3

template.after yml modification


Globals:
  Function:
    Timeout: 3
    Tracing: Active
  Api:
    TracingEnabled: True

Build

$ sam build
Building function 'HelloWorldFunction'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Deploy

$ sam deploy --guided

Configuring SAM deploy
======================

	Looking for samconfig.toml :  Found
	Reading default arguments  :  Success

	Setting default arguments for 'sam deploy'
	=========================================
	Stack Name [sam-app]:
	AWS Region [us-east-1]: ap-northeast-1
	#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
	Confirm changes before deploy [Y/n]: y
	#SAM needs permission to be able to create roles to connect to the resources in your template
	Allow SAM CLI IAM role creation [Y/n]: y
	HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
	Save arguments to samconfig.toml [Y/n]: y

	Looking for resources needed for deployment: Found!

		Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-1rzppw621dkka
		A different default S3 bucket can be set in samconfig.toml

	Deploying with following values
	===============================
	Stack name                 : sam-app
	Region                     : ap-northeast-1
	Confirm changeset          : True
	Deployment s3 bucket       : aws-sam-cli-managed-default-samclisourcebucket-1rzppw621dkka
	Capabilities               : ["CAPABILITY_IAM"]
	Parameter overrides        : {}

Initiating deployment
=====================

	Saved arguments to config file
	Running 'sam deploy' for future deployments will use the parameters saved above.
	The above parameters can be changed by modifying samconfig.toml
	Learn more about samconfig.toml syntax at
	https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html

	Deploying with following values
	===============================
	Stack name                 : sam-app
	Region                     : ap-northeast-1
	Confirm changeset          : True
	Deployment s3 bucket       : aws-sam-cli-managed-default-samclisourcebucket-1rzppw621dkka
	Capabilities               : ["CAPABILITY_IAM"]
	Parameter overrides        : {}

Initiating deployment
=====================
HelloWorldFunction may not have authorization defined.
Uploading to sam-app/8476bddd8c14756a7c801a61352b828d.template  1142 / 1142.0  (100.00%)

Waiting for changeset to be created..

CloudFormation stack changeset
------------------------------------------------------------------------------------------------
Operation                        LogicalResourceId                ResourceType
------------------------------------------------------------------------------------------------
+ Add                            HelloWorldFunctionHelloWorldPe   AWS::Lambda::Permission
                                 rmissionProd
+ Add                            HelloWorldFunctionRole           AWS::IAM::Role
+ Add                            HelloWorldFunction               AWS::Lambda::Function
+ Add                            ServerlessRestApiDeployment47f   AWS::ApiGateway::Deployment
                                 c2d5f9d
+ Add                            ServerlessRestApiProdStage       AWS::ApiGateway::Stage
+ Add                            ServerlessRestApi                AWS::ApiGateway::RestApi
------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:ap-northeast-1:************:changeSet/samcli-deploy1597324152/18c44433-12d1-4e03-9fb8-00737d018991


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

2020-08-13 22:09:28 - Waiting for stack create/update to complete

CloudFormation events from changeset
-------------------------------------------------------------------------------------------------
ResourceStatus           ResourceType             LogicalResourceId        ResourceStatusReason
-------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS       AWS::IAM::Role           HelloWorldFunctionRole   -
CREATE_IN_PROGRESS       AWS::IAM::Role           HelloWorldFunctionRole   Resource creation
                                                                           Initiated
CREATE_COMPLETE          AWS::IAM::Role           HelloWorldFunctionRole   -
CREATE_IN_PROGRESS       AWS::Lambda::Function    HelloWorldFunction       -
CREATE_IN_PROGRESS       AWS::Lambda::Function    HelloWorldFunction       Resource creation
                                                                           Initiated
CREATE_COMPLETE          AWS::Lambda::Function    HelloWorldFunction       -
CREATE_IN_PROGRESS       AWS::ApiGateway::RestA   ServerlessRestApi        Resource creation
                         pi                                                Initiated
CREATE_IN_PROGRESS       AWS::ApiGateway::RestA   ServerlessRestApi        -
                         pi
CREATE_COMPLETE          AWS::ApiGateway::RestA   ServerlessRestApi        -
                         pi
CREATE_IN_PROGRESS       AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   -
                         yment                    yment47fc2d5f9d
CREATE_IN_PROGRESS       AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   Resource creation
                         yment                    yment47fc2d5f9d          Initiated
CREATE_IN_PROGRESS       AWS::Lambda::Permissio   HelloWorldFunctionHell   Resource creation
                         n                        oWorldPermissionProd     Initiated
CREATE_IN_PROGRESS       AWS::Lambda::Permissio   HelloWorldFunctionHell   -
                         n                        oWorldPermissionProd
CREATE_COMPLETE          AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   -
                         yment                    yment47fc2d5f9d
CREATE_IN_PROGRESS       AWS::ApiGateway::Stage   ServerlessRestApiProdS   -
                                                  tage
CREATE_COMPLETE          AWS::Lambda::Permissio   HelloWorldFunctionHell   -
                         n                        oWorldPermissionProd
CREATE_IN_PROGRESS       AWS::ApiGateway::Stage   ServerlessRestApiProdS   Resource creation
                                                  tage                     Initiated
CREATE_COMPLETE          AWS::ApiGateway::Stage   ServerlessRestApiProdS   -
                                                  tage
CREATE_COMPLETE          AWS::CloudFormation::S   sam-app                  -
                         tack
-------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------
Key                 HelloWorldFunctionIamRole
Description         Implicit IAM Role created for Hello World function
Value               arn:aws:iam::************:role/sam-app-HelloWorldFunctionRole-IIPXQC9S1XKJ

Key                 HelloWorldApi
Description         API Gateway endpoint URL for Prod stage for Hello World function
Value               https://tws0qc6nbc.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/

Key                 HelloWorldFunction
Description         Hello World Lambda Function ARN
Value               arn:aws:lambda:ap-northeast-1:************:function:sam-app-
HelloWorldFunction-1OH75PSLQUSLC
-------------------------------------------------------------------------------------------------

Successfully created/updated stack - sam-app in ap-northeast-1

Run

curl https://tws0qc6nbc.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
{"message": "hello world"}

Succeeded.

Verification

Now, let's check the X-Ray Service Map, Traces, and analytics in the management console.

Service Map

You can check how the API Gateway and Lambda are called in order from the client and the latency of each.

xray1.png

Traces

Here you can check the status of the trace. Here you can filter according to each condition.

xray2.png

Analytics

Finally, analytics, as the name implies, can perform a variety of analyzes and comparisons.

xray3.png

Try tracing AWS services

Now let's trace the AWS service. This time, let's add a trace for PutItem to DynamoDB.

For details, refer to "[AWS] Try creating an API Gateway + Lambda + DynamoDB sample with Serverless Application Model (SAM)". I would like to explain only the part related to X-Ray with points.

Add code

First, create a directory for Function separately from HelloWorld, and create files in it.

$ mkdir sam_ddb
$ touch sam_ddb/app.py
$ touch sam_ddb/requirements.txt

First, the body of the Lambda function, but the important part here is the patch ['boto3'] part. By doing this, you will be able to trace boto3 AWS service calls.

sam_ddb/app.py


import json
import boto3
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch
from datetime import datetime

patch(['boto3'])

@xray_recorder.capture('put_item ddb')
def lambda_handler(event, context):
    event_body = json.loads(event["body"])
    dynamodb = boto3.resource("dynamodb")

    table = dynamodb.Table("Demo")
    table.put_item(
        Item={
            "Key": event_body["key"],
            "CreateDate": datetime.utcnow().isoformat()
        }
    )

    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "succeeded",
        }),
    }

sam_ddb/requirements.txt


aws-xray-sdk
boto3

Finally, edit template.yml. The parts that need to be changed are Resources and ʻOutputs`. Please note that IAM Role requires permissions for X-Ray.

template.yml before change


AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app

  Sample SAM Template for sam-app

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3
    Tracing: Active
  Api:
    TracingEnabled: True

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.8
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

template.after changing yml


AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app

  Sample SAM Template for sam-app

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3
    Tracing: Active
  Api:
    TracingEnabled: True

Resources:
  DynamoTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Demo
      AttributeDefinitions:
        - AttributeName: Key
          AttributeType: S
        - AttributeName: CreateDate
          AttributeType: S
      KeySchema:
        - AttributeName: Key
          KeyType: HASH
        - AttributeName: CreateDate
          KeyType: RANGE
      ProvisionedThroughput: 
        ReadCapacityUnits: 5
        WriteCapacityUnits: 5

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.8
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

  SamDdbFunction:
    Type: AWS::Serverless::Function
    Properties:
      Role: !GetAtt SamDdbFunctionIamRole.Arn
      CodeUri: sam_ddb/
      Handler: app.lambda_handler
      Runtime: python3.8
      Events:
        SamDdb:
          Type: Api
          Properties:
            Path: /ddb
            Method: post

  SamDdbFunctionIamRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - 'lambda.amazonaws.com'
            Action:
              - 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/CloudWatchLogsFullAccess'
      Policies:
        - PolicyName: 'SamDdbPolicy'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:PutItem
                Resource: !GetAtt DynamoTable.Arn
              - Effect: Allow
                Action:
                  - xray:PutTraceSegments
                  - xray:PutTelemetryRecords
                Resource: '*'

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn
  SamDdbApi:
    Description: "API Gateway endpoint URL for Prod stage for SAM DDB function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/ddb/"

Build and deploy

Run sam build and sam deploy --guided. (Details omitted)

Run

Since we have added one API this time, we will also call the Hello World API for comparison.

$ curl https://fp8nyhpv87.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
{"message": "hello world"}

Next, let's call the Post API to write to DynamoDB.

$ curl -X POST -H "Content-Type: application/json" -d '{"key": "demo-data"}' https://fp8nyhpv87.execute-api.ap-northeast-1.amazonaws.com/Prod/ddb/
{"message": "succeeded"}

X-Ray confirmation

First, from the Service Map. The arrangement is twisted for some reason, but you can see that the access part to DynamoDB is cut out and displayed. You can see that the writing part of the DB takes 620ms.

xray4.png

The same is true for Traces. The DynamoDB part can be traced independently.

xray5.png

Check DynamoDB just in case

The data is registered perfectly.

ddb.png

Summary

Tracing with X-Ray enables various analyses. You can easily add it after the fact, but if possible, we recommend that you introduce it from an early stage.

Sample code repository

https://github.com/hito-psv/sam-demo-003

Recommended Posts

[AWS] Try tracing API Gateway + Lambda with X-Ray
[AWS] Create API with API Gateway + Lambda
[AWS SAM] Create API with DynamoDB + Lambda + API Gateway
LINE BOT with Python + AWS Lambda + API Gateway
Create API with Python, lambda, API Gateway quickly using AWS SAM
Easy REST API with API Gateway / Lambda / DynamoDB
Send images taken with ESP32-WROOM-32 to AWS (API Gateway → Lambda → S3)
View images on S3 with API Gateway + Lambda
Amazon API Gateway and AWS Lambda Python version
Try AWS Lambda Destinations
AWS CDK-Lambda + API Gateway (Python)
AWS Lambda with PyTorch [Lambda import]
Quickly take a query string with API Gateway-> Lambda (Python)
AWS Amplify + API Gateway + Lambda + Python returns a normal response
[AWS] Try adding Python library to Layer with SAM + Lambda (Python)
Try automating Start / Stop for EC2 instances with AWS Lambda
Try giving AWS Lambda environment variables?
Using Lambda with AWS Amplify with Go
Notify HipChat with AWS Lambda (Python)
I tried ChatOps with Slack x API Gateway x Lambda (Python) x RDS
[Python] I wrote a REST API using AWS API Gateway and Lambda.
How to create a serverless machine learning API with AWS Lambda
LINE BOT (Messaging API) development with API Gateway and Lambda (Python) [Part 2]
[AWS] Using ini files with Lambda [Python]
I tried to delete bad tweets regularly with AWS Lambda + Twitter API
Try implementing XOR with Keras Functional API
Try using Dropbox API v2 with Go
I tried to make "Sakurai-san" a LINE BOT with API Gateway + Lambda
Understanding from the mechanism Twilio # 3-1 --AWS API Gateway + Lambda implementation Walkthrough (Part 1)
[AWS] Link Lambda and S3 with boto3
Connect to s3 with AWS Lambda Python
[AWS] Do SSI-like things with S3 / Lambda
Try assigning or switching with Python: lambda
Python + Selenium + Headless Chromium with aws lambda
I just did FizzBuzz with AWS Lambda
Try Tensorflow with a GPU instance on AWS
Regular serverless scraping with AWS lambda + scrapy Part 1.8
Serverless scraping using selenium with [AWS Lambda] -Part 1-
Serverless application with AWS SAM! (APIGATEWAY + Lambda (Python))
Try slack OAuth authentication with flask (Slack API V2)
I tried connecting AWS Lambda with other services
Infrastructure construction automation with CloudFromation + troposphere + AWS Lambda
Pass Cognito Id to Lambda via API Gateway
A note that connects to Lambda via AWS API Gateway (HTTP API) to process POST data
Dynamic HTML pages made with AWS Lambda and Python
[First API] Try to get Qiita articles with Python
[AWS] Play with Step Functions (SAM + Lambda) Part.3 (Branch)
Try to make RESTful API with MVC using Flask 1.0.2
Deploy Python3 function with Serverless Framework on AWS Lambda
Create a Layer for AWS Lambda Python with Docker
[AWS] Play with Step Functions (SAM + Lambda) Part.1 (Basic)
Try automating Qiita's like monitoring with Lambda + DynamoDB + CloudWatch
I want to AWS Lambda with Python on Mac!
Manage your Amazon CloudWatch loggroup retention with AWS Lambda
Make ordinary tweets fleet-like with AWS Lambda and Python
[AWS] Play with Step Functions (SAM + Lambda) Part.2 (Parameter)
Made "Unofficial Apple Refurbished Product Introduction" BOT with LINE Messaging API (v2) + API Gateway + lambda (python)