Migrating AWS Lambda to Pulumi Project from Serverless
Introduction
In this article, We will be looking at how to migrate an AWS lambda function deployed as a Serverless application into a Pulumi project without modifying the underlying lambda code.
This article assumes that you have a basic understanding of serverless framework1 and pulumi2.
The code snippets and the projects are available on github here
Our existing Serverless Framework implementation
Let us first imagine that we have a serverless application with a single lambda function (createTodo) deployed in AWS. To make it interesting, we will use the code from typescript todo rest api example serverless examples repository.
The project structure could look something like this:
todos-api
├─ .eslintrc.json
├─ functions
│ └─ create.ts
├─ package.json
├─ package-lock.json
├─ serverless.yml
└─ tsconfig.json
serverless.yml could look as follows:
And we deploy with the following command
$ cd todos-api
$ serverless deploy
This would create the following AWS resources:
- CloudFormation template used to provision the stack
- S3 bucket where zip files of the function are stored
- Lambda serverless application
- Lambda function, belonging to the application
- CloudWatch log group with log streams for each instance of the function
- Dynamodb Table resource
- Role attached to the function with the following policies:
- An "assume role" policy with a permission for AWS Lambda service to assume this role
- A policy allowing the CloudWatch log group to create log streams and put log events to these streams
- A policy allowing dynamodb PUT operation
- REST API Gateway with:
- the
/newPOST endpoint integrated with the function - a permission to invoke the function
- the
After we implement the Lambda function with Pulumi, we will have pretty much the same set of resources, except for the:
- CloudFormation template
- S3 bucket
- Lambda serverless application
So you can remove them after you've moved your function.
Converting to pulumi project
At the end of the conversion, we would have the following structure.
By having the above structure, we are separating the business logic and infrastructure code. This is useful, especially when the project grows with various services and resources.
Initialize pulumi
$ cd todos-api
$ npm i -D -E @pulumi/pulumi @pulumi/aws
Above command adds the necessary pulumi dependencies as a dev dependency to already existing package.json file.
After adding pulumi dependencies, create a pulumi config file (Pulumi.yaml) to initialize a new pulumi project. The same can also be accomplished via pulumi templates using pulumi new command.
Pulumi.yaml file could look as follows:
Now, we will create a new stack for the above pulumi project. Let's name it as dev
$ pulumi stack init
The above command will create a new pulumi stack named dev. Also, it is suggested to set a region for the stack using below command.
$ pulumi config set aws:region eu-central-1
This will create a stack file Pulumi.dev.yaml with the following content.
config:
aws:region: eu-central-1
With this, we are done initializing pulumi project and a dev stack.
Infrastructure code
Before proceeding, let's create an empty directory named infrastructure along with index module to place the infra code.
$ mkdir -p infrastructure && touch infrastructure/index.ts
Also, let's prepare a script to help packaging node modules and code archive files. The script could look as follows:
Additionally, we could add a simple npm script command to our package.json file so that we could run packaging via npm.
Now, let's migrate the resources one by one.
Resources: DynamoDB Table
To provision dynamodb table,
💡 We could also import the existing table instead of creating a new one.
Resources: IAM Role
To create a role similar to the one from serverless,
Lambda function
To create lambda function resource with a node modules layer,
Api Gateway
Finally, we are going to hookup above lambda function with api gateway.
💡 Do also checkout pulumi crosswalk for aws on effortlessly creating aws gateway. I opted for the above conventional approach as I do not want to alter the source code of the lambda.
Deployment
We are now ready to deploy these resources via pulumi. We are going to run our packaging script first before issuing pulumi up.
$ npm run package
$ pulumi up -y
The output for the above command will look like below:
Updating (dev):
Type Name Status
+ pulumi:pulumi:Stack todos-api-dev created
+ ├─ aws:apigateway:RestApi dev-todos-api-rest created
+ ├─ aws:iam:Role dev-todos-api-executionRole created
+ ├─ aws:dynamodb:Table dev-todos-api created
+ ├─ aws:lambda:LayerVersion dev-todos-api-lambda-layer-nodemodules created
+ ├─ aws:apigateway:Resource dev-todos-api-resource created
+ ├─ aws:iam:RolePolicy dev-todos-api-executionRole-policy created
+ ├─ aws:apigateway:Method dev-todos-api-method created
+ ├─ aws:lambda:Function dev-todos-api-createTodo created
+ ├─ aws:lambda:Permission dev-todos-api-createTodo-permission created
+ ├─ aws:apigateway:Integration dev-todos-api-integration-post created
+ └─ aws:apigateway:Deployment dev-todos-api-deployment created
Outputs:
createTodoApiUrl: "https://xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com/dev"
Resources:
+ 12 created
Duration: 27s
Cleanup
Our lambda function is now entirely managed by pulumi. So, we can remove the serverless stack, and it's resources.
$ serverless remove
$ npm uninstall serverless serverless-plugin-typescript serverless-pseudo-parameters
$ rm serverless.yml
The code snippets and the projects are available on github here
This post is highly inspired by Moving Lambda function from Serverless to Terraform
-
serverless framework - zero-friction serverless development; easily build apps that auto-scale on low cost, next-gen cloud infrastructure. ↩︎
-
Pulumi - Modern Infrastructure as Code. By leveraging familiar programming languages for infrastructure as code, Pulumi makes you more productive, and enables sharing and reuse of common patterns. A single delivery workflow across any cloud helps developers and operators work better together. ↩︎