Building with AWS Lambda, How to Troubleshoot Size Issues with Deployments

Based on our experience working with various clients, we often come across the issue of "Wow, this Lambda function is getting quite large" more frequently than expected. As developers, when building REST APIs with Lambda, it's easy to run into situations where the size of your Lambda deployment package exceeds the recommended limits if best practices aren't followed.

In this guide, we aim to provide you with some essential rules for developing with the Serverless Framework. You can also find other helpful guides for packaging and deploying functions using AWS SAM and AWS CDK at the following links:

• Managing AWS Lambda applications in SAM for scale

• Managing AWS Lambda applications with CDK for scale

Now, let's define the scenario we'll be discussing in this guide. We have a Lambda service responsible for handling shipping quotes and generating labels for a client application.

First let’s define the scenario, we have a lambda service that handles shipping quote and label generation for a client application.

david@davids-MBP-2 .serverless % ls -l
total 80112
...serverless and CF stuffs
-rw-r--r--  1 david  staff  89878598 Jun 30 13:48 swank-logistics-service.zip

With Serverless package, we can see that the total zip size of this service is roughly 90MB

Already over the 50MB limit for manual uploads on lambdas, which is not ideal. How can we reduce the overall package size and make this lambda more maintainable and manageable?

Whatever the approach, we must always remember the limits of Lambda package size:

Deployment package size

• 50 MB (zipped, for direct upload)

• 250 MB (unzipped, including layers)

• 3 MB (console editor)

So let’s solve this problem, there are a few different paths you can take to reach the goal of more manageable and maintainable lambdas.

Using excluded patterns

The Serverless docs outline the ability to ignore files and folders from your lambda package. Remember the only thing you should be deploying to AWS are modules that are required for your code to run, everything else is obsolete.

package:
  patterns:
    - !*.pyc
    - !__pycache__/**
    - !pandas/tests/**
    - !numpy/tests/**
    - !.installed.cfg

or you can exclude everything and only include what you need

package:
   patterns:
      - !*/**
      - src/*
      - requirements.txt
      // other required patterns

For a python lambda, this is a good starting point for files and folders to ignore. Build on this and play around with stripping away unnecessary files and folders from your zip file in the .serverless folder

Introduce Lambda Layers ... sort of

Why sort of? Because it only works if the unzipped size of your code + the lambda layer are under 250MB. And sometimes, with multiple beefy packages in your deployment package, this can be a tricky solution.

We walk through these steps in detail under the following Serverless guide

Everything you need to do can be summarized in the following steps

1. Create a folder for your lambda layers with a whatever package management tool you use

2. Define the lambda layer in your serverless.yml

3. Deploy the service

I know, easy right?

But keep the following in mind. The overall size of your lambda layer matters. Remember that 250MB unzipped size limit we had before? This includes the unzipped lambda layer.

Time for some Refactoring

At this point there isn't much else we can do, the package size is huge even after removing some dependencies and files. The lambda layer was huge so it brought the unzipped size of 250MBs. Aw man, what do we do now!

Time to clean up the mess. You'll have to take a look at how you've structured the function within your micro-service and see which of them can be stripped out to their own individual service.

Please note: This will affect the configuration of your application in API Gateway, which means you'll have to coordinate with client side teams and frontend devs to make sure the right endpoints are being hit.

Let's take the following scenario

Take the following scenario. We have 8 lambda functions under the poorly named. “Big lambda service”. This service installs Numpy, pandas and Matplotlib to calculate the odds of Elon’s next rocket exploding.

Within this services, 4 lambdas handle crud operations, two are cron jobs, and the other two were just thrown in here because we didn’t know what else to do with them.

Every new service is going to need it’s own API, otherwise we run the risk of overwriting api mappings when we deploy our code.

Throw it in Elastic Container Registry

Now, this is the ultimate solution I discovered after exploring various options for multiple projects. By using ECR (Elastic Container Registry), we can deploy our Lambdas without any concerns about their size. However, please note that this approach may significantly increase your deployment times, as you need to include the step of building the Docker image in your deployment process.

Your configuration will need to be modified from its current state, which might resemble something like this:

functions:
  hello:
    image:
      name: appimage
      command:
        - app.hello <- very important

With ECR you can specify a command for each lambda function which will enable you to run multiple lambdas from one ECR container.

This is just a small highlight of some of the work we did with the Serverless framework for managing and deploying lambdas to the cloud.

In other studies we will discuss other frameworks for managing and deploying lambdas and micro services up to the cloud.

Thank you for reading!