PoshJosh's Blog

AWS Lamda - Limitations and Use Cases

March 09, 2020

AWS Lambda Limitations

Runtime environment limitations as of 21/01/2020:

  • The disk space is limited to 512 MB.
  • The default deployment package size is 50 MB.
  • Memory range is from 128 to 3008 MB. Previously, the maximum amount of memory available to your functions was 1536 MB.
  • Maximum execution timeout for a function is 15 minutes.
  • Request limitations by Lambda: Request and response body payload size are maximized to 6 MB.
  • The event request body can be up to 128 KB.
  • Lambda Cold Start. Takes some time for the Lambda function to handle the first request, because Lambda has to start a new instance of the function. This means infrequently-used serverless code may suffer from greater response latency. (This can be mitigated by periodically pinging your function.)
  • Application dependencies can be troublesome, especially if third-party libraries link to external packages like C programs for Python code. This becomes a problem with the 50 MB package size limitation.
  • Monitoring Lambda via cloud watch logs can get costly if you need that sort of thing.

Depending on how you look at this limitation, it’s not all doom and gloom. These limitations can be looked at from a different perspective as a design choice that AWS made.

In order to have a serverless model and architecture, it’s imperative to have a system that’s fast and scalable. Meaning restrictions have to be put in place in order to strike an optimal balance i.e. trade-offs.

Thus, if you decide to use Lambda, you need to be sure about your use cases that they fall within the bounds of the limitations.

There are workarounds like persisting your data or state machines that address the limitations but it’s not the objective of this article to address the limitations or provide solutions.

Use cases

In combination with other AWS products

Lambda integrates very well with most other AWS services.

Manipulating objects in an S3 bucket, processing events from a Kinesis Stream, database items from a DynamoDB table or messages in an SQS queue, responding to REST API requests, etc. These are all examples of services that work seamlessly with AWS Lambda. For a full list of integrations, please refer to the official documentation

Serverless Website or Mobile App Backend

While static content can be stored in S3 and CloudFront, dynamic API requests can be served by AWS Lambda in combination with API Gateway or AppSync. With this configuration, it is possible to have an entire website in serverless platforms.

Unpredictable, high-variance load

Lambda is usually a good fit for workloads whose demand is unpredicable and highly variable, due to its highly scalable performance. Traditional infrastructures are provisioned accounting for peak demand. This leads to waste on idle resources. Although auto-scaling can mitigate, it may not be able to cope with rapid spikes in demand.

File Manipulation

A Lambda function can provide a quick and stable way to manipulate files. Images stored in an S3 bucket, for example, could be automatically converted from JPG to PNG, or have its size and quality setting reduced for optimized web navigation. This could work for virtually any kind of file: text, video, compressed, etc.

Artificial intelligence

Implementing and maintaining an infrastructure to run AI systems on a large scale can be difficult. Some machine learning frameworks and libraries, such as Scikit Learn, SciPy, NumPy, spaCy, etc. can run smoothly on AWS Lambda. Models that are too big to deploy with the Lambda package can be stored in S3 and retrieved on demand. It’s possible to keep the model in memory for a warm start in the next invocations served by the same Lambda container.

Disaster recovery

AWS Lambda can be used to automate tasks such as EBS snapshot and AMI creation to backup resources when configuring EC2 instances. Backup images can be stored in S3, for example. Lambda can also be used to restore backup images and run CloudFormation templates.

Extract, Transform, Load (ETL)

ETL jobs can be easily automated and scaled with AWS Lambda. A JSON object can be stored in S3, normalized in Lambda to insert in an RDS database, or validated to save in a DynamoDB table, for example. Later, Lambda could optimize the JSON using a columnar format to store back in S3 and serve fast and efficient analytical queries with AWS Athena.

References


Written byChinomso IkwuagwuExcélsior

Limited conversations with distributed systems.

Modifying legacy applications using domain driven design (DDD)

Gherkin Best Practices

Code Review Best Practices

Hacking Cypress in 9 minutes

Some common mistakes when developing java web applications

How to make a Spring Boot application production ready

SQL JOINS - A Refresher

Add Elasticsearch to Spring Boot Application

Add entities/tables to an existing Jhipster based project

CSS 3 Media Queries - All over again

Maven Dependency Convergence - quick reference

Amazon SNS Quick Reference

AWS API Gateway Quick Reference

Amazon SQS Quick Reference

AWS API Gateway Quick Reference

AWS Lambda Quick Reference

Amazon DynamoDB - Quick Reference

Amazon Aurora

Amazon Relational Database Service

AWS Database Services

AWS Security Essentials

Amazon Virtual Private Cloud Connectivity Options

Summary of AWS Services

AWS Certified Solutions Architect - Quick Reference

AWS CloudFront FAQs - Curated

AWS VPC FAQs - Curated

AWS EC2 FAQs - Curated

AWS Achritect 5 - Architecting for Cost Optimization

AWS Achritect 4 - Architecting for Performance Efficiency

AWS Achritect - 6 - Passing the Certification Exam

AWS Achitect 3 - Architecting for Operational Excellence

AWS Achitect 2 - Architecting for Security

AWS Achitect 1 - Architecting for Reliability

Amazon DynamoDB Accelerator (DAX)

Questions and Answers - AWS Certified Cloud Architect Associate

Questions and Answers - AWS Certified Cloud Architect Associate

AWS Connectivity - PrivateLink, VPC-Peering, Transit-gateway and Direct-connect

AWS - VPC peering vs PrivateLink

Designing Low Latency Systems

AWS EFS vs FSx

AWS Regions, Availability Zones and Local Zones

AWS VPC Endpoints and VPC Endpoint Services (AWS Private Link)

AWS - IP Addresses

AWS Elastic Network Interfaces

AWS Titbits

Jenkins on AWS - Automation

Jenkins on AWS - Setup

Jenkins on AWS - Best practices

Introduction to CIDR Blocks

AWS Lamda - Limitations and Use Cases

AWS Certified Solutions Architect Associate - Part 10 - Services and design scenarios

AWS Certified Solutions Architect Associate - Part 9 - Databases

AWS Certified Solutions Architect Associate - Part - 8 Application deployment

AWS Certified Solutions Architect Associate - Part 7 - Autoscaling and virtual network services

AWS Certified Solutions Architect Associate - Part 6 - Identity and access management

AWS Certified Solutions Architect Associate - Part 5 - Compute services design

AWS Certified Solutions Architect Associate - Part 4 - Virtual Private Cloud

AWS Certified Solutions Architect Associate - Part 3 - Storage services

AWS Certified Solutions Architect Associate - Part 2 - Introduction to Security

AWS Certified Solutions Architect Associate - Part 1 - Key services relating to the Exam

AWS Certifications - Part 1 - Certified solutions architect associate

AWS Virtual Private Cloud (VPC) Examples

Curated info on AWS Virtual Private Cloud (VPC)

Notes on Amazon Web Services 8 - Command Line Interface (CLI)

Notes on Amazon Web Services 7 - Elastic Beanstalk

Notes on Amazon Web Services 6 - Developer, Media, Migration, Productivity, IoT and Gaming

Notes on Amazon Web Services 5 - Security, Identity and Compliance

Notes on Amazon Web Services 4 - Analytics and Machine Learning

Notes on Amazon Web Services 3 - Managment Tools, App Integration and Customer Engagement

Notes on Amazon Web Services 2 - Storages databases compute and content delivery

Notes on Amazon Web Services 1 - Introduction

AWS Auto Scaling - All you need to know

AWS Load Balancers - How they work and differences between them

AWS EC2 Instance Types - Curated

Amazon Web Services - Identity and Access Management Primer

Amazon Web Services - Create IAM User

Preparing Jenkins after Installation

Jenkins titbits, and then some

Docker Titbits

How to Add Chat Functionality to a Maven Java Web App

Packer - an introduction

Terraform - an introduction

Versioning REST Resources with Spring Data REST

Installing and running Jenkins in Docker

Automate deployment of Jenkins to AWS - Part 2 - Full automation - Single EC2 instance

Automate deployment of Jenkins to AWS - Part 1 - Semi automation - Single EC2 instance

Introduction to Jenkins

Software Engineers Reference - Dictionary, Encyclopedia or Wiki - For Software Engineers