Repository navigation
Comparison between AWS Web Adapter and (Python) Mangum + Fastapi #283
Description
Activity
I don't have exact numbers to share. In general, a framework adds a bit of cold-start time because it starts a full-blown web server.
Fastapi with Uvicorn usually adds about 100~200ms cold-start time. Usually it is a worthy trade off for the productivity and portability gain.
The main goal of this project is to provide an easy on-ramp for people new to Lambda, and wants to start building a Serverless Web App using familar tools and frameworks. In addition, people found that this tool help to migrate existing web apps to Lambda without requiring major refactoring of the exiting code base, such as this one.
I agree, portability is the key-word.
I wouldn't be so humble to say this is just an easy on-ramp, perhaps this is a true game-changer! (I wish I'd known about this before. I went the Mangum way (which I do not regret)).Regarding the overhead, 100-200 ms doesn't bother me at all.
My question is: how about the full startup (cold-start) time (consider no db connection, just a hello-world using: minimal Dockerfile (python Alpine?), uvicorn, fastapi). I'd like to know that number (which is very hard to measure from the outside of AWS)Moreover: would you say that in absolute numbers, an adapter made of Mangum vs the AWS lambda adapter, is Mangum going to be faster? by many ms?
I will do a test in this weekend and post the resultes here.
Here come the results: the cold start time is actually pretty close for LWA and Mangum, within 100ms.
Test setup: two Lambda functions (256MB memory) with Http API endopoints, triggered every 10 minutes to ensure we hit cold start every time. I collected 144 data pointes over 12 hours (excluding the first a few outliers caused by the cold cache in Lambda). The latenency data is from Http Api's IntegrationLatency metric, which is the end-to-end latency of Lambda invoke.
Here is the Http Api IntegrationLantecy over different percentiles. LWA is faster at high percentil p99 and p100. Mangum is faster at lower percentil P90 to P0. The difference is less than 100ms.
Here is the lantency graph over 12 hours.
And here are two samples of x-ray traces.
-
LWA
-
Mangum
Reacted by Carlos Garcés, gabriels1234 and Seth Miller-
Here is the function code and docker files used in the test.
- LWA + FastAPI
- the code
from fastapi import FastAPI app = FastAPI() @app.get("/") async def root(): print("in root method") return {"message": "Hello World"}
- the Dockerfile
FROM public.ecr.aws/docker/library/python:3.11.5-slim COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.1 /lambda-adapter /opt/extensions/lambda-adapter ENV PORT=8000 WORKDIR /var/task COPY requirements.txt ./ RUN python -m pip install -r requirements.txt COPY *.py ./ CMD exec uvicorn --port=$PORT main:app
- Mangum + FastAPI
- the code
from fastapi import FastAPI from mangum import Mangum app = FastAPI() @app.get("/") async def root(): print("in root method") return {"message": "Hello World"} handler = Mangum(app, lifespan="off")
- the Dockerfile
FROM public.ecr.aws/docker/library/python:3.11.5-slim ENV PORT=8000 WORKDIR /var/task COPY requirements.txt ./ RUN python -m pip install -r requirements.txt COPY *.py ./ ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ] CMD ["main.handler"]
Reacted by gabriels1234Thanks so much!
The numbers you got are good enough to start testing it on real projects.
The only metric missing would be from a warm-start, would those 100ms be equally present with the lambda adapter vs mangum?No, for warm-start, it is very close: almost identical at p50.
- LWA
Summary: Total: 10.5154 secs Slowest: 0.6041 secs Fastest: 0.0103 secs Average: 0.0188 secs Requests/sec: 950.9877 Total data: 250000 bytes Size/request: 25 bytes Response time histogram: 0.010 [1] | 0.070 [9906] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.129 [92] | 0.188 [0] | 0.248 [0] | 0.307 [0] | 0.367 [0] | 0.426 [0] | 0.485 [0] | 0.545 [0] | 0.604 [1] | Latency distribution: 10% in 0.0144 secs 25% in 0.0157 secs 50% in 0.0174 secs 75% in 0.0194 secs 90% in 0.0221 secs 95% in 0.0265 secs 99% in 0.0631 secs Details (average, fastest, slowest): DNS+dialup: 0.0003 secs, 0.0103 secs, 0.6041 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0206 secs req write: 0.0000 secs, 0.0000 secs, 0.0024 secs resp wait: 0.0181 secs, 0.0102 secs, 0.6035 secs resp read: 0.0000 secs, 0.0000 secs, 0.0006 secs Status code distribution: [200] 10000 responses
- Mangum
Summary: Total: 10.0350 secs Slowest: 0.1506 secs Fastest: 0.0105 secs Average: 0.0191 secs Requests/sec: 996.5162 Total data: 250000 bytes Size/request: 25 bytes Response time histogram: 0.011 [1] | 0.025 [9522] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.039 [340] |■ 0.053 [23] | 0.067 [8] | 0.081 [5] | 0.095 [0] | 0.109 [1] | 0.123 [49] | 0.137 [45] | 0.151 [6] | Latency distribution: 10% in 0.0143 secs 25% in 0.0157 secs 50% in 0.0175 secs 75% in 0.0197 secs 90% in 0.0221 secs 95% in 0.0243 secs 99% in 0.1120 secs Details (average, fastest, slowest): DNS+dialup: 0.0009 secs, 0.0105 secs, 0.1506 secs DNS-lookup: 0.0001 secs, 0.0000 secs, 0.0239 secs req write: 0.0000 secs, 0.0000 secs, 0.0023 secs resp wait: 0.0181 secs, 0.0104 secs, 0.1389 secs resp read: 0.0000 secs, 0.0000 secs, 0.0008 secs Status code distribution: [200] 10000 responses
Reacted by Jan@gabriels1234 I'm closing this issue. Feel free to open new ones if additional questions come up.
I just learned about LWA today and had this exact same question. Thank you so much @bnusunny for the thorough comparison. Super helpful!!
Reacted by Harold Sun- locked and limited conversation to collaborators
on Sep 26, 2024
Hi, Just saw the presentation and was thinking, performance-wise (mainly cold-start time) how would this compare to using the Mangum adapter + a framework (such as fastapi).
Thanks!