In the 2017 Re:Invent lecture Become a Serverless Black Belt: Optimizing Your Serverless Applications, the speaker says that Lambda shouldn't be used for orchestration code since waiting time in Lambda is not efficient. It is a much better fit for local compute bound operations like transforming data coming through Kinesis Firehose. However, it would be nice to have the benefits of serverless scaling and resiliency when writing APIs that do a decent bit of waiting like 20 second calls to legacy systems.
In the past there has been an "insert code here" cloud PaaS built around VMs and containers, but not yet around functions as a service.
- VM: Elastic Beanstalk and Google App Engine Flexible
- Container: Google App Engine Standard
- FaaS: Sort of Lambda, but not efficient for all workloads.
As you go down the list, you get more responsive scaling and more efficient economics. If AWS was to release the first FaaS-based PaaS, it could use that to differentiate itself from the other cloud providers. Even though Lambda revenue might decrease in the short term, API Gateway would probably increase due to people hosting higher traffic APIs on Lambda. Also more open source attention would be drawn to running Express, Hapi, and the other web/api frameworks on Lambda.
- Max Parallel Requests: The user would need to set a maximum parallel requests option. The default of 0 would maintain the current behavior of a new Lambda invocation on each request. Most cases would probably be between 10-20 parallel requests before you would want to scale to another Lambda invocation.
- Max Runtime:
- Keep the current hard limit 5 minute max runtime and use the max duration parameter to limit a single event's processing time. For example, with a 30 second max duration, the Lambda could (but not required to if all current requests had completed) continue to launch tasks up until the 4:30 mark.
- Keep the current parameter controlling the invocation runtime and introduce a new parameter limiting a single event's processing time.