rf-lambda-executor

Run Robot Framework Tests via AWS Lambda

Find more documentation about RF and AWS Lambda here.

It is parallel test execution project that splits up your test cases into multiple Lambda functions and executes them in parallel.

It uses multiple AWS Ressources:

• AWS Lambda to execute Robot Framework tests and use its API
• AWS S3 to store the test cases and the test results
• AWS DynamoDB to store the run status
• AWS SQS to communicate between the Lambda functions
• AWS CloudFormation to create the AWS Ressources

You can find the sample project here and you can clone it with the command below:

git clone https://github.com/manykarim/rf-lambda-executor

It will be featured at RoboCon 2023 in the my talk To Infinity and Beyond - Serverless scaling with AWS Lambda.

Project Structure

.
├── distributor
│   ├── app.py
│   ├── __init__.py
│   ├── Listener
│   │   ├── DistributorListener.py
│   │   └── __init__.py
│   └── requirements.txt
├── executor
│   ├── app.py
│   ├── Dockerfile
│   ├── __init__.py
│   └── requirements.txt
├── LICENSE
├── merger
│   ├── app.py
│   ├── __init__.py
│   └── requirements.txt
├── poetry.lock
├── pyproject.toml
├── README.md
├── samconfig.toml
└── template.yaml

The most important parts are:

• the distributor , executor and merger folders which contain the Lambda functions
• the template.yaml file which contains the CloudFormation stack

template.yaml

The template.yaml file is the main file for the CloudFormation stack. It contains all the AWS Ressources that will be created and the Lambda functions that will be deployed.
It is a YAML file using the SAM Specification.

Lambda Functions

The Lambda functions are located in the distributor , executor and merger folders.

Distributor

The distributor is the first Lambda function that will be executed.
It is responsible for splitting up the test cases into smaller chunks and sending them to the executor Lambda function.
It is triggered by an HTTP Post request to the API Gateway.

It will

• download the test cases from the S3 Bucket Testsbucket
• split up the tests into N small chunks of similar size
• create a DynamoDB entry for each chunk
• send each chunk to a SQS queue to the executor Lambda function

The dependencies are defined in the distributor/requirements.txt file.

Executor

Is the second Lambda function that will be executed. It is responsible for executing the test cases and sending the results to the merger Lambda function. It is triggered by a message in the SQS queue.

It will

• download the test cases from the S3 Bucket Testsbucket
• execute the test cases
• upload the results to the S3 Bucket Resultsbucket
• send the results to the merger Lambda function
• update the DynamoDB entry
• check if all chunks are finished and trigger the merger Lambda function

Due to the size of the dependencies (e.g. robotframework-browser) the executor Lambda function is deployed as a Docker container.
The Dockerfile is located in the executor folder.

Merger

Is the third Lambda function that will be executed. It is responsible for merging the results from the executor Lambda functions and sending the results to the S3 Bucket Resultsbucket. It is triggered either by the executor Lambda function or by a HTTP Get request to the API Gateway.

It will

• download the results from the S3 Bucket Resultsbucket
• merge the results using rebot
• upload the results to the S3 Bucket Resultsbucket in the folder final
• update the DynamoDB entry

The dependencies are defined in the merger/requirements.txt file.