Skip to content

Commit d2e0eb9

Browse files
lukqwlukqw
committed
init commit
0 parents  commit d2e0eb9

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+33577
-0
lines changed

‎.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.venv
2+
files
3+
cdk
4+
.DS_STORE
5+
assets/video.mov

‎README.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# MNIST handwritten digit recognition model running on a local serverless SageMaker endpoint
2+
3+
| Key | Value |
4+
|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
5+
| Environment | <img src="https://img.shields.io/badge/LocalStack-deploys-4D29B4.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAKgAAACoABZrFArwAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAALbSURBVHic7ZpNaxNRFIafczNTGIq0G2M7pXWRlRv3Lusf8AMFEQT3guDWhX9BcC/uFAr1B4igLgSF4EYDtsuQ3M5GYrTaj3Tmui2SpMnM3PlK3m1uzjnPw8xw50MoaNrttl+r1e4CNRv1jTG/+v3+c8dG8TSilHoAPLZVX0RYWlraUbYaJI2IuLZ7KKUWCisgq8wF5D1A3rF+EQyCYPHo6Ghh3BrP8wb1en3f9izDYlVAp9O5EkXRB8dxxl7QBoNBpLW+7fv+a5vzDIvVU0BELhpjJrmaK2NMw+YsIxunUaTZbLrdbveZ1vpmGvWyTOJToNlsuqurq1vAdWPMeSDzwzhJEh0Bp+FTmifzxBZQBXiIKaAq8BBDQJXgYUoBVYOHKQRUER4mFFBVeJhAQJXh4QwBVYeHMQJmAR5GCJgVeBgiYJbg4T8BswYPp+4GW63WwvLy8hZwLcd5TudvBj3+OFBIeA4PD596nvc1iiIrD21qtdr+ysrKR8cY42itCwUP0Gg0+sC27T5qb2/vMunB/0ipTmZxfN//orW+BCwmrGV6vd63BP9P2j9WxGbxbrd7B3g14fLfwFsROUlzBmNM33XdR6Meuxfp5eg54IYxJvXCx8fHL4F3w36blTdDI4/0WREwMnMBeQ+Qd+YC8h4g78wF5D1A3rEqwBiT6q4ubpRSI+ewuhP0PO/NwcHBExHJZZ8PICI/e73ep7z6zzNPwWP1djhuOp3OfRG5kLROFEXv19fXP49bU6TbYQDa7XZDRF6kUUtEtoFb49YUbh/gOM7YbwqnyG4URQ/PWlQ4ASllNwzDzY2NDX3WwioKmBgeqidgKnioloCp4aE6AmLBQzUExIaH8gtIBA/lFrCTFB7KK2AnDMOrSeGhnAJSg4fyCUgVHsolIHV4KI8AK/BQDgHW4KH4AqzCQwEfiIRheKKUAvjuuu7m2tpakPdMmcYYI1rre0EQ1LPo9w82qyNziMdZ3AAAAABJRU5ErkJggg=="> |
6+
| Services | S3, SageMaker, Lambda, |
7+
| Integrations | AWS SDK |
8+
| Categories | Serverless, S3 website, Lambda function URLs, SageMaker, Machine Learning, JavaScript, Python | |
9+
| Level | Intermediate |
10+
11+
## Introduction
12+
13+
This is a sample application that demonstrates how to use SageMaker on LocalStack.
14+
A simple web frontend allows users to draw a digit and submit it to a locally running SageMaker endpoint.
15+
The endpoint returns a prediction of the digit, which is then displayed in the web frontend.
16+
Request handling is performed by a Lambda function, accessible via a function URL, that uses the SageMaker SDK to invoke the endpoint.
17+
18+
Here's a short summary of AWS service features we use:
19+
* S3 website
20+
* Lambda function URLs
21+
* SageMaker serverless endpoint
22+
23+
Here's the web application in action:
24+
25+
https://user-images.githubusercontent.com/39307517/234326469-7b8b1003-7991-4f28-b465-39653bb47da7.mov
26+
27+
## Architecture overview
28+
29+
![Architecture Diagram](/assets/architecture-diagram.png?raw=True "Architecture Diagram")
30+
31+
32+
## Prerequisites
33+
34+
### Dev environment
35+
36+
Create a virtualenv and install all the development dependencies there:
37+
38+
```bash
39+
python -m venv .venv
40+
source .venv/bin/activate
41+
pip install -r requirements.txt
42+
```
43+
44+
If you'd like to perform training locally, you'll need to install the ml dev dependencies as well:
45+
46+
```bash
47+
pip install -r ml/requirements.txt
48+
```
49+
50+
You'll also need npm/node installed to build the web application. Please install according to official guidelines: https://github.com/nvm-sh/nvm
51+
52+
### Download pytorch container image
53+
As our inference container, we use the PyTorch inference container from the AWS ECR.
54+
55+
```bash
56+
aws ecr get-login-password --region eu-central-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.eu-central-1.amazonaws.com
57+
docker pull 763104351884.dkr.ecr.eu-central-1.amazonaws.com/pytorch-inference:1.10.2-cpu-py38-ubuntu20.04-sagemaker
58+
```
59+
60+
### LocalStack
61+
62+
Start LocalStack Pro with your API key:
63+
64+
```bash
65+
LOCALSTACK_API_KEY=... localstack start
66+
```
67+
68+
## Instructions
69+
70+
You can create the AWS infrastructure on LocalStack by running `python deploy/deploy_app.py`.
71+
Make sure you have activated the python environment from the virtual environment (`.venv`) before running the script.
72+
73+
This script will create the sagemaker endpoint with the model, which it first uploads to a bucket.
74+
The script will also create a lambda function that will be used to invoke the endpoint.
75+
Finally, the script will build the web application and then create a s3 website to host it.
76+
77+
### Using the application
78+
79+
Once deployed, visit http://mnist-website.s3-website.localhost.localstack.cloud:4566
80+
81+
Draw something in the canvas and click on "Guess".
82+
83+
After a few moments the resulting prediction should be displayed in the box to the right.
84+
85+
![Demo Picture](/assets/demo-pic.png?raw=True "Demo Picture")

‎assets/architecture-diagram.png

355 KB
Loading

‎assets/demo-pic.png

40.6 KB
Loading

‎deploy/deploy_app.py

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
import json
2+
import io
3+
import zipfile
4+
import subprocess
5+
import boto3
6+
7+
# aws related
8+
REGION = "eu-central-1"
9+
ENDPOINT_URL = "http://localhost:4566"
10+
11+
AWS_CONFIG = {"region_name": REGION, "endpoint_url": ENDPOINT_URL}
12+
13+
# sagemaker config
14+
SAGEMAKER_CONTAINER_IMAGE_URI = "763104351884.dkr.ecr.eu-central-1.amazonaws.com/pytorch-inference:1.10.2-cpu-py38-ubuntu20.04-sagemaker"
15+
MODEL_BUCKET_NAME = "mnist-model-bucket"
16+
MNIST_MODEL_LOCATION = "ml/results/zip/model.tar.gz"
17+
MNIST_MODEL_KEY = "model.tar.gz"
18+
MNIST_MODEL_NAME = "mnist-model"
19+
MNIST_EP_CONFIGURATION_NAME = "mnist-epc"
20+
MNIST_EP_NAME = "mnist-endpoint"
21+
22+
# lambda config
23+
LAMBDA_NAME = "MnistHandlerLambda"
24+
25+
# website config
26+
WEBSITE_BUCKET_NAME = "mnist-website"
27+
28+
29+
def create_sagemaker_endpoint():
30+
# create s3 bucket for model hosting
31+
s3 = boto3.client("s3", **AWS_CONFIG)
32+
s3.create_bucket(
33+
Bucket=MODEL_BUCKET_NAME,
34+
CreateBucketConfiguration={"LocationConstraint": REGION},
35+
)
36+
37+
# upload ml model to s3
38+
s3.upload_file(
39+
Filename=MNIST_MODEL_LOCATION,
40+
Bucket=MODEL_BUCKET_NAME,
41+
Key=MNIST_MODEL_KEY,
42+
)
43+
44+
# create role for sagemaker to access s3
45+
iam = boto3.client("iam", **AWS_CONFIG)
46+
47+
document = {
48+
"Version": "2012-10-17",
49+
"Statement": [
50+
{
51+
"Effect": "Allow",
52+
"Principal": {"Service": "sagemaker.amazonaws.com"},
53+
"Action": "sts:AssumeRole",
54+
}
55+
],
56+
}
57+
sagemaker_role = iam.create_role(
58+
RoleName="sagemaker-role", AssumeRolePolicyDocument=json.dumps(document)
59+
)
60+
iam.attach_role_policy(
61+
RoleName="sagemaker-role",
62+
PolicyArn="arn:aws:iam::aws:policy/AmazonSageMakerFullAccess",
63+
)
64+
s3_policy = {
65+
"Version": "2012-10-17",
66+
"Statement": [
67+
{
68+
"Effect": "Allow",
69+
"Action": [
70+
"s3:GetObject",
71+
"s3:PutObject",
72+
"s3:DeleteObject",
73+
"s3:ListBucket",
74+
],
75+
"Resource": "arn:aws:s3:::*",
76+
}
77+
],
78+
}
79+
iam.put_role_policy(
80+
RoleName="sagemaker-role",
81+
PolicyName="sagemaker-s3-access",
82+
PolicyDocument=json.dumps(s3_policy),
83+
)
84+
85+
# create sagemaker model and endpoint
86+
sm = boto3.client("sagemaker", **AWS_CONFIG)
87+
88+
model_data_url = f"s3://{MODEL_BUCKET_NAME}/model.tar.gz"
89+
90+
sm.create_model(
91+
ModelName=MNIST_MODEL_NAME,
92+
PrimaryContainer={
93+
"Image": SAGEMAKER_CONTAINER_IMAGE_URI,
94+
"Mode": "SingleModel",
95+
"ModelDataUrl": model_data_url,
96+
},
97+
ExecutionRoleArn=sagemaker_role["Role"]["Arn"],
98+
)
99+
100+
sm.create_endpoint_config(
101+
EndpointConfigName=MNIST_EP_CONFIGURATION_NAME,
102+
ProductionVariants=[
103+
{
104+
"VariantName": "single-variant",
105+
"ModelName": MNIST_MODEL_NAME,
106+
"ServerlessConfig": {
107+
"MemorySizeInMB": 6144,
108+
"MaxConcurrency": 8,
109+
},
110+
},
111+
],
112+
)
113+
114+
sm.create_endpoint(
115+
EndpointName=MNIST_EP_NAME,
116+
EndpointConfigName=MNIST_EP_CONFIGURATION_NAME,
117+
)
118+
119+
120+
def create_lambda():
121+
# create function for handling requests which are passed onto sagemaker endpoint
122+
lambda_client = boto3.client("lambda", **AWS_CONFIG)
123+
124+
# Zip up the Lambda code from the specified directory
125+
with io.BytesIO() as buffer:
126+
with zipfile.ZipFile(buffer, mode="w") as zip_file:
127+
zip_file.write("lambda/index.py", arcname="index.py")
128+
zip_content = buffer.getvalue()
129+
130+
# Create the Lambda function
131+
response = lambda_client.create_function(
132+
FunctionName=LAMBDA_NAME,
133+
Runtime="python3.9",
134+
Role="arn:aws:iam::000000000000:role/lambda-role",
135+
Handler="index.lambda_handler",
136+
Code={"ZipFile": zip_content},
137+
Timeout=300,
138+
Environment={"Variables": {"SAGEMAKER_ENDPOINT_NAME": MNIST_EP_NAME}},
139+
)
140+
141+
response = lambda_client.create_function_url_config(
142+
FunctionName=LAMBDA_NAME,
143+
AuthType="NONE",
144+
Cors={
145+
"AllowCredentials": True,
146+
"AllowMethods": [
147+
"*",
148+
],
149+
"AllowOrigins": [
150+
"*",
151+
],
152+
"MaxAge": 123,
153+
},
154+
)
155+
156+
return response["FunctionUrl"]
157+
158+
159+
def build_webapp(lambdaUrl: str):
160+
for env in ["production", "development"]:
161+
f = open(f"web/.env.{env}", "w")
162+
f.write(f"REACT_APP_LAMBDA_URL={lambdaUrl}")
163+
f.close()
164+
165+
# build web app
166+
subprocess.run("npm run build", cwd="./web", shell=True)
167+
168+
169+
def host_website():
170+
# create s3 bucket for hosting the web app
171+
s3 = boto3.client("s3", **AWS_CONFIG)
172+
s3.create_bucket(
173+
Bucket=WEBSITE_BUCKET_NAME,
174+
CreateBucketConfiguration={"LocationConstraint": "eu-central-1"},
175+
)
176+
177+
# Set the bucket policy for static website hosting
178+
policy = {
179+
"Version": "2012-10-17",
180+
"Statement": [
181+
{
182+
"Sid": "PublicReadGetObject",
183+
"Effect": "Allow",
184+
"Principal": "*",
185+
"Action": "s3:GetObject",
186+
"Resource": f"arn:aws:s3:::{WEBSITE_BUCKET_NAME}/*",
187+
}
188+
],
189+
}
190+
s3.put_bucket_policy(Bucket=WEBSITE_BUCKET_NAME, Policy=json.dumps(policy))
191+
192+
# Upload the website files to the bucket
193+
subprocess.run(f"awslocal s3 sync web/build s3://{WEBSITE_BUCKET_NAME}", shell=True)
194+
195+
# Enable static website hosting for the bucket
196+
s3.put_bucket_website(
197+
Bucket=WEBSITE_BUCKET_NAME,
198+
WebsiteConfiguration={
199+
"ErrorDocument": {"Key": "index.html"},
200+
"IndexDocument": {"Suffix": "index.html"},
201+
},
202+
)
203+
204+
# Print the URL of the static website
205+
print(
206+
f"S3 static website URL: http://{WEBSITE_BUCKET_NAME}.s3-website.localhost.localstack.cloud:4566"
207+
)
208+
209+
210+
def main():
211+
create_sagemaker_endpoint()
212+
213+
lambdaUrl = create_lambda()
214+
215+
build_webapp(lambdaUrl)
216+
217+
host_website()
218+
219+
220+
if __name__ == "__main__":
221+
main()

‎lambda/index.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import json
2+
import os
3+
import boto3
4+
5+
ENDPOINT_NAME = os.environ['SAGEMAKER_ENDPOINT_NAME']
6+
sm = boto3.client('runtime.sagemaker')
7+
8+
def lambda_handler(event, context):
9+
payload = event['body']
10+
print("received payload: ", payload, type(payload))
11+
12+
payload_json = json.loads(payload)
13+
input_data = payload_json['data']
14+
15+
print("invoking endpoint %s with input: %s of type %s" %(ENDPOINT_NAME, input_data, type(input_data)))
16+
response = sm.invoke_endpoint(
17+
EndpointName=ENDPOINT_NAME,
18+
ContentType='application/json',
19+
Body=input_data,
20+
)
21+
22+
response = response['Body'].read().decode('utf-8')
23+
print("got response: ", response, type(response))
24+
25+
res = {"result": response}
26+
27+
return {
28+
"headers": {
29+
"Access-Control-Allow-Origin": "*",
30+
"Access-Control-Allow-Credentials": True,
31+
"Content-Type": "application/json",
32+
},
33+
"statusCode": 200,
34+
"body": json.dumps(res),
35+
}

0 commit comments

Comments
 (0)