Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Istio Crypto Scheduled Workflow #79

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Istio Crypto Scheduled Workflow #79

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
2e338d2
added istioCrypto Deoploy Script
asubedy Jul 6, 2023
43791af
added scheduled workflow for istioCrypto
asubedy Jul 6, 2023
1311093
added cryptoDesign and and httpbin
asubedy Jul 6, 2023
2df0a0f
fixed httpbin spelling
asubedy Jul 6, 2023
79fbbe9
removed unnecessary space
asubedy Jul 7, 2023
69f382f
components register check added
asubedy Jul 17, 2023
ddc266a
proper url for checking registration
asubedy Jul 17, 2023
9f67fc4
added csr sript
asubedy Jul 20, 2023
355fa95
added correctly deployable crypto Design
asubedy Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 153 additions & 0 deletions .github/workflows/scheduled-istioCrypto-benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
name: Scheduled Benchmark Tests on Self-hosted Runner
on:
# for triggering manually, provide a test configuration file name or a performance profile name
workflow_dispatch:
inputs:
profile_name:
description: "performance profile to use"
required: false
profile_filename:
description: "test configuration file"
required: false
# scheduled to run on everyhour
schedule:
- cron: '0 * * * *'

jobs:
# Scheduled Benchmark Test
start-runners-scheduled:
name: Start self-hosted CNCF CIL runners for scheduled test
timeout-minutes: 60
runs-on: ubuntu-latest
if: ${{ github.event_name == 'schedule' }}
# strategy:
# fail-fast: false
# matrix:
# service-mesh: ['istio', 'linkerd', 'osm']
# load-generator: ['fortio', 'wrk2']
# test-configuration: ['load-test','soak-test']
outputs:
github_run_id: ${{ env.GITHUB_RUN_ID }} # use this github_run_id as a suffix for CIL machines and runners
steps:
- name: Checkout Code
uses: actions/checkout@v2

- name: Configure CNCF CIL credentials
run: |
chmod +x .github/workflows/scripts/self-hosted-credentails.sh
.github/workflows/scripts/self-hosted-credentails.sh ${{ secrets.CNCF_CIL_TOKEN }}
shell: bash

- name: Create registration token for CNCF CIL runner
id: getRegToken
run: |
reg_token=$(curl -s -X POST -H "Accept: application/vnd.github.v3+json" \
-H 'Authorization: token ${{ secrets.GH_ACCESS_TOKEN }}' \
https://api.github.com/repos/${{github.repository}}/actions/runners/registration-token | jq -r .token)
echo REG_TOKEN=$reg_token >> $GITHUB_ENV
echo REPOSITORY=${{github.repository}} >> $GITHUB_ENV
shell: bash

# The hostname will be like istioCrypto-fortio-load-1997512481
- name: Start CNCF CIL runner
id: start-cil-runner
run: |
echo GITHUB_RUN_ID=${{ github.run_id }} >> $GITHUB_ENV
chmod +x .github/workflows/scripts/start-cil-runner.sh
.github/workflows/scripts/start-cil-runner.sh ${{ secrets.cncf_cil_token }} istioCrypto-fortio-load
shell: bash

scheduled-test:
name: Scheduled Istio-Crypto Benchmark Test
timeout-minutes: 60
needs:
- start-runners-scheduled
runs-on: istioCrpto-fortio-load-${{ github.run_id }}
asubedy marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ github.event_name == 'schedule' }}
strategy:
# fail-fast: false
# matrix:
# service-mesh: ['istio', 'linkerd', 'osm']
# load-generator: ['fortio', 'wrk2']
# test-configuration: ['load-test','soak-test']
steps:
- name: Install dependencies
run: |
echo "Current user: $(whoami)"
echo "Installing kubectl..."
curl -LO https://dl.k8s.io/release/v1.23.2/bin/linux/amd64/kubectl
sudo install -o smp -g smp -m 0755 kubectl /usr/local/bin/kubectl
echo "Installing docker..."
sudo apt update -y
sudo apt install -y jq unzip apt-transport-https ca-certificates software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
sudo apt-cache policy docker-ce
sudo apt install -y docker-ce
sudo systemctl status docker
sudo mkdir -p ~/.kube
sudo chmod 777 ~/.kube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube


- name: Start minikube
run: |
minikube start --memory 5000
kubectl get po -A

- name: Checkout Code
uses: actions/checkout@v2

- name: Deploy Istio Crypto design and deploy httpbin application
run: |
chmod +x .github/workflows/scripts/istioCrypto.sh
.github/workflows/scripts/istioCrypto.sh
shell: bash

- name: Get Date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d-%H.%M.%S')"

- name: Run Benchmark Tests
uses: layer5io/meshery-smp-action@master
with:
provider_token: ${{ secrets.MESHERY_TOKEN }}
platform: docker
profile_filename: ${{ matrix.test-configuration }}.yaml
endpoint_url: ${{env.ENDPOINT_URL}}
service_mesh: ${{env.SERVICE_MESH}}
load_generator: ${{ matrix.load-generator }}
profile_name: 'istioCrypto-${{ matrix.test-configuration }}'
test_name: '${{ steps.date.outputs.date }}'

stop-runner-scheduled:
name: Stop self-hosted runner
timeout-minutes: 60
needs:
- start-runners-scheduled # required to get output from the start-runner job
- scheduled-test # required to wait when the main job is done
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
service-mesh: ['istio', 'linkerd', 'osm']
asubedy marked this conversation as resolved.
Show resolved Hide resolved
load-generator: ['fortio', 'wrk2']
test-configuration: ['load-test','soak-test']
if: ${{ always() && github.event_name == 'schedule' }} # required to stop the runner even if the error happened in the previous jobs
steps:
- name: Checkout Code
uses: actions/checkout@v2

- name: Stop CNCF CIL runner
run: |
chmod +x .github/workflows/scripts/stop-cil-runner.sh
.github/workflows/scripts/stop-cil-runner.sh ${{ secrets.cncf_cil_token }} ${{ matrix.service-mesh }}-${{ matrix.load-generator }}-${{ matrix.test-configuration }}-${{ needs.start-runners-scheduled.outputs.github_run_id }}
shell: bash

- name: Remove CNCF CIL runner from github repository
if: always()
run: |
runner_id=$(curl -s -H 'Authorization: token ${{ secrets.GH_ACCESS_TOKEN }}' -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${{github.repository}}/actions/runners | jq '.runners[] | select(.name == "${{ matrix.service-mesh }}-${{ matrix.load-generator }}-${{ matrix.test-configuration }}-${{ needs.start-runners-scheduled.outputs.github_run_id }}") | {id}' | jq -r .id)
curl -X DELETE -H 'Authorization: token ${{ secrets.GH_ACCESS_TOKEN }}' -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${{github.repository}}/actions/runners/$runner_id
shell: bash
71 changes: 71 additions & 0 deletions .github/workflows/scripts/istioCrypto.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env bash


# Istio, crypto deployment

export MESH_NAME='Istio'
export SERVICE_MESH='ISTIO'

# Check if mesheryctl is present, else install it
if ! [ -x "$(command -v mesheryctl)" ]; then
echo 'mesheryctl is not installed. Installing mesheryctl client... Standby... (Starting Meshery as well...)' >&2
curl -L https://meshery.io/install | ADAPTERS=istio PLATFORM=kubernetes bash -
fi

sleep 200
kubectl get pods -n meshery

echo "Meshery has been installed."
# mesheryctl system login --provider None

# mesheryctl mesh deploy istio --watch

sleep 200

kubectl get all -n istio-system

# so that istio operator gets initialized
mesheryctl system stop

sleep 400

mesheryctl system start

# Applying/deploying crpyto pattern
mesheryctl pattern apply -f ./AadiCrypto.yaml
asubedy marked this conversation as resolved.
Show resolved Hide resolved

sleep 200


kubectl get all -n istio-operator

# deplyoing httbin application
mesheryctl app onboard -f ./httbin.yaml -s "Kubernetes Manifest"
asubedy marked this conversation as resolved.
Show resolved Hide resolved

sleep 100

export INGRESS_NAME=istio-ingressgateway
export INGRESS_NS=istio-system

kubectl get svc "$INGRESS_NAME" -n "$INGRESS_NS"


export INGRESS_HOST=$(kubectl -n "$INGRESS_NS" get service "$INGRESS_NAME" -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export INGRESS_PORT=$(kubectl -n "$INGRESS_NS" get service "$INGRESS_NAME" -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
export SECURE_INGRESS_PORT=$(kubectl -n "$INGRESS_NS" get service "$INGRESS_NAME" -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
export TCP_INGRESS_PORT=$(kubectl -n "$INGRESS_NS" get service "$INGRESS_NAME" -o jsonpath='{.spec.ports[?(@.name=="tcp")].port}')


export GATEWAY_URL=http://$INGRESS_HOST:$INGRESS_PORT/headers
Copy link
Member

@gyohuangxin gyohuangxin Jul 18, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asubedy For crypto test case, the GATEWAY_URL should be https://$INGRESS_HOST:$SECURE_INGRESS_PORT.
And we need to generate a self-signed certificate to use in performance profile, so we can create another shell script which can be named as generate_csr.sh , I can provide the script we used before:


# generate CA cerficate
openssl genrsa -out fortio.com.key 2048
openssl req -new -x509 -days 365 -key fortio.com.key -subj "/C=CN/ST=GD/L=SZ/O=fortio.com, Inc./CN=fortio.com Root CA" -out fortio.com.crt

# generate CSR
openssl req -newkey rsa:2048 -nodes -keyout httpbin.fortio.com.key -subj "/C=CN/ST=GD/L=SZ/O=fortio.com, Inc./CN=*.fortio.com" -out httpbin.fortio.com.csr
openssl x509 -req -extfile <(printf "subjectAltName=IP:10.239.241.168,DNS:fortio.com,DNS:www.fortio.com") -days 365 -in httpbin.fortio.com.csr -CA fortio.com.crt -CAkey fortio.com.key -CAcreateserial -out httpbin.fortio.com.crt

# upload key and crt as a secret
kubectl create -n istio-system secret tls httpbin-fortio-credential --key=httpbin.fortio.com.key --cert=httpbin.fortio.com.crt

Then we can use the URL and fortio.com.crt in performance file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gyohuangxin I created the file, so we run this script after deploying Istio Crypto and deploying httpbin application right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asubedy Yes, it's correct.


# mesheryctl perf apply --file $GITHUB_WORKSPACE/.github/$perf_filename -t ~/auth.json --url "$endpoint_url" --mesh "$service_mesh" --name "$test_name" --load-generator "$load_generator" $perf_profile_name -y

echo "Service Mesh: $MESH_NAME - $SERVICE_MESH"
echo "Gateway URL: $GATEWAY_URL"
echo "ENDPOINT_URL=$GATEWAY_URL/productpage" >> $GITHUB_ENV
echo "SERVICE_MESH=$SERVICE_MESH" >> $GITHUB_ENV


asubedy marked this conversation as resolved.
Show resolved Hide resolved