Skip to content

Master #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jaycelmarie wants to merge 21 commits into
base: main
Choose a base branch
from
Open

Master #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e69084b
Made changes to payload.json
jaycelmarie Dec 22, 2022
78f58c3
Modified project name under worker.py
jaycelmarie Dec 22, 2022
4d44e6d
Edited worker.py
jaycelmarie Dec 22, 2022
eebaff5
add
jaycelmarie Dec 22, 2022
e7acd48
key
jaycelmarie Dec 22, 2022
d7f352e
key
jaycelmarie Dec 22, 2022
c08f401
new
jaycelmarie Dec 22, 2022
0e7bf44
Changes to payload
jaycelmarie Dec 22, 2022
fa794b7
Project name modified
jaycelmarie Dec 22, 2022
9c2d9ca
project name
jaycelmarie Dec 22, 2022
de6921a
changes
jaycelmarie Dec 22, 2022
d3a87db
key
jaycelmarie Dec 22, 2022
68ba101
key
jaycelmarie Dec 22, 2022
8090a60
us
jaycelmarie Dec 22, 2022
ed90d64
us
jaycelmarie Dec 22, 2022
0564cda
payload modified
jaycelmarie Dec 22, 2022
650f262
worker modified
jaycelmarie Dec 22, 2022
5f6d0f6
key
jaycelmarie Dec 22, 2022
b09bd2e
keykey
jaycelmarie Dec 22, 2022
132db95
new generated key
jaycelmarie Dec 22, 2022
05cb075
payload modified
jaycelmarie Dec 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .key
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ya29.a0AX9GBdXkw688sLrr06WpCzorsvDoUhsuifwY37ypZy2ToTnkdhHhDvIxIndLvL5a20WHckuOGTkAa6xHC2gtwCbD89x5kY3ds8PliKBdrv_PboCSnnZkC1MoY7a4UvHsW0pQeT8mRtg2tXFmIKA2G7DN_B0etuRV1jFZuDJybjVL_uaT0b4atoWRmMct4JcXRQPfaqfgj5uG_lQ71iYAnDniu7_QL4pKQH5ucBQaCgYKAVYSARISFQHUCsbCoc4FXCeLcTq2mRy6W7bvtw0238
14 changes: 7 additions & 7 deletions payload.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"deviceName": "slave1",
"initializeParams": {
"diskSizeGb": "10",
"diskType": "projects/spark-371009/zones/us-west4-b/diskTypes/pd-balanced",
"diskType": "projects/valued-context-372415/zones/us-west4-b/diskTypes/pd-balanced",
"labels": {},
"sourceImage": "projects/debian-cloud/global/images/debian-11-bullseye-v20221102"
},
Expand All @@ -26,12 +26,12 @@
"guestAccelerators": [],
"keyRevocationActionType": "STOP",
"labels": {},
"machineType": "projects/spark-371009/zones/europe-west1-b/machineTypes/e2-medium",
"machineType": "projects/valued-context-372415/zones/europe-west1-b/machineTypes/e2-medium",
"metadata": {
"items": [
{
"key": "startup-script",
"value": "apt update\ncurl --output spark.tgz https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz\napt -y install default-jdk\ntar -xzvf spark.tgz\n./spark-3.3.1-bin-hadoop3/sbin/start-slave.sh spark://spark-master.europe-west1-b.c.spark-371009.internal:7077"
"value": "sudo apt update\ncurl --output spark.tgz https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz\nsudo apt -y install default-jdk\ntar -xzvf spark.tgz\n./spark-3.3.1-bin-hadoop3/sbin/start-slave.sh spark://spark-master.europe-west1-b.c.valued-context-372415.internal:7077"
}
]
},
Expand All @@ -45,7 +45,7 @@
}
],
"stackType": "IPV4_ONLY",
"subnetwork": "projects/spark-371009/regions/europe-west1/subnetworks/default"
"subnetwork": "projects/valued-context-372415/regions/europe-west1/subnetworks/default"
}
],
"reservationAffinity": {
Expand All @@ -59,7 +59,7 @@
},
"serviceAccounts": [
{
"email": "569439294570-compute@developer.gserviceaccount.com",
"email": "176933669329-compute@developer.gserviceaccount.com",
"scopes": [
"https://www.googleapis.com/auth/devstorage.read_only",
"https://www.googleapis.com/auth/logging.write",
Expand All @@ -78,5 +78,5 @@
"tags": {
"items": []
},
"zone": "projects/spark-371009/zones/europe-west1-b"
}
"zone": "projects/valued-context-372415/zones/europe-west1-b"
}
2 changes: 1 addition & 1 deletion worker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def addWorker(token, num):
tdata=json.load(p)
tdata['name']='slave'+str(num)
data=json.dumps(tdata)
url='https://www.googleapis.com/compute/v1/projects/spark-371009/zones/europe-west1-b/instances'
url='https://www.googleapis.com/compute/v1/projects/valued-context-372415/zones/europe-west1-b/instances'
headers={"Authorization": "Bearer "+token}
resp=requests.post(url,headers=headers, data=data)
if resp.status_code==200:
Expand Down