Skip to content

Latest commit

 

History

History
38 lines (30 loc) · 1.13 KB

aws_setup.md

File metadata and controls

38 lines (30 loc) · 1.13 KB

Initial AWS Setup

Log into AWS

  1. AWS Login URL
  2. Use the name and temporary password in your email.

Install Spark Locally

  1. Go to the spark download page
  2. Download
  3. Untar the package
  • OS X: Archive Utility
  • *NIX: cd ~/Downloads; tar zxvf spark-1.2.1-bin-hadoop2.4.tgz
  • Windows: Install OS X or *NIX

Set up the Spark Cluster Login Scripts

  1. Place your acces key id and secret access key in the empty quotes in the first two lines
  2. Put SOMETHING in the AWSPROFILE line
  3. Download HackademyKey.pem, and place it in the ~/.ssh directory
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWSPROFILE=""
export MYKEYPAIR="HackademyKey"
export MYKEYPAIRFILE="$HOME/.ssh/${MYKEYPAIR}.pem"
cd /opt/spark/ec2
./spark-ec2 --key-pair=${MYKEYPAIR} --identity-file=${MYKEYPAIRFILE} login hackademy

Verify That you can Create EC2 Cluster