-
Notifications
You must be signed in to change notification settings - Fork 1
Use Spark (python) from CLI
andreacondorelli edited this page Sep 1, 2015
·
1 revision
To launch from CLI:
spark-submit mySparkCode.py argv1 argv2 argv3
While in the '.py' file you must also create the spark context:
from pyspark import SparkContext, SparkConf
conf = SparkConf()
sc = SparkContext(conf=conf)
To read an input from CLI:
import sys
argList = argv[1:]
-
Sparking
- Spark
- Spark Streaming
- Contributors