spark-streaming-constantinputdstreams.adoc

ConstantInputDStreams

ConstantInputDStream is an input stream that always returns the same mandatory input RDD at every batch time.

ConstantInputDStream[T](_ssc: StreamingContext, rdd: RDD[T])

ConstantInputDStream dstream belongs to org.apache.spark.streaming.dstream package.

The compute method returns the input rdd.

Note	rdd input parameter is mandatory.

The mandatory start and stop methods do nothing.

Example

val sc = new SparkContext("local[*]", "Constant Input DStream Demo", new SparkConf())
import org.apache.spark.streaming.{ StreamingContext, Seconds }
val ssc = new StreamingContext(sc, batchDuration = Seconds(5))

// Create the RDD
val rdd = sc.parallelize(0 to 9)

// Create constant input dstream with the RDD
import org.apache.spark.streaming.dstream.ConstantInputDStream
val cis = new ConstantInputDStream(ssc, rdd)

// Sample stream computation
cis.print