Skip to content

Commit f46949b

Browse files
authored
Merge pull request #125 from marius-tw/master
123 | mh | fixes #123
2 parents dc517ba + 93b8e3c commit f46949b

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

versioned_docs/version-2.0/beyond-the-batch/checkpointing.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ Checkpoints in Spark can be leveraged to their full benefit if we fulfill the fo
2626
2. Use a source (e.g. File Source, Kafka Source) from which data can be replayed (this is useful for incomplete data in a certain micro-batch).
2727
3. Processing logic is consistent and idempotent (the same result is obtained when given the same input data)
2828

29-
Let us now look at an example of how checkpoints are actually created. We will use the same Stateful Streaming Wordcount code from previous sections[hyperlink] and add checkpointing code to it.
29+
Let us now look at an example of how checkpoints are actually created. We will use the same Stateful Streaming
30+
Wordcount code from previous [sections](https://data-derp.github.io/docs/2.0/beyond-the-batch/stateful-vs-stateless-streaming/) and add checkpointing code to it.
3031

3132
1. In the writestream code of Stateful Streaming Wordcount example, add one more option to enable checkpoints when the program runs as shown below (make sure that you are giving a location somewhere in the user directory of your local machine as the other locations might not give the privilege to your program to create a directory and write data to it)
3233
![Checkpoint-Code.png](./assets/Checkpoint-Code.png)

0 commit comments

Comments
 (0)