Data_Streaming_and_Realtime_Analytics

Week	Topic	Materials Link
1	Introduction + review on big data	https://drive.google.com/drive/folders/1pCP8i125ZpJEwhxtUQFqxcTLwWE1um-5?usp=sharing
2	Review on big data (continue) & data gathering	https://drive.google.com/drive/folders/1pCP8i125ZpJEwhxtUQFqxcTLwWE1um-5?usp=sharing
3	Data streaming distribution ( Kafka I )	https://drive.google.com/drive/folders/1C-mIez_Mc0i1tjGtxDFuJSRujNnh6qoG?usp=sharing
4	Data streaming distribution ( Kafka II )	https://drive.google.com/drive/folders/1rc0S6aT-jAXDY4qahIJTDDI0w4jsqTZm?usp=sharing https://github.com/ekaratnida/kafka-connect
5	Processing I (Kafka streams)	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week03
6	Processing II (Kafka streams)	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week04
7	Processing III (Spark streaming)	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week06/pyspark https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/blob/main/Week07/Week7_twitter.ipynb
-	No Midterm exam	-
9	Online machine learning (classification)	https://riverml.xyz/dev/introduction/installation/ Sagemaker: https://ssc.io/pdf/modin711s.pdf Continual learning: https://www.sciencedirect.com/science/article/pii/S0893608019300231
10	Online machine learning (clustering)	https://riverml.xyz/dev/introduction/installation/
11	Mobile IoT I	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week05
12	Mobile IoT II	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week08 (Quiz II)
13	Game I	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week10
14	Game II	https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week10
15	- Trading - Recommendation system - Embedding	- https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week14 (Quiz III) - https://dl.acm.org/doi/pdf/10.1145/3132847.3133163?casa_token=hZ-azg6-eTYAAAAA:gtUUSX0FVu4QvN__SmAfL99x4lB2onSxR_S8f1qkbY8V6LY_EwSypym6pRCXKj_ZFxVRL2VLmAcdCg - https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture

Install git

https://git-scm.com/download/win

Medium

https://pok-ekarat.medium.com/%E0%B8%AA%E0%B8%A3%E0%B9%89%E0%B8%B2%E0%B8%87%E0%B8%A3%E0%B8%B0%E0%B8%9A%E0%B8%9A-real-time-monitoring-%E0%B9%80%E0%B8%9A%E0%B8%B7%E0%B9%89%E0%B8%AD%E0%B8%87%E0%B8%95%E0%B9%89%E0%B8%99%E0%B8%94%E0%B9%89%E0%B8%A7%E0%B8%A2-python-influxdb-%E0%B9%81%E0%B8%A5%E0%B8%B0-grafana-625480a6e511

TBD

https://www.influxdata.com/what-is-time-series-data/

https://github.com/tklouie/PyData_LA_2018

Visualization

https://github.com/scriptkiid/Python-Plotly-Websockets-Dashboard

Last quiz (2563/2)

Quiz1 TF-IDF: https://www.youtube.com/watch?v=ulo_U6-qUwA

Quiz2 Mobile activity analytics: https://youtu.be/wjQ_gu9ityc

Quiz3 Game behavior analytics: https://youtu.be/UMXkgR5WOcY

Project

https://docs.google.com/document/d/1rxWEqy7B7MG0ILRNUJ32semZESPlA_f4bYdsYDJTPOI/edit

Interesting papers

• https://arxiv.org/pdf/2104.00087.pdf

Miscellaneous

What is avro?

https://www.confluent.io/blog/avro-kafka-data/#:~:text=Avro%20is%20an%20open%20source,programming%20language%20of%20your%20choice.

What is Serdes?

https://www.clairvoyant.ai/blog/apache-kafka-serde#:~:text=The%20Apache%20Kafka%20provides%20a,present%20in%20kafka%2Dcleints%20jar.

Streaming Algorithms in Machine Learning

https://sagemaker-examples.readthedocs.io/en/latest/scientific_details_of_algorithms/streaming_median/streamingMedian.py.html

Use case

1. https://www.youtube.com/watch?v=nIZ4tTlWIAc