Skip to content

ekaratnida/Data_Streaming_and_Realtime_Analytics

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

902 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data_Streaming_and_Realtime_Analytics

Week Topic Materials Link
1 Introduction + review on big data https://drive.google.com/drive/folders/1pCP8i125ZpJEwhxtUQFqxcTLwWE1um-5?usp=sharing
2 Review on big data (continue) & data gathering https://drive.google.com/drive/folders/1pCP8i125ZpJEwhxtUQFqxcTLwWE1um-5?usp=sharing
3 Data streaming distribution ( Kafka I ) https://drive.google.com/drive/folders/1C-mIez_Mc0i1tjGtxDFuJSRujNnh6qoG?usp=sharing
4 Data streaming distribution ( Kafka II ) https://drive.google.com/drive/folders/1rc0S6aT-jAXDY4qahIJTDDI0w4jsqTZm?usp=sharing https://github.com/ekaratnida/kafka-connect
5 Processing I (Kafka streams) https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week03
6 Processing II (Kafka streams) https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week04
7 Processing III (Spark streaming) https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week06/pyspark https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/blob/main/Week07/Week7_twitter.ipynb
- No Midterm exam -
9 Online machine learning (classification) https://riverml.xyz/dev/introduction/installation/
Sagemaker: https://ssc.io/pdf/modin711s.pdf
Continual learning: https://www.sciencedirect.com/science/article/pii/S0893608019300231
10 Online machine learning (clustering) https://riverml.xyz/dev/introduction/installation/
11 Mobile IoT I https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week05
12 Mobile IoT II https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week08 (Quiz II)
13 Game I https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week10
14 Game II https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week10
15 - Trading
- Recommendation system
- Embedding
- https://github.com/ekaratnida/Data_Streaming_and_Realtime_Analytics/tree/main/Week14 (Quiz III)
- https://dl.acm.org/doi/pdf/10.1145/3132847.3133163?casa_token=hZ-azg6-eTYAAAAA:gtUUSX0FVu4QvN__SmAfL99x4lB2onSxR_S8f1qkbY8V6LY_EwSypym6pRCXKj_ZFxVRL2VLmAcdCg
- https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture

Install git

https://git-scm.com/download/win

Medium

https://pok-ekarat.medium.com/%E0%B8%AA%E0%B8%A3%E0%B9%89%E0%B8%B2%E0%B8%87%E0%B8%A3%E0%B8%B0%E0%B8%9A%E0%B8%9A-real-time-monitoring-%E0%B9%80%E0%B8%9A%E0%B8%B7%E0%B9%89%E0%B8%AD%E0%B8%87%E0%B8%95%E0%B9%89%E0%B8%99%E0%B8%94%E0%B9%89%E0%B8%A7%E0%B8%A2-python-influxdb-%E0%B9%81%E0%B8%A5%E0%B8%B0-grafana-625480a6e511

TBD

https://www.influxdata.com/what-is-time-series-data/

https://github.com/tklouie/PyData_LA_2018

Visualization

IMAGE ALT TEXT HERE

https://github.com/scriptkiid/Python-Plotly-Websockets-Dashboard

Last quiz (2563/2)

Quiz1 TF-IDF: https://www.youtube.com/watch?v=ulo_U6-qUwA

Quiz2 Mobile activity analytics: https://youtu.be/wjQ_gu9ityc

Quiz3 Game behavior analytics: https://youtu.be/UMXkgR5WOcY

Project

https://docs.google.com/document/d/1rxWEqy7B7MG0ILRNUJ32semZESPlA_f4bYdsYDJTPOI/edit

Interesting papers

Buy Me a Coffee at ko-fi.com

Miscellaneous

What is avro?

https://www.confluent.io/blog/avro-kafka-data/#:~:text=Avro%20is%20an%20open%20source,programming%20language%20of%20your%20choice.

What is Serdes?

https://www.clairvoyant.ai/blog/apache-kafka-serde#:~:text=The%20Apache%20Kafka%20provides%20a,present%20in%20kafka%2Dcleints%20jar.

Streaming Algorithms in Machine Learning

https://sagemaker-examples.readthedocs.io/en/latest/scientific_details_of_algorithms/streaming_median/streamingMedian.py.html

Use case

  1. https://www.youtube.com/watch?v=nIZ4tTlWIAc

About

This workshop teaches how Apache Kafka works and how you can use it to build applications that react to events as they happen.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • HTML 93.5%
  • Jupyter Notebook 6.2%
  • Python 0.2%
  • Java 0.1%
  • CSS 0.0%
  • JavaScript 0.0%