Skip to content

Commit

Permalink
Merge pull request #28 from ahuang11/update_docs
Browse files Browse the repository at this point in the history
revamp docs and remove bokeh requirementss
  • Loading branch information
ahuang11 authored Apr 6, 2024
2 parents bc77907 + 6942503 commit 1ba531a
Show file tree
Hide file tree
Showing 16 changed files with 131 additions and 42 deletions.
15 changes: 11 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,12 @@ Connect multiple streams together to provide further context.
```python
from streamjoy import stream, connect

URL_FMTS = {
"visible": "https://www.goes.noaa.gov/dimg/jma/fd/vis/{i}.gif",
"infrared": "https://www.goes.noaa.gov/dimg/jma/fd/rbtop/{i}.gif",
}

if __name__ == "__main__":
URL_FMTS = {
"visible": "https://www.goes.noaa.gov/dimg/jma/fd/vis/{i}.gif",
"infrared": "https://www.goes.noaa.gov/dimg/jma/fd/rbtop/{i}.gif",
}
visible_stream = stream(
[URL_FMTS["visible"].format(i=i) for i in range(1, 11)],
intro_title="Himawari Visible",
Expand All @@ -146,6 +147,12 @@ if __name__ == "__main__":

You can also render images directly from datasets, either through a custom renderer or a built-in one, and they'll also run in parallel!

The following example requires xarray, cartopy, matplotlib, and netcdf4.

```bash
pip install xarray cartopy matplotlib netcdf4
```

```python
import numpy as np
import cartopy.crs as ccrs
Expand Down
1 change: 0 additions & 1 deletion docs/assets/design.svg

This file was deleted.

2 changes: 1 addition & 1 deletion docs/example_recipes/air_temperature.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Highlights:
- Imports `streamjoy.xarray` to use the `stream` accessor.
- Passes the `uri` to `stream` as the first argument to save the animation to disk.

```python
```python hl_lines="2 5"
import xarray as xr
import streamjoy.xarray

Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/co2_timeseries.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Highlights:
- Uses a custom `renderer` function to create each frame of the animation.
- Uses `Paused` to pause the animation at notable dates.

```python
```python hl_lines="4 19 114"
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.ticker import AutoMinorLocator
Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/gender_gapminder.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Highlights:
- Updates `fps` to 30 to create a smoother animation.
- Uses `connect` to concatenate the two heterogeneous streams (different keyword arguments with different titles) together.

```python
```python hl_lines="21-23 31 34"
import pandas as pd
from streamjoy import stream, connect

Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/nice_orbit.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Highlights:
- Uses `wrap_matplotlib` to automatically handle saving and closing the figure.
- Uses a custom `renderer` function to create each frame of the animation.

```python
```python hl_lines="45 46"
import numpy as np
import matplotlib.pyplot as plt
from numba import njit
Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/oisst_globe.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Highlights:
- Uses the built-in `default_xarray_renderer` under the hood
- Uses `renderer_kwargs` to pass keyword arguments to the underlying `ds.plot` method.

```python
```python hl_lines="19 32"
import cartopy.crs as ccrs
from streamjoy import stream

Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/sea_ice.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Highlights:
- Uses `pattern` to filter for only the sea ice concentration images.
- Uses `intro_title` and `intro_subtitle` to provide context at the beginning of the animation.

```python
```python hl_lines="3 6-9"
from streamjoy import stream, connect

connect(
Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/sine_wave.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Highlights:
- Uses `wrap_matplotlib` to automatically handle saving and closing the figure.
- Uses a custom `renderer` function to create each frame of the animation.

```python
```python hl_lines="5-6 14"
import matplotlib.pyplot as plt
import numpy as np
from streamjoy import stream, wrap_matplotlib
Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/stream_code.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Highlights:
- Uses a custom `renderer` function to create each frame of the animation.
- Propagates `formatter`, `max_line_length`, and `max_line_number` to the custom `renderer` function.

```python
```python hl_lines="51 102-104"
from textwrap import dedent

import numpy as np
Expand Down
2 changes: 1 addition & 1 deletion docs/example_recipes/temperature_anomaly.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Highlights:
- Uses a custom `renderer` function to create each frame of the animation.
- Uses `Paused` to pause the animation at notable dates.

```python
```python hl_lines="16 17 112"
import pandas as pd
import matplotlib.pyplot as plt
from streamjoy import stream, wrap_matplotlib, Paused
Expand Down
57 changes: 39 additions & 18 deletions docs/best_practices.md → docs/how_do_i.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
# Best practices
# How do I...

## ⏸️ Take advantage of `Paused`, `intro_pause`, `ending_pause`
## 🖼️ Use all resources with `max_frames=-1`

By default, StreamJoy only renders the first 50 frames to prevent accidentally rendering a large dataset.

To render all frames, set `max_frames=-1`.

```python
from streamjoy import stream

stream(..., max_frames=-1)
```

## ⏸️ How to pause animations with `Paused`, `intro_pause`, `ending_pause`

Animations can be good, but sometimes you want to pause at various points of the animation to provide context or to emphasize a point.

Expand All @@ -21,7 +33,7 @@ stream(..., renderer=plot_frame)

Don't forget there's also `intro_pause` and `ending_pause` to pause at the beginning and end of the animation!

## 📊 Decorate custom `renderer` with `wrap_*` decorators
## 📊 Reduce boilerplate code with `wrap_*` decorators

If you're using a custom `renderer`, you can use `wrap_matplotlib` and `wrap_holoviews` to automatically handle saving and closing the figure.

Expand All @@ -45,7 +57,7 @@ from streamjoy import stream
stream(..., intro_title="Himawari Visible", intro_subtitle="10 Hours Loop")
```

## 💾 Save to memory for testing purposes
## 💾 Write animation to memory instead of file

If you're just testing out the animation, you can save it to memory instead of to disk by calling write without specifying a uri.

Expand All @@ -55,7 +67,9 @@ from streamjoy import stream
stream(...).write()
```

## 🚪 Use accessors for `pandas` and `xarray` objects
## 🚪 Use as a method of `pandas` and `xarray` objects

StreamJoy can be used directly from `pandas` and `xarray` objects as an accessor.

```python
import pandas as pd
Expand All @@ -64,10 +78,10 @@ import streamjoy.pandas
df = pd.DataFrame(...)

# equivalent to streamjoy.stream(df)
df.stream(...)
df.streamjoy(...)

# series also works!
df["col"].stream(...)
df["col"].streamjoy(...)
```

```python
Expand All @@ -77,13 +91,13 @@ import streamjoy.xarray
ds = xr.Dataset(...)

# equivalent to streamjoy.stream(ds)
ds.stream(...)
ds.streamjoy(...)

# dataarray also works!
ds["var"].stream(...)
ds["var"].streamjoy(...)
```

## ⛓️ Use `sum` to join homogeneous streams and `connect` for hetereogenous
## ⛓️ Join streams with `sum` and `connect`

Use `sum` to join homogeneous streams, i.e. streams that have the same keyword arguments.

Expand All @@ -101,13 +115,13 @@ from streamjoy import stream, connect
connect([stream(..., **kwargs1), stream(..., **kwargs2)])
```

## 🎥 When to use `.mp4` vs `.gif`
## 🎥 Decide between writing as `.mp4` vs `.gif`

If you need a comprehensive color palette, use `.mp4` as it supports more colors.

For automatic playing and looping, use `.gif`. To reduce the file size of the `.gif`, set `optimize=True`, which uses `pygifsicle` to reduce the file size.

## 📦 Wrap `streamjoy` functionality under `__name__ == "__main__"`
## 📦 Prevent `RuntimeError` by using `__name__ == "__main__"`

If you run a `.py` script without it, you might encounter the following `RuntimeError`:

Expand All @@ -128,9 +142,16 @@ The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
```

To patch, simply wrap your `stream` call in `if __name__ == "__main__":`.

```python
if __name__ == "__main__":
stream(...)
```

It's fine without it in notebooks though.

## ⚙️ Change the default `config` settings
## ⚙️ Set your own default settings with `config`

StreamJoy uses a simple `config` dict to store settings. You can change the default settings by modifying the `streamjoy.config` object.

Expand All @@ -155,7 +176,7 @@ import streamjoy
streamjoy.config = {"max_frames": -1}
```

## 🔧 Explicitly set keyword arguments
## 🔧 Use custom values instead of the defaults

Much of StreamJoy is based on sensible defaults to get you started quickly, but you should override them.

Expand All @@ -167,11 +188,11 @@ StreamJoy will warn you on some settings if you don't override them:
No 'max_frames' specified; using the default 50 / 100 frames. Pass `-1` to use all frames. Suppress this by passing 'max_frames'.
```

## 🧩 Use `processes=False` for rendering HoloViews objects
## 🧩 Render HoloViews objects with `processes=False`

This is done automatically! However, in case there's an edge case, note that the kdims/vdims don't seem to carry over properly to the subprocesses when rendering HoloViews objects. It might complain that it can't find the desired dimensions.

## 📚 Use `threads_per_worker` if flickering
## 📚 Prevent flickering by setting `threads_per_worker`

Matplotlib is not always thread-safe, so if you're seeing flickering, set `threads_per_worker=1`.

Expand All @@ -181,7 +202,7 @@ from streamjoy import stream
stream(..., threads_per_worker=1)
```

## 🖥️ Specify `client` if using a remote cluster
## 🖥️ Provide `client` if using a remote cluster

If you're using a remote cluster, specify the `client` argument to use the Dask client.

Expand All @@ -192,7 +213,7 @@ client = Client()
stream(..., client=client)
```

## 🪣 Use `fsspec` to read/write intermediate files on a remote filesystem
## 🪣 Read & write files on a remote filesystem with `fsspec_fs`

```python
fs = fsspec.filesystem('s3', anon=False)
Expand Down
5 changes: 0 additions & 5 deletions docs/package_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,6 @@ I also was thinking of naming this `streamio` and `streamit`, but the former was

Below is a diagram of the package design. The animation part is actually quite simple--most of the complexity comes with handling various input types, e.g. URLs, files, and datasets.

<figure>
<img src="https://github.com/ahuang11/streamjoy/raw/main/docs/assets/design.svg" alt="StreamJoy package design" style="width:100%">
<figcaption>StreamJoy package design</figcaption>
</figure>

```mermaid
graph TD
A[Start] --> Z{Input Type}
Expand Down
64 changes: 62 additions & 2 deletions docs/supported_formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ stream(URL_DIR, uri="air_temperature.mp4", pattern="air.sig995.194*.nc")
<source src="https://github.com/ahuang11/streamjoy/assets/15331990/93cb0c1b-46d3-48e6-be2c-e3b1487f9117" type="video/mp4">
</video>

## 🧮 Numpy Array
## 🧮 Numpy NdArray

```python
from streamjoy import stream
Expand All @@ -45,6 +45,14 @@ stream(array, max_frames=-1).write("newtonscradle.mp4")

## 🐼 Pandas DataFrame or Series

!!! note "Additional Requirements"

You will need to additionally install `pandas` and `matplotlib` to support this format:

```bash
pip install pandas matplotlib
```

```python
from streamjoy import stream
import pandas as pd
Expand All @@ -61,20 +69,46 @@ stream(df, uri="gapminder.mp4", groupby="Country", title="{Year}")
</video>

## 🐻‍❄️ Polars DataFrame

!!! note "Additional Requirements"

You will need to additionally install `polars`, `pyarrow`, `hvplot`, `selenium`, and `webdriver-manager` to support this format:

```bash
pip install polars pyarrow hvplot selenium webdriver-manager
```

```python
from streamjoy import stream
import polars as pl

df = pl.read_csv(
"https://raw.githubusercontent.com/franlopezguzman/gapminder-with-bokeh/master/gapminder_tidy.csv"
)
df = df.query("Country in ['United States', 'China', 'South Africa']")
df = df.filter(pl.col("Country").is_in(['United States', 'China', 'South Africa']))
stream(df, uri="gapminder.mp4", groupby="Country", title="{Year}")
```

<video controls="true" allowfullscreen="true">
<source src="https://github.com/ahuang11/streamjoy/assets/15331990/fb07014c-8c63-46da-9ca0-1592e4649ccd" type="video/mp4">
</video>

## 🗄️ XArray Dataset or DataArray

!!! note "Additional Requirements"

You will need to additionally install `xarray` and `matplotlib` to support this format:

```bash
pip install xarray matplotlib
```

For this example, you will also need to install `pooch` and `netcdf4`:

```bash
pip install pooch netcdf4
```

```python
from streamjoy import stream
import xarray as xr
Expand All @@ -89,6 +123,32 @@ stream(ds, uri="air.mp4", cmap="RdBu_r")

## 📊 HoloViews HoloMap or DynamicMap

!!! note "Additional Requirements"

You will need to additionally install `holoviews` to support this format:

```bash
pip install holoviews
```

For the bokeh backend, you will need to install `bokeh`, `selenium`, and `webdriver-manager`:

```bash
pip install bokeh selenium webdriver-manager
```

For the matplotlib backend, you will need to install `matplotlib`:

```bash
pip install matplotlib
```

For this example, you will also need to install `pooch`, `netcdf4, `hvplot`, and `xarray`:

```bash
pip install pooch netcdf4 hvplot xarray
```

```python
import xarray as xr
import hvplot.xarray
Expand Down
Loading

0 comments on commit 1ba531a

Please sign in to comment.