Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The bike-rentals-auto-ml image has 1.6 GiB #95

Open
adelton opened this issue Sep 22, 2023 · 3 comments
Open

The bike-rentals-auto-ml image has 1.6 GiB #95

adelton opened this issue Sep 22, 2023 · 3 comments
Labels
kind/question Further information is requested priority/normal An issue with the product; fix when possible

Comments

@adelton
Copy link
Contributor

adelton commented Sep 22, 2023

Unlike the tensorflow-housing image that has 120.6 MiB (at https://quay.io/repository/rhoai-edge/tensorflow-housing?tab=tags&tag=latest, and also reproduced in my setup), the bike-rentals-auto-ml reported size is 1.6 GiB, at https://quay.io/repository/rhoai-edge/bike-rentals-auto-ml?tab=tags and again in my setup -- see #93 (comment) and #93 (comment).

Assuming both just ship a runtime without bundling the AI model, there seems to be some serious bloat in bike-rentals-auto-ml.

The pipelines/README.md should likely explain why those container images differ by an order of magnitude in size, what is so special about the second one that it warrants the extra bits.

@piotrpdev
Copy link
Member

Assuming both just ship a runtime without bundling the AI model, there seems to be some serious bloat in bike-rentals-auto-ml.

Can you elaborate? Both include the serving runtime and model.

The serving runtime image alone for bike-rentals is almost 900MB compressed: seldonio/mlserver:1.3.5-slim. It is large, but supports a wide range of models: https://github.com/SeldonIO/MLServer#inference-runtimes

# Create the MLServer container. Use the slim image, since we are providing an environment tarball.
#
FROM docker.io/seldonio/mlserver:1.3.5-slim

The bike-rentals image also contains all of the dependencies for the model:

# Download model dependencies and create a portable tarball
# The tarball is placed inside the model directory.
RUN . ~/miniconda3/bin/activate && \
conda env create -n mlflow-env -f model/conda.yaml && \
conda activate mlflow-env && \
pip install mlserver-mlflow && \
conda list && \
conda deactivate && \
conda activate && \
conda install conda-pack && \
conda-pack -n mlflow-env -o model/environment.tar.gz
# Create the MLServer container. Use the slim image, since we are providing an environment tarball.
#
FROM docker.io/seldonio/mlserver:1.3.5-slim

See #27 for seldonio/mlserver vs openvino/mlserver.

@adelton
Copy link
Contributor Author

adelton commented Sep 26, 2023

Our discussions typically revolved around the model being huge. In this case though, the model is small (54 MB) and the runtime is huge.

I wonder if we are able to demonstrate the behaviour with a runtime that might not be so generic but whose size would be more palatable.

@piotrpdev
Copy link
Member

seldonio/mlserver and openvino/mlserver are popular and cover most of the model frameworks and types/flavors. If we're looking for the best compatibility using these along with MLflow is probably the best option.

We can take the KServe approach and write our own runtimes if we want many smaller runtimes instead of a couple of large ones ¯\(ツ)/¯.

@piotrpdev piotrpdev added the kind/question Further information is requested label Oct 5, 2023
@piotrpdev piotrpdev added the priority/normal An issue with the product; fix when possible label Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/question Further information is requested priority/normal An issue with the product; fix when possible
Projects
None yet
Development

No branches or pull requests

2 participants