LLM Inference Repository

This repository hosts code related to Large Language Models (LLM) inference, including references and examples. It aims to provide resources for leveraging LLMs in various applications.

MK1 Endpoint

Inside the mk1 folder, you will find code for setting up a FastAPI endpoint using MK1 for inference with Flywheel. The endpoint allows you to deploy and interact with LLM models efficiently and securely.

Getting Started with Poetry

To set up the development environment using Poetry:

1. Install Poetry if you haven't already:

   curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

2. Clone this repository and navigate to the project directory:

   git clone https://github.com/albertoecf/llm_inferece.git
   cd llm_inference

3. Install dependencies using Poetry:

   poetry install

4. Activate the virtual environment created by Poetry:

   poetry shell

Now you're ready to start working with the LLM inference code in this repository! Refer to individual folders and files for specific instructions and examples.