Skip to content

Latest commit

 

History

History
26 lines (20 loc) · 822 Bytes

README.md

File metadata and controls

26 lines (20 loc) · 822 Bytes

my-gpt-backend

Python backend for My-GPT.

Requirements:

  • CMake and Make
  • Python 3.8+ and Pip 3
  • RabbitMQ
  • CUDA Toolkit and related drivers (if your system has a GPU)
  • Orca-Mini-3B model file

Setup:

git clone https://github.com/aadarsh-ram/my-gpt-backend.git
cd my-gpt-backend

python -m venv venv
. venv/bin/activate

chmod +x ./requirements-setup.sh
./requirements-setup.sh
  • Download the model file, convert it to GGUF format using this script.
  • Update .env file with the path to the model file in the MODEL_PATH variable. Make any necessary changes for the RabbitMQ server.
  • Run the backend with python3 app.py