Example K8 Manifest

[sudslab]

Anyone have luck getting MPT-7b or Falcon 7B running in Kubernetes?

• AWS, GCP, etc
• Example Manifest.yaml
• Machine Type
• GPU Support
• MLOps platform
• etc

Or would you recommend looking at Sagemaker? We currently run both a staging & production AWS EKS Cluster but don't have a node group specifically tailored to run an LLM