prem-operator

README

📡 Prem Operator: On premise AI with Kubernetes

🗺 Overview

Deploy AI models and apps to Kubernetes without hitting every pitfall.

🚀 Quick Start

Deploy the operator and chat with Hermes in 2 or 3 steps. Note that you need a NVIDIA GPU with 8GB of RAM.

1. Install the NVIDIA Operator

2. Install the Prem Operator

   $ helm install latest oci://registry-1.docker.io/premai/prem-operator-chart
   

3. Deploy the Hermes + Big-AGI example and forward the ports

   $ kubectl apply -f examples/big-agi.yaml
   $ kubectl port-forward services/big-agi-service 3000:3000
   

If you browse to localhost:3000 you'll be able to begin chatting as soon as LocalAI downloads the model.

https://github.com/richiejp/prem-operator/assets/988098/0f06b254-a1a0-4ae5-815a-ed84998f5c89

💌 Get in touch, or Wanna Contribute

Feel free to create an issue here or reach out to us on Prem's Discord on the "#operator" channel.

💖💖 We'd love to hear from you! 💖💖

🔗 Links

• Quick Starts
  • 🦙+🖥️Llama 3 + TUI + CLI
  • 🌿+φElia TUI + Phi-2
• Guides
  • 🦜️🔗Langchain
  • 🧩Ingress
  • 🌐Managed Clusters (GCP, AWS)
  • 📜Getting started
  • 📦Deployment
  • 👩‍💻Developer
  • 🧪vLLM
• Topics
  • 🔍Frequently Asked Questions
  • 🤝Contributing
  • 🎗️Issues
  • 🔧Troubleshooting
• Blogs
  • 💡Introducing Prem-Operator

Simplified Architecture Diagram

📝 License

Prem-Operator is under Apache 2.0 license. See the LICENSE file for details.