GitHub - predibase/lorax: Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

LoRAX (LoRA eXchange) is a framework that allows users to serve thousands of fine-tuned models on a single GPU, dramatically reducing the cost of serving without compromising on throughput or latency.

📖 Table of contents

• 📖 Table of contents


• 🌳 Features


• 🏠 Models


• 🏃‍♂️ Getting started with Docker
  
  
  • Launch LoRAX Server
  
  
  • Prompt via REST API
  
  
  • Prompt via Python

... See more