GitHub - SeldonIO/MLServer: An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
github.com
GitHub - SeldonIO/MLServer: An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
Deep-ML
deep-ml.comLoRAX (LoRA eXchange) is a framework that allows users to serve thousands of fine-tuned models on a single GPU, dramatically reducing the cost of serving without compromising on throughput or latency.
📖 Table of contents
📖 Table of contents
- 📖 Table of contents
- 🌳 Features
- 🏠 Models
- 🏃♂️ Getting started with Docker
- Launch LoRAX Server
- Prompt via REST API
- Prompt via Python
predibase • GitHub - predibase/lorax: Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
OpenLLMetry is a set of extensions built on top of OpenTelemetry that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Datadog, Honeycomb, and others.
It's built and maintained by Traceloop under the Apache 2.0 license.
The repo con... See more
It's built and maintained by Traceloop under the Apache 2.0 license.
The repo con... See more
traceloop • GitHub - traceloop/openllmetry: Open-source observability for your LLM application, based on OpenTelemetry
Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax. EasyLM can scale up LLM training to hundreds of TPU/GPU accelerators by leveraging JAX's pjit functionality.
Building on top of Hugginface's transformers and datasets, this repo provides an easy to use and easy... See more
Building on top of Hugginface's transformers and datasets, this repo provides an easy to use and easy... See more