Salad - GPU Cloud | 10k+ GPUs for Generative AI

dstack is an open-source toolkit and orchestration engine for running GPU workloads. It's designed for development, training, and deployment of gen AI models on any cloud.

Supported providers: AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, and DataCrunch.

Latest news ✨

• [2024/01] dstack 0.14.0: OpenAI-compatible endpoints preview (Release)


• [2023/12] dst

... See more

In 2019, OpenAI announced GPT-2 with this post: https://t.co/jjP8IXmu8D Today (~5 years later) you can train your own for ~$672, running on one 8XH100 GPU node for 24 hours. Our latest llm.c post gives the walkthrough in some detail: https://t.co/XjLWE2P0Hp... See more

Deploying a Generative AI model requires more than a VM with a GPU. It normally includes:

• Container Service : Most often Kubernetes to run LLM Serving solutions like Hugging Face Text Generation Inference or vLLM.


• Compute Resources : GPUs for running models, CPUs for management services


• Networking and DNS : Routing traffic to the appropriate servic

... See more

GPU-accelerated databases are mind-blowing! ​ Imagine a database natively integrated with best-in-class AI foundational models: ​ • Zero warmup latency • Massive GPU-backed scalability • Ability to process your data with any model • Ability to train and fine-tune models on your data ​ There are 1.7 million deployments of PostgreSQL worldwide, one o... See more