r/LocalLLaMA - Reddit

r/LocalLLaMA - Reddit

RelatedHighlights

My recommendation for anyone looking to do ML work, especially training LLMs, is to use a cloud service like Lambda Labs. You'll spend less time training and you'll still be able to code while it's going on.

The 36GB RAM is dynamically shared between your system and your GPU. If you're planning to run containers and an IDE and a browser alongside yo... See more

r/MachineLearning - Reddit

Training great LLMs entirely from ground zero in the wilderness as a startup — Yi Tay

Yi Tay yitay.net

Thumbnail of Training great LLMs entirely from ground zero in the wilderness as a startup — Yi Tay

Thinking Like an AI

Ethan Mollick oneusefulthing.org

Thumbnail of Thinking Like an AI

eneral-purpose models

1.1B: TinyDolphin 2.8 1.1B. Takes about ~700MB RAM and tested on my Pi 4 with 2 gigs of RAM. Hallucinates a lot, but works for basic conversation.

2.7B: Dolphin 2.6 Phi-2. Takes over ~2GB RAM and tested on my 3GB 32-bit phone via llama.cpp on Termux.

7B: Nous Hermes Mistral 7B DPO. Takes about ~4-5GB RAM depending on contex

... See more

r/LocalLLaMA - Reddit