Interface > Data > Model: The “interface” and “data” layers will further distinguish market leaders while the “models” layer becomes increasingly commoditized and pushed to the edge. As a growing number of our everyday use-cases of powerful GenerativeAI models fall below the frontier of “the best models,” they will be enabled by cheaper commoditize... See more

core components of Deep RL that enabled success like AlphaGo: self-play and look-ahead planning.

Self-play is the idea that an agent can improve its gameplay by playing against slightly different versions of itself because it’ll progressively encounter more challenging situations. In the space of LLMs, it is almost certain that the largest portion o... See more

pair-preference-model-LLaMA3-8B by RLHFlow: Really strong reward model, trained to take in two inputs at once, which is the top open reward model on RewardBench (beating one of Cohere’s).

DeepSeek-V2 by deepseek-ai (21B active, 236B total param.): Another strong MoE base model from the DeepSeek team. Some people are questioning the very high MMLU sc... See more

The right way to use ChatGPT going forward might be to follow the programmer’s maxim that if you do it three times, you should automate it, except now the threshold might be two and if something is nontrivial it also might be one. You can use others’ versions, but there is a lot to be said for rolling one’s own if the process is easy. If it works w... See more

We are excited about (i) NLP use cases (ii) built by industry experts that (iii) serve large, tech-laggard industries with intensely manual workflows for rich text data that (iv) often sit across multiple systems of record.

Jazz.ToolsPowerSyncFireproofAutomergeDXOSElectricSQLand of course, Berlin's own: Yjs .

Study finds RLHF reduces LLM creativity and output variety : A new research paper posted in /r/LocalLLaMA shows that while alignment techniques like RLHF reduce toxic and biased content, they also limit the creativity of large language models, even in contexts unrelated to safety.

How do models represent style, and how can we more precisely extract and steer it?

A commonly requested feature in almost any LLM-based writing application is “I want the AI to respond in my style of writing,” or “I want the AI to adhere to this style guide.” Aside from costly and complicated multi-stage finetuning processes like Anthropic’s RL with... See more

The multiple cantilevered AI overhangs:

Compute overhang. We have much more compute than we are using. Scale can go much further.

Idea overhang. There are many obvious research ideas and combinations of ideas that haven’t been tried in earnest yet.

Capability overhang. Even if we stopped all research now, it would take ten years to digest the new capa... See more