AI - Ongoing Model Improvements

While LLMs continue to devour web-scraped data, they’ll increasingly consume their own digital progeny as AI-generated content continues to flood the internet. This recursive loop, experimentally confirmed, erodes the true data landscape. Rare events vanish first. Models churn out likely sequences from the original pool while injecting their own un... See more

t doesn’t help that the two most impressive implementations of AI for real work - Claude’s artifacts and ChatGPT’s Code Interpreter - are often hidden and opaque

One way of thinking about this is Daniel Kahneman’s simple model of thinking: System 1 and System 2. System 1 thinking is fast and intuitive. Current AI models’ pattern recognition and next-token prediction are good examples of this. System 2 thinking is slow and analytical, akin to genuine reasoning and understanding. It is System 2 thinking where... See more

GPT-4 and 4-Turbo have always been available for free in Microsoft Copilot. 4o is now free in ChatGPT. Claude 3.5 Sonnet is now free in Claude. So there are many that have never subscribed to a premium plan that have been using the best models all along still

First, operationalisation takes a long time, often revealing hidden potential in existing technologies. As Jack Clark notes, “if we stopped all AI progress today, there’s a huge capability overhang”. Even without further model development, building the right scaffolding can unlock surprising capabilities. This scaffolding isn’t just software; it in... See more

But there's just something about the immediacy and interactivity of Claude's "Artifacts" window, combined with the model's less stilted tone that brings it home.

For the most avarege users, these seemingly minor difference matter so much more than raw performance on abstract LLM benchmarks.

I still think that we are thinking about scaling of network-based intelligence looking at this particular network. An extended network of machines and humans in larger networks facilitated by smaller and increasingly embodied devices might in themselves add an AI-augmented neocortex to our (human-machine) systems. Intelligence will continue to emer... See more

it is kind of surprising that none of the major AI labs seem to have put out any deep documentation aimed at non-specialists. There are some guides for programmers or serious prompt engineers, but remarkably little aimed at non-technical folks who actually want to use these systems to do stuff - the vast majority of users

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more