A rough analogy to the current LLM process is that making a new model is like baking a cake. You figure out your data and algorithms—like mixing the batter—and then you pretrain the model, that is, run it on a large number of computers for several months—like putting it in the oven—and then at the end you do some “post training”—like frosting and d... See more

The thing is, once it's clear what to do, the technical parts may be challenging, but they're pretty much achievable. What's really complicated is everything that happens before that. Before there's a working piece of software, a bunch of people need to communicate through different languages, since there are only humans in between the customer and... See more

Take the humble “document” as an example. For decades, document editing programs like word processors effectively emulated a printed sheet of paper, onto which the user typed with an emulated typewriter. Other software tools like spreadsheets did better, managing to escape complete skeuomorphism in favor of an infinite canvas. Notion is another goo... See more