Language Is Our Latent Space

I think the best way to understand what an “idea” really is as a node in a network of other ideas, and large language models like GPT-3 are the best maps we currently have of that network. In general, having a map makes you much better at navigating terrain.

Language Models Represent Space and Time , Wes Gurnee and Max Tegmark argue that “modern LLMs acquire structured knowledge about fundamental dimensions such as space and time, supporting the view that they learn not merely superficial statistics, but literal world models”, based on some analyses of the alleged capacity of LLMs to understand geograp... See more