In general, people confabulating invent plausible sounding justifications which have no basis in fact. This is usually not conscious with an intent to deceive but instead appear to strongly believe in the story they have just reported. This behaviour is identical to what LLMs do. When they are forced to give an answer using a fact they do not know,... See more
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
An analysis of GPT-4V, a large multimodal model with visual understanding, discussing its capabilities, input modes, working modes, prompting techniques, and potential applications in various domains.
What a modern LLM does during training is, essentially, very very quickly skim the textbook, the words just flying by , not spending much brain power on it.
Rather, when you or I read that math textbook, we read a couple pages slowly; then have an internal monologue about the material in our heads and talk about it with a few study-buddies; read an