Ahead of AI #12: LLM Businesses and Busyness
- What a modern LLM does during training is, essentially, very very quickly skim the textbook, the words just flying by , not spending much brain power on it.
- Rather, when you or I read that math textbook, we read a couple pages slowly; then have an internal monologue about the material in our heads and talk about it with a few study-buddies; read an
SITUATIONAL AWARENESS - The Decade Ahead • I. From GPT-4 to AGI: Counting the OOMs
In streaming settings, StreamingLLM outperforms the sliding window recomputation baseline by up to 22.2x speedup.
mit-han-lab • GitHub - mit-han-lab/streaming-llm: Efficient Streaming Language Models with Attention Sinks
o, alternative pathways to building Type-2 reasoning-capable AI systems, likely using neurosymbolic approaches, have become much more attractive. People like Gary Marcus have argued for neurosymbolic approaches for decades. Such approaches combine the pattern recognition of neural nets, like LLMs, with symbolic reasoning’s logic and rules. Vinod Kh... See more