The Alignment Problem
Unexpected Rewards When researchers test people using expected and unexpected rewards, there is greater activation of anticipation and reward structures in the brain when the reward is unexpected (Berns 2001). Basically, people have a much stronger response to unexpected rewards than they do to ones they know are coming.
Julie Dirksen • Design for How People Learn (Voices That Matter)
Go. The problem is colossal for a simple reason: it is only at the very end that the system receives a single reward signal, indicating whether the game was won or lost. During the game itself, the system receives no feedback whatsoever—only the final checkmate counts.
Stanislas Dehaene • How We Learn: Why Brains Learn Better Than Any Machine . . . for Now
The world is combinatorially weird and fractally interesting. And therefore, omnivorous curiosity is the only proper response. ... let’s optimize instead for the interesting, the strange, and the weird. Ideas and topics that ignite our curiosity are worthy of our attention, because they might lead to advances and insights that we can’t anticipate.
📡 No.317 — From utopian Star Trek to absurdist Douglas Adams? ⊗ How to fix “AI’s original sin” ⊗ Islands of coherence
Studying babies makes us realize that the biological computers on this planet differ from the man-made computers in this regard, as well. They don’t just compute, learn, reason, and know. They are driven to do all these things and are designed to take intense pleasure in doing so.