Sublime
An inspiration engine for ideas
With all the challenges in ethics and computation, and the knowledge needed from fields like linguistics, psychology, anthropology, and neuroscience, and not just mathematics and computer science, it will take a village to raise to an AI.
Gary Marcus • Deep Learning Is Hitting a Wall
For at least four reasons, hybrid AI, not deep learning alone (nor symbols alone) seems the best way forward:
- So much of the world’s knowledge, from recipes to history to technology is currently available mainly or only in symbolic form.
- Deep learning on its own continues to struggle even in domains as orderly as arithmetic. A hybrid system may have
Gary Marcus • Deep Learning Is Hitting a Wall
Classical computer science, of the sort practiced by Turing and von Neumann and everyone after, manipulates symbols in a fashion that we think of as algebraic, and that’s what’s really at stake.
Gary Marcus • Deep Learning Is Hitting a Wall
But symbols on their own have had problems; pure symbolic systems can sometimes be clunky to work with, and have done a poor job on tasks like image recognition and speech recognition; the Big Data regime has never been their forté. As a result, there’s long been a hunger for something else.
Gary Marcus • Deep Learning Is Hitting a Wall
o, alternative pathways to building Type-2 reasoning-capable AI systems, likely using neurosymbolic approaches, have become much more attractive. People like Gary Marcus have argued for neurosymbolic approaches for decades. Such approaches combine the pattern recognition of neural nets, like LLMs, with symbolic reasoning’s logic and rules. Vinod Kh... See more
Azeem Azhar • 🧠 AI’s $100bn question: The scaling ceiling
Artificial Intelligence: A Guide for Thinking Humans
amazon.com
Since the generative AIs have been trained on the entirety of human work — most of it mediocre — it produces “wisdom of the crowd”-like results. They may hit the mark but only because they are average.
Kevin Kelly • Interview: Kevin Kelly, Editor, Author, and Futurist
There are serious holes in the scaling argument. To begin with, the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.