Deep Learning Is Hitting a Wall

With all the challenges in ethics and computation, and the knowledge needed from fields like linguistics, psychology, anthropology, and neuroscience, and not just mathematics and computer science, it will take a village to raise to an AI.

"There is no one way the mind works, because the mind is not one thing. Instead, the mind has parts, and the different parts of the mind operate in different ways: Seeing a color works differently than planning a vacation, which works differently than understanding a sentence, moving a limb, remembering a fact, or feeling an emotion.” Trying to squ... See more

Deep learning systems are black boxes; we can look at their inputs, and their outputs, but we have a lot of trouble peering inside. We don’t know exactly why they make the decisions they do, and often don’t know what to do about them (except to gather more data) if they come up with the wrong answers...Hybrids that allow us to connect the learning

... See more

For at least four reasons, hybrid AI, not deep learning alone (nor symbols alone) seems the best way forward:

1. So much of the world’s knowledge, from recipes to history to technology is currently available mainly or only in symbolic form.


2. Deep learning on its own continues to struggle even in domains as orderly as arithmetic. A hybrid system may have

... See more

To think that we can simply abandon symbol-manipulation is to suspend disbelief.

Belittling unfashionable ideas that haven’t yet been fully explored is not the right way to go.

But symbols on their own have had problems; pure symbolic systems can sometimes be clunky to work with, and have done a poor job on tasks like image recognition and speech recognition; the Big Data regime has never been their forté. As a result, there’s long been a hunger for something else.

Classical computer science, of the sort practiced by Turing and von Neumann and everyone after, manipulates symbols in a fashion that we think of as algebraic, and that’s what’s really at stake.