The problem of sparsity also has safety implications. If, in some future time, you’re developing a superintelligent AI with vast capabilities, driven by ε-greedy reinforcement learning, and you decide you’ll reward it with one point if it cures cancer and zero otherwise, watch out—because it will have to try a lot of random things before it stumble

... See more