Maintaining the speed of search gets harder as Enigma.io adds more data; by mid-2013, the company had 100,000 datasets (essentially, 100,000 large data tables) and more than 20 billion individual data points. On the other hand, figuring out how to incorporate new datasets gets easier over time, because there are only so many different kinds of data

present significant challenges. Behind the scenes the industry is moving from a rigid model of individual servers or designated banks of servers towards a much more scalable and dynamic model that can respond almost instantly to changes and shifts in computational demand.

Loading big chunks of text into a “blob” field in a database is possible, but it really isn’t the best use of the database or the best way to handle such data. MapReduce can help.