Iteration of the Prime Intellect
One of the byproducts of the massive telecommunications buildouts of the 80s and 90s is that there is an enormous quantity of unused fiber optic cable criscrossing the country. Some estimates have the capacity of the unused cable at several times that of the currently operating networks. Over the past few years, Google (who else?) has been quietly buying a lot of this ‘dark’ fiber. People have noticed, people have guessed, but no one really knew what they were going to do with it. Now Cringely explains why:
Google hired a pair of very bright industrial designers to figure out how to cram the greatest number of CPUs, the most storage, memory and power support into a 20- or 40-foot box. We’re talking about 5000 Opteron processors and 3.5 petabytes of disk storage that can be dropped-off overnight by a tractor-trailer rig. The idea is to plant one of these puppies anywhere Google owns access to fiber, basically turning the entire Internet into a giant processing and storage grid.
Why is this a big deal? If you read The Metamorphosis of Prime Intellect this might sound very familiar to you. Indeed, the results are similar. Google will now have the ability to do all of the things it wants to do, faster and more clearly. They’re building what is essentially a “virtual multicast” network, which means they can take really bandwidth and computation-intensive tasks and put them out on repeater clusters that are very close, network-wise, to you. That way they don’t have to handle 50 million streaming videos from their central site, they just push out to 300 (and growing) sites, who stream it to 170,000 each. This is an enormous win; in bandwidth terms it is similar to finding a media codec that gave two orders of magnitude better compression. In computational terms it’s even better. This enables a whole new generation of applications, most of which Cringely touches on. But I don’t think we’ll understand the full ramifications of this change for many years to come.
