As I See It: The Hunt For Perfect Storage
June 11, 2018 Victor Rozek
I have a garage so full of stuff my car winters outdoors. For all I know, D.B. Cooper stashed his loot in there, which is why it has never been found. After a fruitless hour searching for a power washer attachment, I would have been happy to set the whole mess ablaze.
If the first half of my life was about acquiring stuff, the second half is about trying to get rid of it. Without much success. Abundance, of course, is not the worst difficulty ever to befall humankind, but like too much of any good thing, indulging yourself is great – until it isn’t.
The quandary is that living space, like disc space, is fixed; while acquisition, like data, is dynamic and growing. Which is how I ended up building a $2,000 shed to house $500 worth of junk. But since misery is grateful for company, I’m pleased to report I’m not the only one who is congestion-challenged. The titans of Information Technology share the same frustrations managing excess. OK, so my stuff is junk and their stuff is mostly junk data, but it’s no less problematic for that, and it’s amassing faster than the national debt. And as hard as getting rid of stuff can be, keeping data in near perpetuity presents its own set of travails.
An hour’s worth of video is uploaded to YouTube every second; sixty hours of video every minute. Over 4 billion videos are viewed each day. And all that stuff needs to be squeezed into some version of an organized virtual garage. Google has an estimated one million servers storing 10 exabytes to 15 exabytes of data. Backup must be a bitch. Columnist or CEO, the great equalizer is that we’re all drowning in something, and we’re all looking for better storage.
Usually, newer technology provides a fix for problems created by older technology. But storing and retrieving mountains of data across timeframes that span centuries must account for the fact that every new solution renders previous solutions obsolete and eventually unusable. Today’s data is written and stored on technology that will soon be as archaic as the telegraph. At best, current storage media has a half-life and will eventually degrade. Formatting advances and hardware incompatibilities make data transient. So if you have something precious stored on 5¼ -inch floppies, good luck reading them once eBay runs out of floppy drives. (As a last resort, however, you can always check for floppy drives in my garage.)
Perhaps the bigger problem is the accelerating volume of data being produced. Here are some daunting figures courtesy of Forbes: More data has been created in the past two years than in the entire previous history of the human race. By the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet. By then, our accumulated digital universe will grow from 4.4 zettabytes today, to around 44 zettabytes. That’s 44 trillion gigabytes of data, or roughly the amount of junk crammed into my shed.
Since there are physical limits to how much data can be squeezed onto today’s storage options, and practical limits to how long storage media lasts before it degrades, research has shifted to molecular science, experimental physics, and even biology to solve the challenges of long-term storage, preservation, and retrieval.
Recently a team of engineers, chemists, and physicists collaborated to create a so-called wet information storage system that suspends nanoparticles in liquid and can store up to 1 terabyte of data in a tablespoon of solution. There are, however, a number of remaining challenges, perhaps the most notable is the ability to read the data back without having to use a scanning tunneling microscope.
Other researchers are experimenting with atomic-sized defects in diamonds that can store vast amounts of data and offer endless re-write ability with virtually zero degradation of data over time. As hazily explained in a phys.org article titled, Defects in Diamond: A unique platform for optical data storage in 3D, researchers “seek to circumvent data storage limits by exploiting the charge state and spin properties of the Nitrogen-Vacancy (NV) center in diamond.” Yeah, that was my next guess. There is one caveat however: the diamonds must be kept in the dark.
Another group in the United Kingdom developed a silver dollar-sized quartz disc designed to store up to 360 terabytes of data. And if durability is a concern, researchers claim their invention has the potential to keep that data viable for billions of years, give or take an eon. And when the solar system collapses and Earth burns to a cinder, these discs will survive longer than whatever is still living, enduring heat up to 1,000 degrees Celsius. Makes me feel warm all over.
Data storage at the atomic level also shows a great deal of promise. At least in theory, all the books ever written could be inscribed on material the size of a postage stamp. You can lick your way through War and Peace.
But perhaps the most fashionable, if not the most viable storage medium being researched is DNA. Over five years ago two separate groups of researchers were able to store data on synthetic DNA. Initially, the amount of data stored was modest, but the potential density is said to be “almost unfathomable.”
Under ideal conditions, the molecule can survive up to 6.8 million years, although the DNA strands begin to decay at roughly 1.5 million years, so if you need to retrieve your data, best hurry. The downside to DNA storage is its prohibitive cost.
All of this is beyond impressive, but if you only need to keep your data for, say, a modest 50 years, good old magnetic tape is still your best bet.
If my experience is any indication, stuff always expands to fill all available space, and then will invariably spill into unavailable space; or, in the case of data, into unwanted hands. I frankly fail to see the virtue of keeping every scrap of useless data for an indefinite period of time.
The larger question is: Why bother with any of it? After all, we’re not only talking about saving things of scientific or historic import. Why grant eternal life to vicious tweets, meaningless emails, social media posts extolling the joys of eating bacon, web sites that sell PEZ dispensers, and all the other insignificant minutiae that inhabits cyberspace. Will anyone really care hundreds of years from now?
At some point it makes better sense to clean out the garage than to build more sheds.
