While it is widely known that Pompeii was destroyed by the eruption of Mount Vesuvius, only serious historians will recall that this catastrophic event took place on August 24, 79 AD. If you are curious what typical Pompeiians ate for breakfast that morning nearly 2000 years ago, there are a few conserved bits of papyrus that will yield that information. Through the preservation and study of centuries-old artifacts, we have retained insight into the lives of our early ancestors.
Information archival has changed a lot since the days of dusty old books and scraps of paper. As of 2024, there is an estimated 11.23 zettabytes (11.23 trillion gigabytes) of digital data stored worldwide. We have placed our trust in the cloud and the digital storage hardware it requires. (see Read This If You Can for more on the cloud). Post things on the Internet with care, we are told - once there they will live on forever.
Except that they don’t. Today there are over a billion websites, and a quarter of a million new ones are created every day. According to a study by the Pew Research Center, 38% of web pages that existed in 2013 are gone. Two-thirds of the links in web pages from the last 9 years lead to the frustrating “404” message. Researchers recently noted that half of all hyperlinks in Supreme Court opinions lead to content that is either missing or has changed since it was originally linked. Sophisticated search engines like ChatGPT-4 make the Internet ever more accessible, but with link rot increasing, signs of Artificial Dementia are becoming apparent.
Although Indiana Jones with his iconic hat and relentless pursuit of historical objects isn’t coming to save us, there is hope. The American non-profit Internet Archive, founded in 1996 by Brewster Kahl, has loaded up a handful of data centers around the world with 866 billion web pages, 42 million books, 13 million videos along with other assorted digital stuff. Their Wayback Machine is a trove of information that might otherwise be lost. This is an encouraging start, but there are problems.
Like with any non-profit, there is financial stress. This massive collection must be defended against cyberattacks (as recently as May 2024, the Internet Archives was tied up in a large DDoS attack). The owners of much of the intellectual property being preserved just don’t want their stuff made available for free, or perhaps at all. And finally, there is the storage technology itself – digital storage, much like human memory, is fragile by nature and can expire either through a slow aging process, or a sudden catastrophic event.
Most of us store our digital stuff on HDD’s or SSD’s. The typical lifetime for these media is 5-10 years, barring any sudden failures. Often, our storage demands grow so quickly that we upgrade our equipment before reaching those limits. Either way, these storage devices are not normally accessible to others. For protection, it is common to purchase cloud storage, which typically also lives on HDD’s and SSD’s. Tape storage is still widely used for digital archiving, and this extends the life of the data to a few decades. The closest thing to true archival storage is a high-grade M-DISC, which in theory can last hundreds or even thousands of years. Their widespread adoption seems unlikely - storing data on an HDD runs about 1.4 cents per gigabyte vs 5 cents per gigabyte for M-DISCS.
For a better appreciation of the problem, try typing https://example.com/fix-broken-links into your browser. We humans are a bit like the Internet – as we age, short term memory routinely fades, while longer term memory perseveres. While there is no cure for dementia – human or digital – there are treatments like the Internet Archives which can slow the progress and ease the symptoms.
From time to time, we all wonder how many farm animals were raised in Northumberland, England in the 11th Century. Well maybe not all of us, but for those who do there is the Domesday Book (an actual physical book). It is the oldest document residing in the UK National Archives, and has the information you are looking for, provided you understand Medieval Latin. For that, there is The Dictionary of Medieval Latin from British Sources (another legitimate hard copy tome).
The incredible technology of the Internet may have lured us into an archival complacency that is slowly destroying the connection we have to our history and our ancestors.
Author Profile - Paul W. Smith - leader, educator, technologist, writer - has a lifelong interest in the countless ways that technology changes the course of our journey through life. In addition to being a regular contributor to NetworkDataPedia, he maintains the website Technology for the Journey and occasionally writes for Blogcritics. Paul has over 50 years of experience in research and advanced development for companies ranging from small startups to industry leaders. His other passion is teaching - he is a former Adjunct Professor of Mechanical Engineering at the Colorado School of Mines. Paul holds a doctorate in Applied Mechanics from the California Institute of Technology, as well as Bachelor’s and Master’s Degrees in Mechanical Engineering from the University of California, Santa Barbara.
