Self-decoding format

Hi, I've been thinking for a while, what would be a cost-efficient method of storing large amounts of data in a way that someone far in the future would be able to read? It should be eminent to anyone that digital data is not at all permanent. Most data becomes functionally unreadable within twenty years. Certain blu-ray formats are probably the best for the purpose of long-term readability, as they are capable of surviving extreme atmospheric conditions. It is conjectured that the "M-disc" format, when stored properly, could keep for hundreds of years. Now suppose someone is in the future, and they are only dimly aware of what computers were. We can probably assume that they know the rudiments of electronics, physics, and chemistry, and they have the resources necessary to reconstruct a basic blu-ray reader. Perhaps some manuals on the subject could be included in some kind of sealed package, so that the package itself is self-decoding. Some amount of language barrier would be unavoidable, but it could be lessened by only using clearly written and simple sentences. In any case: supposing they have a blu-ray reader that is capable of giving them the raw bit stream(perhaps at a reduced rate), how would we get from there to reading mass quantities of data? My first thought is to put Morse code on the inner rings, containing instructions on how to decode the rest of the disc. I imagine that a primitive blu-ray player would simply be able to show, visually, whether a 1 or 0 is read. Thus it should be encoded to be read at low RPM and with some redundancy (for instance, perhaps short = 20 zeros, long = 40 ones). This morse code data might contain instructions on building a machine that can decode the rest of the data. Multiple formats besides morse code could also be used, for instance a fixed-width encoding. I think a computer would be a little out of the scope for that layer, more effective might be a simple typewriter-like printer that can output alphabetical characters onto a long roll of paper. In any case, after the initial Morse-encoded disc entailing how to build this machine and the purpose of the disc, what might come next is a very long stream of text designed to be read by such a machine, again at a low RPM. Ultimately, the goal would be to get the future reader decoding compressed raster images and interpreting some kind of basic file system. For that, probably what would be necessary is a basic computer, or at the very least a very complicated digital circuit. So I imagine that the middle part of the disc would need to include quite a lot of information on basic sciences, as well as exact descriptions of the format used in the outer layers, so that the readers could eventually decode the outer information of the disc. The importance of self-decoding formats is necessary not only if the future reader has lost the technology of computers, but also if the future readers' computers have advanced so much that reading the disc amounts to archaeology. In any case, I'm sure they would be very grateful that someone bothered to make it easier for them.

another possibility for long-term storage: etching directly on a relatively inert metal (such as gold) and possibly sealing it in a very long-lasting plastic coating to prevent wear. The letters could be printed microscopically, in which case all that is needed for reading is someone proficient enough to grind a half-decent magnifying lens. The disadvantage is that this is pricey, and it is also liable to eventual vandalism as inert metals are often considered valuable.

>>5979 couldn't you just store it with a printed copy of the technical specifications?

Microfiche container with instructions to a civilization able to read one to install industries in order to build a computer device capable of decoding raw data printed on the rest of the microfiche batch. Pros: natural language and math is easier to decode for humans, be it past of future. Cons: does not sound as cool as post-apocalyptic soyberpunk fantasy OP tries to imagine it.

I think that the optical technology required to read a blue ray disk is a bit advanced to ask of a civilization that hasn't developed enough linguistics to decode a foreign language. I think I would make a monolith, probably out of aluminum plated in tungsten carbide, and I'd etch into it pictograms, each one displaying different ores, and how to construct and use to the tools to refine them, and then displaying human figures offering them up to the monolith. The last one: neodymium; the monolith has a magnetic lock, and inside are rows and rows of aluminum core tungsten carbide plated tablets, double-sided, half of them pictogram and grapheme pairs, the other half instructional text on how to turn the metals they have gathered for the monolith to build electronics. The first thing I have them build? A speaker. Speaker are relatively minor arcana in terms of mechatronics and it will allow a larger group of people to continue to research the monolith as few will be able to put the time in to learn to read what with surviving and all but my spoken language will have a relatively low difficulty of acquisition, allowing it to propagate widely and unfortunately, probably all but wiping out the natural language that was dominant before. There will be record-like disks inside the tablets, that can be removed by chipping the tungsten carbide on the side, where there's no aluminum underneath to brace it. The rest is easy. After this step you can even leave blu-rays if you give them the verbal instructions on how to build a player (Good luck on getting a good enough vacuum on a small enough laser diode without industrialization though, not to mention the optical sensor.)

>>6052 microfiche degrades within a few decades, even when properly stored.

You don't know their architecture. Best ou can do is store data in a simple format, leave specs and docs in several formats

>>5979 Didn't they have to deal with this on the Voyager Golden Record? To figure out how to explain the playback to an alien civilization. https://en.m.wikipedia.org/wiki/Voyager_Golden_Record#Playback

>>5979 >what would be a cost-efficient method of storing large amounts of data in a way that someone far in the future would be able to read? It's called a book, and has been used for thousands of years. An even more impressive alternative is a clay tablet. A clay tablet is great if the data really needs to last for eons, because when it dries it becomes stone, and there are some really old stones around, so we already know this high tech solution works.

>>6286 Also, Microsoft's developing a glass storage medium which could last for centuries: https://arstechnica.com/gadgets/2019/11/microsofts-project-silica-offers-robust-thousand-year-storage/ Wasn't there an older thread about data loss and long-term storage?

>>5980 reminds me of pic rel >>6300 yes but I can't find it, it was something like "things rot". There is this one too https://www.lainchan.org/%CE%A9/res/39549.html

>>6297 The vast, overwhelming majority of books never last one century. They are lost to fire, flood, insects, or people just not having space for them. Do you know that libraries in the US routinely destroy books and always have? Clay tablets take up even more space. There's no reason anyone would keep that around in their attic after you're dead and your great-grandchildren have sold the place. Or it's condemned, or someone decides it's in the way of a new freeway.

Engraving stone. With an increasingly complex depiction of mathematical and natural elements and their grammatical rules. I figure that something like chemistry is not all that hard to get across, for anyone with an atomic model can connect it to some depiction of orbitals that is to them presented. More interesting is the mathematical constructions needed to interpret a digital signal. My thoughts on this have been more geared towards sending a signal to outer space, where there is no stone and no obvious interpretation. In that case I imagine the signal should start with a identifiably pattern that establishes, before anything, the length of the syntactic (or symbolic) unit (ie whether it's 8-bit or 6-bit, where a bit means a higher or lower wave intensity, or something). Say: 000100100011010001010110011110001001101010111100110111101111. Anyone with the curiosity and the time would soon realize it's a binary sequence of 4-bits. Thereafter more symbols can be added, eventually laying out a language, grammar and all, most importantly, a purely mathematical one that does not rely on phonetics, or even on any form of assumed share experience.

so biological molecules have chirality, left or right handedness. but right handed became dominant in DNA and left handed doesn't exist in nature. so some guy made left handed DNA and put it in a pond, came back and a year later it was still there because no other biological organisms could break it down and use it, because it's not the right handedness. I don't think this is long term viable though, but it's neat. if long term readability is important, you need to go big not small, placing blocks like QR code stonehenge that is almost impossible to move, but has redundancy incase they do. I don't think anyone has included this in an encoding/decoding, but predicting how something will decay would also be useful, so it can still be used.

Seems like the sort of question the Long Now Foundation have been asking for a couple decades now, although it doesn't seem like they've quite cracked it yet; their page on their supposed "Long Server" project (http://longserver.org/) is, ironically, gathering dust. However... >>5980 ...they basically did this as a proper prototype for their Rosetta Project (https://rosettaproject.org/). It's a metal disk with very fine text engraving (readable at 650x magnification) that stores 13,000 pages and (they claim) can last thousands of years. They also made a bunch of copies of a smaller wearable version that stores 1,000 pages. Possible means of bootstrapping more complex data storage decoding. Agree that it's not quite "cost-efficient" though (although mass production could help a little). More outlandish ideas: start a cult/religion whose purpose is to keep data readable over generations indefinitely. (Wait, isn't this just oral traditions with extra steps?) Or hope that the Internet Archive is still around in a thousand years' time. :P