Why Aren't You Archiving Things Lains?

Something I've honestly found very alarming for a while is how very little people give a soykaf about whether data stays or goes. More often then not, websites go down forever, and with that pieces of history are lost forever. More often then not, people get embarrassed and delete their old works which some enjoyed. Many great youtube videos get pulled due to copyright claims and are lost forever. I doubt everything lost on what.cd has been restored. With this "if it's obsolete, throw it away" mentality, so much old hardware and software, especially software made available via download has been lost, and many drivers have been lost forever. Archive.org and other organizations can only do so much to save the our technological past for future generations to see, and what will we do if something happens to them? They are allowed to operate by the "generosity" of governments and other actors. As Net Neutrality dies, we need to focus more effort than ever on archiving stuff like this so corporations and governments wont be fully able to paint our perception of the world quite like they did in the pre-internet days with television. The history of libraries and archives is rather dark. "but how important could one little website/piece of obscure or outdates software be?" you ask. These things are worth saving because they give pieces to a puzzle that will never be fully completed. They show facts, mindsets, progress, and events of the time, and much more. I'm sure you all have seen what happens when people have their own versions of what happened. You end up with crazy activists, holidays like Columbus Day, inaccurate representations of the past, and so much more. It can be harmless and insignificant, or it can be dangerous. These things may not seem worth saving now, but I assure you there will be people in the future who wish we saved more of this stuff. So why aren't you at least archiving things you like, lainons?

> I doubt everything lost on what.cd has been restored. whatcd was so hard to access, it's not worth the effort at that point. information wants to be free, but private trackers are the exact opposite of free. Entropy is a law, info is subject to entropy like everything else, it is generated and lost on a regular basis. I think that's just a fact of life, I have accepted this. I am archiving books I like and value. I'm not thinking that I am saving these for future, it's for my personal archive so that I can use them as references.

>>5394 >it's not worth the effort at that point. So many audio tracks were exclusive to that site, and are now lost forever. Tell me why that effort is not worth it.

>>5392 >The history of libraries and archives is rather dark. You lost me there. Are you saying those were censored ? As far as I know, in my own country, archives were only ever lost to fire and rot (my dad still had fun tracing his family tree to the XIIth century). Maybe a few book were removed from the libraries, I guess. But it's not like the masses frequented those so much that it would be worth censoring much in the first place. Now regarding the broader point. I bloody hate data loss, on a very visceral level. But I don't think my own obsession is justified. There is so much data from our own past missing, and we neither realize it nor miss it. I'm not speaking about great historical events, from which we still have partial data (be it roman conquests or moon landing). I'm speaking about daily life of people, the song they sung in taverns, the slang never written, the joke they told. All this that is akin to what we find today on the web. Now, for each piece of data, two possibilities : Either it has a value, recognised by enough people that one or several will archive it. Then it stay, at least for a bit longer. Either it does not, or no one recognizes it. Then it will die with its current incarnation, be it discontinued website, out of print DVD, forgotten book left to rot at the back of a library. And then it's gone forever, no one in the future will access the data or even know of its existence ; but then again, no one did even in the present. Of all those lost tracks from whatcd, how many had never been listened to ? Or only listened to to be discarded as trash ? A human life can only use so much data, and there are only so many of us here. Some data has to go. You speak of history manipulation, and it sure isn't a good thing, but will it really be avoided by more complete archives ? I imagine Columbus landing and its consequences in the following centuries are pretty well documented, but did it stop Columbus holiday, whatever that is ? (wikipedia does not seem to paint it as especially bad.) See my above point, people don't visit the archives in the first place, and the historians who do have little effect on popular opinion, especially when facing an hostile government. So. Archive what you deem worth it, and don't worry about leaving a complete puzzle behind. A few well chosen pieces are more telling than a picture too large for anyone to behold. Especially when the issue of indexing and making the archives accessible will arise. I repeat : how do you intend millions of anon to make their archived data usefully accessible ? Without some standard of archiving that takes this into account (I guess torrent and p2p in general are a good way to introduce redundancy in data while keeping it accessible.), anything beyond personal hoarding seems doomed to die with you. If this exists, make it known, if it does not, develop it. I'm sure you'd find people to help here. All this is coming from my own perspective, which is of a borderline crippling refusal to see data go, hence me trying very hard to convince myself that's it's ok to throw away this one soykafty pamphlet that I received ten years ago and stored ever since.

>>5397 When my country was formed in 1990 they literally threw loads of books into basements of questionable archiving quality.

All we can do is do our part in preserving the wired. For instance with youtube, download videos and upload them to the Internet Archive if you think there's a chance they will disappear soon. That's not a great answer I understand. We need mass efforts to do more. IA cannot do everything. I just feel that not enough people care; they're too occupied with the present to worry about preservation. We have truly become a disposable society, not just in materials, but ideas and information.

>>5404 OP here, this is very understandable, there is only little you can do. I should have elaborated. I don't expect every anon to be like me and to archive a lot of stuff, I just wish there were more people who saved stuff at least every now and then, stuff that they enjoy or found worthwhile. I can't count the times I've tried to find something I enjoyed in the past, only to find that it's gone without trace.

>>5392 >So why aren't you at least archiving things you like, lainons? Desu, I mirrored libgen 2 moths ago.

I seed a lot of things, and I have about a gig of random images I found funny and about 20 gigs of various PDFs.

oh and about 100 gigs of movies and TV shows.

>why aren't you archiving things If it's not commonly available its probably fake or outdated.

I archive rom dumps, e-books, some sites about anarchy, lots of electronics projects some wikis some dEEp web sites also I image some laptop OSes (those who come with freeDOS and all the other ones with UNIX) with a large amount of memes and youtube channels. And copy lots and lots of CDs/DVDs. If I think the information can be useful I will archive it.

I have archived 2TB worth of anime, manga **and hentai**. I am planning archiving other content when I'll get new HDDs

I am archiving on blu rays.

>>5434 and yet real, valuable information(usually of the historical variety) gets lost all the time.

>>5438 Where do you store all of this?

>>5477 They are not really big at the moment I store them in an internal 2TB hard drive with a 1TB external drive for backing up some of it I am thinking of buying a NAS eventually

>>5394 >whatcd was so hard to access, it's not worth the effort at that point. information wants to be free, but private trackers are the exact opposite of free. That's like saying books on difficult material or using complicated language should be discarded, just because they're not intellectually available to the average person. What.cd was freely accessible to those who could prove their competence. Maintaining a userbase consisting of people who were knowledgable about music and interested in its archival enabled the site to be the vast library it was.

>>5395 Different lainon here, but they might not be lost forever, given the fact someone already has them on their HDD, it's just a matter of if one of those people decide to share it again on the internet, be it on a form of torrent, or anything

I try to archive things I deem worth it, recently I imaged some of my old CDs with games, more or less proprietary software, including the ones from magazines, I'm mirroring binaries and sources of any software I like/find useful now or possibly in the future, I have a tick of pasting URLs of every document that is likely to disappear sooner or later into Archive.org and archive.is (including MS Technet, device drivers for Windows and various documentation that's available from hardware/software vendors, but very prone to disappear or get altered at owner's discretion). Also my own works, be it music, graphics, 3D stuff, code, photos, logs, whatever

>>5395 I'm sure people shared those tracks elsewhere.

>>5481 False analogy. A better analogy would be non-public libraries (with content unavailable elsewhere) with stupid elitist membership policies.

>>5590 Because reading a website to learn a thing or two and then answer a few questions is too hard. Getting into WCD was very easy. What was hard tho is that you had to maintain a ratio (which could be made easy, by contributing and participating in the site's events. After you got to seed a few things everything became easy).

>>5590 Like a university library as opposed to a public one. >>5606 I never saw the point to begin with. Torrents were useful because anyone could use them and you could easily tell which one was the quality one unless it was something super obscure. For super obscure things, it helps to have the whole world (or as much of it as possible) sharing what they have to increase the chances of actually finding that one thing you want. I don't see how any of this is helped by limiting the number of people to share from. It just doesn't seem practical.

I'm a low level data hoarder Recently I told my dad I need a new hard drive to archive stuff and he said "Can't you find that stuff online already?" fuarrrk you dad, just fuarrrk you. Getting real, I already archive media that I like, I have tons of books, movies, records and comics, and some other minor stuff. I'm doing my part, right?

>>5609 >I don't see how any of this is helped by limiting the number of people to share from. Simple: quality over quantity. Also the ratio rules were written in such a way that people were encouraged to upload. Sure, opening it up to everybody would increase the chance that *somebody* who has that obscure release visits the site, but the ratio rules make that person more likely to go through the trouble of actually uploading. >It just doesn't seem practical. It may seem counter-intuitive but you can't argue with the results. What.cd was the best place to find music: legal or illegal, public or private. For whatever reason, this system just works very well. That being said, It'll probably eventually be replaced by some other scheme as it's drawbacks become more apparent (We are already seeing some issues with the whatcd replacements. I.e. if you get a bunch of dedicated pirates with giant music collections together, it can be very difficult for newcomers to build ratio).

>>5611 > Simple: quality over quantity It sounds like you were searching for commonly shared media which could be found elsewhere anyway. The quality versions were also available on public sources. > What.cd was the best place to find music Not for the stuff I was looking hard for. Sure I like mainstream music sometimes, no problem to find that. But that's not all I like. Some recordings were entirely impossible to find before file sharing, and merely extremely difficult after. I spent about eight years looking for one 7" record, then found the mp3s on Soulseek.

>>5610 Yeah I get it. Just because something is there now doesn't mean it will be in a year. More people archiving and sharing media increases its likelihood of survival.

>>5612 The "quality" I'm talking about is more than just the bitrate of the audio files. Pick an album, pick a file type, and there's a pretty good chance it'd be on what.cd perfectly tagged and thoughtfully organized. Yeah, sometimes you have to go digging through old blogs and file dumps, but I've never looked for something not on what.cd and found in on some public tracker. Soulseek is totally quantity over quality... which is great for obscure scene releases but bad for creating a cohesive library. To be fair, what.cd vaporized in a day while Soulseek is still going strong. That's a pretty good argument for public > private for the purposes of archiving. I just wish we didn't have to compromise on the high community standards just to get something with a little longevity.

>>5614 I'm starting to understand you better now. > perfectly tagged and thoughtfully organized Ok, Soulseek is bad for that. I usually double check that stuf online after downloading anything. Some users on there are more reliable for that kind of quality. There is a sort of meritocracy too, though a different kind. If you have a good shared folder (or a feminine sounding screen name), you get priority to download from people, by each user's discretion though.

>>5610 Have you tried explaining to your dad why the content already being available misses the point of what you want to do?

>>5613 >>5628 I did told my dad "That stuff is online just because some one took the care of archiving it and posting it once the previews source was taken down" He understood that.

>>5617 i realize i was kind of making an off-topic point about why i like what.cd, but it didn't really have much to do with the OP (apologies for that). maybe a better thing to consider is that organization and ease of access ("quality") matters in archiving for the same reason why a library isn't just doomsday vault full of stacks of books. the more people have the data, the more likely it is to persist, and the easier it is for people to find the data they're looking for, the more likely they are to download a copy. i'm interested in whether or not peer to peer is even a good way to make an archive. the obvious alternative is something like library genesis, except with wiki-style metadata editing. in this case the data is actually stored on the server(s) and the redundancy comes from mirroring and federation rather than direct user participation. there's upsides and downsides. the most obvious downside is that if this were applied to anything besides books, it would have a serious target painted on it's back by the copyright agencies. it wouldn't be hard for these agencies to take down the 15 or so federated mirrors, and they would have the unambiguous legal right to do so (i.e. the pirate bay "we just host links, not data" gray area doesn't apply).

>>5631 wanted to add: another downside is that it isn't hard to host a mirror if all it contains is text documents. that's quite a few terabytes, sure, but it's orders of magnitude smaller than the storage requirements of a music or video database. i think that's the main reason why any sort of public music/video archive would have to be peer to peer in some capacity.

>>5631 > the more people have the data, the more likely it is to persist, and the easier it is for people to find the data they're looking for, the more likely they are to download a copy. I avoid the word "elite" usually but it applies now. Maybe a core of devoted nerds who are dedicated to preserving quality copies of things could be the kernel from which these copies spread... > i'm interested in whether or not peer to peer is even a good way to make an archive ....and since so many people are going to use p2p anyway, at least they can be receiving and spreading those "elite" versions.

>>5633 yeah, i think i'm with you on p2p > first-party hosting. how about a public torrent tracker with wiki-style metadata editing, and a moderation hierarchy? allow tracker uploads from anyone, but leave the organization of these trackers to the "elites". perhaps moderator rank would be conferred in a similar way to invites on private trackers (a combination of seeding requirements and demonstrated knowledge of the technology). normal visitors would have input too, but it would be limited to a per-torrent rating system and talk pages like on wikipedia.

I'm really interested in archiving sites and wikis I visit often for offline use. What scripts/software do you all use to archive lots of pages at once?

I have comics and books that I really care about, but I don't have a good way to archive those. DVDs and gamess are easy. I really should archive my stuff more.

How can I save an entire webpage?

>>6538 Making a "shallow copy" is trivial. But for a meaningful archive you'd need to write a crawler.

>>6538 wget

look, does anyone have the ZhCn files for Wotlk 3.3.5a? I feel like as time goes on it will only get harder to obtain, but I need it to live, and I will never let it die.

>>5451 hm, were you part of /eternalarchive/? Back when the TPP was more of a threat it seemed very interesting to me, but I never actually saw the eepsites or torrents they made. Come to think of it, I don't even know if the project still exists in current year.