/AAD/ - Archiving And Donating computer resources general - /g/ (#106119069) [Archived: 453 hours ago]

Anonymous
8/2/2025, 9:06:46 PM No.106119069
1753564395309837
1753564395309837
md5: fe73c677f7370e80b8b888ba50355fc8🔍
Useful archiving efforts and other projects to help out with for people new to and interested in archiving:

HIGH priority (If you don't help archive these automatically, the data will probably be lost forever):

1. http://warrior.archiveteam.org/
Help out automatically archive things being shut down right now by running ArchiveTeam Warrior program (or specific containers) in the background
Requirements: Few GB of space, some bandwidth and small amount of CPU power, more info: https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior

If you learn that a site or any online data is in danger of shutting down, read through this page and contact ArchiveTeam on their IRC if required in order to have it archived: https://wiki.archiveteam.org/index.php/Projects

2. Help out automatically forward URLs you browse that are not archived on https://archive.org to them for archival with a browser extension
https://github.com/internetarchive/wayback-machine-webextension
Replies: >>106122530 >>106122555
Anonymous
8/2/2025, 9:06:54 PM No.106119071
MEDIUM priority (Important overall)

3. Seed torrents for as long as possible, rare data forever. Make sure to look up a guide for your router to PORT FORWARD your torrent client port, to substantially increase your upload (and your download) speed. In low population torrent swarms, if no one is port forwarded then you might not be able to connect to each other at all and exchange any data despite having it.
Requirements: As much or as little bandwitdh you want (you can set the limits if you need to)
https://github.com/qbittorrent/qBittorrent (Recommended client, especially to replace uTorrent)

4. Archive web pages you want to have a local copy of with a "Web Extension for saving a faithful copy of a complete web page in a single HTML file with a single click"
https://github.com/gildas-lormeau/SingleFile

5. Archive videos with "GUI front-end for youtube-dl, yt-dlp and other compatible video downloaders"
https://github.com/axcore/tartube

6. "Capture or record any area of your screen and share it with a single press of a key"
https://github.com/ShareX/ShareX

7. Archive entire websites you want to have a local copy of
https://www.httrack.com/
Anonymous
8/2/2025, 9:07:54 PM No.106119078
8. Publish the data that you have archived that isn't easily or at all available online. You can easily create torrents yourself in your torrent client and then share the magnet link to it anywhere online for anyone to access and, as long as DHT (Distributed Hash Table, decentralized way to share torrents without the need for any specific tracker) is enabled in settings (on by default), your files will be searchable on DHT by DHT crawlers, local or online (for example https://btdig.com/, where you can actually also search for FILE NAMES within all DHT torrents)
(archive.org also creates torrents for all uploads automatically but their torrents shouldn't be relied on because of an error-prone implementation and since they can also break when more files are uploaded or if the item's metadata changes, which includes even getting a new comment on the item)


OTHER useful things:

- In your torrent client settings add the best trackers to be automatically added for all of your newly added torrents (helps more easily connect to peers, especially in obscure torrents)
https://github.com/ngosang/trackerslist

- Look into running a node for I2P (anonymous private network within the global internet)
Requirements: Mostly bandwidth, more info: https://geti2p.net/en/faq
https://geti2p.net/

- Look into running Tor/Hyphanet(Freenet)/IPFS/YaCy/SearXNG nodes

- Easily capture and digitize all data and metadata from optical media (CDs, DVDs, Blu-rays...) with Media Preservation Frontend (MPF)
https://github.com/SabreTools/MPF

- "A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI"
https://github.com/bitmagnet-io/bitmagnet

- "ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline"
https://github.com/ArchiveBox/ArchiveBox
Anonymous
8/2/2025, 9:08:56 PM No.106119085
- Look into donating your PC resources to be used more intensively in projects:
BOINC (Berkeley Open Infrastructure for Network Computing: https://boinc.berkeley.edu/projects.php
GIMPS (Great Internet Mersenne Prime Search): https://www.mersenne.org/


- Additional archiving tools: https://github.com/iipc/awesome-web-archiving

- Additional links to archiving and similar communities:
https://wiki.archiveteam.org/index.php/Archiveteam:IRC
https://www.reddit.com/r/Archiveteam
https://www.reddit.com/r/DataHoarder
https://www.reddit.com/r/DataHoarder/wiki/index/ - Hardware and software for data hoarding FAQ
https://www.reddit.com/r/lostmedia
https://www.reddit.com/r/GamePreservationists
https://www.reddit.com/r/torrents
https://www.reddit.com/r/qBittorrent
https://annas-archive.se/torrents
>>>/t/

What are you archiving or want to archive?
Do you have or know anyone who has some rare interesting data or media not available online?
Anonymous
8/2/2025, 9:09:56 PM No.106119092
1753157375886580
1753157375886580
md5: 300e7dd9ec66b9a140d1bf619f0047f9🔍
Updated the OP to include
>- Easily capture and digitize all data and metadata from optical media (CDs, DVDs, Blu-rays...) with Media Preservation Frontend (MPF)
https://github.com/SabreTools/MPF
Anonymous
8/2/2025, 9:12:23 PM No.106119106
An actually useful thread
Thanks
Anonymous
8/3/2025, 12:42:58 AM No.106120900
Bump
Anonymous
8/3/2025, 12:58:59 AM No.106121018
>7. Archive entire websites you want to have a local copy of
>https://www.httrack.com/
I'm assuming this only works on static pages?

Honestly I primarily have an issue with finding content worth archiving, most content online seems to be one-time stuff that I won't ever use again. Especially media. What good is archiving it then locally? (It's different for public resources)
Really the only thing I seem to actively archive are books and text files, which barely need space to begin with.
Replies: >>106121357
Anonymous
8/3/2025, 1:39:20 AM No.106121357
>>106121018
>I'm assuming this only works on static pages?
No, but all tools have problems with very JavaScript heavy websites
>Honestly I primarily have an issue with finding content worth archiving
Surely you at least have some obscure content like YouTube videos, music or whatever that you like that can be gone any moment for example
Replies: >>106121480
Anonymous
8/3/2025, 1:53:01 AM No.106121480
>>106121357
>Surely you at least have some obscure content like YouTube videos, music or whatever that you like that can be gone any moment for example
I wonder. I guess books and music, but I don't know many videos I consider worth preserving as I wouldn't rewatch them most likely.
Obviously OC, personal projects, code and pictures, but from the open internet it's fairly limited all things considered.
I have some old websites that I found that are ancient and outdated, preserving those may be worth it as they may be gone at any random moment.
Anonymous
8/3/2025, 4:18:24 AM No.106122530
>>106119069 (OP)
Ah, you are back anon! Thank god. Been here a few times, always a good time. Thread is going to be needed more than ever.

>What are you archiving or want to archive?
A few chan threads, websites that are sensitive to current legislation and are likely to be ID Blocked. Few games, 2.2 ratio, anything I like to get my hands on. I wanted to get a copy of the WEF(slash)intelligence website, but HTTrack does NOT like going at that. I'm thinking of switching over to wget, but I don't know how to use it

>Do you have or know anyone who has some rare interesting data or media not available online?
Me. A few happenings from this website over the past 5 years, some comfy stuff. Honestly, a lot of it is just compressed archives of porn that probably need to be decompressed and updated. Porn isn't interesting, but I can imagine "smut dealer" will be on the employment records in 10 years time. I am also trying to get as much ASMR content from YouTube as I can.

I also need a solution to recursive chan thread archiving. chanthreaddownloader stopped being dependable, and I need something, a tool, where it'll follow child threads automatically with the full site structure. Any ideas?
Anonymous
8/3/2025, 4:22:22 AM No.106122555
>>106119069 (OP)
recently ive been filling all the harddrives i can find with all the media i couldl ever want to consume. Im doing it because im scared of dead internet theory and i want to download the media before bad entities are able to seamlessly generate propaganda into media and corrupt it
Anonymous
8/3/2025, 4:48:55 AM No.106122714
Is there a torrent with a good curated spread of books? I'm thinking the Western canon and notable popular fiction, reference materials, and a good spread of non-fictional works? I've got a budget of about 10-20TB