OSDev.org

The Place to Start for Operating System Developers
It is currently Tue Mar 19, 2024 3:59 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: How does webarchive get content
PostPosted: Mon Feb 08, 2021 11:03 am 
Offline
Member
Member

Joined: Fri Nov 22, 2019 5:46 am
Posts: 590
How/where does the web-archive page get old and lost web page contents?

The page(s) must obviously be stored by someone _before_ they disappear.

I mean, does some person or software store randomly webpages and checks if they disappear? Or does some person save webpages on his own behalf and later contributes them to the archive?

And how do they know in advance which pages will disappear?

Greetings
Peter


Top
 Profile  
 
 Post subject: Re: How does webarchive get content
PostPosted: Mon Feb 08, 2021 11:43 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1590
The web archive employs a program to download the pages and builtin assets (I believe those are called "spiders"). And then they have a ton of storage somewhere to store this stuff on. Obviously they don't know which sites are going to disappear. They just sample some pages by some algorithm. If they managed to sample a page you want before it got memory holed, then you are in luck. You can also somehow request a certain page be added.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: How does webarchive get content
PostPosted: Mon Feb 08, 2021 2:27 pm 
Offline
Member
Member
User avatar

Joined: Thu Aug 11, 2005 11:00 pm
Posts: 1109
Location: Tartu, Estonia
One can also manually save pages with web.archive.org/save/URL.

_________________
Programmers' Hardware Database // GitHub user: xenos1984; OS project: NOS


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Solar and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group