I just had a thought recently, how much of a risk is there that all these accumulated years of discussions and analysis, etc, could get lost if some mistake at the hosting company or some hacking fiasco were to happen? I thought of this while reading through the Evangelion 2.XX Storyboards vs Final topic and thinking "Wow, this is so cool... This needs to be preserved forever". Would it be possible for random users like us to download backups of at least the text of the forums? (Huge as even that would surely be.)

Well, there's always the approach we took to making backups of the AnimeNation Evangelion forum in its heyday -- write a simple web-crawler that just visits every thread and slurps down the contents.

I wonder how much data all ~600k posts would take up in a text file.... This needs to be safeguarded somehow. I mean, the forum is an almost ten-year-long collective effort; it would be a shame to lose it all because there's no backup.

I'm not sure how expensive it would be given the volume of data but would it be viable to get space on a "cloud drive" external to the site's own servers and use that to back up everything?

MoroNoKimi wrote:I'm not sure how expensive it would be given the volume of data but would it be viable to get space on a "cloud drive" external to the site's own servers and use that to back up everything?

Text takes up a very small amount of space. The images on this forum probably make up a larger portion of the data than the text does.

Ornette could probably carry around a copy of the entire forum on one of these.

We do have a daily database backup run at midnight (server time)

cron job wrote:This is a message from the evageeks.org database backup script
Began backup process on: Sat May 4 00:00:02 PDT 2013
Ended backup process on: Sat May 4 00:11:35 PDT 2013
=========================
The database backup has hopefully been successful

and the backup size is

04-May-2013 00:11 269M

for the whole site -- which is tiny in this day and age.

Mr. Tines wrote:and the backup size is: 239MB

Seriously? Ah, I take it the images are saved as links, right? Even still, that means that each post is saved in an average of about 400 bytes! Does it take 11 minutes because the data is compressed?

Yes, (most) images aren't in the database, only their URLs; and this is the resulting bzipped tarball size.

Well, that's quite a relief to know. It feels a bit stupid to think they might not have some backup, but hearing confirmation is always a plus.