NationStates

I'd like to compile an archive all NationStates daily dumps as far back as possible. I have started collecting them but my archive only goes back 15 months and is missing a few weeks of data.

If anyone who has them would be willing to share them it would appreciated. I'll also consider monetary rewards if anyone has many (>100) that I am missing to cover bandwidth/labor for the collection ($10-$100).

My archive: www.capitalistparadise.com/nationstates/archive/

Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia

Ballotonia wrote:Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia

I planning on moving them to amazon glacier. I'll set up anyone who needs to upload them with an amazon s3 key so they can upload them. I'll cover expenses.

I'll TG you the instructions when I get home.

Edit: TG Sent.

Wish we could help Afforess, but we load the database and dump the dump. All we have is yesterday.

NewTexas wrote:Wish we could help Afforess, but we load the database and dump the dump. All we have is yesterday.

(Image)

Ballotonia has kindly uploaded dumps from 2010ish to present so far. I was hoping to be able to get dumps from farther back - I want to set up a public repository. Kinda sad to see all that history gone

History is alive and well at NSHistory with weekly extracts back to May, 2003. However, there were no dumps back then; all we came up with was scraping All Nations and All Regions. So, we have a whopping 3 columns of data on nations and 3 columns on regions. With the advent of the dumps, we keep a bit more.

NewTexas wrote:History is alive and well at NSHistory with weekly extracts back to May, 2003. However, there were no dumps back then; all we came up with was scraping All Nations and All Regions. So, we have a whopping 3 columns of data on nations and 3 columns on regions. With the advent of the dumps, we keep a bit more.

When exactly did Daily Dumps become available?

Also is a raw dump of your database possible?

Afforess wrote:Also is a raw dump of your database possible?

Perhaps a telegram conversation or an exchange of email / IRC names would be appropriate about now.

Ballotonia wrote:Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia

Ballo, did that dump include the one's I gave you? Just so I don't go digging 'em up..

Unibot III wrote:

Ballotonia wrote:Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia

Ballo, did that dump include the one's I gave you? Just so I don't go digging 'em up..

Yep.

Still uploading stuff, BTW. Not done yet It keeps timing out when left unattended, and I have to sleep/go to work, etc...

Afforess; I also have raw region XML files from 2004-2006. Do you want those too? These are not datadumps (did not exist back then, if I recall correctly) but a collection of all regional XML files downloaded separately in a single day.

Ballotonia

Ballotonia wrote:

Unibot III wrote:
Ballo, did that dump include the one's I gave you? Just so I don't go digging 'em up..

Yep.

Still uploading stuff, BTW. Not done yet It keeps timing out when left unattended, and I have to sleep/go to work, etc...

Afforess; I also have raw region XML files from 2004-2006. Do you want those too? These are not datadumps (did not exist back then, if I recall correctly) but a collection of all regional XML files downloaded separately in a single day.

Ballotonia

Yeah, raw region XML files would be interesting too.

Credit Ballotonia for much of the archives: http://nsarchive.net/

Would be cool if that could be linked to somewhere, link in the API docs page.

Added link to NS Archive from API doc.

[violet] wrote:Added link to NS Archive from API doc.

Thanks. I just finished uploading all of the archives. I have a program set up to move new daily dumps into the s3, so it should stay up to date .