NATION

PASSWORD

Historical Daily Dumps

Bug reports, general help, ideas for improvements, and questions about how things are meant to work.
Post a reply
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Historical Daily Dumps

Postby Afforess » Tue Sep 17, 2013 10:12 am

I'd like to compile an archive all NationStates daily dumps as far back as possible. I have started collecting them but my archive only goes back 15 months and is missing a few weeks of data.

If anyone who has them would be willing to share them it would appreciated. I'll also consider monetary rewards if anyone has many (>100) that I am missing to cover bandwidth/labor for the collection ($10-$100).

My archive: www.capitalistparadise.com/nationstates/archive/
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
User avatar
Ballotonia
Senior Admin
 
Posts: 5494
Founded: Antiquity
Democratic Socialists

Postby Ballotonia » Tue Sep 17, 2013 10:17 am

Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia
"Een volk dat voor tirannen zwicht zal meer dan lijf en goed verliezen, dan dooft het licht…" -- H.M. van Randwijk
Top
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Postby Afforess » Tue Sep 17, 2013 10:19 am

Ballotonia wrote:Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia

I planning on moving them to amazon glacier. I'll set up anyone who needs to upload them with an amazon s3 key so they can upload them. I'll cover expenses.

I'll TG you the instructions when I get home.

Edit: TG Sent.
Last edited by Afforess on Tue Sep 17, 2013 2:37 pm, edited 2 times in total.
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
User avatar
NewTexas
Spokesperson
 
Posts: 181
Founded: Antiquity
Civil Rights Lovefest

Postby NewTexas » Tue Sep 17, 2013 4:42 pm

Wish we could help Afforess, but we load the database and dump the dump. All we have is yesterday.

Image
Big Tex
Governor of Texas

Author: NSDossier
Top
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Postby Afforess » Tue Sep 17, 2013 5:00 pm

NewTexas wrote:Wish we could help Afforess, but we load the database and dump the dump. All we have is yesterday.

(Image)


:o

Ballotonia has kindly uploaded dumps from 2010ish to present so far. I was hoping to be able to get dumps from farther back - I want to set up a public repository. Kinda sad to see all that history gone :(
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
User avatar
NewTexas
Spokesperson
 
Posts: 181
Founded: Antiquity
Civil Rights Lovefest

NSHistory

Postby NewTexas » Tue Sep 17, 2013 6:00 pm

History is alive and well at NSHistory with weekly extracts back to May, 2003. However, there were no dumps back then; all we came up with was scraping All Nations and All Regions. So, we have a whopping 3 columns of data on nations and 3 columns on regions. With the advent of the dumps, we keep a bit more.

:ugeek:
Big Tex
Governor of Texas

Author: NSDossier
Top
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Postby Afforess » Tue Sep 17, 2013 6:18 pm

NewTexas wrote:History is alive and well at NSHistory with weekly extracts back to May, 2003. However, there were no dumps back then; all we came up with was scraping All Nations and All Regions. So, we have a whopping 3 columns of data on nations and 3 columns on regions. With the advent of the dumps, we keep a bit more.

:ugeek:


When exactly did Daily Dumps become available?

Also is a raw dump of your database possible? ;)
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
User avatar
Frisbeeteria
Senior Game Moderator
 
Posts: 27796
Founded: Dec 16, 2003
Capitalizt

Postby Frisbeeteria » Tue Sep 17, 2013 6:36 pm

Afforess wrote:Also is a raw dump of your database possible?

Perhaps a telegram conversation or an exchange of email / IRC names would be appropriate about now.
Last edited by Frisbeeteria on Tue Sep 17, 2013 6:36 pm, edited 1 time in total.
Top
User avatar
Unibot III
Negotiator
 
Posts: 7113
Founded: Mar 11, 2011
Democratic Socialists

Postby Unibot III » Tue Sep 17, 2013 7:11 pm

Ballotonia wrote:Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia


Ballo, did that dump include the one's I gave you? Just so I don't go digging 'em up..
Last edited by Unibot III on Tue Sep 17, 2013 7:11 pm, edited 1 time in total.
[violet] wrote:I mean this in the best possible way,
but Unibot is not a typical NS player.
Milograd wrote:You're a caring, resolute lunatic
with the best of intentions.
Org. Join Date: 25-05-2008 | Former Delegate of TRR

Factbook // Collected works // Gameplay Alignment Test //
9 GA Res., 14 SC Res. // Headlines from Unibot // WASC HQ: A Guide
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✯ ✯ ✯ ✯ ✯ ✯ Duty is Eternal, Justice is Imminent: UDL ✯ ✯ ✯ ✯ ✯ ✯
Top
User avatar
Ballotonia
Senior Admin
 
Posts: 5494
Founded: Antiquity
Democratic Socialists

Postby Ballotonia » Tue Sep 17, 2013 11:48 pm

Unibot III wrote:
Ballotonia wrote:Where would one upload them to?

I got about a year of additional data dumps in my personal archive. Note NS itself doesn't store them.

Ballotonia


Ballo, did that dump include the one's I gave you? Just so I don't go digging 'em up..


Yep.

Still uploading stuff, BTW. Not done yet :p It keeps timing out when left unattended, and I have to sleep/go to work, etc...

Afforess; I also have raw region XML files from 2004-2006. Do you want those too? These are not datadumps (did not exist back then, if I recall correctly) but a collection of all regional XML files downloaded separately in a single day.

Ballotonia
Last edited by Ballotonia on Tue Sep 17, 2013 11:51 pm, edited 1 time in total.
"Een volk dat voor tirannen zwicht zal meer dan lijf en goed verliezen, dan dooft het licht…" -- H.M. van Randwijk
Top
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Postby Afforess » Wed Sep 18, 2013 8:38 am

Ballotonia wrote:
Unibot III wrote:
Ballo, did that dump include the one's I gave you? Just so I don't go digging 'em up..


Yep.

Still uploading stuff, BTW. Not done yet :p It keeps timing out when left unattended, and I have to sleep/go to work, etc...

Afforess; I also have raw region XML files from 2004-2006. Do you want those too? These are not datadumps (did not exist back then, if I recall correctly) but a collection of all regional XML files downloaded separately in a single day.

Ballotonia



Yeah, raw region XML files would be interesting too.
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Postby Afforess » Thu Sep 19, 2013 4:08 pm

Credit Ballotonia for much of the archives: http://nsarchive.net/

Would be cool if that could be linked to somewhere, link in the API docs page. ;)
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
User avatar
[violet]
Executive Director
 
Posts: 16205
Founded: Antiquity

Postby [violet] » Mon Sep 23, 2013 7:53 pm

Added link to NS Archive from API doc.
Top
User avatar
Afforess
Ambassador
 
Posts: 1105
Founded: Jun 22, 2009
Ex-Nation

Postby Afforess » Tue Sep 24, 2013 9:16 am

[violet] wrote:Added link to NS Archive from API doc.


Thanks. I just finished uploading all of the archives. I have a program set up to move new daily dumps into the s3, so it should stay up to date .
Minister of the Interior, Capitalist Paradise

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Top
Post a reply

Advertisement

Remove ads

Return to Technical

Who is online

Users browsing this forum: Agedincum, Bagong Timog Mindanao, Dazchan, Ehrijeters, Jackuul, Kractero, Rocain Founder, Romanovskaya, Saint Spartacus, The Ice States, The Soviet Onions and the Baltics

Powered by phpBB® Forum Software © phpBB Group

Advertisement

Remove ads