NATION

PASSWORD

NationStates API (nationdata/regiondata)

Bug reports, general help, ideas for improvements, and questions about how things are meant to work.

Advertisement

Remove ads

User avatar
Coffee and Crack
Bureaucrat
 
Posts: 59
Founded: Aug 17, 2008
Ex-Nation

Postby Coffee and Crack » Thu Apr 15, 2010 9:00 pm

I just use a cronjob every hour for my population tracking script.

User avatar
NewTexas
Spokesperson
 
Posts: 174
Founded: Antiquity
Civil Rights Lovefest

Postby NewTexas » Thu Apr 15, 2010 9:11 pm

Coffee and Crack wrote:I just use a cronjob every hour for my population tracking script.


That is not the same as starting a job that will make an request every second for 56,000 seconds in a row, sleep 30,000 seconds in a row, rinse and repeat, 365 times a year. That is the alternative without the one file if you want the whole world.

:p
Big Tex
Governor of Texas

Author: NSDossier

User avatar
[violet]
Site Admin
 
Posts: 15658
Founded: Antiquity

Postby [violet] » Fri Apr 16, 2010 4:11 pm

I might make two XML files, one containing all regions and the other containing all nations -- just in case anyone wants one but not the other. Sound OK?

User avatar
Winnipeg
Bureaucrat
 
Posts: 52
Founded: Antiquity
Benevolent Dictatorship

Postby Winnipeg » Fri Apr 16, 2010 9:30 pm

Would work for me as long as we can request each one once every 24 hours in case we want both.
Robin 'Winnipeg' Morrison
Creator of WinniVision(TM)
Home of the West Pacific

User avatar
[violet]
Site Admin
 
Posts: 15658
Founded: Antiquity

Postby [violet] » Sun Apr 18, 2010 4:36 pm

Yep, of course.

I'll look at getting this in this week.

User avatar
Winnipeg
Bureaucrat
 
Posts: 52
Founded: Antiquity
Benevolent Dictatorship

Postby Winnipeg » Mon Apr 19, 2010 6:41 am

Cool beans, looking forward to it!
Robin 'Winnipeg' Morrison
Creator of WinniVision(TM)
Home of the West Pacific

User avatar
Nanocyberia
Bureaucrat
 
Posts: 64
Founded: Dec 15, 2007
Democratic Socialists

Postby Nanocyberia » Mon Apr 19, 2010 4:14 pm

Hello,

I'd like to see a population tracker right within NS, instead of the API. Anywhere from the last 3 days to the last 7... (And then I could make my own tracker where I just copy-and-paste the values into a static HTML file, rather than doing it everyday.) Nothing fancy, either, so it doesn't need to be a graph or anything.
I'd personally also fancy seeing the detailed gov stats somewhere in NS, too, but just single, daily updates; no tracking.

User avatar
Winnipeg
Bureaucrat
 
Posts: 52
Founded: Antiquity
Benevolent Dictatorship

Postby Winnipeg » Mon Apr 19, 2010 6:09 pm

I doubt you'll see that happen anytime soon, and it almost seems kinda pointless for the population itself since it's nearly a static movement every day. I mean you'd have a line graph that went up by 5-8 Million every day it would look pretty stable at a very low incline I'm sure.

Plus NS doesn't use a Database for storing data, so holding this data would probably end up being much harder on the server than it would be for a Database setup.
Robin 'Winnipeg' Morrison
Creator of WinniVision(TM)
Home of the West Pacific

User avatar
Nanocyberia
Bureaucrat
 
Posts: 64
Founded: Dec 15, 2007
Democratic Socialists

Postby Nanocyberia » Tue Apr 20, 2010 3:42 pm

Winnipeg wrote:I mean you'd have a line graph that went up by 5-8 Million every day it would look pretty stable at a very low incline I'm sure.
Graphs are way too hard... ;) A table would be sufficient; it needs to be dynamically generated, so that only that last four values are kept, for example. Conceptually, it would be like this :
Code: Select all
dump pop. count to table;
if row count >= 5, delete top/bottom row; // delete top row if most recent value is in bottom row; else, other way round :)

I'm pretty sure you don't a database for that... (In fact, if one is feeling really adventurous, you could set up the above script to allow custom values for the conditional statement; let's say the default is no more than three rows. After that, take your pick between a drop-down menu with pre-set choices or an input field with a maximum value.)
Last edited by Nanocyberia on Tue Apr 20, 2010 4:04 pm, edited 1 time in total.

User avatar
Fit battion
Spokesperson
 
Posts: 161
Founded: Dec 15, 2007
Ex-Nation

Postby Fit battion » Thu Apr 22, 2010 7:39 am

A graph for regional population might be nice (if necessary I'll make a seperate thread), where it automatically changes at update and calculates pop and stuff.
Cheese

User avatar
North Mack
Ambassador
 
Posts: 1213
Founded: Apr 27, 2005
Left-wing Utopia

Postby North Mack » Thu Apr 22, 2010 8:43 am

Oh look, a straight line...

All the information you want to see is available outside of NS through various calculators, and not really necessary to make the admins and techies put so much work into something that already exists.
T H E S T E L L A R R E P U B L I C O F N O R T H M A C K
Forged in Fire, Reforged in Blood
[ DeviantArt | NS FT Discord | The Local Cluster | FT Advice and Assistance Thread | The State of the Galaxy ]

User avatar
Coffee and Crack
Bureaucrat
 
Posts: 59
Founded: Aug 17, 2008
Ex-Nation

Postby Coffee and Crack » Thu Apr 29, 2010 6:40 pm

Any update on the super lists?

User avatar
[violet]
Site Admin
 
Posts: 15658
Founded: Antiquity

Postby [violet] » Thu Apr 29, 2010 6:44 pm

Got delayed by major server issues this week, hoping to get to it soon.

User avatar
[violet]
Site Admin
 
Posts: 15658
Founded: Antiquity

Postby [violet] » Sat May 01, 2010 11:32 pm

OK, around 02-May-10 22:20 PDT two files should appear:

http://nationstates.net/pages/nations.xml.gz
http://nationstates.net/pages/regions.xml.gz

... and thereafter be updated once every 24 hours. These will contain the XML of all regions and nations in the game. Please let me know if you find these useful, any feedback regarding data format, etc.

One difference to the regular API is that each item is <NATION> (or <REGION>), not <NATIONINFO> (or <REGIONINFO>).
Last edited by [violet] on Sat May 01, 2010 11:33 pm, edited 1 time in total.

User avatar
NewTexas
Spokesperson
 
Posts: 174
Founded: Antiquity
Civil Rights Lovefest

Postby NewTexas » Sun May 02, 2010 5:48 am

Unable to download either file. "website is unavailable or not found"

:geek:
Big Tex
Governor of Texas

Author: NSDossier

User avatar
North Mack
Ambassador
 
Posts: 1213
Founded: Apr 27, 2005
Left-wing Utopia

Postby North Mack » Sun May 02, 2010 10:49 am

[violet] wrote:OK, around 02-May-10 22:20 PDT two files should appear:



She said they aren't there yet...
T H E S T E L L A R R E P U B L I C O F N O R T H M A C K
Forged in Fire, Reforged in Blood
[ DeviantArt | NS FT Discord | The Local Cluster | FT Advice and Assistance Thread | The State of the Galaxy ]

User avatar
Winnipeg
Bureaucrat
 
Posts: 52
Founded: Antiquity
Benevolent Dictatorship

Postby Winnipeg » Sun May 02, 2010 1:12 pm

[violet] wrote:OK, around 02-May-10 22:20 PDT two files should appear:

http://nationstates.net/pages/nations.xml.gz
http://nationstates.net/pages/regions.xml.gz

... and thereafter be updated once every 24 hours. These will contain the XML of all regions and nations in the game. Please let me know if you find these useful, any feedback regarding data format, etc.

One difference to the regular API is that each item is <NATION> (or <REGION>), not <NATIONINFO> (or <REGIONINFO>).


Thanks Violet, a fitting release date for the API update I think :p My 29th Birthday.
Robin 'Winnipeg' Morrison
Creator of WinniVision(TM)
Home of the West Pacific

User avatar
NewTexas
Spokesperson
 
Posts: 174
Founded: Antiquity
Civil Rights Lovefest

Postby NewTexas » Mon May 03, 2010 4:45 pm

[violet] wrote:Please let me know if you find these useful, any feedback regarding data format, etc.


We would like to let you know, but three different tools were unable to gunzip them into anything meaningful.

Anybody else have any luck?
Big Tex
Governor of Texas

Author: NSDossier

User avatar
Pythagosaurus
Cute Purple Dinosaur
 
Posts: 549
Founded: Nov 24, 2004
Civil Rights Lovefest

Postby Pythagosaurus » Mon May 03, 2010 5:04 pm

`file` is your friend. It's gzipped twice. 8)

User avatar
New South Hell
Spokesperson
 
Posts: 161
Founded: Feb 15, 2008
Ex-Nation

Postby New South Hell » Mon May 03, 2010 5:42 pm

I was able to process the regions file with WinZip once I renamed it to .tgz. This is a trick I discovered some time ago that often helps for .gz files downloaded from the Web.

- nsh -

User avatar
New South Hell
Spokesperson
 
Posts: 161
Founded: Feb 15, 2008
Ex-Nation

Postby New South Hell » Tue May 04, 2010 2:33 pm

I have to correct my previous post. When I downloaded the nations file, I found that what I needed to do was unzip it, rename the extracted file to have a .tgz extension, and then unzip it a second time. I suspect this is what I did with the regions file as well - but that I reported it incorrectly because I had a bit of confusion between the original .gz file and its contents which were still very clearly compressed.

User avatar
NewTexas
Spokesperson
 
Posts: 174
Founded: Antiquity
Civil Rights Lovefest

Postby NewTexas » Tue May 04, 2010 2:51 pm

Thank you very much for the tip that it was gzipped twice. We never would have guessed that. And we don't even want to know why it is gzipped twice. We have it working and loaded in a database now and a mechanism for doing it daily. The only slight gotcha is that we are going to have to do a purge periodically to get rid of all the dead soldiers.

Now our tool will only hit the actual feeds twice for each request. One to get the latest nation feed for the request and one to get the latest region feed for the request. All the rest of the processing will be done out of the database.

The load times were much better than using the feeds.

Regions
start: 2010-05-04T10:53:34.347
end: 2010-05-04T10:54:03.763

Nations
start: 2010-05-04T15:11:09.120
end: 2010-05-04T15:29:04.533

So, it took
30 seconds to load 6,300+ regions
and
18 minutes to load 52,000+ nations

Much better than the 1 per second, which would have been:
6300 seconds to load 6,300+ regions
and
867 minutes to load 52,000+ nations


WooHootie! :ugeek:

Way to go [violet]!

:clap:

<now, to recode the rest of our app to use the db instead of the feeds>
<sigh>
Big Tex
Governor of Texas

Author: NSDossier

User avatar
Pythagosaurus
Cute Purple Dinosaur
 
Posts: 549
Founded: Nov 24, 2004
Civil Rights Lovefest

Postby Pythagosaurus » Tue May 04, 2010 2:56 pm

I wouldn't count on it staying gzipped twice. It was almost certainly a mistake. Somebody probably just forgot to test his/her code.

User avatar
[violet]
Site Admin
 
Posts: 15658
Founded: Antiquity

Postby [violet] » Tue May 04, 2010 7:38 pm

Hmm, odd. The file is not double-zipped, but it winds up that way on your computer when you download it via HTTP.

User avatar
[violet]
Site Admin
 
Posts: 15658
Founded: Antiquity

Postby [violet] » Tue May 04, 2010 11:10 pm

Possibly fixed for update in 22 hours from now, but possibly not because I'm not really sure what's going on.

PreviousNext

Advertisement

Remove ads

Return to Technical

Who is online

Users browsing this forum: No registered users

Advertisement

Remove ads