NationStates API (nationdata/regiondata)

[Flanderlion]

https://www.nationstates.net/cgi-bin/ap ... 6;start=20 and just iterate through. Would be much nicer if they added at least the daily census in the daily dumps so there would be a much easier way of collecting the bulk data rather than a million requests. Re the actual coding, it's kind of more of something for you to figure out, API gives you a way to access the data, but it's for the user to put it together and edit it in their desired way. For non-technical users there is the existing site.

Also - https://www.nationstates.net/pages/api.html#terms - you need to set a userAgent when using the API (or you can just use your browser manually) and if you scrape there are significantly more stringent criteria - viewtopic.php?p=16394966#p16394966

I've got a feeling most of the sheets/excel things were illegal because they don't set a userAgent, but they might've added the ability to set those in the intervening years between the script rules changes/Elite being illegal and 2023.

[Klaus Devestatorie]

Fetching data via =IMPORTXML on google sheets has thus far not refused me access to any data I have requested, but after some fiddling around to try and see what my user agent actually is, I have determined that google sheets is only sending a non specific message indicating google sheets use for a user agent. Is there a way to specify a user agent directly in the API URL?

[Trotterdam]

I'm not trying to make 500 API calls, though. I'm just trying to make one API call that has all the data I want, with no intention of making this call more than once a day.

You can't. You'll have to either put in the work to process multiple API calls, or give up on your goals.

There doesn't appear to be an option to let me select any group of 20 nations other than the 20 highest rated nations for that census, so I can't request by region in successive calls.

You can: https://www.nationstates.net/cgi-bin/api.cgi?region=ordo_imperialia&q=censusranks;scale=1;start=21.

However, this is somewhat risky. If the ordering changed between API calls (such as from a nation answering an issue just as you're fetching the data, or from a nation entering or leaving the region), then you could get a particular nation listed twice, while another nation might get skipped over entirely. Doing it the way I suggested at least guarantees that you get data from approximately the same time on every nation in the region, even if it might not all be from exactly the same time.

[Klaus Devestatorie]

I'm not trying to make 500 API calls, though. I'm just trying to make one API call that has all the data I want, with no intention of making this call more than once a day.

You can't. You'll have to either put in the work to process multiple API calls, or give up on your goals.

There doesn't appear to be an option to let me select any group of 20 nations other than the 20 highest rated nations for that census, so I can't request by region in successive calls.

You can: https://www.nationstates.net/cgi-bin/api.cgi?region=ordo_imperialia&q=censusranks;scale=1;start=21.

However, this is somewhat risky. If the ordering changed between API calls (such as from a nation answering an issue just as you're fetching the data, or from a nation entering or leaving the region), then you could get a particular nation listed twice, while another nation might get skipped over entirely. Doing it the way I suggested at least guarantees that you get data from approximately the same time on every nation in the region, even if it might not all be from exactly the same time.

I reckon I've found a way to handle API calls at a decent clip via some manual spreadsheet data work, and I can resolve the nation movement mid-request issue by only bothering with specific nations of interest instead of trying to handle every nation in a region no matter what- but the user agent issue has me on hold for now. Hopefully there's either a way to either directly declare a user identifier in the API url, or that this would be added in the future- my research has confirmed that there's no function in google sheets for a user agent. I'm fiddling with the excel data import wizard instead in case this has the ability to set a user agent- is it possible to set this via the advanced power query editor, per this example?

Code: Select all

let
    Source = Csv.Document(Web.Contents("https://www.nationstates.net/cgi-bin/api.cgi?a=useragent", [Headers=[UserAgent="nation=Klaus Devestatorie"]]),[Delimiter=",", Columns=1, Encoding=1252]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}})
in
    #"Changed Type"

[Bowzin]

-snip-

Try User-Agent for the header key, instead of UserAgent.

[Klaus Devestatorie]

-snip-

Try User-Agent for the header key, instead of UserAgent.

Champion! Thank you, I'm in business

[Hesskin Empire]

So I was messing about with some code and realized that the new WA nations shard has a comma (,) as a delimiter while the residents (NATION) shard has a colon (:) as a delimiter... was this on purpose to differentiate between shards in some way? I know it caused me some time debugging before realizing lmao.

(Note: I haven't checked other shards so I don't know if they tend to share delimiters or have unique ones)

[Bowzin]

So I was messing about with some code and realized that the new WA nations shard has a comma (,) as a delimiter while the residents (NATION) shard has a colon (:) as a delimiter... was this on purpose to differentiate between shards in some way? I know it caused me some time debugging before realizing lmao.

(Note: I haven't checked other shards so I don't know if they tend to share delimiters or have unique ones)

Delegates are , delimited as welll

[Eluvatar]

So I was messing about with some code and realized that the new WA nations shard has a comma (,) as a delimiter while the residents (NATION) shard has a colon (:) as a delimiter... was this on purpose to differentiate between shards in some way? I know it caused me some time debugging before realizing lmao.

(Note: I haven't checked other shards so I don't know if they tend to share delimiters or have unique ones)

It was intended to be consistent with the newer shards and with what is more common in the wider world of data.

[Hesskin Empire]

So I was messing about with some code and realized that the new WA nations shard has a comma (,) as a delimiter while the residents (NATION) shard has a colon (:) as a delimiter... was this on purpose to differentiate between shards in some way? I know it caused me some time debugging before realizing lmao.

(Note: I haven't checked other shards so I don't know if they tend to share delimiters or have unique ones)

It was intended to be consistent with the newer shards and with what is more common in the wider world of data.

So will we see the delimiters changed on the shards to be consistent at some point? I don't know much on setting up APIs themselves so I have no idea how difficult that process would be.

[Roavin]

No, because we don't want to break existing scripts. But going forward, we're going to stick to commas for new stuff.

[Sandaoguo]

Would it be possible to add a "Founded In" stat to the nation API, recording the region in which the nation was founded? Ideally within the default set of shards delivered by the base parameter.

[Bowzin]

Would it be possible to add a "Founded In" stat to the nation API, recording the region in which the nation was founded? Ideally within the default set of shards delivered by the base parameter.

Seems like its marked as "in development"

[Trotterdam]

Why would we track that? I don't want a region that I stayed in for 1.5 seconds before moving out to be a permanent mark on my record.

[Sandaoguo]

Why would we track that? I don't want a region that I stayed in for 1.5 seconds before moving out to be a permanent mark on my record.

It’s an API call, not your government record… But anyways, it’s a data point worth collecting to monitor the impact of Frontiers and the new flow of newly founded nations around the game.

[Roavin]

Not documented (yet), but check this out: api.cgi?q=newnationdetails

[Darcania]

Not documented (yet), but check this out: api.cgi?q=newnationdetails

Thank you!

[Darcania]

Apologies for the second post in a row, but is there a chance we could get a governortitle shard or something of the like? I notice some regions have custom names for their office of Governor but I can't find that info in the API.

[Merni]

The "governor" shard for regions doesn't seem to be documented.

[Udenstar]

I would like to have a record of my activity from when my nation was created to current. On the activities page, it seems to only show the past 6 or 7 days (possibly a specific number of events?) with no option to go back further. Instead of cluttering up that page, could it be possible to get more log entries via the API?

Data from Issues seem to be found on those individual issues. What isn't included is how World Assembly changes impact the nation or record of nation moves, gaining/losing endorsements, etc.

If these data points are kept in the NationStates database, why can't they be available to the player?

[United Calanworie]

I would like to have a record of my activity from when my nation was created to current. On the activities page, it seems to only show the past 6 or 7 days (possibly a specific number of events?) with no option to go back further. Instead of cluttering up that page, could it be possible to get more log entries via the API?

Data from Issues seem to be found on those individual issues. What isn't included is how World Assembly changes impact the nation or record of nation moves, gaining/losing endorsements, etc.

If these data points are kept in the NationStates database, why can't they be available to the player?

We purge happenings logs after... I want to say seven days? If you wish to keep information about your nation, you need to poll the happenings API at a relatively constant rate, probably once per day, in order to retrieve all that information. The data structure is as follows:

Code: Select all

<NATION id="testlandia">
<HAPPENINGS>
<EVENT>
<TIMESTAMP>1683674301</TIMESTAMP>
<TEXT>
Following new legislation in @@testlandia@@, real power lies not in the debating room but with those who decide what is debated.
</TEXT>
</EVENT>
</HAPPENINGS>
</NATION>


[Eluvatar]

If these data points are kept in the NationStates database, why can't they be available to the player?

Happenings are not (currently) kept in the database, but in something custom. If we were to put them in the database, with an eye to longer term retention, we'd need to write a lot of code for converting between the text of the happenings and a more compressed record for storage.

[Lemmingtopias]

Does NS keep a log of the amount of unique issues answered, or options answered?

And if so, is that something that could be added as a shard alongside the current answered shard?

[Roavin]

Does NS keep a log of the amount of unique issues answered, or options answered?

And if so, is that something that could be added as a shard alongside the current answered shard?

Do you mean something like "issue 123 was answered 123321 times"?

[Lemmingtopias]

Does NS keep a log of the amount of unique issues answered, or options answered?

And if so, is that something that could be added as a shard alongside the current answered shard?

Do you mean something like "issue 123 was answered 123321 times"?

No. I mean something like, Testlandia has answered 123 unique issues.