NationStates API (nationdata/regiondata)

[[violet]]

^^ Bug fixed. Your region restricted recruitment to "API only" and the system was blocking your attempt to create an API template since it wasn't technically an API request. It should work now.

[Auralia]

[violet], would it be possible to add a private shard that simply resets the CTE timer for a nation? I'd like to write a web-based autologin script, but the main NationStates website doesn't (and shouldn't) support CORS requests. As it stands I can make the request (using hidden POST form hackery) but I can't examine it to verify that it succeeded. A dedicated API call would solve that problem.

[Greater Tern]

[violet], would it be possible to add a private shard that simply resets the CTE timer for a nation? I'd like to write a web-based autologin script, but the main NationStates website doesn't (and shouldn't) support CORS requests. As it stands I can make the request (using hidden POST form hackery) but I can't examine it to verify that it succeeded. A dedicated API call would solve that problem.

I think any of the private shards reset the CTE timer when accessed, since using them counts as a login.

[Imperium Anglorum]

[violet], would it be possible to add a private shard that simply resets the CTE timer for a nation? I'd like to write a web-based autologin script, but the main NationStates website doesn't (and shouldn't) support CORS requests. As it stands I can make the request (using hidden POST form hackery) but I can't examine it to verify that it succeeded. A dedicated API call would solve that problem.

Oh, I did that already (it's in beta though), https://github.com/ifly6/ifly-Nation-Manager . It is just a list of nations and relevant attributes (encrypted), with a connector to query each one. When it comes to the private shards, I just fetch the nation Notifications.

---

V, will there be a way to restore nations via the API, or will that have to be done via interaction with the site?

[Auralia]

I think any of the private shards reset the CTE timer when accessed, since using them counts as a login.

It looks like you're correct! In that case, [violet], would you mind adding CORS support to the private shards API? I believe you would need to add "Password", "Pin", and "Autologin" to an "Access-Control-Allow-Headers" header in API responses.

V, will there be a way to restore nations via the API, or will that have to be done via interaction with the site?

This seems unlikely, given that restoring nations has gameside effects in places other than your own nation.

[[violet]]

[violet], would you mind adding CORS support to the private shards API? I believe you would need to add "Password", "Pin", and "Autologin" to an "Access-Control-Allow-Headers" header in API responses.

People have asked for CORS stuff before but I've never been able to figure it out with enough confidence to feel like I know what I'm doing.

Can your script not see whether the request returns HTTP 200 vs 403/509?

[Auralia]

[violet], would you mind adding CORS support to the private shards API? I believe you would need to add "Password", "Pin", and "Autologin" to an "Access-Control-Allow-Headers" header in API responses.

People have asked for CORS stuff before but I've never been able to figure it out with enough confidence to feel like I know what I'm doing.

Can your script not see whether the request returns HTTP 200 vs 403/509?

No -- without CORS support the browser blocks JavaScript from receiving any response whatsoever, including the HTTP return code.

For example, when trying to get Auralia's notices, the Chrome Developer Console prints the following:

Fetch API cannot load https://www.nationstates.net/cgi-bin/ap ... uralia&v=9. Request header field password is not allowed by Access-Control-Allow-Headers in preflight response.

...and JavaScript gets a "TypeError: Failed to fetch" with no other information.

You've already added CORS support to most of the API by including an "Access-Control-Allow-Origin" header in the response that contains the origin of the request. Actually, I believe it was in response to a request from me.

The problem is that the private shards API requires that you add an additional header for authentication (Password, Pin, or Autologin). Browsers don't let JavaScript send cross-origin requests with arbitrary headers unless authorized by the destination through an "Access-Control-Allow-Headers" header in the response. You can read more about this header on MDN.

Also, if you want the X-Pin header in the response to be visible to the JavaScript running in the browser, you need to include that header in a "Access-Control-Expose-Headers" header in the response as well. Again, you can take a look at MDN.

As a side note, all custom headers should generally be prefaced with "X-" in order to avoid conflicting with new headers defined in future versions of HTTP.

[[violet]]

The part I have trouble with is I know this security model exists for a reason, and when I start creating exceptions, I'm unclear on the security implications of the holes I just poked in it.

I can certainly see it's handy for third-party scripts to be able to make cross-domain requests and read the return data. But what class of attacks does enabling "Access-Control-Allow-Headers" allow? There must be some, or it wouldn't exist.

[[violet]]

As a side note, all custom headers should generally be prefaced with "X-" in order to avoid conflicting with new headers defined in future versions of HTTP.

Is this not the case already?

[Imperium Anglorum]

As a side note, all custom headers should generally be prefaced with "X-" in order to avoid conflicting with new headers defined in future versions of HTTP.

Is this not the case already?

I think he's referring to the authentication example here? https://www.nationstates.net/pages/api.html#nationapi

[Auralia]

The part I have trouble with is I know this security model exists for a reason, and when I start creating exceptions, I'm unclear on the security implications of the holes I just poked in it.

I can certainly see it's handy for third-party scripts to be able to make cross-domain requests and read the return data. But what class of attacks does enabling "Access-Control-Allow-Headers" allow? There must be some, or it wouldn't exist.

If I understand it correctly, the original purpose of blocking cross-domain requests was to prevent malicious websites from masquerading as the user on other websites. Let's say you log into a banking website and then visit a malicious website. In the absense of the same-origin policy, the JavaScript running on the website could send requests to the banking website withdrawing money, and the bank would think they were coming from you because they are coming from your browser using your session cookie.

This attack doesn't really make sense in the context of the NationStates API, though. Most of the shards are public, so it doesn't matter who accesses them. Moreover, the private shards do not rely on cookies for authentication, so a malicious website can't masquerade as the user unless the user actually provides them with their password.

As a side note, all custom headers should generally be prefaced with "X-" in order to avoid conflicting with new headers defined in future versions of HTTP.

Is this not the case already?

Not entirely. Per the API documentation, the request authentication headers for private shards are "Pin", "Password" and "Autologin", which are not prefaced with "X-". However, the response uses the header "X-Pin".

[Caelapes]

Moreover, the private shards do not rely on cookies for authentication, so a malicious website can't masquerade as the user unless the user actually provides them with their password.

I'm pretty sure you can use private shards with a login token/PIN (I forget the exact term - it's been a while since I've looked at the private shards) which is identical to the one stored in your cookies for auto-login.

[Auralia]

Moreover, the private shards do not rely on cookies for authentication, so a malicious website can't masquerade as the user unless the user actually provides them with their password.

I'm pretty sure you can use private shards with a login token/PIN (I forget the exact term - it's been a while since I've looked at the private shards) which is identical to the one stored in your cookies for auto-login.

That's true, but it doesn't affect my underlying point. The NationStates private shards API doesn't actually look at cookies for authentication. It uses custom headers which will not be included automatically by a browser in a request. The script actually has to know what credentials to use and specify them manually.

[[violet]]

This attack doesn't really make sense in the context of the NationStates API, though. Most of the shards are public, so it doesn't matter who accesses them. Moreover, the private shards do not rely on cookies for authentication, so a malicious website can't masquerade as the user unless the user actually provides them with their password.

That much I understand, but not why the ability to post headers is also denied by default. That doesn't have anything to do with cookies yet has apparently been deemed risky enough to block by browser vendors.

Per the API documentation, the request authentication headers for private shards are "Pin", "Password" and "Autologin", which are not prefaced with "X-". However, the response uses the header "X-Pin".

Ah, I see. I'll update the documentation and we'll accept both X- prefaced and non-prefaced versions.

[Auralia]

This attack doesn't really make sense in the context of the NationStates API, though. Most of the shards are public, so it doesn't matter who accesses them. Moreover, the private shards do not rely on cookies for authentication, so a malicious website can't masquerade as the user unless the user actually provides them with their password.

That much I understand, but not why the ability to post headers is also denied by default. That doesn't have anything to do with cookies yet has apparently been deemed risky enough to block by browser vendors.

One user on StackOverflow suggests that it's just browsers being cautious about exposing web sites to any kind of request that used to be blocked by the same-origin policy. Some servers may react badly when certain kinds of headers are included in a request, so browsers require that they explicitly opt-in to receiving those headers. If you really wanted to cause mayhem you could simply use curl or wget, but perhaps the browser vendors are just being friendly.

One other possibility that I thought of is to prevent phishing. If part of your public API requires custom headers for some special restricted purpose, you may want to prevent browsers from accessing that part of the API so as to ensure that no user can be fooled into thinking that they are communicating with you directly rather than through a go-between.

[Auralia]

Any thoughts, [violet]? I'd appreciate knowing for certain whether you're planning to open up the private shards API via CORS so that I can know whether it's necessary to wrap my script in an Electron wrapper or something.

[Really stateless nation]

Is there any API to get (anti-)top N countries from non-today World Census? For example, I can want to get names of top 10 countries for Corruption and names of anti-top 7 countries for Nudity (i.e. 7 countries with the most clothed citizens).

[Greater Tern]

Is there any API to get (anti-)top N countries from non-today World Census? For example, I can want to get names of top 10 countries for Corruption and names of anti-top 7 countries for Nudity (i.e. 7 countries with the most clothed citizens).

You can use the censusranks shard with the scale modifier to get the top 10 for a given census: https://www.nationstates.net/cgi-bin/ap ... s;scale=51

You can add the start modifier on top of that to change the start index: https://www.nationstates.net/cgi-bin/ap ... 1;start=50

I'm not sure if there's a way to get the last 10 nations for a given census.

[Imperium Anglorum]

Right now, if I use this, I'll get my endorsements.

Code: Select all

https://www.nationstates.net/cgi-bin/api.cgi?nation=imperium_anglorum&q=endorsements

Would it be possible to have something like this:

Code: Select all

https://www.nationstates.net/cgi-bin/api.cgi?nation=imperium_anglorum+alsted&q=endorsements

So one can query multiple nations for the same shard?

[[violet]]

So one can query multiple nations for the same shard?

No, this isn't supported. The API is structured that way because loading additional nations is one of the slower areas of our code.

[Imperium Anglorum]

So one can query multiple nations for the same shard?

No, this isn't supported. The API is structured that way because loading additional nations is one of the slower areas of our code.

As the Daily dump does not show endorsement data, is the only way I can get that data for my region by querying the API for every single nation?

[Agadar]

"beforetime" and "sincetime" parameters are now supported in the Happenings API. See the doc for examples.

Violet, for how long are happenings stored for? It seems I can't get any more happenings that are older than the 17th of January.

[Trotterdam]

for how long are happenings stored for?

7 days, as you probably suspected from that date.

[Agadar]

for how long are happenings stored for?

7 days, as you probably suspected from that date.

That's real disappointing to learn, as this means I can't make regional rankings of nations with the most endorsements given, as there is no other way for me to find out who endorsed who during a specific period of time outside of happenings, which are apparently limited to up to 7 days ago.

[Imperium Anglorum]

That's real disappointing to learn, as this means I can't make regional rankings of nations with the most endorsements given, as there is no other way for me to find out who endorsed who during a specific period of time outside of happenings, which are apparently limited to up to 7 days ago.

Query lots of nations into lots of files which you can store.