NationStates API (nationdata/regiondata)

[Valentine Z]

Will a nation population ever hit 100 billion?

/ Not a mod/admin, but I think this would have fit more in NS History thread. Anyway, I will bite, I can't resist a stat-related question.

Kyupaa currently holds the record as the largest nation in terms of population. 40.582 billion. As of now, assuming that the population growth (~6 million per day on average) has not changed for the foreseeable future, I am projecting that it will take even Kyuppa a total of...

(100 billion - 40.582 billion) / (6 million / day) = ~27 years!

See you in 2048.

[The Unified Missourtama States]

Will a nation population ever hit 100 billion?

This is kind of the wrong thread, but we can assume yes, it certainly wouldn't be a storage problem, as population is stored in millions (5 million=5 in the nation data), however perhaps the bigger problem is that a nation would need to be about 45 years old to get to 100 billion!

[Darcness]

The exact limit will vary based on a nation's population.

I'm not looking at the code right this second but as best as I understand the limit is shared between posting dispatches, telegrams, and RMB messages. If your nation is sending telegrams concurrently with creating dispatches, that could be an issue.

I'm running into this even with a 'X-ratelimit-requests-seen' header value of 2, and that is just a single 'prepare' and 'execute' pairing for a single Dispatch add. There is no concurrent activity against the API from this nation.

[The Northern Light]

The exact limit will vary based on a nation's population.

I'm not looking at the code right this second but as best as I understand the limit is shared between posting dispatches, telegrams, and RMB messages. If your nation is sending telegrams concurrently with creating dispatches, that could be an issue.

I'm running into this even with a 'X-ratelimit-requests-seen' header value of 2, and that is just a single 'prepare' and 'execute' pairing for a single Dispatch add. There is no concurrent activity against the API from this nation.

Confirming that I am running into the same issue. I privately flagged this to Eluvatar a couple of days ago, but worth posting in public so that other users of the dispatch API are aware.

As far as I can tell, it is currently impossible to post dispatches with the dispatch API.

[Eluvatar]

I'm running into this even with a 'X-ratelimit-requests-seen' header value of 2, and that is just a single 'prepare' and 'execute' pairing for a single Dispatch add. There is no concurrent activity against the API from this nation.

Confirming that I am running into the same issue. I privately flagged this to Eluvatar a couple of days ago, but worth posting in public so that other users of the dispatch API are aware.

As far as I can tell, it is currently impossible to post dispatches with the dispatch API.

I've found the bug. Preparing the command updates the ratelimiting time.

I will fix it soon (TM) but in the meantime I believe a viable workaround would be to wait sufficient seconds between the prepare call and the execute call.

Edit: It should be fixed.

[The Northern Light]

Confirming that I am running into the same issue. I privately flagged this to Eluvatar a couple of days ago, but worth posting in public so that other users of the dispatch API are aware.

As far as I can tell, it is currently impossible to post dispatches with the dispatch API.

I've found the bug. Preparing the command updates the ratelimiting time.

I will fix it soon (TM) but in the meantime I believe a viable workaround would be to wait sufficient seconds between the prepare call and the execute call.

Edit: It should be fixed.

It seems to be working now. Thanks!

[August]

Bug report: census scores retrieved with the history mode enabled are multiplied by 100 after Sep 8.

Conch Kingdom nation count, single-value (not history) - correct
Conch Kingdom nation count, history mode - correct until Sep 8
Conch Kingdom nation count graph on the HTML site - all correct
Conch Kingdom influence census graph on the HTML site - all correct
Conch Kingdom influence census graph generated from API data - fubared

I used CK as an example, but this seems to apply to all regions. Even though the incorrect values date back to two months ago, this problem just started today. My application makes history queries all the time with no issues.

This seems to have happened at least two more times recently, each time correcting itself exactly 24 hours later. Going off memory and one graph I pulled up, so I may have missed one or two occurrences.

Apr 1, 2021, 0600 ET - Apr 2, 2021, 0600 ET
Apr 10, 2021, 0600 ET - Apr 11, 2021, 0600 ET

This is wreaking havoc on my application, so if it is not a quick fix, I need to write some kind of filter for census data.

[Bormiar]

Yes, I know. I was referring more generally to why bots can’t do it. I’m sure it’s been suggested before, but I don’t know why it was knocked down.

That's a separate issue. I don't remember anyone ever requesting an API shard for posting to RMBs before. (One exists for posting Dispatches.) It wasn't knocked down, it just hasn't been discussed.

For the most part, I'd be most interested in a suppress private shard for nations with communications. I think from there users (especially in large UCRs and GCRs) can create automoderators which handle mundane things a human at a computer doesn't want to do. For example:

• Automatic management of things that many regions don't like but are still acceptable per NS rules, such as double posting.
• The obvious: suppressing things like spam and flames (probably just check for curse words) before a player can send them over to moderation, moderation receives it, notices it and takes action.
• Temporary RMB bans (i.e. suppressing all posts) where banjection seems overkill or is impossible (either because of too much influence or because it's TRR)

I don't mean to press this as though it should be high on the admins' list, but I'd definitely use it if TRR agreed to it. Suppressing spam can be a particularly big pain.

Note: I doubt this shard would send too many requests, because even in large regions you'd only have to make a request to see recent posts every half hour or so.

---

To clarify: is the rate limit per script (i.e. useragent), per device, per user, or per IP? For example, if I have a script running in the background and wanted to do some testing with another script, would I have to coordinate between the scripts to ensure I stay within the rate limit?

[Flanderlion]

To clarify: is the rate limit per script (i.e. useragent), per device, per user, or per IP? For example, if I have a script running in the background and wanted to do some testing with another script, would I have to coordinate between the scripts to ensure I stay within the rate limit?

Per user. But given there should only be one user per device/useragent/IP at a time, I'd also ensure I didn't break the rate limit per those either.

[[violet]]

Bug report: census scores retrieved with the history mode enabled are multiplied by 100 after Sep 8.

This seems to have happened at least two more times recently, each time correcting itself exactly 24 hours later.

It happens when a World Census scale requires renormalization, because nations have started recording scores that are too high or too low to handle easily. So we need to adjust both the stored value and the multiplier. But because there are a quarter-million nations in the game, and their values are cached in a number of places, this takes a little while to complete, and in the meantime, sometimes you see a bad combination of value and multiplier.

This doesn't happen often - only three or four times so far, I think - but may happen again, and cause another period where some scores in some nations will be temporarily wrong.

[August]

It happens when a World Census scale requires renormalization, because nations have started recording scores that are too high or too low to handle easily. So we need to adjust both the stored value and the multiplier. But because there are a quarter-million nations in the game, and their values are cached in a number of places, this takes a little while to complete, and in the meantime, sometimes you see a bad combination of value and multiplier.

This doesn't happen often - only three or four times so far, I think - but may happen again, and cause another period where some scores in some nations will be temporarily wrong.

Thanks for the response. Will it always be a multiplication by 100? My planned solution is to monitor a census shard for a locked region, so that the true value (say, an Economy score of 50) never changes, and apply a correction to the data in my program if the API returns something other than 50.

[Bormiar]

It happens when a World Census scale requires renormalization, because nations have started recording scores that are too high or too low to handle easily. So we need to adjust both the stored value and the multiplier. But because there are a quarter-million nations in the game, and their values are cached in a number of places, this takes a little while to complete, and in the meantime, sometimes you see a bad combination of value and multiplier.

This doesn't happen often - only three or four times so far, I think - but may happen again, and cause another period where some scores in some nations will be temporarily wrong.

Thanks for the response. Will it always be a multiplication by 100? My planned solution is to monitor a census shard for a locked region, so that the true value (say, an Economy score of 50) never changes, and apply a correction to the data in my program if the API returns something other than 50.

Why would it always be a multiplication by a hundred? They said sometimes the stats are too high, which would require division. EDIT: Your correction factor can just be the new value divided by the old value (in this case 50).

By the way, does the 28 second happenings delay have to apply to happenings with filters that don’t relate to triggering (e.g. laws)?

[Flanderlion]

By the way, does the 28 second happenings delay have to apply to happenings with filters that don’t relate to triggering (e.g. laws)?

Laws at the time of implementing (which looks closer to 2012 I think), legislation did relate to triggering - as they used to process at update. That changed when we got 4 issues a day in 2016. So it used to be important to have the delay, now not so much.

[August]

Why would it always be a multiplication by a hundred?

I have no idea why, but all three or four times I have seen it happen so far, the values returned by the API have been 100x greater than their true values.

Your correction factor can just be the new value divided by the old value (in this case 50).

I was planning to do this, but it would be nice to know what I can expect, in case something else breaks in the future. This is such a rare (but destructive) occurrence that I cannot simply wait and figure it out experimentally.

[Bormiar]

Why would it always be a multiplication by a hundred?

I have no idea why, but all three or four times I have seen it happen so far, the values returned by the API have been 100x greater than their true values.

Your correction factor can just be the new value divided by the old value (in this case 50).

I was planning to do this, but it would be nice to know what I can expect, in case something else breaks in the future. This is such a rare (but destructive) occurrence that I cannot simply wait and figure it out experimentally.

What exactly is this for that you can’t let it break, save the data and experiment later?

[August]

What exactly is this for that you can’t let it break, save the data and experiment later?

I cannot let it break because it is a live system used interactively by a lot of people. Even restoring it from a save point would be disruptive.

I do not need to experiment with the data. When I said "figure it out experimentally," I meant "wait for this problem to occur so many times that I can predict how it will affect the data, from empirical evidence."

[Sedgistan]

Suppressing spam can be a particularly big pain.

I'm strongly opposed to this suggestion - scripted post suppression. I don't think you've considered how else it would be used (raid a region, auto suppress every post with no effort required). Nor is it desirable to have the suppression of non-rulebreaking content made automated. Posts that are a mere mild nuisance should require effort to suppress.

[Bormiar]

Suppressing spam can be a particularly big pain.

I'm strongly opposed to this suggestion - scripted post suppression. I don't think you've considered how else it would be used (raid a region, auto suppress every post with no effort required). Nor is it desirable to have the suppression of non-rulebreaking content made automated. Posts that are a mere mild nuisance should require effort to suppress.

I had considered it but I guess I forgot the solution: simply require that the RO has had communications for a certain period of time (like in welcome tgs).

[[violet]]

Why would it always be a multiplication by a hundred?

I have no idea why, but all three or four times I have seen it happen so far, the values returned by the API have been 100x greater than their true values.

That's probably the most common case, where nations just keep accumulating higher scores until we realize we should scale everything down by 100 to fit the numbers into the database. But sometimes we may do different scalings.

[The Glorious Hypetrain]

To clarify: is the rate limit per script (i.e. useragent), per device, per user, or per IP? For example, if I have a script running in the background and wanted to do some testing with another script, would I have to coordinate between the scripts to ensure I stay within the rate limit?

Per user. But given there should only be one user per device/useragent/IP at a time, I'd also ensure I didn't break the rate limit per those either.

Wanted to ask a follow up question w/r/t rate limits on the telegram API and this, would the following use case be in compliance? (acknowledging I need to abide by the total 50 per 30 rule regardless)

1. Script A sending a recruitment telegram every 180 seconds
2. While A is running, Script B starts and begins sending a information telegram to region members every 30 seconds for about 10 minutes

I saw this in the API docs below, but wanted to confirm that my interpretation was correct before proceeding (that both can run rather than one cancelling out the other via timer)

The Telegrams API Rate Limit works by checking the amount of time that has passed since your last successful request. For example, if you sent a telegram via the API 60 seconds ago, you can now successfully send a non-recruitment telegram (since 60 is greater than 30), but not a recruitment telegram (since 60 is less than 180).

Thanks!

[[violet]]

Wanted to ask a follow up question w/r/t rate limits on the telegram API and this, would the following use case be in compliance? (acknowledging I need to abide by the total 50 per 30 rule regardless)

1. Script A sending a recruitment telegram every 180 seconds
2. While A is running, Script B starts and begins sending a information telegram to region members every 30 seconds for about 10 minutes

I saw this in the API docs below, but wanted to confirm that my interpretation was correct before proceeding (that both can run rather than one cancelling out the other via timer)

The Telegrams API Rate Limit works by checking the amount of time that has passed since your last successful request. For example, if you sent a telegram via the API 60 seconds ago, you can now successfully send a non-recruitment telegram (since 60 is greater than 30), but not a recruitment telegram (since 60 is less than 180).

Thanks!

Yes, this is fine.

I'm assuming both scripts use the same API key, so as not to violate API Term #2:

Do Not Sneakily Exceed Rate Limits
You must not deliberately attempt to avoid being rate-limited, e.g. by splitting your API requests across different IP addresses so that they appear to come from different people, or using puppet nations to obtain multiple Telegrams API Client Keys in order to send telegrams faster.

[Flanderlion]

So in a 30 minute period, they look like they're asking if they could send 10 recruitment and 60 information TGs over that time period? But my interpretation, and what your explanation seems to say, is that over that 30 minute period, you could send 30x+180y=1800, with X being other TGs and Y being recruitment ones. One is saying 'and', but I think the rules are 'or' and you pick and choose essentially.

[Trotterdam]

The way I'd interpret it is that you can send a telegram once every 30 seconds, and no more than 1 out of 6 of those telegrams may be a recruitment telegram.

[The Glorious Hypetrain]

Wanted to ask a follow up question w/r/t rate limits on the telegram API and this, would the following use case be in compliance? (acknowledging I need to abide by the total 50 per 30 rule regardless)

1. Script A sending a recruitment telegram every 180 seconds
2. While A is running, Script B starts and begins sending a information telegram to region members every 30 seconds for about 10 minutes

I saw this in the API docs below, but wanted to confirm that my interpretation was correct before proceeding (that both can run rather than one cancelling out the other via timer)

Thanks!

Yes, this is fine.

I'm assuming both scripts use the same API key, so as not to violate API Term #2:

Do Not Sneakily Exceed Rate Limits
You must not deliberately attempt to avoid being rate-limited, e.g. by splitting your API requests across different IP addresses so that they appear to come from different people, or using puppet nations to obtain multiple Telegrams API Client Keys in order to send telegrams faster.

Thank you Violet! They'll come from the same address so #2 will be followed.

[[violet]]

So in a 30 minute period, they look like they're asking if they could send 10 recruitment and 60 information TGs over that time period? But my interpretation, and what your explanation seems to say, is that over that 30 minute period, you could send 30x+180y=1800, with X being other TGs and Y being recruitment ones. One is saying 'and', but I think the rules are 'or' and you pick and choose essentially.

What I'm saying is it's compliant to run multiple Telegram API scripts at once (using the same API key). You won't violate the rules because the server won't permit you to exceed the rate-limit, which is described in the part of the API doc quoted by The Glorious Hypetrain.

This is different to the situation where a script interacts with the HTML site, where the rate limit isn't enforced by the server, but rather policed by admin and moderators, and script authors need to be careful to abide by it or else face a ban.

When you run multiple Telegram API scripts at once, they will of course compete for the same rate-limit allowance--you don't get to send twice as fast by using two scripts.