Roavin's big list of advanced API/script questions

[Roavin]

That's a fair (and understandable) ruling (even though it doesn't help me ;_;). On the 28 seconds - they're picked quite well. It's short enough that reasonable usage for non-R/D isn't affected, and long enough that R/D is affected, from what I can tell from playing the past year and change.

Option 1 is good for most purposes, though it falls short when it gets hot and heavy during update. Option 2 is something I might write up, though it's probably difficult to make a case for it beyond "helps defenders". Option 3 - I think I'll leave that alone for now, per your recommendation. Especially since you're surely looking much closer at any User-Agent with the name "Roavin" in it now

Thanks for your patience! I'll stop being insufferable now.

[HMS Unicorn]

if Ballotonia's ruling about editing dispatches stands, then it also contradicts the response to a GHR I had sent about this exact issue - a GHR that dates after Ballotonia's ruling.

Yes, we're aware of this, and we'll reconcile it.

(In case anyone's feeling worried or confused: we're aware there's an ambiguity in provided rulings and are working to resolve it. In the meanwhile, we will not be penalizing anyone who abides in good faith by rulings given.)

Greetings,

Reconciliation is straightforward: I made a major error in that post, confusing editing and posting a dispatch. It's the ruling for posting dispatches, but written for editing, while they're two totally different things. In doing so I contradicted my own prior postings, the ruling and explanation I handed to other admins/mods, and the rulings I sent out in private messages to other players. Yikes. Please accept my apologies for the caused confusion. The ruling in that post is retracted (I edited it to show the retraction).

Ballotonia

Many thanks for the clarification, appreciated .

I think it's useful to take a step back here.

You will inevitably encounter awkward problems with ratelimits and restrictions if you build a tool that needs to scrape a lot of HTML pages instead of using the API, because we deliberately discourage this. I've mentioned publicly before that I'd like to banish scripts & bots from the HTML site altogether, so any machine-generated request would have to use the API. The API doesn't provide 100% coverage, though, so what we have today is a situation where bots are allowed on the HTML site under a very tight ratelimit, while I encourage tool authors to post in the API thread to ask for stuff they want added.

Have you tried that? Because it would be a much better use of your time if you can get something added to the API and enjoy the 10X higher ratelimit rather than build an HTML-based tool where you have to worry about whether this or that individual scraping request gets counted.

I'd say this is the same for developers as well (or at least for me). API is faster, more robust, and even simpler to use. I'd love to exterminate HTML scraping from my codebase.

I would say that, currently, the main area where API is lacking is with actions that require logging into a nation. The private shards was a great first step towards moving those kind of tasks to the API, so I am really looking forward to private shards being expanded.

(Shameless plug, here is my own wish list for the private shards.)

[[violet]]

Heads-up: After discussion, the team has decided to allow scripts to both edit and publish dispatches. This means that editing & publishing dispatches are not considered "restricted actions," and can be legally performed by a bot on its own without any manual intervention.

As with all bot activity on the HTML site, though, such scripts do need to stay within a 10 requests per minute ratelimit and carry an informative User Agent.

I'll update the Script Rules this week. For clarity, we're also looking at making the rules explicitly define "restricted actions" as "sending a telgram, posting on an RMB, or doing anything that generates a Happenings line in another nation, region, or the WA." They imply this at the moment, but leave some gray areas (such as dispatches).

[Chingis]

Since this thread getting so much action, can I quickly pop on a quick question about whether this (Java):

conn.setRequestProperty("User-Agent", "UA String goes here")

where conn is a URLConnection, is a fine way to set the User Agent and whether you've been actually seeing my user agents floating around (They're the ones that read Kimchi, and if future also Miso)

[[violet]]

I haven't seen any bots identifying as Kimchi.

[Krypton Nova]

Since this thread getting so much action, can I quickly pop on a quick question about whether this (Java):

conn.setRequestProperty("User-Agent", "UA String goes here")

where conn is a URLConnection, is a fine way to set the User Agent and whether you've been actually seeing my user agents floating around (They're the ones that read Kimchi, and if future also Miso)

You could use wireshark or some other packet sniffer to see if it sends as intended. That's of course assuming it is your network or a home network, otherwise that may not be an option.