[Suggestion] Allow script-generated API telegram templates

[HMS Unicorn]

I suggest legalizing the use of scripts to automatically generate API telegram templates (i.e., telegrams sent to "tag:api" and subsequently used with the standard telegram API). The use of such scripts is currently illegal according to this post by Eluvatar.

Please note that this suggestion is specific to API telegram templates, and does not include telegram templates that can be used for manual or stamp-based telegramming.

---

There are several reasons for allowing such scripts.

Constantly-running scripted API telegrams require constant maintenance, in the form of clerical changes for updating references to region officials, URLs, dates, and so on. The process of making such changes can be very tedious, and is also very error-prone: as you edit a telegram draft multiple times to bring it up to date, you may forget to make a change, or introduce a buggy BBcode, or make a mistake copying and installing the new TG ID and secret key in your telegramming script, and so on. Such mistakes can be hard to identify, and it can often be that you only detect them after having sent hundreds of copies of the telegram (or after having made hundreds of buggy TG API calls due to wrong keys).

Likewise, in many cases we want to send one-time mass-TGs through the API, for periodically recurring events (e.g., regional elections, WA vote announcements, etc.). We often use a base template for the text of such telegrams, only replacing specific details as appropriates, like dates and URLs. As in the previous case, creating these API templates manually is tedious and error-prone.

In both cases, being able to automate the template updating process end-to-end would make the maintainers' life much easier, and would remove the potential for bugs and thousands of wasted API calls.

---

To summarize, I would strongly encourage the administration to revisit this rule and permit scripts that create API telegram templates. I think it would be greatly beneficial to the game if such scripts are permitted, as there are several legitimate uses for such scripts. Moreover, such a rule change would not jeopardize the separation between manual and API telegramming, and any downsides with respect to use patterns for API telegram templates can be offset in ways that do not require completely banning such scripts.

Thanks for the consideration!

Edit: Fixed unpaired HR tags.

[Wordy]

Frankly I hate the script arms race that is going on. I have no argument with scripts as such but the fact that they are not well policed and only available to regions with those skilled in creating them.
My argument would be if a script is considered it then is submitted for a legality check and upon approval generally released for all users.

[[violet]]

Regarding the first reason, in essentially all cases where one could use a script create such individualized API telegram templates, they could achieve the exact same effect by having an array of pre-made templates and selecting from among them based on the recipient nation's flag, motto, etc. The only cases where these would not be possible are cases where genuine human creativity is needed for very fine-level customization - and in those cases, a script creating API telegram templates on the fly could not be used either. Therefore, in my opinion, this whole argument for rendering this kind of scripts illegal is entirely moot.

There's a big difference between theory and practice here. In theory, it's the same thing whether a script is generating its own template or merely selecting from a very large array of human-made templates. But in practice, no-one is doing the latter, because that's hard, while a lot of people will do the former, because it would be easy.

I'd even expect that scripts would begin to generate new a template for every message even when they don't need to, just because it's simpler to have a single pathway.

That matters because it's very low-load/storage for us server-side when someone's sending copies of a single template. It's 1,000X more expensive when they're instead sending 1,000 individual TGs, or making single-use templates. With humans, there's a natural brake from the effort required, but automated scripts are relentless and so could become expensive very quickly.

Regarding the second reason, this is a reasonable concern. However, banning script-generated API templates seems like a very extreme solution. The situation the admins fear could be prevented in other ways, without completely eliminating what could be for many regions a very useful functionality.

One possibility is imposing a rate limit on the number of different API templates each API client key can queue copies of in a period of time - say (arbitrary numbers), 100 per month.

Yes, and another is requiring TG stamps for API template creation.

In either case, we're talking about diverting dev time to build systems for something that's only useful to a relatively small number of people, and is actively opposed by some, so it needs a pretty good case.

My preference, where possible, is instead to add more tokens and customization options to templates, since these can be used by everyone, not just scripts.

[Roavin]

Apologies for the gravedig, but this is, I think, fitting in this case:

My preference, where possible, is instead to add more tokens and customization options to templates, since these can be used by everyone, not just scripts.

What do you think of the possibility to add "parameter" macros to templates that can be filled out via the API call?

Here's a silly example:

Hello there %NATION%,

for the safety of the region, we require that WA nations endorse both our delegate %PARAM1% and all of our Guardians. However, you've missed %PARAM3% of them (probably accidentally), so this is a helpful reminder that you still need to endorse:
%PARAM2%

With kind regards,
-- The Guardians

The API call would be

https://www.nationstates.net/cgi-bin/api.cgi?a=sendTG&client=(Client Key)&tgid=(TGID)&key=(Secret Key)&to=roavin&param1=%5Bnation%5DTestlandia%5B%2Fnation%5D&param2=%5Bnation%5DMax+Barry%5B%2Fnation%5D%0D%0A%5Bnation%5DJennifer+Government%5B%2Fnation%5D&param3=2

And would result in:

Hello there Roavin,

for the safety of the region, we require that WA nations endorse both our delegate Testlandia and all of our Guardians. However, you've missed 2 of them (probably accidentally), so this is a helpful reminder that you still need to endorse:
Max Barry
Jennifer Government

With kind regards,
-- The Guardians

[Trotterdam]

I'd even expect that scripts would begin to generate new a template for every message even when they don't need to, just because it's simpler to have a single pathway.

That matters because it's very low-load/storage for us server-side when someone's sending copies of a single template. It's 1,000X more expensive when they're instead sending 1,000 individual TGs, or making single-use templates. With humans, there's a natural brake from the effort required, but automated scripts are relentless and so could become expensive very quickly.

How about a more severe ratelimit on script-generated templates, like one per hour or one per day? That would allow automated generation of regional event announcements, but not personalized text for every single recruitment telegram.

What do you think of the possibility to add "parameter" macros to templates that can be filled out via the API call?

This seems abusable by someone simply making the whole template "%PARAM1%" and then supplying the text as one huge POST/GET request (which is how telegram sending by humans works, anyway).

[Eluvatar]

Presumably parameters would be limited to 40 characters long, but it's still problematic.

[Roavin]

Presumably parameters would be limited to 40 characters long, but it's still problematic.

I should note that in my example, I encoded the [nation]-tags within the parameters (appropriately urlencoded).

[Eluvatar]

Presumably as your parameters would, if intended to be individual nation links, always be nation links, you could use

Code: Select all

[nation]%PARAM1%[/nation]

and thus not need the extra space. If of course you want a list, you want more room than 40 characters, and that's quite problematic.

I'd personally be more inclined to permit meta-templates (or "texts") which can be realized into specific templates through the substitution of parameters than the ability to substitute strings in a template in every TG sent with that template. This would use less space in our database and allow the retention of most current code and displays, just with the addition of some new functionality on top.

[Roavin]

Presumably as your parameters would, if intended to be individual nation links, always be nation links, you could use

Code: Select all

[nation]%PARAM1%[/nation]

and thus not need the extra space. If of course you want a list, you want more room than 40 characters, and that's quite problematic.

Correct. PARAM1 in my example could be wrapped in the tags within the template. PARAM2 is a list, however.

I'd personally be more inclined to permit meta-templates (or "texts") which can be realized into specific templates through the substitution of parameters than the ability to substitute strings in a template in every TG sent with that template. This would use less space in our database and allow the retention of most current code and displays, just with the addition of some new functionality on top.

So, trying to understand this, it appears that the actual TG text is not generated at send time, but rather at render time (which seems to me the most reasonable way of doing it). So in that regard, I can see how it's not a trivial change. Assuming I understood your suggestion correctly, you're proposing that prewritten texts can be plugged into parameters rather than arbitrary strings, then the utility of that to my use case is negligble (though probably useful for OP's use case). At the same time, though, that would require introducing some sort of additional per-telegram-instance data - if that's a template reference or an arbitrary string seems to me, just in terms of effort/infrastructure required, to be the same thing.

If the primary concern is database space from thousands of TGs generated with arbitrary string parameters, then ... yeah, I'm not sure if I have a good suggestion to counter that. Using nullable TEXT columns (so that they're not stored in-row) and a reasonable character length limit would mitigate somewhat but of course I can't judge whether that's enough or not.

[Eluvatar]

Arbitrary string parameters would let your telegram template be

Code: Select all

%PARAM1%

and the entire telegram would actually be in PARAM1 every time.

I have no idea what you mean about prewritten texts getting plugged into parameters.

I have no idea what you mean regarding nullable text columns.

I can attempt to explain again what I meant:

First of all, there will still be telegram templates, which will still have tgid numbers as currently, and can still contain %NATION% or %TOKEN% which will still be decided at render time based on who's viewing the tg.

Second, there will be a new entity, let's call it a 'form letter':

1. A 'form letter' can have arbitrarily many %PARAMETER%s.
2. A human must create any 'form letter'.
3. There will be a way to turn a 'form letter' into an api telegram template, passing in the values to substitute for any and all parameters.
4. There will be a way to combine the statistics for all telegrams based on a 'form letter'.
5. There might be a more severe rate limit for 'form letter' realization into telegram templates than for API telegram sending. Perhaps one telegram template per hour or per day.

[Roavin]

Ah. Yeah that will handle OP's use case beautifully. It won't necessarily handle mine particularly well, but I do understand why admin is hesitant to unwilling to allow arbitrary per-TG string parameters.

[Eluvatar]

If we're just spitballing, I imagine your use case could be handled through the addition of:

Code: Select all

Greetings minion,

I have your instructions for today.

Please endorse:
[list=1]
%FOR NAME IN LIST_PARAM%
[*][nation]%NAME%[/nation]
%END%
[/list]

Thank you for your attention to our inestimably important decrees,
~The Supreme Chair of the Guardian Council of Elders


[Roavin]

Yes!

If these lists can contain tuples, it'd be even better (such as "move %NAME% to %REGION%" for an automated sleeper management system), but it'd work without as well.