Apparent throttling(?) of manual recruitment TGs

[Marshal Pilsudski]

I decided to manually send a test regional recruitment TG to a newly-created puppet of mine to make sure everything worked fine and 30 minutes later it isn't in the new nation's inbox although many other recruitment TGs have gone through. My understanding (which could be wrong since I'm returning to NS after several years) is that recruitment TGs sent using stamps or the API were somewhat throttled to reduce flooding new nations with TGs, but that manually-sent TGs were exempt from this because manual recruitment is generally at a disadvantage to automated methods.

Is some kind of throttling the reason my TG wasn't received or is there some other explanation? Is there a way I can ensure that recruitment TGs go through successfully in the future? It is frustrating to see them fail to be delivered when ones sent using automation go through fine; which gives a significant advantage to regions with members who are either willing to spend RL money on stamps or have the technical prowess to use the API.

EDIT: The TG was finally delivered 56 minutes after it was sent. Is there a reason for this long of a delay?

[Galiantus III]

I don't know if this is related, but I have recently sent out several manual telegrams with very slight differences. I've noticed that in the first 50-100 telegrams I tend to have a high conversion rate (the worst I saw at that point was 2%), then beyond that it plummets. None of my manual telegrams have done poorly at the beginning, only later after I have sent like 500 of the same template. Based on what I know of how the system is supposed to work, there should be no throttling, but based on what I've seen recently I'm questioning whether that is the case. To me it would seem that newer templates are being favored over old ones in some way, and there is an advantage to be had in creating a new template every 100 telegrams sent. I don't think this is intentional, but I'm increasingly convinced something is messing with manual recruiting. I'm considering doing a test just to verify what I'm seeing.

[Merni]

To the OP:
All types of recruitment telegram go through the queue for the respective type (stamp, API, manual) before delivery. See here:

When a recruitment telegram is sent, it goes into one of three queues: the Stamp queue, the API queue and the Manual queue. Delivery is rotated between these queues to give each an equal chance. Your anecdotal evidence is just that.

Because almost no-one send manual recruitment TGs (less than a hundred sent in the last week), they are much more likely to be amongst the first-received telegrams compared to Stamps/API-sent recruitment telegrams, because there likely isn't anyone ahead of you in that queue.

Usually manual recruitment is not widely used, and hence there is little delay for them to be delivered. I don't know if for whatever reason the manual queue was (much) fuller than usual at the time you tested it -- a mod can probably confirm this if so. (You can also check the current status of the various queues using the "tgqueue" shard of the World API.)

[The Nation of the People of the Nation]

I'd like to +1 this. I brought it up a month ago and there seemed to be a number of strange occurrences around that too. viewtopic.php?f=15&t=507311

Can someone please actually look at the telegram system to see what's going on?

[Marshal Pilsudski]

Thanks for the responses. I'm still confused as to why my TG took 56 minutes to send. The manual queue hasn't been anywhere close to as full as either the API or stamps queues recently. Why is it that dozens of API and stamps TGs came through before my manual one did? It seems like the system is more opaque than it appears at first sight; could an admin or mod look into this?

[Marshal Pilsudski]

If the admins are unable or unwilling to clarify this situation further it would be nice if they could at least let me know that.

[Marshal Pilsudski]

Hi, I'm bringing this back up because I have TGs timestamped '3 hours ago' that are still sitting in the manual TG queue. I had been running an API script to recruit but recently moved away from it due to the long queue times and rate limit. Manual recruiting avoids some of the rate limit issues because TGs can be sent to 7 nations at once, but I was really hoping the queue times would shorten dramatically, but this doesn't seem to be the case. Is there a reason for this?

[Sedgistan]

There's nothing to look into or report that is different to what was explained in the thread linked by The Nation of the People of the Nation. The only clarification that needs to be made regarding the post of mine quoted further up is that the "Because almost no-one send manual recruitment TGs" paragraph should be disregarded because it was based on erroneously displayed stats.

So:

When a recruitment telegram is sent, it goes into one of three queues: the Stamp queue, the API queue and the Manual queue. Delivery is rotated between these queues to give each an equal chance.

Any delay to a telegram being received is based on the number of telegrams in the relevant queue ahead of it.

[Marshal Pilsudski]

There's nothing to look into or report that is different to what was explained in the thread linked by The Nation of the People of the Nation. The only clarification that needs to be made regarding the post of mine quoted further up is that the "Because almost no-one send manual recruitment TGs" paragraph should be disregarded because it was based on erroneously displayed stats.

So:

When a recruitment telegram is sent, it goes into one of three queues: the Stamp queue, the API queue and the Manual queue. Delivery is rotated between these queues to give each an equal chance.

Any delay to a telegram being received is based on the number of telegrams in the relevant queue ahead of it.

If the stats are erroneously displayed is the tgqueue shard of the API accurate for trying to guess how quickly a TG will be delivered?

[Sedgistan]

Should be. It's only internal mod-viewable stats that are incorrectly displayed.

[Bernie]

So:

When a recruitment telegram is sent, it goes into one of three queues: the Stamp queue, the API queue and the Manual queue. Delivery is rotated between these queues to give each an equal chance.

Any delay to a telegram being received is based on the number of telegrams in the relevant queue ahead of it.

It doesn't literally rotate though right? It is just each has a 1/3 chance of being chosen? EX: I founded a puppet a couple days ago and got three tag:api recruitment messages in a row, when we know there's obviously a massive stamp queue at all times. Also, have gotten two tag:newXXX in a row more than a couple minutes after founding when it seems likely someone's script would have gotten to me.

I'm not trying to be annoying, and I know admin is always hesitant to reveal too much about how stuff works but I'm not firmly in the camp that when I look at what I can see happening it doesn't seem like the queues are being dolled out equally.

[Sedgistan]

I've asked an admin to clarify. It'd help (my end, at least) if you could let me know which puppet that was.

[Bernie]

I've asked an admin to clarify. It'd help (my end, at least) if you could let me know which puppet that was.

Unfortunately to see if a telegram is manual or api or stamp you have to open the TG. And read TGs get pushed out of inbox so it is long gone. If I see it again and archive the TGs would that work or will you not be able to see they are in a row?

[Bernie]

Sorry for double post.

I also just noticed the FAQ says this under Technical: What special groups can be reached using Telegram Stamps?

• tag: New<Number> will queue a Telegram for attempted delivery to the next <Number> newly created nations. Be aware, however, that new nations are telegrammed very quickly by very many region recruiters. In order to reduce spam, the system throttles recruitment messages so that nations only receive a few at a time; additionally, new nations have a limited amount of space in their Inbox. This means that many recruitment telegrams wired to new nations will arrive late or not at all—and consume Telegram Stamps even so. There is no special priority for recruitment Telegrams based on how quickly or by which method they are sent: All recruitment Telegrams enter the same queue and exit it in random order.

As mentioned above this is wrong. Can the queue process be spelled out here, or at least the last sentence deleted?

Thanks!

[Eluvatar]

If the stats are erroneously displayed is the tgqueue shard of the API accurate for trying to guess how quickly a TG will be delivered?

Yes, that display splits queued messages by queue in exactly the same way that the delivery logic does.

It doesn't literally rotate though right? It is just each has a 1/3 chance of being chosen?

We regularly deliver telegrams. When we do so, we deliver all non-recruitment telegrams (except for feeder welcome TGs which still need to be delayed), then we pick a queue randomly. The chance of a particular queue being picked is, as you say, 1/3.

Within each queue, we go through the queued telegrams in random order, and either:
1. Leave them in the queue because the addressee is a new nation and already has a recent recruitment TG
2. Throw them away because the addressee is blocking recruitment TGs or blocking this recruiter or has already received a recruitment TG from the recruiter's region or is in a class region
3. Deliver them

I think it's the first of the above which is making things unpredictable. We don't pick a nation to deliver recruitment TGs to and then randomly pick which one to deliver, we pick a random recruitment TG and try to deliver it, subject to the above constraints. I would like to collect some more detailed internal statistics about how this works, but right now there's not really much more I could say.

[Old Hope]

If the stats are erroneously displayed is the tgqueue shard of the API accurate for trying to guess how quickly a TG will be delivered?

Yes, that display splits queued messages by queue in exactly the same way that the delivery logic does.

It doesn't literally rotate though right? It is just each has a 1/3 chance of being chosen?

We regularly deliver telegrams. When we do so, we deliver all non-recruitment telegrams (except for feeder welcome TGs which still need to be delayed), then we pick a queue randomly. The chance of a particular queue being picked is, as you say, 1/3.

Within each queue, we go through the queued telegrams in random order, and either:
1. Leave them in the queue because the addressee is a new nation and already has a recent recruitment TG
2. Throw them away because the addressee is blocking recruitment TGs or blocking this recruiter or has already received a recruitment TG from the recruiter's region or is in a class region
3. Deliver them

I think it's the first of the above which is making things unpredictable. We don't pick a nation to deliver recruitment TGs to and then randomly pick which one to deliver, we pick a random recruitment TG and try to deliver it, subject to the above constraints. I would like to collect some more detailed internal statistics about how this works, but right now there's not really much more I could say.

I think I found the problems.
Problem 1:
If you have:
Manual queue: 8000 TG's
Stamps: 80000 TG's
then the Manual queue will NOT resolve ten times faster.
Why?
Well, simply because the manual queue is filtered.
A good portion of these TG's will target a narrow range of nations.
If that range is the same, then there might be even more TG's for a specific nation that are manual than Stamp.
Which means that these TG's take longer!
Whilst the nations with obvious fascist or otherwise undesirable connections will probably only get stamps or API telegrams...
the problem is obviously 1. That you filter out telegrams to new nations already having a recruitment TG...
which is not a problem I want to solve(because it is actually not a problem).

The second problem is obviously

Within each queue, we go through the queued telegrams in random order, and either:

This.
This actually makes things unpredictable.
Instead of giving people the oldest queued telegrams first, you give them random ones.
Newest Recruitment telegram A. Queued 2 seconds ago. First check, top of queue - Sent
Oldest Recruitment telegram B. Queued 2 days ago. Sorry, back to queue, recently sent recruitment telegram.
I - and pretty much everyone else, I think - imagined that queued telegrams would be sorted in order of submission, not randomly, when the queue is selected.

[Eluvatar]

The random ordering is intended to avoid giving a large advantage to the first recruiter. Obviously, recruiting early still has some advantage over recruiting late, because you have more chances for your telegram to get through to their inbox, but the intent is to limit that advantage.

[The North Polish Union]

The random ordering is intended to avoid giving a large advantage to the first recruiter. Obviously, recruiting early still has some advantage over recruiting late, because you have more chances for your telegram to get through to their inbox, but the intent is to limit that advantage.

Why was the decision to set things up this way? At this point I'm just asking the question because it seems that its not the way I would have thought to set things up.

My thought would be that manual recruiters are putting the most effort into recruitment and it would make the most sense to reward that in that the first to send a manual TG to a nation gets priority in the manual queue. This would give regions that manually recruit consistently over a long period of time arguably better return on investment than the current system where TGs are picked randomly from the queue.

It seems like the current state of affairs disincentivizes manual recruiting. Manual recruiters don't get to recruit every new nation since they're not online 24/7, must expend the most time recruiting the nations they do recruit, and then thanks to what appears to be a rather convoluted and opaque algorithm may see their TGs wait hours to be delivered. Obviously from a site financial perspective there's little reason to incentivize manual recruitment over stamps but the status quo seems excessive.

[Imperium Anglorum]

The random ordering is intended to avoid giving a large advantage to the first recruiter. Obviously, recruiting early still has some advantage over recruiting late, because you have more chances for your telegram to get through to their inbox, but the intent is to limit that advantage.

Why was the decision to set things up this way? At this point I'm just asking the question because it seems that its not the way I would have thought to set things up.

My thought would be that manual recruiters are putting the most effort into recruitment and it would make the most sense to reward that in that the first to send a manual TG to a nation gets priority in the manual queue. This would give regions that manually recruit consistently over a long period of time arguably better return on investment than the current system where TGs are picked randomly from the queue.

It seems like the current state of affairs disincentivizes manual recruiting. Manual recruiters don't get to recruit every new nation since they're not online 24/7, must expend the most time recruiting the nations they do recruit, and then thanks to what appears to be a rather convoluted and opaque algorithm may see their TGs wait hours to be delivered. Obviously from a site financial perspective there's little reason to incentivize manual recruitment over stamps but the status quo seems excessive.

Manual telegrams get a separate queue all for itself from stamps or the API.