by Alfonzo » Fri Jan 07, 2022 10:40 pm
by Domais » Fri Jan 07, 2022 10:42 pm
Alfonzo wrote:Hello,
Over the last few hours update has been running very strangely (at least major updates). This all started with small lag last major update but has now escalated to update being significantly halted this current update. Is it possible for this to be looked into?
Thank you.
by Tim-Opolis » Fri Jan 07, 2022 10:50 pm
<Koth - 06/30/2020> I mean as far as GPers go, Tim is one of the most iconic
by Riet Grrrl » Fri Jan 07, 2022 10:51 pm
08/01/2022, 5:33:29 GMT: Amagiri Ayato's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:28 GMT: Makasta being forced to merc's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:26 GMT: Bit of an L's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:25 GMT: Just as I was about to give up's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:23 GMT: Im good at defender's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:21 GMT: Miltray dictator of TL's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:19 GMT: Ohh ohhh ohhhhh's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:16 GMT: Something deep inside's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:14 GMT: Thank you mannal's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:11 GMT: Manunal recurtment i am sorry for you's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:08 GMT: Chadtopia of chadness's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:06 GMT: Suree im a native's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:03 GMT: Krunkoth's influence in Russian Sfsr fell from "Hermit" to "Page".
08/01/2022, 5:33:01 GMT: GCR soverty's influence in Russian Sfsr fell from "Hermit" to "Page".
by Reploid Productions » Fri Jan 07, 2022 10:54 pm
[violet] wrote:Maybe we could power our new search engine from the sexual tension between you two.
by Refuge Isle » Fri Jan 07, 2022 10:57 pm
by Ever-Wandering Souls » Fri Jan 07, 2022 10:57 pm
The Alicorns (Equestria) wrote:Let them stay, no need to badmouth them...From our view a bunch of nations just came in, seized the delegate position, and changed a few superficial things...we play NationStates differently...there's really no reason for us to be butthurt.
http://www.nationstates.net/page=rmb/postid=8944227
http://www.nationstates.net/page=rmb/postid=8951258
Reploid Productions wrote:Raiders are endlessly creative
by Lord Dominator » Fri Jan 07, 2022 11:02 pm
Ever-Wandering Souls wrote:That's a lot of confidence in some mystery bad actor. Like, there's definitely some instability, and my sympathies for this not being a fair face-off at update for y'all to work on building your beachhead, but that's a lot of jumping to conclusions of specific malice against your lib, for what could have a dozen other technical explanations.
[violet] wrote:Should be fixed now. A player decided to rent some Amazon servers to send 50GB worth of bot requests to the Challenge page.
by Refuge Isle » Fri Jan 07, 2022 11:05 pm
Ever-Wandering Souls wrote:That's a lot of confidence in some mystery bad actor. Like, there's definitely some instability, and my sympathies for this not being a fair face-off at update for y'all to work on building your beachhead, but that's a lot of jumping to conclusions of specific malice against your lib, for what could have a dozen other technical explanations.
by Ever-Wandering Souls » Fri Jan 07, 2022 11:18 pm
Refuge Isle wrote:Ever-Wandering Souls wrote:That's a lot of confidence in some mystery bad actor. Like, there's definitely some instability, and my sympathies for this not being a fair face-off at update for y'all to work on building your beachhead, but that's a lot of jumping to conclusions of specific malice against your lib, for what could have a dozen other technical explanations.
95% of this thread has thus far been posting nation updating times. I have a high degree of confidence in those timestamps.
Variance is one thing, a two minute time span becoming a twelve minute timespan when all other points on update appear normal is worth investigation.
The Alicorns (Equestria) wrote:Let them stay, no need to badmouth them...From our view a bunch of nations just came in, seized the delegate position, and changed a few superficial things...we play NationStates differently...there's really no reason for us to be butthurt.
http://www.nationstates.net/page=rmb/postid=8944227
http://www.nationstates.net/page=rmb/postid=8951258
Reploid Productions wrote:Raiders are endlessly creative
by Vincent Drake » Fri Jan 07, 2022 11:27 pm
we're also performing a record number of actions per second for site history (with huge jumps and modern macros/tools)
by Plasma Sage Ghestis » Fri Jan 07, 2022 11:41 pm
by Altmoras » Fri Jan 07, 2022 11:42 pm
Ever-Wandering Souls wrote:Refuge Isle wrote:95% of this thread has thus far been posting nation updating times. I have a high degree of confidence in those timestamps.
Variance is one thing, a two minute time span becoming a twelve minute timespan when all other points on update appear normal is worth investigation.
Certainly not saying it didn't happen, or that it shouldn't be investigated, more just that we're also performing a record number of actions per second for site history (with huge jumps and modern macros/tools), on two decade old spaghetti code, during an update that exists for an hour because 2 tin cans in Canada have to run at 100% capacity for that long to process it - so there's at least plausible other considerations, and we should be asking admin to investigate, before confidently going all in on that.
by Vando0sa » Sat Jan 08, 2022 4:08 am
Plasma Sage Ghestis wrote:Seems myself and potentially a few others had more specific issues pertaining to this topic, some activity feed and game behavior discrepancies. On the initial jump, I jumped with the nation I'm posting from here. At the time of the jump, this nation's activity feed contains my move into A Liberal Haven, a couple people endorsing me, an ejection and ban from the region by GonkDroid, a couple more people endorsing me, then my resignation and later subsequent re-admittance to the WA. In the interim there, I switched a couple times and tried jumping into the region again while the window to do so was still drug open by the massive site lag, then noticed that when I switched back onto this nation that it still behaves as though it's in A Liberal Haven and, from what I can tell, never actually moved to TRR. I confirmed that it can still endorse everyone endorsing Garchyland. My endorsements of these nations appear to still count towards their totals. Other people still appear to be able to endorse me in A Liberal Haven, and all of that activity pertaining to this nation in A Liberal Haven appears to be showing up in TRR's activity feed. It also appears as though this nation never updated in A Liberal Haven as a result of all that, so its influence hasn't changed and there's no corresponding message in the activity feed. The game still shows A Liberal Haven on the side bar as the region this nation is currently in, and I have a telegram from the website informing me of my ban and ejection from the region. Not really sure what to think of all that, but it's what it is. I can grab the relevant snips of activity feeds if they're helpful, otherwise I don't see the point of cluttering the thread when it's easy to click a nation and see the feed.
by Budgie Snugglers » Sat Jan 08, 2022 5:14 am
by Unibot III » Sat Jan 08, 2022 9:29 am
[violet] wrote:I mean this in the best possible way,
but Unibot is not a typical NS player.
Milograd wrote:You're a caring, resolute lunatic
with the best of intentions.
by Plasma Sage Ghestis » Sat Jan 08, 2022 9:34 am
Vando0sa wrote:Plasma Sage Ghestis wrote:Seems myself and potentially a few others had more specific issues pertaining to this topic, some activity feed and game behavior discrepancies. On the initial jump, I jumped with the nation I'm posting from here. At the time of the jump, this nation's activity feed contains my move into A Liberal Haven, a couple people endorsing me, an ejection and ban from the region by GonkDroid, a couple more people endorsing me, then my resignation and later subsequent re-admittance to the WA. In the interim there, I switched a couple times and tried jumping into the region again while the window to do so was still drug open by the massive site lag, then noticed that when I switched back onto this nation that it still behaves as though it's in A Liberal Haven and, from what I can tell, never actually moved to TRR. I confirmed that it can still endorse everyone endorsing Garchyland. My endorsements of these nations appear to still count towards their totals. Other people still appear to be able to endorse me in A Liberal Haven, and all of that activity pertaining to this nation in A Liberal Haven appears to be showing up in TRR's activity feed. It also appears as though this nation never updated in A Liberal Haven as a result of all that, so its influence hasn't changed and there's no corresponding message in the activity feed. The game still shows A Liberal Haven on the side bar as the region this nation is currently in, and I have a telegram from the website informing me of my ban and ejection from the region. Not really sure what to think of all that, but it's what it is. I can grab the relevant snips of activity feeds if they're helpful, otherwise I don't see the point of cluttering the thread when it's easy to click a nation and see the feed.
Oh hey! I just made another thread about this and another weird glitchy thing that happened.. we might have code gremlins..
We removed you from the banlist and banjected again so maybe it fixed it.. still weird though..
by [violet] » Sun Jan 09, 2022 3:14 pm
by Ever-Wandering Souls » Sun Jan 09, 2022 4:37 pm
[violet] wrote:Eluvatar and I have both been looking into this. Thanks again also for the details and charts above, which make it much easier to figure out what we should be looking at.
System load has remained normal since Jan 5, when we blocked a Challenge bot. For the week prior, and especially on Dec 30, 31st, and Jan 3, load from this bot caused update to run long.
Since then, update has more or less hit target length, but it did experience an unusual stall on Jan 7. At this point, I suspect the problem lies in the Activity log, which became backlogged attempting to record a very large number of entries. We haven't identified the root cause, but so far the behavior hasn't repeated.
Update is the single most load-intensive point we have, so it is sensitive to other load-generating events that would otherwise go unnoticed.
The Alicorns (Equestria) wrote:Let them stay, no need to badmouth them...From our view a bunch of nations just came in, seized the delegate position, and changed a few superficial things...we play NationStates differently...there's really no reason for us to be butthurt.
http://www.nationstates.net/page=rmb/postid=8944227
http://www.nationstates.net/page=rmb/postid=8951258
Reploid Productions wrote:Raiders are endlessly creative
by [violet] » Sun Jan 09, 2022 5:54 pm
by Roavin » Sun Jan 09, 2022 6:06 pm
(1) 1/7/2022, 6:27:37 AM GMT+1: Bladetress-22's influence in The Rejected Realms rose from "Zero" to "Unproven".
by Eluvatar » Mon Jan 10, 2022 3:19 pm
[violet] wrote:At this point, I suspect the problem lies in the Activity log, which became backlogged attempting to record a very large number of entries. We haven't identified the root cause, but so far the behavior hasn't repeated.
by [violet] » Mon Jan 10, 2022 4:45 pm
Roavin wrote:Also, to provide some context as to why some individuals think this may be foul play: This is a kind of spike that we haven't seen before, and it only happens right around the time a jump into an occupied region takes place. Even more suspiciously, of the four successive updates where a jump takes place, one of them is not affected, that being the weekday minor update which are traditionally the ones were the least amount of gameplayers are available (and, if this is a malicious actor, they would have to be a R/D gameplayer due to the timing). I hope it's just server gremlins, but I can't fault suspicions that it's not, and at this point we have neither proof that it is nor that it isn't, and should probably drop both speculations and undue celebrations until [v] and Elu know more.
by Roavin » Mon Jan 10, 2022 5:05 pm
[violet] wrote:There was a code change on December 2, 2021 that increased the workload caused by a region move. (The process now has to load the target region twice, not just once.) Doesn't line up particularly well with the issue here, but it's the only relevant system change I can think of. Potentially it might create noticeable slowdown when large numbers of nations are moving into the same region at the same time.
Advertisement
Users browsing this forum: Alcrosnia, Barbartopia, Ducky, Elgoriath, Google [Bot], Khantin, Land Without Shrimp, Moloto Japan, Neuebremen, Omnicontrol, Pyhdon, Reyo, Roydonk, Sateru, Sorcery, Soveriegn, Sto Lat, Tape, The Hurricane, The Mazzars, The Plough Islands, Tianjastan, Vietnam Remnants, Zucksland
Advertisement