NationStates Today

[Sandaoguo]

...Make me Editor...

You're not funny. I'll burn the organization down myself before that happens, if we had it our way you wouldn't even be able to post in this thread.

- Q

What is left to burn down exactly? Y’all just posted saying your entire operations team has been defunct, and the last actual article posted is from 6 months ago by the looks of it

[Clorinda]

You're not funny. I'll burn the organization down myself before that happens, if we had it our way you wouldn't even be able to post in this thread.

- Q

What is left to burn down exactly? Y’all just posted saying your entire operations team has been defunct, and the last actual article posted is from 6 months ago by the looks of it

@-@

king shit

[North East Somerset]

Did the “expert panel” just rank themselves?

Interesting...

[Quebecshire]

Thanks for those insightful graphics, NES!

Miravana pointed this out in the gameplay server when the P20 got published. I can't go back and grab exactly what he said for, uh, reasons, but anyway.

If memory serves correctly: About 8 panel members were on the Power 20. So a little over half of the 15 person panel, and a little under half of the Power 20. It's a common criticism that panel members often end up on the Power 20 itself.

The hard truth is that such an issue is never going to be removed. Mitigated? Yes. I could note several panelists from this version and the previous and explain why they were brought on because "they represent X sphere but are unlikely to be on the list themselves". That isn't a perfect solution though. People decline being on the panel sometimes, and the truth is that the people with the most knowledge to give here often are influential themselves.

Sure, I could throw 15 randos on a panel, but then I feel like the end result would see different complaints later.

[Free Algerstonia]

personally I believe that the power20 results were rigged due to the absence of kazakhstan 3

[RiderSyl]

<snip>

So... if panelists weren't allowed to select themselves, then Roavin and Xoriet would be tied for 1st and Salem would be a close 2nd.

That feels right.

[Quebecshire]

<snip>

So... if panelists weren't allowed to select themselves, then Roavin and Xoriet would be tied for 1st and Salem would be a close 2nd.

That feels right.

Panelists aren't allowed to select themselves. Emphasis mine.

The panellists were first invited to review the fairness of the panel’s composition
and the proposed schedule. Then, a period of discussion was held on the most
important Gameplay events of the last six months to invite thinking about the most
influential players in this period. Then, all panel members could nominate as many
individuals as they wished, not including themselves, including at least one reason
for each nomination.

The 30 nominees with 2+ nominations were shortlisted. The panel agreed to
advance one of the individuals with one nomination. In the voting form, panel
members ranked all 31 of the shortlisted players; this could not include
themselves. If a panellist was on the shortlist, they were to leave the 31st spot
blank. Points were then assigned for each rank, from 31 for first to 1 for thirty-first
on each ballot.

[Malphe II]

Thanks for those insightful graphics, NES!

Miravana pointed this out in the gameplay server when the P20 got published. I can't go back and grab exactly what he said for, uh, reasons, but anyway.

If memory serves correctly: About 8 panel members were on the Power 20. So a little over half of the 15 person panel, and a little under half of the Power 20. It's a common criticism that panel members often end up on the Power 20 itself.

The hard truth is that such an issue is never going to be removed. Mitigated? Yes. I could note several panelists from this version and the previous and explain why they were brought on because "they represent X sphere but are unlikely to be on the list themselves". That isn't a perfect solution though. People decline being on the panel sometimes, and the truth is that the people with the most knowledge to give here often are influential themselves.

Sure, I could throw 15 randos on a panel, but then I feel like the end result would see different complaints later.

'Influence' in NationStates gameplay, or whatever else this is meant to be based off, is directly proportional to levels of involvement & knowledge, so yeah I agree that there's essentially no way to prevent the stock of voters on these awards being mostly identical to the stock of nominees. There are no laypersons in gameplay who know enough to offer informed opinions, without being achieved or sociable enough to get on it themselves. It's why I'll never really see much real value in it, it's invariably an in-group of mutually amicable active gameplayers patting eachother on the back, but as a fun media activity it doesn't really have to be objectively valuable and it seems like yall are trying to make it as objective as you reasonably can.

[Cheesy Tots]

Thanks for those insightful graphics, NES!

Miravana pointed this out in the gameplay server when the P20 got published. I can't go back and grab exactly what he said for, uh, reasons, but anyway.

If memory serves correctly: About 8 panel members were on the Power 20. So a little over half of the 15 person panel, and a little under half of the Power 20. It's a common criticism that panel members often end up on the Power 20 itself.

The hard truth is that such an issue is never going to be removed. Mitigated? Yes. I could note several panelists from this version and the previous and explain why they were brought on because "they represent X sphere but are unlikely to be on the list themselves". That isn't a perfect solution though. People decline being on the panel sometimes, and the truth is that the people with the most knowledge to give here often are influential themselves.

Sure, I could throw 15 randos on a panel, but then I feel like the end result would see different complaints later.

To be fair, Xoriet, Tim, Dilber, Jo, Madjack, and Roavin would all surely be shoe-ins for a Power20, and they make up a third of the panel. Yes they're the most knowledgable, but I guess it depends on the balance you wish to strike between credibility and accuracy.

But ultimately this probably doesn't matter as Power20 isn't to serious of a list and I think the list was generally solid as it tends to be.

edit: malphe ninja'd me i swear

[RiderSyl]

So... if panelists weren't allowed to select themselves, then Roavin and Xoriet would be tied for 1st and Salem would be a close 2nd.

That feels right.

Panelists aren't allowed to select themselves. Emphasis mine.

The panellists were first invited to review the fairness of the panel’s composition
and the proposed schedule. Then, a period of discussion was held on the most
important Gameplay events of the last six months to invite thinking about the most
influential players in this period. Then, all panel members could nominate as many
individuals as they wished, not including themselves, including at least one reason
for each nomination.

The 30 nominees with 2+ nominations were shortlisted. The panel agreed to
advance one of the individuals with one nomination. In the voting form, panel
members ranked all 31 of the shortlisted players; this could not include
themselves. If a panellist was on the shortlist, they were to leave the 31st spot
blank. Points were then assigned for each rank, from 31 for first to 1 for thirty-first
on each ballot.

Badly worded on my part. I should've said "If panelists couldn't be nominated".

[Quebecshire]

Badly worded on my part. I should've said "If panelists couldn't be nominated".

Ah, understandable. Personally I feel that would be more problematic. Banning panels from being nominated at all would skew the thing way more than the current arrangement, because we would either need to block people from the get-go who would otherwise be on it, or invite people who are likely not as familiar with the gameplay scene. Who knows, though.

[Lord Dominator]

At the very least, only 9/62 people have only been on the list while also being a panelist - and only Dilber has managed that more than once (all times actually). This seems reasonable given the circumstances.

Meanwhile, only the aforementioned Dilber + Dakota/Salem, Altino, Tim, Roavin, and Xoriet have had a majority of their listings match up with their appearances on the panel. All of them have been reasonably important/etc at given times, and the most frequent of them have had the longer stretches of it.

There’s probably things to criticize about the list, but I don’t believe the presence of panelists on the list is one of them - though it might be worthwhile to choose people less likely to make the list cut for the panel, if possible.

Edit: Koth also has a majority of appearances while on the panel.

[North East Somerset]

At the very least, only 9/62 people have only been on the list while also being a panelist - and only Dilber has managed that more than once (all times actually). This seems reasonable given the circumstances.

I'm not sure you are reading the graph correctly. The number is actually 26/64 for people who have appeared on the list whilst also being on a panel while they were nominated, at least once. And an astonishing 11/64 times people have only ever appeared in the Power20 when they were themselves on the Panel.

I think its also clear the selection of panellists is far from random, not even from the pool of players the Power20 deems influential, and who could thus be deemed "experts". Certainly there are opportunities for improvement on that front.

[Lord Dominator]

At the very least, only 9/62 people have only been on the list while also being a panelist - and only Dilber has managed that more than once (all times actually). This seems reasonable given the circumstances.

I'm not sure you are reading the graph correctly. The number is actually 26/64 for people who have appeared on the list whilst also being on a panel while they were nominated, at least once. And an astonishing 11/64 times people have only ever appeared in the Power20 when they were themselves on the Panel.

I think its also clear the selection of panellists is far from random, not even from the pool of players the Power20 deems influential, and who could thus be deemed "experts". Certainly there are opportunities for improvement on that front.

My first statistic is strictly your second one, so I thank you for the correction on that. I do agree however there’s room to improve on the panelist front, I just don’t view it as a problem by itself (rather, it’s largely one of optics and possibly mild bias).

[Miravana]

As someone who joined the organizing team for the first time this edition, I would like to note that there was considerable effort to leave certain players off the panel since we believed they would make the list. You site the number 26/64 as a site of failure, but I see that as the decent majority of people on the panels do not end up on the list, which is certainly a good achievement given how much of the actual knowledge of gameplay is held within people who are most definitely going to make the list. If there was any area of improvement, I would think it would be to try to avoid picking panelists who are likely to appear near the 15-25 range on the finished results, based on the organizers own assessments of the list, as those are the most likely to receive any sort of boost for being on the panel and in conversation. That being said, as someone who is privy to the actual voting, most panelists did a very good job of not voting panelists on the board simply because they were on the voting list. Two panelists who were on the voting list failed to make it to the power 20 list, and neither of them would have left heads scratching had they actually made it.


I am hearing a lot of people said "While I do agree with how the results ended up, I am upset about the number of panelists who made it." My question is, if the method that saw less than half the power20 made up of panelists ended up with a generally agreed-upon power 20, is the method really ineffective or biased?

[North East Somerset]

As someone who joined the organizing team for the first time this edition, I would like to note that there was considerable effort to leave certain players off the panel since we believed they would make the list. You site the number 26/64 as a site of failure, but I see that as the decent majority of people on the panels do not end up on the list, which is certainly a good achievement given how much of the actual knowledge of gameplay is held within people who are most definitely going to make the list.

Thats simply not true. 9 of the 15 panellists were on the Power20 this time around. Over the entire history of the Power20, that number is 44 appearances by 77 judges being selected by the panel/s they served on, or 57%.

I am hearing a lot of people said "While I do agree with how the results ended up, I am upset about the number of panelists who made it." My question is, if the method that saw less than half the power20 made up of panelists ended up with a generally agreed-upon power 20, is the method really ineffective or biased?

Again, 9 of the 15 panellists ended up on the Power20. Thats 60%.

[Miravana]

Thats simply not true. 9 of the 15 panellists were on the Power20 this time around. Over the entire history of the Power20, that number is 44 appearances by 77 judges being selected by the panel/s they served on, or 57%.

Read what I said again, less than half of the Individual People who have been on the panel in the history of the publication have appeared on the list in the same year they were on the panel. As you said 26/64.

Again, 9 of the 15 panellists ended up on the Power20. Thats 60%.

What I said is that less than half the power20 is made up of panelists, not that less than half the panelist was on the Power20.

[North East Somerset]

Thats simply not true. 9 of the 15 panellists were on the Power20 this time around. Over the entire history of the Power20, that number is 44 appearances by 77 judges being selected by the panel/s they served on, or 57%.

Read what I said again, less than half of the Individual People who have been on the panel in the history of the publication have appeared on the list in the same year they were on the panel. As you said 26/64.

Again, 9 of the 15 panellists ended up on the Power20. Thats 60%.

What I said is that less than half the power20 is made up of panelists, not that less than half the panelist was on the Power20.

Yes I know what you said, and you are quoting statistics designed to lessen the basic fact that the majority of the Panel ended up in the Power20.

Oh... and thats 9 out of 15 of the panellists, excluding the "Special Managers" who both miracurously made it onto the list as well!

The reason I'm speaking out, is that I am one of the top 5 players in the game to be consistently featured on these lists over the time they have run so far, so it can't be alleged I am either not influential, or bitter about the results.

What I really dont understand is, if a player like me is making it onto the Power20 in 5 out of the 6 rounds ever held - but the panel is a fair representative sample of "experts" - then how I have I never been on the panel? Is the selection of panellists truly random and unbiased?

Some people have served 4+ times in a row on the panel. What I'm pointing out is that the selection of panellists clearly is neither random nor proportional to their expertise. So what is it based on, and why do you think I've never sat on the Panel?

[Miravana]

Read what I said again, less than half of the Individual People who have been on the panel in the history of the publication have appeared on the list in the same year they were on the panel. As you said 26/64.




What I said is that less than half the power20 is made up of panelists, not that less than half the panelist was on the Power20.

Yes I know what you said, and you are quoting statistics designed to lessen the basic fact that the majority of the Panel ended up in the Power20.

Oh... and thats 9 out of 15 of the panellists, excluding the "Special Managers" who both miracurously made it onto the list as well!

The reason I'm speaking out, is that I am one of the top 5 players in the game to be consistently featured on these lists over the time they have run so far, so it can't be alleged I am either not influential, or bitter about the results.

What I really dont understand is, if a player like me is making it onto the Power20 in 5 out of the 6 rounds ever held - but the panel is a fair representative sample of "experts" - then how I have I never been on the panel? Is the selection of panellists truly random and unbiased?

Some people have served 4+ times in a row on the panel. What I'm pointing out is that the selection of panellists clearly is neither random nor proportional to their expertise. So what is it based on, and why do you think I've never sat on the Panel?

I am failing to understand the point you are trying to make. Do we have too many panelists who make the power20, or should NES, someone who was a shoo-in for the list, also have been included? I can only speak for this panel when it was determined Kazaman was knowledgeable about your sphere to give an accurate and fair representation of it. Since you ask about the process I will give it to you here.

The panel is created with an attempt to have as many different spheres and areas of what can be considered "Gameplay" aptly represented.

In many sphere's it is hard to determine an adequate substitute for someone who can be considered a shoo-in, or said panelists may be able to represent multiple spheres and therefore limit the total number of panelists, as I am sure everyone can agree a larger panel would cause only more headache.

I think it is also important to mention that yes there are obviously some panelists who get picked more than others, but that is because they have shown to be good representatives of their spheres and give unbiased votes towards the poles. I am struggling to understand why there is so much complaint against the system if the actual content of the list is not being challenged. If you took away the word panelist from all those involved, I think it is easy to say these are objectively close to as accurate as you can get in a power 20 list, or at least admit there are no real surprises here.

[North East Somerset]

Yes I know what you said, and you are quoting statistics designed to lessen the basic fact that the majority of the Panel ended up in the Power20.

Oh... and thats 9 out of 15 of the panellists, excluding the "Special Managers" who both miracurously made it onto the list as well!

The reason I'm speaking out, is that I am one of the top 5 players in the game to be consistently featured on these lists over the time they have run so far, so it can't be alleged I am either not influential, or bitter about the results.

What I really dont understand is, if a player like me is making it onto the Power20 in 5 out of the 6 rounds ever held - but the panel is a fair representative sample of "experts" - then how I have I never been on the panel? Is the selection of panellists truly random and unbiased?

Some people have served 4+ times in a row on the panel. What I'm pointing out is that the selection of panellists clearly is neither random nor proportional to their expertise. So what is it based on, and why do you think I've never sat on the Panel?

I am failing to understand the point you are trying to make. Do we have too many panelists who make the power20, or should NES, someone who was a shoo-in for the list, also have been included? I can only speak for this panel when it was determined Kazaman was knowledgeable about your sphere to give an accurate and fair representation of it. Since you ask about the process I will give it to you here.

The panel is created with an attempt to have as many different spheres and areas of what can be considered "Gameplay" aptly represented.

In many sphere's it is hard to determine an adequate substitute for someone who can be considered a shoo-in, or said panelists may be able to represent multiple spheres and therefore limit the total number of panelists, as I am sure everyone can agree a larger panel would cause only more headache.

I think it is also important to mention that yes there are obviously some panelists who get picked more than others, but that is because they have shown to be good representatives of their spheres and give unbiased votes towards the poles. I am struggling to understand why there is so much complaint against the system if the actual content of the list is not being challenged. If you took away the word panelist from all those involved, I think it is easy to say these are objectively close to as accurate as you can get in a power 20 list, or at least admit there are no real surprises here.

There are several other players who are a shoo-in, have made several appearances in the list, and who get picked consistently time after time as a panelist, even when there are other panelists also from their same "sphere".

Are you thus saying I would not be a "good representative" or give an "unbiased vote"?

[Miravana]

There are several other players who are a shoo-in, have made several appearances in the list, and who get picked consistently time after time as a panelist, even when there are other panelists also from their same "sphere".

Are you thus saying I would not be a "good representative" or give an "unbiased vote"?

Rather the opposite. I am saying that in your sphere there are other good representatives that were able to vote fairly in your stead. Not ever sphere has obvious choices for the replacement of "shoo-ins", or the shoo-ins in that sphere spread across more regions than their "less influential" peers.

[Unibot III]

I’m just not sure I understand what the value of the panel-based thing is, regardless. In the past when we did “Who is the most influential?” surveys, we’d just open a poll in NSGP. The results were usually interesting and I don’t think they were particularly misinformed — certainly less open to bias.

[Miravana]

I’m just not sure I understand what the value of the panel-based thing is, regardless. In the past when we did “Who is the most influential?” surveys, we’d just open a poll in NSGP. The results were usually interesting and I don’t think they were particularly misinformed — certainly less open to bias.

The main reason I'd caution against this is because then you could vote.

[Tinhampton]

I’m just not sure I understand what the value of the panel-based thing is, regardless. In the past when we did “Who is the most influential?” surveys, we’d just open a poll in NSGP. The results were usually interesting and I don’t think they were particularly misinformed — certainly less open to bias.

The main reason I'd caution against this is because then you could vote.

Unibot is a single voter. You are also a single voter (and all the active gameplayers are something like a few hundred voters).

[Madjack]

For whatever reason people are engaging with Unibot without actually addressing his points. Either engage and address or don't. Here's how easy it is:

The reason the P20 is done on a panel model, has always been done on a panel model and will be done on a panel model in the future if NSToday survives, is because if it were opened to a wider audience it would end up a popularity contest. And whilst popularity and influence can have a close relationship, I would not say they are the same. With a diverse panel that recognises that, you will likely get a more accurate view of just who is influential or not, because it has been my experience of working with this panel that things like popularity were largely set aside during this process, which likely wouldn't be the case in a wider vote.