NATION

PASSWORD

NS Stats - Trends, Correlations, Anti-Correlations, Oh My!

A place to spoil daily issues for those who haven't had them yet, snigger at typos, and discuss ideas for new ones.
User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

NS Stats - Trends, Correlations, Anti-Correlations, Oh My!

Postby Valentine Z » Sun Nov 17, 2019 12:59 am

NS Stats - Trends, Correlations, Anti-Correlations, Oh My!

Being given a green light from a Mod, I was told that this will be the best place for this sort of thing. With that said, let's get down to business.

So, NS Stats! Ahh, some of us don't like it, some of us use it, some of us add modifiers to it to better reflect our nation. With that said, one thing is for sure, that the NS Stats are interesting breeds, and sometimes, you have trends that are obvious, or surprising, or sometimes... actually none at all and might not be related.

And I am going to find these trends!

Join me as we go through each of the "X against Y" trends, and we will find out what kind of secret relationships, if any, these stats have.

-----

Data collection / Disclaimer:

NationStates API is used and the API rate limit followed. The rate limit is 50 requests per 30 seconds (1.6667 requests per second). Using Python, I managed to procure up .csv data from the API, at a rate of around 0.3 - 0.6 requests per second using some delay timers. The data is then processed through an Excel file where I sort, create charts, and the like. I might put in more data as I go along, but this seems like a pretty promising sample space!

-----

I hope that this will spark some interest in the world of NS Stats, even if it is not too useful to check your outcome. It's a pretty fun exercise for me. ^^

I will give some time to discuss on the posted trend, and when the time is right, or when I'm free, or when the discussion slowed down. If any of those happens, I will move to a new trend. Though, if there's interest, I will re-visit the previous trends with more data. After all, this will be a dynamic project!

Moving on to a different trend doesn't mean that the discussion for that trend is over; you can still talk about it, but just that there will be more talk on the most recent trend posted.

Let's get started!
Last edited by Valentine Z on Tue Dec 03, 2019 9:42 am, edited 10 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Sun Nov 17, 2019 12:59 am

List of trends using the 1st Batch of Data (accurate as of 16th to 18th of November 2019).

1st Trawl is accurate as of 16-17th of Nov 2019. Regions trawled through: 10000 Islands, Capitalist Paradise, Europe, Forest, NationStates, Psychotic Dictatorships, The Free Nations Region, Two Letter, Europeia. The Pacifics and other large regions are ignored due to the sheer size of data collection.

2nd Trawl is accurate as of 18th of November 2019, at 19:18. The Rejected Realms' nations are added, making it a total of a whopping n = 10 568.

Number of Data Points: 10 568 nations, 908 848 points across 86 stats. There's a shortcoming that it doesn't really indicate the frequency of the points, so.... if there are a crapton of actual non-puppet nations on, say, the coordinates (70,70), we will not know from the graph alone unless I said so. I will annotate from the data, ignoring raiding/defending/N-Day/Z-Day puppets that doesn't have much in the way of answering issues.

X vs Y.
#1-1: Authoritarianism vs Ideological Radicality
#1-2: Religiousness vs Scientific Advancement
#1-3: Corruption vs Political Freedom

-----

List of trends using the 2nd Batch of Data.

The 1st Trawl of the 2nd Batch has been done as of 24th and 25th of December 2019. There are a whopping 19 526 nations this time, 87 stats, 2 values for each stat (Score and World Rank). As a result, there are a crazy amount of data this time to work with, at a staggering 3 397 524 data points (1 698 762 data points with just nations and stats, without world rank).

Regions included in this trawl, along with my puppets: 10000 Islands, Capitalist Paradise, Europe, Forest, NationStates, Osiris, Texas, The Free Nations Region, The North Pacific, The Secret Order, Two Letters.

#2-1: Niceness (X) vs Compassion (Y) - Requested by Angshire
#2-2: Defense Forces (Y) vs Pacifism (X) - Requested by Angshire
#2-3: Economy (Y) vs Economic Freedom (X) - Requested by Angshire
#2-4: Environmental Beauty (Y) vs Economic Freedom (X) - Requested by Aclion
#2-5: Wealth Gap (Y) vs Economic Freedom (X) - Requested by Aclion
#2-6: Average Income (Y) vs Wealth Gaps (X) - Requested by Angshire
#2-7: Culture (Y) vs Weather (X) - Requested by Angshire
#2-8: Crime (Y) vs Civil Rights (X) - Requested by Angshire
#2-9: Public Education (Y) vs Public Healthcare (X) - Requested by Angshire and Bormiar.
#2-10: Public Education (Y) vs Public Transport (X) - Requested by Angshire
#2-11: Public Healthcare (Y) vs Public Transport (X) - Requested by Angshire

In-queue:
#2-12: Scientific Advancement (Y) vs Human Development Index (X) - Requested by Bormiar
#2-13: Public Healthcare (Y) vs Health (X) - Requested by Bormiar and Angshire
#2-14: Public Education (Y) vs Intelligence (X) - Requested by Bormiar and Sacara
#2-15: Safety vs Inclusiveness vs Compassion - Requested by Bormiar
#2-16: HDI (Y) vs Taxation (X) - Requested by East Gondwana
#2-17: Law Enforcement (Y) vs Crime (X) - Requested by Angshire
#2-18: Law Enforcement (Y) vs Defense Forces (X) - Requested by Angshire
#2-19: Crime (Y) vs Safety (X) - Requested by Angshire
#2-20: Arms Manufacturing (Y) vs Defense Forces (X) - Requested by Pangurstan
#2-21: Civil Rights vs Economic Freedom - Requested by Angshire
#2-22: Political Freedom vs Economic Freedom - Requested by Angshire
#2-23: Corruption vs Economic Freedom - Requested by Angshire
#2-24: Integrity (Y) vs Arms Manufacturing (X) - by Valentine Z

To-do:
Extract the % Expenditure of the nations (military, education, health, etc, etc) - Requested by Candlewhisper Archive.
Last edited by Valentine Z on Sun Jan 19, 2020 7:23 am, edited 32 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Sun Nov 17, 2019 12:59 am

Possible Expansion Space.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

#1-1 - Authoritarianism vs Ideological Radicality

Postby Valentine Z » Sun Nov 17, 2019 1:00 am

Image


Now this is a rather interesting one, as you can see. The Authoritarianism (Auth.) seems to have some sort of an exponential curve as Ideological Radicality (IR). The individual data points would then suggest otherwise that the high IR goes both ways - the high IR also exists for little to no Auth. nations; on the other hand, as you can tell, there's a general trend of the higher the Auth., the larger the IR (or vice versa). One thing is for sure, is that you cannot seem to have a largely dictatorial nation with a very low IR. It has to go up.

There are 385 nations with IR > 30 AND Auth > 3000; 497 nations with IR > 30 AND Auth <= 3000.
There are 414 nations with IR > 30 AND Auth > 2000; 468 nations with IR > 30 AND Auth <= 2000.
There are 466 nations with IR > 30 AND Auth > 1000; 416 nations with IR > 30 AND Auth <= 1000.

So, this all depends on whether you count something to be authoritarian. As the cutoff for Milli-Stalins changes, you will see that both sides of the Authoritative debate has the same IR if you are too extreme. After all, you have radical ideas on both ends.

A dictatorship can be extreme, and so is an anarchy.

Another thing I notice is that there seems to be a gradual decrease on the lower bound of Auth as IR increases. You can see that as IR increases, minimum Auth from these nations go from ~500 to 0.

-----

What do you think? Do you fit in the common trend, or do you have a dictatorship that's pretty average (low IR) ? Or an Anarchy with low IR ?

-----

Edit: More data is added, this time from The Rejected Realms. Effectively, I have twice the number of data to work with. For this trend, it seems to be more or less the same.

Image
Last edited by Valentine Z on Wed Nov 20, 2019 10:22 am, edited 7 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

#1-2 - Religiousness vs Scientific Advancement

Postby Valentine Z » Mon Nov 18, 2019 4:41 am

Ahh, the rather controversial one that got a Beta recently (002b). As a result, the Scientific Advancement (SA), has been bumped up quite significantly while not affecting Religiousness (Religion) that much. Or at least, the religiousness nations now have a bigger scientific advancement.

The statement / trend of "Scientific Advancement being anti-correlative with Religion" is over... at least, so it seems. Okay, it has actually went down, but let's look at the graph.

Image

X-axis is Scientific Advancement, runs from -2000 to +2000. Linear.
Y-axis is Religiousness, runs from 1 to 100 000. Logarithmic.

One thing I noticed is that Religiousness does not seem to be able to run below 1. According to the census at the time of this writing, the lowest ever achieved is 1.05 PPH (Prayers Per Hour). The highest goes to Deep South Borland, at 162 149.12 PPH, but they are not inside the dataset; highest goes to Askatopia, at 89 454.63 PPH.

Anyway, back to the trend... how does it look like? It kind of looks like y = ± x0.5. Interesting thing is that for a decently high scientific advancement, you can enjoy the best of both badges. That is, you can a pretty high scientific advancement with high religiousness combined. However, the extremely religious nations seems to get the other end of the SA spectrum.

Some nations are exceptions to the rule, having -500 SA, while having maybe 1 < x < 10 in religiousness / PPH.

Let's look at the linear graph, then:
Image

Ahh, now there's a certain trend to it. Sure, most of the nations (at least in my dataset) are not too high with religion, but you can see that there's a general decrease in SA as the religiousness goes way too high. On the other hand, high SA usually means moderately religious. The recent Beta didn't boost things too high, but a combination of high SA + high religion are still outliers.

Filtering the results, 550 nations are above 500 for Sci. Adv., that is, SA > 500, these are the results:
99 nations out of this 550, have > 100 PPH.
451 nations out of this 550, have < 100 PPH.
I cut it off at 12 PPH because between these nations, we went from 190.30 PPH to 12 PPH. I take > 100 PPH as being religiousness, but as with #1, different cutoffs give different results.

My own conclusion and theory? High Scientific Advancement and High Religiousness is possible, but it gets harder as your SA goes beyond 1400 - 1500. Extremely extremely highly religiousness nations are in the deep negative end of SA, and this will take some time through a balance of answering issues.

-----

What do you think?

Is your nation highly advanced in terms of science? Are you highly religiousness as well? Or is your nation very secular and atheist, and at the same time, very primitive?
(Also want to say that you guys are allowed to post here, if you thought otherwise. Maybe there was a lack of interest for the time being, but yeah, I have no thread ownership in GI, and you can talk here with me. ^^)
Last edited by Valentine Z on Wed Nov 20, 2019 10:22 am, edited 6 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
The Discipled
Lobbyist
 
Posts: 14
Founded: Jul 18, 2018
Civil Rights Lovefest

Postby The Discipled » Mon Nov 18, 2019 6:38 pm

Delay timers (or rather the thread/sleep collections class) are relatively inefficient, i would have recommended implementing time complexity for nation polling gathering purposes. You can set the delay to that precise mathematical division of 1.6(>how long it is by internal decimal) by doing so, which will increase your total collated information.
Last edited by The Discipled on Mon Nov 18, 2019 6:48 pm, edited 2 times in total.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Mon Nov 18, 2019 7:21 pm

The Discipled wrote:Delay timers (or rather the thread/sleep collections class) are relatively inefficient, i would have recommended implementing time complexity for nation polling gathering purposes. You can set the delay to that precise mathematical division of 1.6(>how long it is by internal decimal) by doing so, which will increase your total collated information.

That is true, since the gathering rate is a lot slower than the legal rate, so... I guess I can see to that later. ^^ Because the time taken also involves appending to the .csv file, so I am not sure how long it takes to trawl on the API itself, while not accounting for the data appends.

EDIT: I tested with 52 nations without the sleep timer, and it took 50.67 seconds (avg. 1.02 requests / sec), so... I would imagine that the append and reading the XML data themselves are longer. I suppose I could lower the sleep timer, just to make sure that I don't take chances in breaking the API limit.
Last edited by Valentine Z on Mon Nov 18, 2019 7:24 pm, edited 1 time in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
The Discipled
Lobbyist
 
Posts: 14
Founded: Jul 18, 2018
Civil Rights Lovefest

Postby The Discipled » Tue Nov 19, 2019 2:26 am

Valentine Z wrote:
The Discipled wrote:Delay timers (or rather the thread/sleep collections class) are relatively inefficient, i would have recommended implementing time complexity for nation polling gathering purposes. You can set the delay to that precise mathematical division of 1.6(>how long it is by internal decimal) by doing so, which will increase your total collated information.

That is true, since the gathering rate is a lot slower than the legal rate, so... I guess I can see to that later. ^^ Because the time taken also involves appending to the .csv file, so I am not sure how long it takes to trawl on the API itself, while not accounting for the data appends.

EDIT: I tested with 52 nations without the sleep timer, and it took 50.67 seconds (avg. 1.02 requests / sec), so... I would imagine that the append and reading the XML data themselves are longer. I suppose I could lower the sleep timer, just to make sure that I don't take chances in breaking the API limit.

It would indeed, since the timer is not only waiting for the response but also the updating of the XML file. This can be solved by having the XML file update in buffers, and utilising a data structure (say an Arraylist) for storing recent statistical changes. This allows you to set the delay timer to begin to poll once there is a free allocation in memory instead of waiting for the XML file to update.

This will allow you to poll and update the XML file at the same time (asynchronous execution), as well as solve your problem of the timer kicking in only on completion of the entire task.

User avatar
Marxist Germany
Ambassador
 
Posts: 1671
Founded: Jun 07, 2018
Corporate Bordello

Postby Marxist Germany » Tue Nov 19, 2019 5:12 am

Is your nation highly advanced in terms of science? Are you highly religiousness as well? Or is your nation very secular and atheist, and at the same time, very primitive?
(Also want to say that you guys are allowed to post here, if you thought otherwise. Maybe there was a lack of interest for the time being, but yeah, I have no thread ownership in GI, and you can talk here with me. ^^)

I am in the top 5% for scientific advancement and top 2% for religiousness, I guess I can say that it is possible to maintain a balance. Also, great idea!
Country represents RL views mostly. Not Marxist anymore.
Author of GA#461, GA#470, GA#477, GA#481, and SC#295

Ex-delegate of The United Federations; citizen of 10000 Islands | Gaming User#0721(Discord)
RP name: Germany
The National Factbook (WIP)
Ambassador Klaus Schmidt
Political Compass
PolitiStates Result
Pro:Laissez-faire, Cultural Nationalism, Guns, Free speech, Christianity, Same-sex marriage, United Ireland.
Anti:Extreme Progressivism, Abortion, Socialism, Interventionism, Mass-migration.
A high school student aged 15 from Ireland, living in Co. Dublin. Interested in politics, gaming, and history.

User avatar
Xupqeu
Lobbyist
 
Posts: 20
Founded: Jul 06, 2015
Left-Leaning College State

Postby Xupqeu » Tue Nov 19, 2019 5:20 pm

Valentine Z wrote:Is your nation highly advanced in terms of science? Are you highly religiousness as well? Or is your nation very secular and atheist, and at the same time, very primitive?
(Also want to say that you guys are allowed to post here, if you thought otherwise. Maybe there was a lack of interest for the time being, but yeah, I have no thread ownership in GI, and you can talk here with me. ^^)

Yes, I'm currently at #1 position for scientific advancement in my region, but my country is really secular, although only because my personal bias in answering issues.

I've seen some country with high scientific advancement and religosity, so I think having high stat in both is possible. But, I've never seen any country with high secularism and primitiveness stat.

Now, if only I can reduce the massive taxation in my country without reducing the scientific advancement...
"Remember, reality is an illusion, the universe is a hologram, buy gold, bye!"

Despite what this country's scientific advancement score implies, Xupqeu is actually a postmoderntech country.

User avatar
TalAkMaChen
Envoy
 
Posts: 245
Founded: Sep 04, 2006
New York Times Democracy

Postby TalAkMaChen » Tue Nov 19, 2019 5:45 pm

Xupqeu wrote:...
I've seen some country with high scientific advancement and religosity, so I think having high stat in both is possible. But, I've never seen any country with high secularism and primitiveness stat.


I've seen similar trends, e.g. in this card collection Back to the Primitive, where often you have the top stats on the cards, including top religiousness with primitiveness (and ignorance as common third high ranking stat).

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Tue Nov 19, 2019 11:06 pm

UPDATE on my personal project: As I realised, Betas have the potential to change and affect the trends in some manner sooner or later. And with a few Betas coming in, as well as more data from new census coming in, this will mean that I will be trawling through the data the second time. Or third... or fourth... n-th time.

However, the good news is that I have no problem with this. I will keep the old data, and if the Beta comes out for a particular stat, I will do potential revisits and see what has changed, or otherwise.

This is certainly a long-term project, and I am excited and looking forward to all the trends that we will find! ^^

Currently, I have 10 568 data points, and there are 168 952 nations as of the time of this writing, which effectively means 6.25 % of the data is represented. I could use a bigger sample space and bump it up to 10% in the future, which can be accomplished by visiting the biggest Pacific there is - The North Pacific.

The Discipled wrote:
Valentine Z wrote:That is true, since the gathering rate is a lot slower than the legal rate, so... I guess I can see to that later. ^^ Because the time taken also involves appending to the .csv file, so I am not sure how long it takes to trawl on the API itself, while not accounting for the data appends.

EDIT: I tested with 52 nations without the sleep timer, and it took 50.67 seconds (avg. 1.02 requests / sec), so... I would imagine that the append and reading the XML data themselves are longer. I suppose I could lower the sleep timer, just to make sure that I don't take chances in breaking the API limit.

It would indeed, since the timer is not only waiting for the response but also the updating of the XML file. This can be solved by having the XML file update in buffers, and utilising a data structure (say an Arraylist) for storing recent statistical changes. This allows you to set the delay timer to begin to poll once there is a free allocation in memory instead of waiting for the XML file to update.

This will allow you to poll and update the XML file at the same time (asynchronous execution), as well as solve your problem of the timer kicking in only on completion of the entire task.

Unfortunately, I will have to humbly admit that I am not that advanced just yet. :lol: I appreciate your input a lot, I really do, but I'm still a full-time student that is not too well-versed in his programming skills, I might add. I'm actually surprised that I got this far. ^^; I will take your words into consideration perhaps in the future to get the data more efficiently, but for this task, I think it will suffice.
Last edited by Valentine Z on Tue Nov 19, 2019 11:11 pm, edited 5 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Flanderlion
Ambassador
 
Posts: 1512
Founded: Nov 25, 2013
Iron Fist Consumerists

Postby Flanderlion » Wed Nov 20, 2019 12:26 am

How accurate do you want the data? As the daily dumps have the census of the day.
As always, I'm representing myself as a citizen, rather than as part of the Government, if I am at the time.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Wed Nov 20, 2019 12:52 am

Flanderlion wrote:How accurate do you want the data? As the daily dumps have the census of the day.

The way I'm collecting is for every single stats of the nations in my list, so... I'm aware of the daily dumps, but those are not the complete set that I kinda want.

I figure that my method, even if slow, could be worth it for that few moments because those are the snapshots of the stats for that particular day, not waiting for the census of the day and missing out several days of progress.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Merconitonitopia
Ambassador
 
Posts: 1588
Founded: Jul 29, 2013
Psychotic Dictatorship

Postby Merconitonitopia » Wed Nov 20, 2019 1:45 am

I would like to see the relationship between corruption and political freedom.
TalAkMaChen wrote:
Xupqeu wrote:...
I've seen some country with high scientific advancement and religosity, so I think having high stat in both is possible. But, I've never seen any country with high secularism and primitiveness stat.


I've seen similar trends, e.g. in this card collection Back to the Primitive, where often you have the top stats on the cards, including top religiousness with primitiveness (and ignorance as common third high ranking stat).

I can see my house from here!
There are a few secular primitives, but they are few and far between. Domestic Corporate America is top 1 percent for both primitiveness and secularism.
It appears that almost all primitive nations are highly religious.

I don't know if there are any clever primitives. It seems that ignorance and primitiveness always go hand in hand. This is true even of primitive countries who top the ranks in education, like Gynostan.
Last edited by Merconitonitopia on Wed Nov 20, 2019 1:46 am, edited 1 time in total.
Land and Kingdom of Merconites
Merconitonitopia

The LORD saw that the wickedness of humankind was great in the earth, and that every inclination of the thoughts of their hearts was only evil continually. And the LORD was sorry that he had made humankind on the earth, and it grieved him to his heart. So the LORD said, "I will blot out from the earth the human beings I have created - people together with animals and creeping things and birds of the air, for I am sorry that I have made them."

User avatar
Bears Armed
GA Secretariat
 
Posts: 19054
Founded: Jun 01, 2006
Civil Rights Lovefest

Postby Bears Armed » Wed Nov 20, 2019 8:15 am

Merconitonitopia wrote:I don't know if there are any clever primitives. It seems that ignorance and primitiveness always go hand in hand. This is true even of primitive countries who top the ranks in education, like Gynostan.

Bears Armed: Top 0.2% for Primitive, Bottom 01% for Ignorance...
https://www.nationstates.net/nation=bea ... etail=rank

But maybe it's only Bears, and not Humans as well, that can manage this?
Last edited by Bears Armed on Wed Nov 20, 2019 8:16 am, edited 1 time in total.
The Confederated Clans of the Free Bears of Bears Armed
(includes The Ursine NorthLands) Demonym = Bear[s]; adjective = ‘Urrsish’.
Our population is approximately 20 million. We do have a national government, although its role is strictly limited. Economy = thriving. Those aren't "biker gangs", they're our traditional cross-Clan 'Warrior Societies'... and are generally respected, not feared.
Author of some GA Resolutions, via Bears Armed Mission; subject of an SC resolution.
Factbook. We have more than 70 MAPS. Visitors' Guide.
The IDU's WA Drafting Room is open to help you.
Author of issues #429, 712, 729, 934, 1120, 1152.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

#1-3 - Corruption vs Political Freedom

Postby Valentine Z » Wed Nov 20, 2019 9:57 am

As requested, here is the trend for Corruption and Political Freedom. So... these two stats. You have classifiers and categories for Political Freedom where it could go from World Benchmark to Corrupted at times. Do you always wonder if 100% political freedom can mean that your country is corrupted? Or was this another one of those general trends again? Let's find out!

Image
Political Freedom: x-axis, Linear.
Corruption: y-axis, Logarithmic.

Image
Political Freedom: x-axis, Linear.
Corruption: y-axis, Linear.

Image
Political Freedom: x-axis, Log. Due to the implications of -Infinity with log(0.00), 0.00 Political Freedom is defaulted to 0.01 for this very instance.
Corruption: y-axis, Log.

So, you can see the general trend once again! As your political freedom increases, your corruption will decrease exponentially. That is, from the first graph, as Political Freedom moves to the right, the corruption follows a "linear" trend on the log graph, so it is actually decreasing exponentially. So far, so good, of course! Then it sort of just shot up again, with 100.00 Political Freedom countries just not getting the "Low Corrupt, High Integrity" attribute.

Then you see that there are also points and exceptions to the rule, as always. Some nations still have a high amount of corruption as political freedom increases, though not as bad as the dictatorships. An explanation can be that if you are too politically free, there's nothing stopping you from collecting bribes for governmental and bureaucratic stuff.

More comment on the data: There is a pool of "horizontal" data of 1.00 Corruption for a range of PF, which, in hindsight, could be due to the puppets that doesn't really move much in terms of issue-answering (TODO: this will be considered in my next trawl, to remove too many puppets from affecting the data too much). It's relatively difficult for me to gauge on the active nations to pick, as opposed to the non-active ones. There are those that live by the NS Stats, while there are other nations that don't really play by the stats since Day One. That is why you are seeing a lot of nations at the 1.38 Corruption mark, because there are many nations that don't answer issues and Corruption and Integrity is something that you will not get a badge at the beginning; you will have to earn it.

Using 1.38 Kickbacks Per Hour (KPH) as a mark, I will look at 1578 nations that have >= 80.00 Political Freedom score. Out of these 1578 nations:
  • 1042 nations have < 1.38 KPH.
  • 153 nations have exactly 1.38 KPH. The Queen's States, most of them. The nations from Two Letter, the region.
  • 383 nations have > 1.38 KPH, and out of these nations, 98 of them are > 10 KPH, and 22 of them are > 100 KPH.
  • Highest Political Freedom with Corruption from the dataset goes to Akabusi of The Rejected Realms as of the time of data collection, at 419.07 KPH for 100.00 Political Freedom and 81.00 Civil Rights. They are an Anarchy with World Benchmark Civil Rights, and Excessive Political Freedom.
  • Highest Political Freedom (100.00) with Lowest Corruption goes to Transylvania of The Rejected Realms, all of this excluding the Two Letter Regions. 100 for both Civil and Political Rights/Freedom, while maintaining a cool 1.51 KPH. This nation never answered a single issue.

Still, you can see the general decrease as you move along, so there's that. A country can be politically free, but whether or not it's corrupt... is perhaps up to the persons.

-----

What do you think?

Does your anarchic nation have a crazily high corruption, or is your anarchy state actually a pretty honest one given the nature of the people?
Last edited by Valentine Z on Wed Nov 20, 2019 10:24 am, edited 8 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Wed Nov 20, 2019 9:51 pm

A few Betas have been implemented (namely the Weaponization and Economy)... I guess it's time for a new trawl.

Implemented Betas: https://www.nationstates.net/page=beta?archive=1
Last edited by Valentine Z on Wed Nov 20, 2019 9:54 pm, edited 1 time in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Free Santa Rosa
Bureaucrat
 
Posts: 62
Founded: Sep 23, 2019
Inoffensive Centrist Democracy

Postby Free Santa Rosa » Thu Nov 21, 2019 6:42 pm

This is amazing. Can I suggest adding a trendline? I will also manifest that I'm used to looking at graph titles as (x-axis) vs. (y-axis) so I was kind of confused for a quick bit.

For a long-term thing, I think it would also be relevant to see how trends go across regions/region tags. Maybe there's a way to collate data across those lines as well?

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Thu Nov 21, 2019 10:25 pm

Free Santa Rosa wrote:This is amazing. Can I suggest adding a trendline? I will also manifest that I'm used to looking at graph titles as (x-axis) vs. (y-axis) so I was kind of confused for a quick bit.

For a long-term thing, I think it would also be relevant to see how trends go across regions/region tags. Maybe there's a way to collate data across those lines as well?


Ahh, sure. I will consider adding a trendline (the one that fits the best) for the future analysis. I forgot to mention that one, sorry about that. My graphs, unless said otherwise, will almost always be Y against X. Now I am thinking of having four graphs as with #3 for better viewing - Linear vs Linear, Log vs Linear, and Linear vs Log, and Log vs Log. I think it will help with those results that are condensed inside.

I could do the regional thing, though... I'm not very sure how to get around doing that just yet. What I can do, however, is to analyse trends from the past with the current one, detailing the Betas that are implemented along the way. :D Because of the nature of the data I am collecting (every single stat from every single nation, not the regional average and stuff), it takes a lot of time to request that data under the limit and at the rate I am going at, collecting for ALL the nations across would take me ~ 2.5 days.

Also of note is that even the regional data (i.e. the nations inside of it) needs to be updated with every trawl because of the nature of nations CTE-ing/moving/getting kicked out/etc... As with 10000 Islands as an example:
- As of 18-11-2019, there were 1173 nations.
- As of 22-11-2019, there were 1168 nations.

I need to find a way to throw an error and continue with the collection if the code cannot find that particular data/nation.

-----

Additional note is that I might be away from a few days sometimes, a week even, owing to the busy schedule that I have with my school. ^^;
Last edited by Valentine Z on Thu Nov 21, 2019 10:40 pm, edited 4 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Trotterdam
Powerbroker
 
Posts: 8138
Founded: Jan 12, 2012
Left-Leaning College State

Postby Trotterdam » Thu Nov 21, 2019 11:46 pm

Valentine Z wrote:Ahh, sure. I will consider adding a trendline (the one that fits the best) for the future analysis. I forgot to mention that one, sorry about that. My graphs, unless said otherwise, will almost always be Y against X. Now I am thinking of having four graphs as with #3 for better viewing - Linear vs Linear, Log vs Linear, and Linear vs Log, and Log vs Log. I think it will help with those results that are condensed inside.
I would suggest using rankings rather than actual values. That would avoid needing to worry about scaling shenanigans, since I'm pretty sure the game uses rather arbitrary scaling for some scores (Civil Rights and Political Freedom are "smooshed" using a piecewise-linear function, for example).

I've thought about this some because I've been considering doing my own correlation analysis for a while. It would be pretty easy to code, if I use the data I already have (i.e., skewed towards actively-issue-answering nations that aren't in the WA) rather than collecting it separately, but I've never found the motivation to actually put in the work to implement it, plus making graphs for every possible census score combination would take up a fair bit of disk space.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Fri Nov 22, 2019 12:57 am

Trotterdam wrote:
Valentine Z wrote:Ahh, sure. I will consider adding a trendline (the one that fits the best) for the future analysis. I forgot to mention that one, sorry about that. My graphs, unless said otherwise, will almost always be Y against X. Now I am thinking of having four graphs as with #3 for better viewing - Linear vs Linear, Log vs Linear, and Linear vs Log, and Log vs Log. I think it will help with those results that are condensed inside.
I would suggest using rankings rather than actual values. That would avoid needing to worry about scaling shenanigans, since I'm pretty sure the game uses rather arbitrary scaling for some scores (Civil Rights and Political Freedom are "smooshed" using a piecewise-linear function, for example).

I've thought about this some because I've been considering doing my own correlation analysis for a while. It would be pretty easy to code, if I use the data I already have (i.e., skewed towards actively-issue-answering nations that aren't in the WA) rather than collecting it separately, but I've never found the motivation to actually put in the work to implement it, plus making graphs for every possible census score combination would take up a fair bit of disk space.

That itself is not a bad idea and I actually gave it some thought. I will check out the API to see if there's a data on rankings instead of stats. Though, this itself has a slight problem, I feel. For instance, the "distances" between the stats are more or less lost (like sometimes, there's a big number gap between 1st and 2nd for Pacifism), and other times, 100.00 Civil Rights nations all needed to be smoshed into "1st".

I kind of wanted to analyse the stat numbers themselves, but sure, I will see to it eventually. Like I said, I am planning this as a long-term project, so... let's see if my coding prowess improves and I can do more things efficiently. ^^
Last edited by Valentine Z on Fri Nov 22, 2019 12:58 am, edited 3 times in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

User avatar
Nouveau Yathrib
Diplomat
 
Posts: 883
Founded: Jul 27, 2016
Corrupt Dictatorship

Postby Nouveau Yathrib » Fri Nov 22, 2019 1:29 am



Excellent work Valentine Z! This confirms a lot of what I've seen with my puppets that exist purely for the sake of being in/reaching specific WA govt categories.

1) more radical = more authoritarian, unless you're trying to become more radical by becoming an Anarchy

2) the most religious nations are very backwards, and the least religious nations tend to be rather advanced. But there are far more religious + advanced nations (Jamilkhuze, Syfenq, and many of my puppets fall into this category) than the reverse

3) it's possible to have low political freedoms without having super high corruption, but to get ultra-low levels of corruption you really need to be in the 70-80 range
I still can't believe that Brazil lost to Germany 1:7. Copy and paste onto your sig if you were alive when this happened.

This account is the predecessor state of Jamilkhuze and Syfenq. This is how they're different, and this is why they exist.

We are currently in the year 2181. About Us | Factbooks | Past and Future History | OOC Info | Public Relations | iiWiki | Q&A

"I am only one, but still I am one. I cannot do everything, but still I can do something.
And because I cannot do everything, I will not refuse to do the something that I can do."

-Edward Everett Hale

User avatar
Trotterdam
Powerbroker
 
Posts: 8138
Founded: Jan 12, 2012
Left-Leaning College State

Postby Trotterdam » Fri Nov 22, 2019 2:33 am

Valentine Z wrote:That itself is not a bad idea and I actually gave it some thought. I will check out the API to see if there's a data on rankings instead of stats.
Definitely. You just need to specify &mode=rank.

Alternatively, you can ask for the raw scores and compute the rankings yourself, ranking among all nations in your sample rather than all nations in the world. This would likely give you more digits of accuracy than the rankings provided by the server itself.

Valentine Z wrote:and other times, 100.00 Civil Rights nations all needed to be smoshed into "1st".
Yeah, figuring how to deal with that situation (there are several ways) is the biggest problem with using rankings.

User avatar
Valentine Z
Negotiator
 
Posts: 6452
Founded: Nov 08, 2015
Liberal Democratic Socialists

Postby Valentine Z » Fri Nov 22, 2019 4:07 am

I got the one with the &mode=rank and as expected, I got back the ranks instead of the scores. Though, turns out.... I actually was getting the Ranks all this time; just that I didn't use them.

As for the scores? At least from the documentation, the ones that I am getting seem to be the raw values themselves.

For instance,
Code: Select all
https://www.nationstates.net/cgi-bin/api.cgi?nation=valentine_z;q=census;scale=all&mode=score


score: Raw value, according to the API Documentation.

-----

I wonder if I can get more precise data, like HDI to > 2 decimal places.
Last edited by Valentine Z on Fri Nov 22, 2019 4:16 am, edited 1 time in total.
For You ヽ(*⌒▽⌒*)ノ
NS Stats - Correlations, Anti-Correlations, Oh My!

Clarissa mistaken for Smurf/Avatar: 8 14/19
Valentijn called She: 35 1/3
Genderbent = Sexually Confused: 2

If you are reading my sig, I want you to have the best day ever ! You are worth it, do not let anyone get you down !
Glory to De Geweldige Sierlijke Katachtige Utopia en Zijne Autonome Machten ov Valentine Z !
♥♥♥ ♥♥♥ ♥♥♥ Q & A Here! | Heights of NS! Add yours now! ♥♥♥ ♥♥♥ ♥♥♥
• Eat my freaking tranquility. - Zenyatta
• Never trouble trouble until trouble troubles you.
• World Map is a cat playing with Australia.
NEWS: Valentine Z-Rapture relationships souring? No fucking shit.

Next

Advertisement

Remove ads

Return to Got Issues?

Who is online

Users browsing this forum: No registered users

Advertisement

Remove ads