[v3.0.1] Spyglass - Open Source Region Timesheet Generator

[Spyglass Development Team]

Spyglass
Open Source NationStates Update Timesheet Generator

About
Spyglass generates Excel Spreadsheets that can be used to predict the update time of a NationStates region, usually to coordinate an invasion, liberation, or other military gameplay action.

• Estimated update times: View estimated major and minor update times for all regions.
• R/D metagame prioritization: Identify regions based the presence/absence of a founder, WA delegate, and password.
• WFE and Embassy preview: Preview world factbook entries and regional embassies to quickly determine if a region has been impacted by the R/D metagame.

To learn more about what information Spyglass provides, view the documentation.

Spyglass is open source. If you want to take part in its development by reporting issues, requesting features, or contributing code, visit us on Github!

Licensing and Credits
Spyglass version 2.0 and onward are licensed under the terms of the GNU General Public License version 3.0.

Spyglass was originally developed by Devi and is now additionally maintained by Khronion, Zizou, and Aav. The icon was contributed by Vleerian.

Downloads

Spyglass downloads are hosted on Github.

You may also use an online version of Spyglass via Google Colab.

Windows and macOS users should use the pre-packaged release files for their operating system. These versions contain all the files needed to run Spyglass. Linux users should see the instructions for setting up a development environment.

Important note: Microsoft Defender Smartscreen may block execution of programs it doesn't recognize, like Spyglass. You may bypass these warnings by clicking "Run Anyway" on the Smartscreen warning dialog.

After downloading the latest version of Spyglass, run the program and follow the on-screen prompts. Spyglass currently runs in a command prompt window, but a graphical user interface is planned for the future. Spyglass will then create an Excel spreadsheet in the same directory where it is located.

Disclaimer: Although Spyglass is designed to comply with NationStates API rules and rate limits, players are ultimately responsible for the behavior of any scripts, tools, or other programs that they use. Players are encouraged to report any issues or unexpected behavior in this forum thread, or by filing an issue on GitHub.

[Spyglass Development Team]

reserved

[Devi]

big props to ziz/aav/khron/souls for getting stuff to a cleaner, more modern state and actually vaguely understanding how github works despite 2016 devi's best attempts to make things a pain <3

[Ever-Wandering Souls]

Thanks Devi <3

To give a tldr/sorta changelog:
-Spyglass had 4 (raider) pages: Devi's main (stuck in 1.3), Khron's 1.4 fork (upkeep vulnerability fixes), and then Aav and Ziz's 1.5 and 1.6 versions (converted to Py3, added GUI, etc).
-We've now gotten together, made some MORE updates(some time fixes, better documentation, better runtime flags, better handling of dependencies when running from source, added GPLv3.0 license), and gotten the Main back up to date - now hosting 2.0.
-2.0 is currently CLI only, but the GUI will be re-added soon in 2.1, alongside some further codebase improvements.
-We also now have both Windows and Mac executables on the release.
-Documentation referencing 'tagging' has been replaced with 'subject to military gameplay actions' so fendas can use it guilt-free.
-We've also created this thread and the relevant dispatch to give Spyglass a proper home NS-Side
-In case you somehow missed it in the past, any version of Spyglass older than 1.4.4 contains a security vulnerability that allows code injection due to improper sanitization. "KD8NPQ" and "Courelli" (NSLeft, I think?) still have forks based on pre-1.4.4 versions.

[Merni]

Very cool! One feature that could make it even more useful specifically for detagging is a column on ROs in the region. (i made a rather crude fork of the 1.4.4 version for this, which some people in defending have found useful.)

[United Calanworie]

Very cool! One feature that could make it even more useful specifically for detagging is a column on ROs in the region. (i made a rather crude fork of the 1.4.4 version for this, which some people in defending have found useful.)

We've got a large backlog of stuff that we're putting together for a 2.1 release, so I'm honestly not sure we'd have time to put this together any time soon. If you want to fork and write it yourself though, feel free to submit a pull request and we'll definitely figure out how it could merge in ^-^

[Merni]

Very cool! One feature that could make it even more useful specifically for detagging is a column on ROs in the region. (i made a rather crude fork of the 1.4.4 version for this, which some people in defending have found useful.)

We've got a large backlog of stuff that we're putting together for a 2.1 release, so I'm honestly not sure we'd have time to put this together any time soon. If you want to fork and write it yourself though, feel free to submit a pull request and we'll definitely figure out how it could merge in ^-^

I'll definitely try that when I have the time, thanks.

[Khronion]

Hi Merni, I appreciate that you are one of the few folks outside of the current development team who have worked with the code! Personally, I'm touched to see that many people across NS are still using Spyglass over its nearly six year history, and would love to see your old code (and any others) make its way back as a pull request in the future.

[Ever-Wandering Souls]

FYI: (probably) all versions of spyglass, ever, broke in the last day or two, likely due to a minor change on the NS end that broke some poor code in Spyglass related to downloading and saving the regions dump. A likely fix has been found, and will be implemented to the repository ASAP.

In the meantime, users can circumvent the bugged portion of the code by manually downloading the current data dump from https://www.nationstates.net/pages/regions.xml.gz and placing that file in the same folder Spyglass runs from - then in the initial options when running Spyglass, selecting the option to use the already-downloaded data dump, NOT download a new one.

[Ever-Wandering Souls]

Update: v2.0.2 resolves the issue. https://github.com/Derpseh/Spyglass/releases/tag/v2.0.2

"Now everyone will HAVE to update their old forks, yay"

[Esfalsa]

Is Spyglass still being actively maintained?

I sent in a pull request last month but haven't seen any activity, so I figured I would check here as to whether GitHub is still the preferred platform for issues/contributions or whether Spyglass is still being actively maintained. (And if there's a better place for inquiries to the maintainers in the future, feel free to let me know.)

[United Calanworie]

Is Spyglass still being actively maintained?

I sent in a pull request last month but haven't seen any activity, so I figured I would check here as to whether GitHub is still the preferred platform for issues/contributions or whether Spyglass is still being actively maintained. (And if there's a better place for inquiries to the maintainers in the future, feel free to let me know.)

Spyglass is still actively maintained. For some reason, GitHub did not notify me (or any of the other maintainers, it appears) that you had made a pull request... despite it saying in the feed area "ProTip! The feed shows you events from people you follow and repositories you watch or star." Feel free to reach out to me offsite (Aav#7546 on Discord) in the future if you've made a PR and it's not been noticed, I check Discord daily.

[United Calanworie]

This is a release notice for version 2.0.3. It fixes the bad behavior raised in #26, and it re-adds custom length command-line flags.(Thank you Esfalsa for #25 making these changes.) It can be found here, with attached binaries for Windows, MacOS (Intel, although Rosetta 2 allows emulation on M1/M2), and Linux (by way of an Ubuntu build). The source code is also attached, and you are always able to run from source with Python 3.8+.

Please report any bugs you may find as GitHub issues, although it is always acceptable to post here as well with the bugs should you not have a GitHub account, or not wish to tie your GitHub account to your NS contributions.

[Spyglass Development Team]

Version 3.0.0 is out. It comes with the following changes:

Changes:
- Command-line flag for not downloading latest daily dump
- Increased speed by switching parsing to lxml from xml.etree.ElementTree
- Switched to argparse instead of sys.argv for command-line flag parsing
- Cleaned up data acquisition so that we don't use eighty billion lists now and new columns can be added with ease

- [x] Code committee approved
- [x] Documentation updated
- readme.md
- dev_environment.md
- [x] build.sh attached
- [x] GitHub Workflow updated to build binaries for new versions

Want this fresh spyglass goodness? Get it here.

[Strike]

This is a nice tool Is there a similar tool/way this can operate that also returns the name of the regions Delegate nation and ROs?

[Ever-Wandering Souls]

This is a nice tool Is there a similar tool/way this can operate that also returns the name of the regions Delegate nation and ROs?

That's not really something sought within the scope of this tool, but that information is within the region dumps. Past dumps can be found in the Archive, and the most recent one can be found on the API Page.

Downloading either of those will give you either a "[date]-regions-xml-gz" (archive) or "regions.xml.gz" (today's), aka an XML tree of the region dumps compressed in the .gz format (if you're unfamiliar, think like a fancy .zip). Common tools like 7zip and PeaZip can handle unpacking a .gz - and it looks like Windows may now even have an inline option, since it's asking me if I want to use "express zip" to do it and I've never installed that, haha.

Once extracted, this is, well, an XML tree. Basic things like Notepad++ can handle viewing that, and you'll see it has delegate and officer fields like such (Equilism as an example):

Code: Select all

<DELEGATE>isaris</DELEGATE>
<DELEGATEVOTES>11</DELEGATEVOTES>
<DELEGATEAUTH>W</DELEGATEAUTH>

<FOUNDER>equility</FOUNDER>
<FOUNDERAUTH>XABCEP</FOUNDERAUTH>

<OFFICERS><OFFICER><NATION>astarial</NATION>
<OFFICE>Cure Squad Security</OFFICE>
<AUTHORITY>BC</AUTHORITY>
<TIME>1509395451</TIME>
<BY>equility</BY>
<ORDER>7</ORDER>
</OFFICER>

<OFFICER><NATION>chaucerin</NATION>
<OFFICE>Co-Founder/Emeritus Rex</OFFICE>
<AUTHORITY>ABCEP</AUTHORITY>
<TIME>1637091203</TIME>
<BY>equility</BY>
<ORDER>7</ORDER>

...that conveys nation name, office name, permissions (abbreviated into a letter) when they were appointed, and by whom.

Code: Select all

X: Executive
W: World Assembly
A: Appearance
B: Border Control
C: Communications
E: Embassies
P: Polls

Excel *can* open XML trees, but the size of this one will likely choke your computer for a minute or 5 even if it's beefy, the result is ugly as sin without a lot of cleanup, and they're ultimately just easier to open and read in something like notepad++ if you're not trimming it down/processing it first.

Also, if you just want the info for one region, you can use API shards:
Delegate: https://www.nationstates.net/cgi-bin/ap ... q=delegate
Delegateauth: https://www.nationstates.net/cgi-bin/ap ... legateauth
Officers: https://www.nationstates.net/cgi-bin/ap ... q=officers

---

Spyglass exists to automate downloading the region dump, trimming out information not needed for R&D, and converting the rest/the info we find relevant into a spreadsheet, while adding estimated minor update times (since the dumps only contain lastupdate which at the time Dumps are created is always Major), and doing some interpretation of the results (i.e. the Main tool flags raidable regions, while the Defender fork highlights regions in need of being de-tagged). Thus it really only cares about things like the WFE, embassies, and delegate permissions, not the delegate's name or full slate of current RO's.

[United Calanworie]

Thinking about this somewhat though, it could potentially be useful to highlight ROs/lack of ROs and their powers in Spyglass.

[Roavin]

while adding estimated minor update times (since the dumps only contain lastupdate which at the time Dumps are created is always Major)

Not anymore! (as of a few minutes ago)

[Spyglass Development Team]

Attn Spyglass Users:

This is a bit of a time of change for NS! We’ve been given relevant new data as recently as today (all hail both minor and major times in dumps!), the preferred format for API rate limiting was updated just today, and Frontiers is fast on the horizon. One thing Frontiers will do is unavoidably break even the basic functions Spyglass, by way of both changing the content of the data dumps and by changing what is defined as a “raidable region” …and fixing that break will require altering the output sheet in a way liable to break derivative tools such as Quickdraw.

Seeing as this is unavoidable, the Spyglass team is taking this as an opportunity to explore other breaking changes as well. For example, whether to offer both “true” and “interpolated” update times (where interpolated can “smooth” random update hangs and provide slightly more reliable predictive data), or whether to add optional columns containing RO information (like how Embassies are optional at present).

In order to begin experimenting with those changes, Spyglass has today release “compatibility version” 3.0.1, which adds minor bug fixes on top of the recently refactored version 3.0. This version (3.0.1) or older are the last version guaranteed to function exactly as Spyglass has up until now, and therefore to work with derivative tools. This version and all prior version are also expected to break entirely when Frontiers releases.

We will then proceed to create experimental test versions that begin to explore potential changes as well as prepare for Frontiers. These test versions will not undergo the complete version of our usual thorough review process , and may contain features that are actively in development and change day-to-day. Use at your own risk, but feel free to try them and give feedback!

Similarly, now is the best chance you’ll have in years to request or submit any output-changing feature adds – though the odds of them being adopted are certainly far more likely if a) they’re helpful to raiders too, and b) you make a pull request yourself.

Shortly after Frontiers releases, we will then fully review and merge in a “new stable version,” concluding this phase of breaking changes, and triggering just one big necessity to update derivative tools.

---

In short: If you want to keep using the same “stable” Spyglass, use only version 3.0.1 or older until Frontiers breaks it. At that time, we will have to update Spyglass in a way that is likely to break derivative tools. If you want to live vicariously, feel free to try experimental versions released in the coming weeks, which eventually will become a new stable version after Frontiers releases, and which may roll in addition changes.

[Merni]

Thinking about this somewhat though, it could potentially be useful to highlight ROs/lack of ROs and their powers in Spyglass.

I certainly agree.

[United Calanworie]

Thinking about this somewhat though, it could potentially be useful to highlight ROs/lack of ROs and their powers in Spyglass.

I certainly agree.

>_>

[United Calanworie]

So um. About that "final" release thing.

I should have really knocked on wood before I said it, because now we have v3.0.2.

The tl;dr of what happened is I'm bad at math. It was literally removing half a line to fix it. Go check it out, update from 3.0.1, yada yada yada. It's not big news, but it should make your times slightly more accurate, especially when talking about big regions.

[Democratic Poopland]

So um. About that "final" release thing.

I should have really knocked on wood before I said it, because now we have v3.0.2.

The tl;dr of what happened is I'm bad at math. It was literally removing half a line to fix it. Go check it out, update from 3.0.1, yada yada yada. It's not big news, but it should make your times slightly more accurate, especially when talking about big regions.

Will there be any updates in the future?

[United Calanworie]

So um. About that "final" release thing.

I should have really knocked on wood before I said it, because now we have v3.0.2.

The tl;dr of what happened is I'm bad at math. It was literally removing half a line to fix it. Go check it out, update from 3.0.1, yada yada yada. It's not big news, but it should make your times slightly more accurate, especially when talking about big regions.

Will there be any updates in the future?

If you had considered checking the github, you could have discovered for yourself that there has, in fact, been a 3.0.3 release.

You could have also read the announcement three posts up where it was made explicitly clear that we are, in fact, working on another release.

[Democratic Poopland]

Will there be any updates in the future?

If you had considered checking the github, you could have discovered for yourself that there has, in fact, been a 3.0.3 release.

You could have also read the announcement three posts up where it was made explicitly clear that we are, in fact, working on another release.

Ah, didn't see. I should of checked. Sorry.