Ongoing Database Problems: Discussion/Questions

[United Calanworie]

Got questions about the ongoing database problems? Ask them here.

You will likely find it useful to read the Admin thread.

[United Calanworie]

We're aware, admin hath been flagged.

[Cupic]

I've made a few changes to the NS API. By which I mean this:

http://www.nationstates.net/pages/api.html

This has previously been known as regiondata.cgi and nationdata.cgi, both of which continue to work at their old addresses. They essentially forward to this one now. which have been retired as of August 2013.

The API exists so that people can build NS-based sites, such as NSDossier and NSEconomy. We want to support such efforts.

Please visit the API documentation page for information. The major change is that very high rates of queries (more than 100/minute) will result in the IP address being denied service for 15 minutes. This is to ensure regular players don't suffer slowdowns because of overzealous scripts.

In general, scripts should not attempt to look up every resident of a region all at once. This can result in dozens, hundreds, or thousands of near-simultaneous requests to the server, which are difficult or impossible to service. If you need information on large numbers of nations, you should trawl them from the API as a speed-limited background job and store the data locally on your server.

Please let me know if you have any questions or concerns!

Background: This is the result of events documented in the "Scripts banned" thread.

Hello! My screen shows a database error, saying "Database connection error at Sun Apr 14 14:55:38 2024" . What do I do?

[Ineva]

I've made a few changes to the NS API. By which I mean this:

http://www.nationstates.net/pages/api.html

This has previously been known as regiondata.cgi and nationdata.cgi, both of which continue to work at their old addresses. They essentially forward to this one now. which have been retired as of August 2013.

The API exists so that people can build NS-based sites, such as NSDossier and NSEconomy. We want to support such efforts.

Please visit the API documentation page for information. The major change is that very high rates of queries (more than 100/minute) will result in the IP address being denied service for 15 minutes. This is to ensure regular players don't suffer slowdowns because of overzealous scripts.

In general, scripts should not attempt to look up every resident of a region all at once. This can result in dozens, hundreds, or thousands of near-simultaneous requests to the server, which are difficult or impossible to service. If you need information on large numbers of nations, you should trawl them from the API as a speed-limited background job and store the data locally on your server.

Please let me know if you have any questions or concerns!

Background: This is the result of events documented in the "Scripts banned" thread.

Hello! My screen shows a database error, saying "Database connection error at Sun Apr 14 14:55:38 2024" . What do I do?

In the words of the late George Harrison, "It's gonna take patience and time."

[Querria]

Oh cool. Thought I had unintentionally done a no-no. Never got a ban, so no idea what it looks like. Hope the [Database error] gets resolved soon.

[Almonaster Nuevo]

What do I do?

Do nothing...
Relax.
Rest in the arms of the dragon.

[Vonum]

How long will this take, approximately? I've got people to message!

[United Calanworie]

How long will this take, approximately? I've got people to message!

It'll take however long it takes. Database crashes aren't anything that we can predict, nor predict how long they take to fix. Admin is actively working on the issue. We're trying some first steps, but if it gets more involved, it could be substantially longer.

[Barbariax]

Panic, the end times have come!!

[Roxium]

Any updates yet?

[United Calanworie]

Good morning everyone. This is a bit of a longer announcement, and it quickly describes where we're at, what kind of data loss we think we'll have, and steps that we're trying to take to mitigate it.

The disk that failed contained several tables on it, but the ones of primary concern to most players are going to be:

• Telegram API keys
• Telegram templates (not the text of the templates themselves, this is an accessory table used to help speed along processes involving templated telegrams being sent)
• The telegram queue (the status of recruitment/campaign telegrams that had recently been sent out)
• Recruitment telegram stats
• And the biggest one: Cards.

I'm going to talk about cards first here as that's the largest community that's going to be affected the most. We've likely lost months worth of data in terms of:

• card ownership
• trades
• bank data
• collections
• bids, asks, auctions

There's a potential that we will be able to recover some of that data. Unfortunately, the disk failed in a rather spectacular manner, and is resistant to our attempts to recover anything from it. We will keep you updated.

As to the telegrams information:
The API keys are gone. You'll have to file a GHR to get a new key once we're back online. Hopefully that's not too much of an issue. We're looking into rebuilding the other tables from the main telegrams table, as it contains all the information that is required to build the other tables. It should be possible to recover relatively quickly. It's likely that mass telegrams will be restored to whatever state they were in up to approximately a week ago for the largest telegrams.

We're continuing our efforts at data recovery, and will let you know when we know more re. data loss. In the meantime, we're also trying to bring the game back up with critically impacted areas (e.g. cards, telegrams) disabled while we do our best to restore the data that we can.

[Barinive]

I thought it was an in-progress update until i saw the technical difficulties screen [Posting this to test if nations on forums still work]
Btw, i know that the forums and main game are different websites, but are nations dependent on the main game or is it a different account?

[New Westmore]

[COMTEST]

anyway, are we also expecting to have lost data as well regarding things like RMB messages and cosmetic stuff like our flags and dispatches?

[United Calanworie]

[COMTEST]

anyway, are we also expecting to have lost data as well regarding things like RMB messages and cosmetic stuff like our flags and dispatches?

As of my post -- no. We have to rebuild the RMB page index, but that doesn't have anything to do with the messages themselves, merely their display.

[The Southern Dependencies]

What's the expected time for recovery of the site other than "a few days", if such an estimation can be given?

[Troc]

how bout cards?

[United Calanworie]

What's the expected time for recovery of the site other than "a few days", if such an estimation can be given?

We don't have one. Hopefully within a few days.

[Valentine Z]

how bout cards?

As per Aav's post, yeah that is affected big time.

[United Calanworie]

how bout cards?

I'm going to talk about cards first here as that's the largest community that's going to be affected the most. We've likely lost months worth of data in terms of:

• card ownership
• trades
• bank data
• collections
• bids, asks, auctions

There's a potential that we will be able to recover some of that data. Unfortunately, the disk failed in a rather spectacular manner, and is resistant to our attempts to recover anything from it. We will keep you updated.

[The Southern Dependencies]

What's the expected time for recovery of the site other than "a few days", if such an estimation can be given?

We don't have one. Hopefully within a few days.

Understandable, thanks for the help anyways! ^-^

[New Westmore]

[COMTEST]

anyway, are we also expecting to have lost data as well regarding things like RMB messages and cosmetic stuff like our flags and dispatches?

As of my post -- no. We have to rebuild the RMB page index, but that doesn't have anything to do with the messages themselves, merely their display.

I see. Another question - will the whole site be revamped cosmetically and technologically as well once the database is fixed, or will it remain untouched in that regard?

[Indo States]

I'm wondering if dispatches are also affected by the database failure?

[Ferranghia]

Unfortunately, the disk failed in a rather spectacular manner,

Part of me hopes for a very flashy sparks shower as the disk blasted open filling the room with smoke and that pleasant smell of burned electronics

[The Southern Dependencies]

Will there be any attempt to modernise what is, quite clearly, in my opinion, an outdated system? This website is, after all, very ancient.

[United Calanworie]

As of my post -- no. We have to rebuild the RMB page index, but that doesn't have anything to do with the messages themselves, merely their display.

I see. Another question - will the whole site be revamped cosmetically and technologically as well once the database is fixed, or will it remain untouched in that regard?

Will there be any attempt to modernise what is, quite clearly, in my opinion, an outdated system? This website is, after all, very ancient.

Yeah uh. Our tech team time has been spent putting out fires and repairing the servers, not creating new shiny things like CSS.