Jolt Archive

[Maltropia]

I know a lot of people who're going to love this.

[Reploid Productions]

A Christmas miracle? I thought the stuff Jolt gave [violet] and Max was basically unusable entirely!

[[violet]]

A Christmas miracle? I thought the stuff Jolt gave [violet] and Max was basically unusable entirely!

There was a bit of a tussle over it; initially they refused to release it. But we had a signed agreement saying that was our data, and kept pestering, and in the dying days they sent us a 6GB dump. I've always wanted to put up a read-only archive somewhere but never managed to do it.

[[violet]]

OK, I now have a MySQL dump (1MB compressed) that contains the full table structure and up to 200 records from each table. I've sanitized email addresses and limited or removed data where it seems to be private, e.g. PMs.

Please email me at admin@nationstates.net if you would like a copy of this sample.

[Vakolic]

OK, I now have a MySQL dump (1MB compressed) that contains the full table structure and up to 200 records from each table. I've sanitized email addresses and limited or removed data where it seems to be private, e.g. PMs.

Please email me at admin@nationstates.net if you would like a copy of this sample.

done

[Shadow Afforess]

OK, I now have a MySQL dump (1MB compressed) that contains the full table structure and up to 200 records from each table. I've sanitized email addresses and limited or removed data where it seems to be private, e.g. PMs.

Please email me at admin@nationstates.net if you would like a copy of this sample.

Progress So Far:

https://dl.dropboxusercontent.com/u/498 ... rchive.png

https://dl.dropboxusercontent.com/u/498 ... 0forum.png

The index page, and each of the forum thread list pages each generate right now. I need to break the thread list into pages...and also generate the individual thread contents.

The program is a command line tool that connects to a mysql database and generates static html files from the database (following the schema violet shared). It uses bootstrap 3 as the base of the look & feel, and the templates for each type of page can be customized. I'll release it to github shortly.

Edit:

Github repository: https://github.com/Afforess/jolt-archiver

[Lamoni]

I would love being able to look over, and c/p my old Jolt stuff.

[Shadow Afforess]

Alright, after some work, I have a basic static html generator for the jolt forums. It's not fancy, but it creates and index of the forums, the threads in each forum, and the contents of each thread... replicating a basic forum layout. It does not allow any searching or ability to view poster/user information. I might add that later, assuming [violet] approves of the work so far.

It's a java command line tool (so you need at least the JRE installed). You can compile it from the source (using Java & Maven) if you want to verify that it has no nasty surprises, or can use a precompiled jar provided by me.

It requires 5 command line arguments - the hostname of the database, port, database schema name, database user, and database password. Also there is a sixth optional flag, -verbose if you want it to print out text as it generates the site. It will take several minutes to generate.

Example command line usage:

Code: Select all

java -jar joltarchiver-1.0.0-SNAPSHOT.jar localhost 3306 jolt afforess mypassword

It will create a folder named "templates" in the same directory as the jar. It generates the bootstrap 3 files and forum layout. It also includes the css and 3 template files (extension ends with .template) which the homepage, forum page, and thread views are produced from. You can edit these 3 files and re-run the program to modify the look and feel of the static html produced, beyond simple css edits. Grid.css contains any modifications made to the base bootstrap 3 css. It also uses 1 jquery plugin, to produce the pagination. Everything is self-contained, no resources are fetched from the web, so you can safely host and drop the files onto a simple webhost and serve the static files from there.

I'm sure there are bugs & bbcode parsing bugs I couldn't spot with the small 1mb sample you gave me. Let me know how it goes and I will address issues as they come up!

[[violet]]

Alright, after some work, I have a basic static html generator for the jolt forums. It's not fancy, but it creates and index of the forums, the threads in each forum, and the contents of each thread... replicating a basic forum layout.

Looks very nice! However I can't find the "contents of each thread" part... each thread page seems to be empty except for the thread title. So I can see the same output from your screenshots earlier, but no posts.

[Shadow Afforess]

Alright, after some work, I have a basic static html generator for the jolt forums. It's not fancy, but it creates and index of the forums, the threads in each forum, and the contents of each thread... replicating a basic forum layout.

Looks very nice! However I can't find the "contents of each thread" part... each thread page seems to be empty except for the thread title. So I can see the same output from your screenshots earlier, but no posts.

Hm.. If you are using it on the tiny sample you gave me, most of the threads listed have no posts (at least, not in the sample) and most of the posts don't line up with any of the threads included in the threads table. There were 3 threads that had 1 post though, which showed it did work. "rape-isn't-about-sex-or-is-it" amusingly is the longest thread that does have content.

Are you running it on the sample or the entire database? If it's the entire database, then clearly there is a problem. If it's the sample, then it's working fine.

[[violet]]

Ah, right, right. Just running the sample at the moment.

[Hyperion]

Amazing tech stuff. I'd love to see the work of my ancestors.

[[violet]]

Thanks to Afforess's converter, I've got a sample up!

NationStates Forum Archive

For now, it's just the Technical and NationStates forums. The converter takes a long time to run, so this is a useful opportunity to spot parsing errors and provide feedback!

[Northrop-Grumman]

Thanks to Afforess's converter, I've got a sample up!

NationStates Forum Archive

For now, it's just the Technical and NationStates forums. The converter takes a long time to run, so this is a useful opportunity to spot parsing errors and provide feedback!

Whoa, nostalgia rush! There goes my evening!

(It looks awesome!)

[The Orson Empire]

Thanks to Afforess's converter, I've got a sample up!

NationStates Forum Archive

For now, it's just the Technical and NationStates forums. The converter takes a long time to run, so this is a useful opportunity to spot parsing errors and provide feedback!

So...that's what Nationstates looked like back on the Jolt forums.

[[violet]]

So...that's what Nationstates looked like back on the Jolt forums.

No, it looked like this. Or actually it looked different depending on which version it was. The archive linked above has the content but not the same look.

Edit: This is the oldest version I can find.

[Zwangzug]

[violet] and Afforess are made of awesome, win, etc. I am in awe.

On a slightly nitpicky note, trawling back through the NationStates threads (around page 129), the dawn of history appears to be Sat, 26 Aug 2006 07:45 PM ? I'm not sure how they're sorted beyond that, I'm seeing rather few sports threads so I suspect they may be out of order before this.

Edit: time resumes on page 148.

[The Orson Empire]

So...that's what Nationstates looked like back on the Jolt forums.

No, it looked like this. Or actually it looked different depending on which version it was. The archive linked above has the content but not the same look.

Edit: This is the oldest version I can find.

I see.

It has a much more...rugged look then today's forums.

[Emporer Pudu]

I can't wait to see more of this.

[Astrolinium]

Is there any chance that perhaps a search feature might be added in the future?

[[violet]]

Is there any chance that perhaps a search feature might be added in the future?

Afforess says he may add one, but given a little time to index, Google will do a good job, too.

[Astrolinium]

Is there any chance that perhaps a search feature might be added in the future?

Afforess says he may add one, but given a little time to index, Google will do a good job, too.

Excellent.

2006 is a bit fuzzy, especially because I was 11 and really shouldn't even have been on the forums in the first place.

[Herrebrugh]

I haven't encountered myself yet, I'm afraid

But it's good fun seeing people ask questions about the UN just before the first of April 2008

[[violet]]

trawling back through the NationStates threads (around page 129), the dawn of history appears to be Sat, 26 Aug 2006 07:45 PM ? I'm not sure how they're sorted beyond that, I'm seeing rather few sports threads so I suspect they may be out of order before this.

Edit: time resumes on page 148.

Mmm, the frozen-in-time threads are all blank, too.

Edit: Looks like a problem with the database rather than the converter. They exist in the thread table (with that time) but there are no corresponding posts in the post table.

Edit 2: I think they may be the shadows of moved threads.

Edit 3: Yes, well, seems weird, but when a thread is moved, it looks like VBulletin creates a new row in table 'thread' with a new 'threadid' and the same 'title' but with a 'pollid' value of the old 'threadid'. But the converter can't just assume rows with 'pollid' are moved threads, because sometimes 'pollid' is used to store the ID of actual polls. Presumably there is some other flag I haven't noticed yet.

[United States of Natan]

wow! this is incredible! thanks, afforess and violet, for allowing us to see past forum posts.