Dispatch searching

[Tinhampton]

There are ~1.5m dispatches. Why will allowing people to search all of them cause the site to lag?

[Frisbeeteria]

There are ~1.5m dispatches. Why will allowing people to search all of them cause the site to lag?

I don't know if any proposed search will happen in the database where I think dispatches are stored, or if it's in the flat files are where nations are stored. Searching the flat files slows down the site. It just does. I don't know why, but the 2002 architecture this site is built on has limitations that more modern sites don't have to deal with. We've had several mod tools rejected for this reason.

I could of course be completely wrong, but I'm just setting expectations based on what I believe [v] has said in the past.

[United Calanworie]

There are ~1.5m dispatches. Why will allowing people to search all of them cause the site to lag?

Because flat file searches are slow. Fortunately, from what I can tell, dispatches are in a database -- a SELECT query is substantially faster than loading and reading the flat files.

[Panagouge]

And if dispatch searching via bot/script/api/whatever is a concern, making that illegal is easy enough.

[Ioudaia]

A page=search is ideal imo because then you can bookmark or share topics you were looking for as needed.

Oooh! That's a great idea!

[United Calanworie]

And if dispatch searching via bot/script/api/whatever is a concern, making that illegal is easy enough.

Fortunately, it doesn't even have to be made illegal -- we just make it technically infeasible to do so.

[The Orwell Society]

I love this idea

[Ever-Wandering Souls]

would we prefer to not even bother with that and instead always search *everything*

Maybe dispatches are stored differently, but there are around 8 million dead nations. Searching all of them (as opposed to the 300K active nations) would almost certainly slow the game every time someone searched. Not only should it not be the default, it shouldn't even be an option.

...would it have to search each nation file? Are dispatches not something that's been database'd? The forums have tens of millions of posts (well, at least millions, F7 purges will have trimmed the total number somewhat), and searches against that database run mostly just fine.

Edit - ah, I missed page 2 of the thread, where Aav clarifies that they should be databased. Thank the Gods, lol. I'd favor "everything" 100% then.

[Adastra]

How would you envision such an interface?

I think the following advanced search functionality (among others) would be nice:
- the ability to search only Dispatch Titles, only Dispatch Content, or in Both
- the ability to narrow down to Dispatches to those made by members of a specific region
- the ability to narrow down Dispatches to those made by nations already marked on your Dossier
- the ability to narrow down Dispatches between, prior, or after certain dates since last update (or since publishing if possible)
- the ability to distinguish between factbook and non-factbook Dispatches (likely already made possible by a 'search by category' function)

Where should this interface be accessible from? page=dispatches? Do we want a page=search that comes with "advanced" search options while page=dispatches just has a little keyword-search box?

I think the search being from page=dispatches is the best location.
I also think page=dispatches/nation=NATION should offer the same search functionality, but narrowed down to the dispatches of that nation.
Is there such thing as a page=dispatches/region=REGION? for dispatches from that region's members?

With search functionality, I wonder if it's possible to add to one's dossier new dispatches based on certain keywords, but I can't think of a smooth implementation (without implementing a keywords/tags system)

[Ever-Wandering Souls]

There *are* categories already, like gameplay->reference, which aren't keywords per se, but do help narrow things down a bit.

If making full dispatch body text searchable isn't an option, then a middle ground might be a new "short description" sort of field, seperate from the title, and making that indexed. Give people a tweet's worth of chars to list tags or keywords, going forwards, so they can have a snazzy title but still provide useful info for a search.