NSCode | [list] and dispatch content leakage

[Aeioux]

Under certain circumstances, that require poor formatting of the target dispatch and of the post it is referenced by, contents of a dispatch linked via plain text url (without the [url] tag) "leak" outside the dispatch preview box, consequently breaking its footer. This behavior can be seen in dispatches, RMB posts and telegrams.

Steps to reproduce:
1. Create a dispatch with the following contents:

Code: Select all

[box]
    [list]
    [/box]
[/list]
Lorem ipsum


[box] tag can be replaced with: [floatleft], [floatright], [sidebar], [align], [box], [size] may be some other tags. This tag must be closed before the [/list]. Content placed after the [/list] will leak outside the preview container. Example dispatch: https://www.nationstates.net/page=dispatch/id=1403142

2. Link this dispatch with the following code:

Code: Select all

[list]https://www.nationstates.net/page=dispatch/id=1403142


3. Preview/post the telegram/dispatch/post and observe the "Lorem ipsum" string outside the dispatch preview container.


What happened here:
1. [list] tag is always replaced by NS BBCode processor with <ul>/<ol>, regardless of the existence of the closing tag [/list]. Same applies to the closing tag [/list].
2. [box] and other such tags are a little trickier - while they are checked for their closing counterpart tag, that appears to be done via regex or independent stacks, which allows for the classic invalid bracket sequence situation to occur, such as ([)] in this case, only with tags instead of brackets.
3. This results in the following server-rendered invalid html:

Code: Select all

<ul>
    <div class="dispatchsample">
        <div class="dispatch" data-main-url="page=dispatch/id=1403142">
            <div class="dispatchsamplemain">
                <div class="dispatchheadline">
                    <h2><a href="page=dispatch/id=1403142" class="quietlink">DEBUG_DISPATCH_TARGET</a></h2>
                    <p class="dispatchauthorline">by
                        <a href="nation=aeioux" class="nlink">
                            <img src="/images/flags/uploads/aeioux__695322t1.png" class="smallflag" alt="">
                            <span class="nnameblock">
                                <span class="ntype">The Corporate States of</span>
                                <span class="nname">Aeioux</span>
                            </span>
                        </a>
                    </p>
                </div>
                <div class="nscodebox">
                    <ul>
                    </div>
                </ul>
                    Lorem Ipsum
                <br>
            </div>
        </div>
    </div>


4. The web browser (Chrome, Firefox) attempts to fix it as follows, effectively moving the offending </ul> tag and the rest of the content after the dispatch preview container:

Code: Select all

<ul>
    <div class="dispatchsample">
        <div class="dispatch" data-main-url="page=dispatch/id=1403142">
            <div class="dispatchsamplemain">
                <div class="dispatchheadline">
                    <h2><a href="page=dispatch/id=1403142" class="quietlink">DEBUG_DISPATCH_TARGET</a></h2>
                    <p class="dispatchauthorline">by
                        <a href="nation=aeioux" class="nlink">
                            <img src="/images/flags/uploads/aeioux__695322t1.png" class="smallflag" alt="">
                            <span class="nnameblock">
                                <span class="ntype">The Corporate States of</span>
                                <span class="nname">Aeioux</span>
                            </span>
                        </a>
                    </p>
                </div>
                <div class="nscodebox">
                    <ul>
                    </ul>
                </div>
            </div>
        </div>
    </div>
</ul>
Lorem Ipsum
<br>

Expected behavior:
1. Improperly matched tags should not be converted to html
2. [list] tag should be properly matched with [/list] tag
3. Dispatch content should stay within the preview box
4. Server-rendered html should not be invalid

And it also would be nice to not have the full dispatch content posted in its preview box, especially since the majority of its content is not visible. Limiting the preview to first few hundred characters should be okay, but it would require proper handling of the BBCode tag stack.

[SherpDaWerp]

Steps to reproduce:
1. Create a dispatch with the following contents:

Code: Select all

[box]
    [list]
    [/box]
[/list]
Lorem ipsum


This sounds like a case of "garbage in, garbage out".

If you write misnested tags, you can't be surprised when the output is also misnested. It's not like this is unknown, either: [box] tags break stuff so often there's even a moderation megathread for fixing broken posts.

[Aeioux]

Steps to reproduce:
1. Create a dispatch with the following contents:

Code: Select all

[box]
    [list]
    [/box]
[/list]
Lorem ipsum


This sounds like a case of "garbage in, garbage out".

If you write misnested tags, you can't be surprised when the output is also misnested. It's not like this is unknown, either: [box] tags break stuff so often there's even a moderation megathread for fixing broken posts.

While your description indeed matches the situation, i don't think it should be treated as normal. Since a lot of people have problems understanding formatting tags, it is best to reduce cases of unexpected behavior, otherwise these, well, bugs could be treated by users as features - for example, that [list] doesn't appear to require a closing tag.

The code from the first step was encountered in a live dispatch that has a much bigger list and an [align] instead of [box], and since it appears to work just fine, no one even realizes there's a mistake. I'm not asking for a syntax highlighter in these textareas (though, implementable), but in this example, if either the [align] or the [list] (depending on the parser strategy) were kept as text in this case, it would clearly show that that tag is misplaced. So my suggestion is to make BBCode processing more strict in terms of syntax, so that NS is not filled with so many markup-invalid pages (which by the way affects search engines indexing rating).

I'm not familiar with these forums and that thread specifically, but it appears like a lot of effort is...directed on fixing Y problems, instead of fixing an X which causes the Y to happen.

[SherpDaWerp]

This sounds like a case of "garbage in, garbage out".

If you write misnested tags, you can't be surprised when the output is also misnested. It's not like this is unknown, either: [box] tags break stuff so often there's even a moderation megathread for fixing broken posts.

While your description indeed matches the situation, i don't think it should be treated as normal. Since a lot of people have problems understanding formatting tags, it is best to reduce cases of unexpected behavior, otherwise these, well, bugs could be treated by users as features - for example, that [list] doesn't appear to require a closing tag.

The code from the first step was encountered in a live dispatch that has a much bigger list and an [align] instead of [box], and since it appears to work just fine, no one even realizes there's a mistake. I'm not asking for a syntax highlighter in these textareas (though, implementable), but in this example, if either the [align] or the [list] (depending on the parser strategy) were kept as text in this case, it would clearly show that that tag is misplaced. So my suggestion is to make BBCode processing more strict in terms of syntax, so that NS is not filled with so many markup-invalid pages (which by the way affects search engines indexing rating).

I'm not familiar with these forums and that thread specifically, but it appears like a lot of effort is...directed on fixing Y problems, instead of fixing an X which causes the Y to happen.

As long as it doesn't break anything major (and 90% of the time, it doesn't - as you said, these bugs are often undetectable) I don't see a problem with treating them as "features", bad practice as it may be.

Enforcing stricter standards on BBCode would just result in people typing invalid syntax and getting frustrated when it doesn't post. If an additional check is added and the post/dispatch/whatever fails to send, citing "your BBCode is invalid", what will happen is someone will just go to Technical and complain "why won't my post send?!". A syntax highlighter might work, although I have no idea what would be involved in implementing that; I'm thinking some form of auto-tag-closing wherein you type

Code: Select all

[box]

and a greyed-out

Code: Select all

[/box]

appears at the end of your text. Typing additional tags would place their greyed-out close tags in the proper order, and pressing "tab" would fill in the first closing tab as actual typed text. Having said that, if someone relies wholly on clicking the buttons above posts forumside, then their tags are guaranteed to be properly formatted, but such a system doesn't exist gameside so there's still problems.

NS is still the first option that comes up when you search "nationstates", so I doubt search indexing is that much of a problem.

A lot of effort is directed towards fixing the symptoms rather than the cause, yes, but invalid BBCode will always exist short of your "block it" solution, and there's a lot more Moderation man-hours to fix posts than Admin man-hours to write code.