We need to talk about Discourse

(eyelight) #1

Can I start with a practical demonstration?

Navigate over to a nice long thread, say the Weekend Challenge, currently its 19 posts and can be found here. Browse it a little, go to the bottom, then go back to the top. Now turn off your internet connection, if on a phone that means turn off wifi and then turn off mobile data, then try scrolling down to the bottom of the page…

Don’t worry I’ll wait…

Ah, you can’t get to the bottom can you, because the oh so clever over engineered website dynamically attempts to load the content each time you scroll past it, so without an active network connection you get stuck.

On a simple text / chat site this may not be much of an issue (*), but when the page contains ten or twenty 2MB images you are dynamically loading 20MB every time you scroll, even though you have already downloaded the images already, maybe even numerous times, so if you’re scrolling up and down gleefully in awe at the beautiful artwork then this page can quickly turn into a monstrous 200MB or more web page. So if you are without wifi on a phone then you are munching through some serious mobile data, never mind the bandwidth that blenderartists.org is wasting / abusing!
My next gripe is an age old problem, one many of you may never have even encountered, go ahead and turn off javascript (maybe refresh too, as if you were browsing without scripts in the first place), ooooh, wheres half the content gone, no youtube links, lots of details missing, no ads, etc… By default this is how I browse the internet, its a lot safer, you’re not tracked by third parties and far less annoying!
Finally for now, its potentially troll city. With dynamic content being loaded on the fly, I can potentially change my post ad infinitum and the changes will be almost animated seamlessly on your page. I could even pass secret messages if I knew my counterpart was actively on the page.
None of this is really tested yet just theoretical, so the first main point may actually be based on page rather than post but the issue is still the same because you don’t know where the page boundaries are. The second JS point is obviously true and the third is an almost given, but you could argue that it is a feature rather than an issue.
Right, back to bed to continue my nightmare, sweet dreams!

Challenge #787 (20/07/18) Entries CLOSED
(eyelight) #2

Oh, I did intend to see what page saving was like, maybe tomorrow in the safety of daylight

(Martynas Žiemys) #3

Oh come on… It’s 2018. If Blenderartist uses 200 MB of my I-don’t-even-know-how-many-GB mobile data plan, I am fine with that. However I cannot hit ‘reply’ when writing a comment on the phone. That’s a bigger issue. It works if I load desktop version of the site.

This button area selects the text being written when touched. There is no way to leave the reply from that screen.

(Simon Storl-Schulke) #4

Thats just how it works in Web 2.0. Javascript is essential for almost everything. Most of the advanced features here wouldn’t even work without JS. Either accept that and trust BA not to send your Info to Wladimir Putin - or live without the features it offers.

Pretty sure most browsers load that into the cash and don’t have to reload it every time.

(vilvei) #5

Most browsers have so eager caching policies, that someyimes it gets hard to update some resources. But then again, I would be surpriced if this kind of platform doesn’t use modern techniques to have proper offline experience, with eg. pwa: https://developers.google.com/web/progressive-web-apps/

(Photox) #6

It’s not practical to scroll through long threads on my phone. It’s painful. And all to load an ad. Definitely a step backwards there.

(Martynas Žiemys) #7

Adaway works fine for blocking ads on android. I find it quite time saving as I can walk my dog in the morning and look through the forums. The old one with pages was harder to use on the phone for me personally. I mean if we don’t count the inability to post replies :smiley:

(Photox) #8

I’ll check out adaway, sounds good.

Um, yeah, not being able to reply, that’s straight up broken.

(Martynas Žiemys) #9

It works with system’s hosts file or something like that, but the device needs to be rooted. Very nice app. No more ads forever. :slight_smile: I hope somebody fixes the reply button. Is it just me? Can somebody confirm it does not work for everyone?

(Photox) #10

I am replying now on a pixel, Android 7.1.2

The reply button is an icon, a curved arrow.

bottom right, looks like the one in the desktop, but does not have the text ‘Reply’

(BeerBaron) #11

Not on my end, all the data is cached by my browser, as expected. Maybe your mobile browser starts evicting stuff from the cache earlier.

Either way, 200MB of images in one thread isn’t necessarily what this forum software was designed for.

Loading a lot of HTML content (not just images) actually has a ton of overhead, it’s not so much about the bytes downloaded, but about what has to be kept in RAM and layout. At some point, you really have to load stuff in and out dynamically and that doesn’t work without Javascript.

If HTML/Browser technology wasn’t so terrible, we would be able to load large documents. Alas, we can’t and this is about as good as it gets.

(enenra) #12

This is the so called “progress”. No matter how powerful the hardware becomes and no matter how fast the connection becomes some developers will find a way to use it all up for silly animations and unnecessary gimmicks.
The sluggishness of modern websites just beggars belief. You can have hardware capable of rendering 100fps in modern games but some websites will bring it to its knees.

(Daedalus_MDW) #13

this happens to me too sometimes on android. close the tab and open it again, you should be able to post the saved reply. since switching to edge i havent had this happen, but then again i dont post much.

i use edge mobile browser since they got adblock plus built-in. way faster then mobile firefox with ublock origin.

(BeerBaron) #14

No, that’s not the problem. HTML is just slow, layouting is slow, markup has ridiculously high memory overhead. Even a performance-oriented web developer can not get rid of that fundamental overhead. It doesn’t show up on small pages, but it definitely becomes a problem with large pages.

That’s what you get when you have a consortium pile up features into “a standard” at the highest level with no consideration for performance and scalability. It can’t be made fast anymore without breaking everything.

(eyelight) #15

Yes the overhead can be terrible, especially when developers infinitely wrap elements in elements and then dynamically alter the DOM on the fly recalculating all those relative elements, pretending they are using script to manage a cache when in reality they are thrashing the garbage collector and fragmenting the memory structures of a heavy DOM.

Javascript has some beautifully simple features, but the simplicity of use has large overheads that many don’t even realise happens never mind use efficiently. I may have a test of how much unused boilerplate script libraries are loaded into the script engine this weekend, but then again I may just debug my own Irrlicht GUI elements. A quick look now indicates 400 lines before the HTML starts and that includes 8 script links and numerous CSS links.

Sometimes its good to just clear the page and start afresh with a simple functional page with architectonic form, who knows?

(Stefan Werner) #16

In practice, it’s almost always scripting that’s the slowdown and not HTML or CSS. Do the experiment and turn off JavaScript in your browser for a day. You’ll be surprised by how fast some web pages suddenly become. Or, sadly, how many web pages fail to deliver basic static content without JavaScript.

When the layout engine has to wait for replies from 15 different remote servers before it can render the page, the one with the highest latency becomes the bottleneck for the entire site.

(eyelight) #17

Arrrrgh, I had navigated over to the Weekend Challenge to review the entries, saw that skw had replied, so switched tabs to the other one I’d opened to view the scripts and my last post was still there in the reply box. How could I get rid of it, even clicking the standard new reply button just set focus on the existing (already posted) response textarea, yeah, closed the tab / page and started again!

This thing is like the EU, an over bureaucratic set of expensive luvies. Sorry to steal the phrase but, “looks good, doesn’t work!”

(Daedalus_MDW) #18

theres a trash icon or cancel or something.

(BeerBaron) #19

… which Discourse isn’t doing.

That’s not a very fair comparison. Javascript isn’t slow, it’s manipulating the HTML dynamically (and the resulting layout recalculations) that has the overhead. Of course if you turn off all dynamic manipulation, then it’s “fast” again, but not because of Javascript. In a day and age when this technology is created for complex user interfaces and real applications, that’s a real problem. Those need to be able to have dynamic HTML.

Anyway, my point is that even without Javascript, a “huge” document, which a long forum thread can easily produce, would be too heavy even without any Javascript. The only alternative there is to split up the page and make the users click through them manually.

That’s not to say that many websites couldn’t be better, it’s to say that there isn’t much room for improvement for Discourse itself.

(eyelight) #20


So if you visit the site without JS, virtually everything is hidden by default and then turned on using JS.

As for the element wraps, yes, every element is wrapped and each inner item too, a very common practice in this modern age of slower webpages on faster processors. A lot of this happens dynamically after the page is loaded using JS, you can see it by using an inspector.

JS is relatively slow, its a script running through a scripting engine. Then it manipulates the DOM mainly using (long) string names. In comparison a fast script engine exposes pointers to data you search for once (i.e. Lua in game engines). A script engine requires all the script to be parsed each time to set the state machine up, hence why mentioning all the includes. Yes, its the current trendy method, but its still removing the control from the user and pushing the calls to the hard work into innocent looking anonymous scripts.

Your argument seems to me is, “we decided and implemented change, now you are stuck with it and we will prevaricate until you capitulate”, welcome to Mordor


BTW, the real applications that use JS, don’t have at least a meg of JS includes, they use pointers, not string ordered DOMs