On web standards, frameworks and programming paradigms (Patreon)

If you ignore the rest of this long, deeply technical and utterly subjective blog post, at least watch this video.

https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript

The talk was given in 2014, but is written as if it was given in 2035. It's funny and prescient.

The gist is that JavaScript is one of the worst programming languages in the world; and it's also the most popular, and most important languages, one that paved the way for a future where JavaScript is everywhere and nowhere.

Web standards

Pathfinder Second Edition was first announced at GenCon in August 2018.

When I decided, nearly 6 years ago, to develop the Pathfinder 2e character sheets as HTML-based rather than PDF-based, I knew that I had a lot of work ahead of me. But I naively thought that the days of incompatible browsers were drawing to an end. I'd done my time in the Internet Explorer mines, and looked forward to leaving those behind for an era of compatible modern browsers that developers could rely on.

Well, it turns out, even 6 years later, web standards are still not very standard.

I've been fixing a lot of bugs lately, and a good number of them are due to differences between Chrome and Firefox. Even in one browser there are countless quirks and weird behaviours that can trip you up. And this matters because increasingly, our software world is being built on these new platforms.

The browser is the new operating system.

The age of jQuery, and the age of React

I have a fond rant within me on the history and importance of jQuery, the library that - for a time, at least - papered over the cracks and made the internet work. It would not be an exaggeration to say that jQuery made the modern internet possible.

Though it lasted for decades, the age of jQuery is over now, or at least drawing to a close. It's still used on a great many websites of course, but all the excitement now is in reactive frameworks (React, Vue, etc). There was a brief buzz around static site builders, but they never really hit the big time.

And while I find so many of the details of reactive frameworks annoying, there is something there worth paying attention to: reactive programming is akin to functional programming, applied to the data layer. It establishes a clear directionality of calculation, and so saves developers from the risk of confusing, circular behaviours.

The short version: reactive programming is like a spreadsheet. When you change a value, any calculated values based on it change in response, without you having to manually ask for them to recalculate. Subsequent calculations based on those also change, and so on. The current state of all data is guaranteed to reflect the state of the inputs.

As is so often the case in programming, discipline saves you from yourself. A language, framework or environment with strict rules to stop you making mistakes can be annoying in the short term, but it pays back tenfold in disasters avoided.

On the web, reactive programming is typically paired with component libraries - this is what React is. This is because components are a tidy way of building massive complex web pages out of neat, reusable parts.

A web component typically combines an HTML template with some specific CSS and JavaScript for that component. A component may be as small as a checkbox or as large as the whole page; components can contain other components, all the way down. Global data is discouraged, instead components should communicate locally, passing data up and down the tree until it gets where it's needed. And because it's reactive.

There are more reactive programming frameworks and variations thereof than I could possibly list, never mind actually try. It seems like more are made very day. I've even been tempted to write my own.

Downsides of this approach

The most popular libraries are constantly changing. Reach has an ever-flowing conveyer belt of mods, while Vue has reinvented its component format twice in the last few years. The fact is, while reactive web programming is definitely a step in the right direction, we're still busily exploring what that actually means. None of the options on the market can really claim to be the right answer.

The downside of this approach is that a web page built with these is basically empty until the massive JavaScript library runs that fills it with content. Compared to the simple web pages of the past, reactive libraries are big, slow and heavy.

There's also some tension here between the desire of these reactive web frameworks to put everything into a single page web app, versus the desire of semantic HTML (and SEO) to have separate pages for each purpose. Part of me misses the days of static sites, and wishes static site builders had won.

Users of reactive web frameworks hope that the current churn of new ideas will settle down into a stable, reliable platform. It won't. We're in an intermediate stage of history, when new ideas are tried out in search of the right answer. The current reactive frameworks will keep mutating, will keep bucking under the bronco, because they are only an incomplete step in the right direction.

Unit testing

My new job (as of 2021) involves a lot of testing. It deals with other people's money, so we really like to be very careful.

An important part of this is "unit testing", which is an automated test that checks that a given part of a program, in isolation, does its job. To do its job properly, unit tests needs to be automated. Every time you build or push, it runs the tests. If you fuck up, you find out about it immediately.

But what is a "unit"? In an object-oriented language a unit a class, more or less... but only because that's the easiest unit of code to test.

JavaScript has not traditionally been amenable to unit testing, because it was messily epigenetic on top of the DOM. Component libraries solve this problem, potentially, by breaking the HTML, CSS and JavaScript into units that can be tested in isolation.

The next age

What will the next era of web development be about? My money is on WASM.

(did you watch that video? you really should.)

Web Assembly is a way of writing code in any language, from COBOL to Rust, and compiling it to a common executable format. That format can be run in the browser, on the server, or anywhere else. It runs nearly as fast as native code, which is a damn sight faster than most interpreted languages.

In a sense, this isn't new. Java and C# offer something similar. The Java Virtual Machine dates to 1991, and the .Net runtime to the early 2000s. Both established relatively fast virtual machine compilation targets that were independent of the underlying architecture. The promise of both was to ignore the differences between platforms by writing for one virtual platform that worked everywhere.

But both are high-level garbage-collected designs, which restricts the languages you can actually compile to them; and perhaps more importantly, they're both owned by mega corporations that don't play well with others. Somehow, presumably through dark magic and sacrifices, WASM has been agreed by every browser maker as the standard compilation target for web-attached libraries, something both Java and C# aspired to and neither achieved.

If you've ever seen a web page that does some 3D rendering in a way that looks too good to be JavaScript, and is too responsive to be done on the server, you've probably used WASM. HeroForge, for example.

While it's been around for a while, and been getting consistently better for that time, there are two things holding WASM back. The first is WASI, which is the set of APIs for WASM to access files as if it was an actual application. This is progressing nicely.

The second is higher-order conventions for actually building things in it. Right now, WASM is powerful if you have the technical wherewithal to compile to it, but it's out of reach for the 99% of developers who don't want to dig that deep.

My predication

So here's my prediction:

The next big step in web development will be a reactive component framework that compiles its components to WASM and uses that to generate content.

It'll render content in three generations: compile time, server and client. Those parts where the content is known at compile time will be pre-rendered, those parts known at request time will be rendered on the server, and those parts that react in response to user behaviour will be rendered in the browser. All three generations will use the same compiled component format and the programmer will often be unaware of which it's doing because they're just writing data sources and components.

This solution combines the advantages of reactive component frameworks with those of static site builders. It's fast to load, as the static HTML page contains real content, so you don't need to load a whole library before anything appears. It's friendly to SEO, making it suitable to use on public pages. It's flexible enough to build deep single-page apps.

The compiled server part, also WASM, will be suitable for pushing to serverless edge services like CloudFlare. This will help it be fast and dynamic at the same time.

It'll be opinionated. Rather than unlimited flexibility and plugins, it'll prescribe the use of eg Sass, Typescript etc. The advantage of making that decision is that you can optimise for a neat solution.

Like functional or borrow-based programming, or even modern Java and C#, it will make a firm distinction between which values are fixed and which are variable.

While React's component format is JavaScript with embedded HTML, and Vue's is HTML with embedded JavaScript, the format of this new framework will be a segmented file with specific sections for each piece.

For performance, the compiler and server will probably be written in Rust - but not until after the first version has proved itself. It'll start in something slower and easier to prototype, like Node.js or Python.

But it won't be written until the current generation of reactive frameworks has had more time to fully explore the space of possible solutions. There's more turbulence ahead.