The Demise of the Mildly Dynamic Website

Currently available for hire — experienced software engineer with a cryptography and security background. Reach out via email or LinkedIn. (June 2024)

Early websites. In the beginning, website HTML was crafted by hand. Your average personal — or corporate — website might consist of hand-edited HTML subsequently uploaded, probably via FTP, to a web server which knew only how to serve static files.

Since web pages in a set often include shared elements like headers and footers, maintaining this was obviously troublesome. I have no doubt each and every person charged with maintaining such a website found their own solutions. Maybe they wrote simple Perl scripts to generate pages with common headers. If they weren't programmers, maybe they used some tool like Frontpage, Dreamweaver or Hotdog that (possibly) offered some useful functionality in this regard. Or maybe they just stuck it out and edited every page manually when they wanted to change the header or footer. You might think that sounds nuts, but I'm fairly sure it was quite common.

At some point “Server Side Includes” (SSI) came along, offering extremely basic file inclusion functionality to allow common headers and footers. CGI was developed, allowing web pages to be produced from dynamic scripts. While groundbreaking for enabling dynamic web applications, I got the impression most ordinary websites did not make use of it for serving pages. Many web servers required or at least recommended CGI scripts to be placed in a special cgi-bin directory, making URLs rather cumbersome. One major CGI application was the blogging platform Movable Type, which featured a CGI and Perl-based administration webapp which automatically regenerated static files from a database whenever changes were made.

Rise of PHP. What was really significant however was the explosion in popularity of PHP. A great deal has been written about PHP, much of it negative, but there are some interesting attributes of PHP to note which are relevant here:

• PHP has the world's simplest deployment model for web applications: copy some files to a directory. This deployment model is so simple, and so intuitively understandable, that almost anyone capable of using an FTP client (the existing market of people making their own homepages, in other words) could deploy a PHP application.

  Of course, PHP inherited this particular attribute from CGI, but without the cumbersome special cgi-bin directory which was often (though not always) required, and often with greater performance. The lack of shebang lines probably also reduced compatibility issues people faced trying to transfer CGI scripts from Windows to Unix environments or vice versa.

• Because PHP originated as a templating language, the initial behaviour of a PHP file is simply to print its contents. This means, in turn, that you can just “dip in and out” of PHP as you need it. Compare this in turn with even the simplest Perl or Python CGI script. For the latter, you're writing a program which outputs HTML; with PHP, you're writing HTML with some occasionally interwoven code. If a particular page doesn't need to do anything dynamic, it needn't contain any code at all.

• PHP scripts don't consume any resources (persistent processes, etc.) when they're not being used.

Even more interestingly, these two attributes combined to create something very special in PHP¹: support for the “hackish state”, an environment optimal for casual tinkering with server-side scripting — without having to bring out a whole framework and take the time to set it up. If you had an idea for some kind of dynamic, server-side behaviour, you could probably have it prototyped in PHP in 15 minutes. It might be crude, but it would be real. But the “hackish state” of web development that PHP supported went beyond this in a way that, having experienced it, I find hard to describe; perhaps the closest analogy is environments designed to enable programmatic art, like the interesting recent development of live music programming environments. You could call it a kind of jamming. Though there are countless server-side web frameworks, PHP is uniquely adapted to do server-side development in a jamming state of mind.

Of course, this doesn't mean that what you produce in such a state is going to be something robust, maintainable, or something you want to use in production. But these requirements can be solved later, after the initial prototyping and a period of uninhibited, spontaneous creativity; just as a great illustration might be preceded by, and necessarily flow from, rough concept art and pencil sketches.

The mildly dynamic website. PHP enabled dynamic web applications for the masses. But an interesting and particular effect of the rise of PHP was that it enabled and led to the rise of what I'm going to call “mildly dynamic” websites.

Broadly, these are websites which are still web pages, not web applications; they're pages of essentially static information, personal websites, blogs, and so on, but they are slightly dynamic. They might have a style selector at the top of each page, causing a cookie to be set, and the server to serve a different stylesheet on every subsequent page load. Perhaps there is a random quote of the day at the bottom of each payload. Perhaps each page has a comments section. Perhaps each page load shows a different, randomly chosen header image. Anyone remember shoutboxes?

Of course, this is all minor functionality, with the possible exception of comments sections. It's not a big deal. Still, these random details gave a certain character and individuality to websites of this era that is hard to find today. They were, inimitably, the product of jamming. (Many use the term “indieweb” today in reaction to their own perceptions of a change in the character of the web from how they once remembered it; these random details are one such part of my own memories.)

(As an aside, WordPress and similar PHP blogging platforms are probably the principal exception to the static site trend here. These are a mixed bag; while on the one hand, they're still oriented towards publishing static content, they do offer mildly dynamic functionality like comments systems. On the other hand, they're not really oriented towards adding your own mildly dynamic functionality. Though of course you can add custom functionality with plugins, they're not really an environment that encourages jamming, or, say, embedding dynamicity inside a blog post right then and there. Others have noted that the rise of blogging platforms such as these led to a greater uniformity in websites, because of the rigid publishing paradigm of chronologically ordered blog posts they imposed on those who use them. You can go against the flow and fight the system, of course, but this takes effort, and most won't, so the net result is that websites become oddly samey.)

The dynamicity gap. Eventually, the era of PHP websites passed and static site generators became the new fad. Everything old is new again! First you had people who made static websites discovering dynamic websites, and then you had people who made dynamic websites discovering static websites. A consequence of this, however, has been the demise of the mildly dynamic website.

In short, here's the problem: the mildly dynamic functionality I mention above is too minor to be worth bringing in a web application framework, and spawning and maintaining persistent processes just to serve a website.

Or, to put it another way:

You can read the “Effort” axis as whatever you like here; size, complexity, resource consumption, maintenance burden. What PHP offered is an increase in this effort proportional to the amount of dynamicity to be added to a website. This is something no framework can do (and I include PHP frameworks in this²). If you have a static website and you want to make it dynamic and you don't use PHP, pretty much all options available to you imply a massive increase in complexity that must be paid up front before you can do anything at all. You bring in a massive framework for the dynamic equivalent of “Hello World”, and then on top of that (if it's a non-PHP framework, meaning persistent processes are probably involved) the process for deploying your website radically changes and becomes far more complex.

It's not surprising at all, therefore, that people tend not to do this nowadays.

This in turn has led to much of the functionality which might have previously been implemented on the server side on a “mildly dynamic” website being moved to JavaScript. A blog is generated by a static site generator, but then how does one support comments? Answer: Specialist third-party services like Disqus come into existence. Now, you can have comments on your website just by adding a <script> tag, and not have to traverse the painful vertical line of making your website itself even slightly dynamic. This has driven increased dependence on third party services; dynamicity itself becomes something to be outsourced.

Fundamentally, this is what we lost when we moved beyond PHP. Nowadays, we have an abundance of frameworks for web applications to choose from; yet PHP remains is the only server-side technology really offering a linear increase in effort for a linear increase in dynamicity if you are starting at zero. We probably shouldn't be surprised, therefore, by the trend of moving everything to client-side JavaScript. You see, client-side JavaScript shares something in common with PHP: it plots the same line on the graph above. It is, besides PHP, the other compelling technology for mildly dynamic websites, because you can start using as little JavaScript as you like without paying an upfront cost in frameworks and infrastructure.

Suppose you are writing an article about some concept and you want to put some special dynamic widget in the middle of the article, which people can play with to gain an understanding of the concept. This is just one article you're writing, one of countless others; you're not going to spin up an application server (and maintain it indefinitely) just for this throwaway toy widget. The only reasonable implementation options are JavaScript and PHP. This support for jamming is something these two languages have in common, and should be seen as a positive attribute, yet is rarely mentioned.

Or, suppose a company makes a webpage for looking up products by their model number. If this page were made in 2005, it would probably be a single PHP page. It doesn't need a framework — it's one SELECT query, that's it. If this page were made in 2022, a conundrum will be faced: the company probably chose to use a statically generated website. The total number of products isn't too large, so instead their developers stuff a gigantic JSON file of model numbers for every product made by the company on the website and add some client-side JavaScript to download and query it. This increases download sizes and makes things slower, but at least you didn't have to spin up and maintain a new application server. This example is fictitious but I believe it to be representative. There's little room for mildly dynamic functionality in the modern ecosystem of static site generators and kitchen-sink process-based web frameworks. I might call this the dynamicity gap.

It's a common refrain that the adoption of technology in the web development space seems to be excessively fad driven. In the last decade we have seen a completely misguided move to “Single Page Apps”, which break completely without JavaScript. It's the fad, you see. Yet now the new fad is “Server Side Rendering”; the notion that (amazing, I know) one actually generates web pages on the server — and serves them to the client! From the way people talk about this you would think this concept has never been introduced to them before and is completely novel to them. The sad thing is, it may well be the case. It's long been fairly apparent to me that the average modern web developer has no comprehension of what the web actually is³. The web seems to be hostage to a CADT development model in which, periodically, everything old becomes new and everything new becomes old, as the old, forgotten way of doing things becomes sufficiently forgotten that when rediscovered, it suddenly becomes new and shiny. Perhaps “the grass is always greener” also applies to static vs. dynamic websites, and to client-side vs. server-side. It's possible we will see the fad pendulum swing back toward server-side; though the dynamicity gap suggests that if this does happen, it will probably be constrained to full-fat web applications, not (barring a sudden return to PHP) ordinary websites.

AWS Lambda: CGI But It's Trendy. Recently we've seen the rise in popularity of AWS Lambda, a “functions as a service” provider. From my perspective this is literally a reinvention of CGI, except a) much more complicated for essentially the same functionality, b) with vendor lock-in, c) with a much more complex and bespoke deployment process which requires the use of special tools.

What captured people's imaginations about AWS Lambda is that it lets you a) give any piece of code an URL, and b) that code doesn't consume resources when it's not being used. Yet these are also exactly the attributes possessed by PHP or CGI scripts. In fact, it's far easier for me to write a PHP script and rsync it to a web server of mine than for me to figure out the extensive and complex tooling for creating, maintaining and deploying AWS Lambda functions — and it comes without the lock-in to boot. Moreover, the former allows me to give an URL to a piece of code instantly, whereas with the latter I have to figure out how to setup AWS API Gateway plumbing correctly. I'm genuinely curious how many people find AWS Lambda interesting because they've never encountered, or never properly looked at, CGI.

Of course, it's fair to say if Lambda does offer anything that PHP/CGI doesn't it is high availability. That's something you can't say about a solitary PHP server. But it's not as though it's infeasible to create a highly available PHP cluster either — and I can still deploy a script and give it an URL in 30 seconds with nought but rsync. Moreover, the average server is reliable enough that a lack of HA just doesn't seem to matter in many circumstances. This is especially true when one is talking about niche dynamic functionality which is only occasionally used. These are often the same circumstances where Lambda might be used — things called rarely enough it's not worth keeping a daemon spun up. The PHP websites I've published have never been highly available and I've never lost sleep over that; the ability to just give a piece of code an URL in 30 seconds, without complex deployment tooling, proprietary APIs or vendor lock-in seems to me a lot more valuable for the things I do.

Future directions. It seems like more technologies which plug the dynamicity gap on the server-side — languages like PHP — would be valuable. The CGI/PHP deployment and execution model clearly excites people as demonstrated by the popularity of AWS Lambda, yet we still see relatively little new development in this space; the ability to dip in and out of code in web pages to offer mildly dynamic functionality on the server side without frameworks is empowering yet is largely offered by PHP alone. For that matter, the popularity of WordPress is probably in part because, being written in PHP, it inherits these advantages in turn.

Theoretically, there are many plugins for webservers adding support for scripting using any scripting language you can name. These are sometimes used to host full-blown web applications but I don't see them being used to facilitate mildly dynamic functionality. mod_perl and mod_lua, for example, don't really offer any kind of functionality allowing you to dip and out of Perl/Lua code inside HTML. mod_python does offer “Python Server Pages” which does support this using an ASP-like syntax, but it seems rarely used. Moreover it seems largely replaced by mod_wsgi, which provides no comparable functionality.

This website is statically generated by a ~700-line Node.js script which I wrote specifically for the purpose. I considered using an off-the-shelf static site generator but ultimately, since I knew exactly what output I wanted, I concluded it would take more effort to wrestle any such tool into doing exactly what I want than to just write my own tool. I'm idly pondering bringing PHP back into the mix, however — so that I can make it mildly dynamic, and offer things like comments once more.