Memoirs from the old web: The KEYGEN element

The <keygen/> element is probably one of the weirdest elements ever added to HTML:

<form method="POST">
  <keygen
    name="key"
    challenge="...optional challenge string..."
    keytype="RSA" />
  <button type="submit">Submit</button>
</form>

The purpose of the <keygen/> element was to allow a web browser to generate a private/public keypair upon submitting a form, in a way that allowed a web browser to be enrolled in a new client certificate.

Anyone who has setup a website will appreciate that SSL/TLS requires a web server to have an SSL certificate authenticating its identity. However, while TLS requires a server certificate, it also has optional support for client certificates, whereby a client connecting to a TLS server must also authenticate its identity to the server. Just as in the case of server certificates, this is done by presenting a certificate and proving ownership of the corresponding private key.

Client certificates are rarely used on the web today, though the client certificate functionality of TLS is popular in some non-web contexts¹. Yet there is—or was—more infrastructure in web browsers to handle client certificates than many realise. This infrastructure, and the <keygen/> element dates back to Netscape 4.0. Possibly, when cryptographic functionality such as SSL was first being added to web browsers, there was substantially more optimism about the viability of client certificates on the web than there is now.

Why are client certificates rarely used? There are likely several reasons, but probably the most obvious one is that the UI for handling them has been truly abysmal. The bare minimum functionality and UI for enrolling a web browser in a new client certificate was added to Netscape 4.0 and seems to have received basically no maintenance or enhancement since.

This unloved feature was finally axed in Firefox 68, released in 2019, which removed support for <keygen/>. Chrome has likewise dropped support. It was never supported by IE. Thus, you can't use <keygen/> today, but this article will look at how it worked.

How did it work? At its core, the <keygen/> tag is simply a form control, which is given a name="" attribute like any other form control. A public key (for example, an RSA public key) is generated and sent in the field when the form is submitted, so the server can learn the web browser's public key.

The trick, as it were, is that when the public key is generated and submitted by the web browser, the corresponding private key is stored in a local database by the web browser — indeed, if the private key were simply thrown away, there would be little point.

In general, the public key sent to the server is then used by the server to generate a client certificate, which is delivered back to the user. The obvious way to do this is to redirect the user to a page where they can follow a link to the certificate file, but in principle a client certificate could be delivered at this point using some other non-web method.

When the user clicks a link to this X.509 client certificate in their browser, a special thing happens: the browser asks if the user wants to install the client certificate. When the browser tries to install the certificate, it discovers that it has the corresponding private key which it saved in a local database earlier, when the <keygen/> tag was submitted. This completes the enrolment process, and allows the user to access websites which require a client certificate.

It's interesting to note that the private key stored locally by the <keygen/> tag isn't visible to the user in any of Firefox's certificate or key management UI. As far as I can tell, there's no way to identify the presence of these locally stored private keys, or delete them.

UI issues. The UI issues with <keygen/> were severe, but probably the worst of these occurs not even during enrolment, but after it. A TLS web server can configure itself either to not request a client certificate (most websites), to request but not require one, or to require one. When visiting a website which requests or requires a client certificate, Netscape-heritage browsers would pop up a dialog asking you to choose what client certificate to use. If the website requires a client certificate but you don't have any suitable ones installed in the browser, the connection would simply fail with an SSL error screen.

Thus, if a website were configured to require a client certificate, the UI is terrible for users not yet enrolled, who get only an SSL error screen. There is no opportunity for websites to provide a message like “you have not registered yet, go here”. Since some website needs to be available to new users to handle enrolment, this basically requires you to have two different domains — say, newusers.example.com and secure.example.com, with only the latter requiring a certificate. Of course, if the user accidentally loses their client certificate or tries to access the secure site from a new device, they again get an SSL error which is extremely uninformative and unhelpful.

The UI if a website is configured to only request an optional client certificate is not that much better. While this allows successful access to the website for new users, the UI issues are similar to those of HTTP authentication; the user is prompted to choose a client certificate to send (if they have one) using a browser-specific UI, and this choice is then cached. Of course, most users will probably not know what a client certificate is, so this dialog is in itself confusing. Moreover, this creates an issue when the user completes enrolment and wishes to change from not using a client certificate to using a client certificate, as (much like browsers do not provide any way to clear credentials submitted for HTTP authentication) there is no way to manually re-trigger the certificate selection dialog, short of (for example) restarting the browser; there is no “logout” mechanism. The same issue occurs if you have multiple client certificates and want to change between them.

Actual deployment. Actual deployment of client certificate authentication on the web is probably found mostly in internal intranets. Though the <keygen/> enrolment mechanism has been removed from browsers today, web browsers can still use client certificates installed manually.

There was, however, one real-world deployment of the <keygen/> element on a public website which I remember all too well — in large part because of the poor UI made dealing with the website a frequent headache. Namely, the certificate authority StartSSL, part of StartCom.

StartSSL was a CA which built its name on giving away free SSL certificates when everyone else was charging for them. It was essentially a poor man's Let's Encrypt before Let's Encrypt had even been conceived. It wasn't possible to obtain certificates automatically — instead, you had to use their website. StartSSL had decided not to use usernames and passwords, but client certificates as the basis of authentication to their website; presumably the fact that they were an SSL company made them overenthusiastic about adopting client certificates despite the terrible UI in web browsers.

The site used the <keygen/> element to complete enrolment in a client certificate. Once you obtained the certificate, if I recall correctly, you were transferred to another subdomain configured to require a client certificate. If you hadn't installed the client certificate properly, you got an SSL error. I remember dealing with this process going wrong often and having to troubleshoot Firefox's client certificate handling. All in all, the main thing I took away from StartSSL's site is that using client certificates for website authentication is barely viable even for a highly technical audience. The UI for using and enrolling client certificates was basically shipped in Netscape 4 and never changed or improved since, an unloved and forgotten feature. Of course this is a vicious circle; nobody uses client certificates since the UI is terrible, and nobody spends time improving the UI because nobody uses the feature.

Incidentally, StartSSL was given the CA death penalty in 2016; you can find the details on Wikipedia.

What does it look like? I installed Firefox 1.0.8 (released 2006) under Wine to demonstrate <keygen/>:

(Yes, the <blink> element wasn't the only circumstance in which Netscape-heritage browsers would blink at you! The “please wait” dialog was likely included due to the significantly longer time key generation might take back when the feature was first implemented. Note also that you can't even select an RSA key size greater than 2048 bits.)

After the key generation is complete, the form is submitted as usual. The field data submitted for the <keygen/> field is a base64-encoded ASN.1 structure which is described on the MDN page. This format is known as the Netscape Signed Public Key and Challenge (SPKAC) format. OpenSSL has support for dumping it, though somewhat peculiarly, it can only handle it if provided in Base64-encoded form prefixed with the string SPKAC=:

$ (echo -n 'SPKAC='; cat test.der | base64 -w 0) | openssl spkac
Netscape SPKI:
  Public Key Algorithm: rsaEncryption
    RSA Public-Key: (2048 bit)
    Modulus:
        00:e2:8b:a3:c2:75:1d:d8:ee:eb:63:4a:8a:3c:f1:
        74:b6:4c:3a:90:bf:c7:eb:d6:6f:0f:e3:78:8f:91:
        b2:62:76:05:d5:cd:80:e5:ec:0a:bc:a0:1b:13:74:
        a1:28:79:8c:f6:4a:4c:1f:87:90:51:50:a0:b6:2e:
        13:a4:50:e0:1d:72:a1:87:ed:93:4c:f5:9b:e1:d9:
        3c:e6:61:30:50:f3:0f:0f:b4:b1:1a:0f:93:8c:54:
        e2:5f:bb:17:30:80:71:d8:bf:da:79:04:e0:20:9f:
        c5:18:bd:1e:42:9e:99:69:5c:22:0e:42:ff:86:8e:
        b8:b0:09:9a:e3:7d:f2:4e:2a:b7:ea:75:84:46:5c:
        3f:a0:60:b6:bb:f0:0c:81:c0:cf:ff:19:bd:18:b4:
        7a:a3:dd:09:55:a9:b7:8f:e3:1c:d5:42:86:e5:ce:
        ee:2d:80:5b:ef:d4:6c:61:fd:cb:1c:01:b2:02:51:
        e8:38:3a:d3:74:04:13:d2:de:30:83:91:50:3e:51:
        62:fd:e3:59:4e:d4:88:4a:6f:17:5c:4b:51:bd:ff:
        66:df:d1:b7:37:87:02:e9:c1:06:74:38:86:c6:ca:
        c6:53:61:b1:b0:cc:f9:08:fd:95:08:bd:e2:0f:2b:
        41:a6:68:68:e8:2b:5c:13:e6:0e:00:90:26:e1:ef:
        4d:93
    Exponent: 65537 (0x10001)
  Challenge String: example challenge string
  Signature Algorithm: md5WithRSAEncryption
      64:c1:e7:2d:57:c5:57:1d:df:79:45:94:47:c2:31:1f:77:3f:
      a9:f7:b8:78:e1:13:12:5d:2b:0c:89:3d:fd:a6:9b:e7:38:87:
      ee:b9:c9:54:66:29:1c:2c:35:c0:ae:f6:6d:57:d7:6c:2d:6f:
      ed:83:38:ae:e5:7e:84:a3:35:8a:9c:95:8a:4b:b1:04:a3:37:
      45:09:5f:82:ba:77:7e:4b:f2:a4:41:2e:1f:8a:1c:d9:2f:0b:
      ab:11:6d:fd:87:07:f9:0f:75:ad:28:4f:f9:14:c3:12:10:9e:
      71:8d:a4:59:22:7b:f7:d2:67:86:e3:76:13:31:ef:a9:ce:bb:
      2f:65:cd:b6:2d:b1:4a:78:38:df:53:f9:d8:17:89:92:9a:1e:
      bf:9e:62:a3:be:85:63:98:22:13:ef:e6:78:f6:ec:a6:9a:35:
      47:30:d9:a9:9a:68:02:1f:f0:37:12:55:e9:b3:6f:8f:07:43:
      5a:5a:a1:99:bb:07:30:58:28:1a:74:9e:1d:42:80:f7:49:8b:
      a5:4d:8e:94:08:cb:be:fa:b9:b5:41:5f:22:f1:f6:ba:3a:a9:
      3d:14:c3:44:10:3d:2e:36:03:99:38:d1:27:39:d9:c3:ca:c9:
      fe:ee:76:9e:ee:c8:40:1e:c8:b7:d7:79:0a:9f:c3:87:53:34:
      b5:1b:7f:e8

The challenge string comes from the challenge="" attribute of the <keygen/> element.

Internet Explorer. <keygen/>, like <embed> and <blink>, was a proprietary tag invented by Netscape, and was never supported by IE. Instead, IE required the use of an ActiveX object via JavaScript (new ActiveXObject("X509Enrollment.CX509EnrollmentWebClassFactory")) to obtain equivalent functionality.

Current situation. <keygen/> support has been removed from all modern web browsers. The MDN docs provide a good overview of what the element did. The HTML5 specification briefly documented <keygen/>, although it no longer does and recommends the use of the Web Cryptography API instead. Around 2011, a proposal for “TLS Origin-Bound Certificates (OBC)” was floated which was essentially a proposal for a variant of client certificates with better browser UI as a replacement for cookies, but this never saw wide adoption. It was later superceded by a “TLS Channel ID” proposal which also failed to gain adoption.

It is worth noting that TLS client certificates completely break corporate firewalls which seek to “secure” traffic by performing man-in-the-middle attacks on it. These systems generally work by installing a rogue CA in enterprise-controlled end devices to allow for traffic interception. However, the firewall can't forge a client certificate, which is after all validated by the website being connected to. Personally I see this as a feature², but it probably also has served to inhibit further deployment of client certificates.

Nowadays, current efforts to advance the use of cryptography on the web seem to be focused on replacing passwords with hardware or virtual tokens using APIs such as WebAuthn. There is a clear parallel with client certificates here, being that the need for passwords is reduced or eliminated, but since the feature is a subject of modern interest and increasing trends rather than an unloved bygone, it has much better UI, usability, and browser support.

Further reading. This article by Simon Tatham provides an interesting discussion about how the functionality of the <keygen/> element can be emulated in browsers today. The WHATWG blog tag for keygen provides a chronicle of the abortive effort to standardise <keygen/> and links to many mailing list discussions that went on during this process, as well as a writeup of how <keygen/> works.