The different kinds of authentication protocols

There are almost as many authentication protocols out there as there are application protocols, making it a confusing landscape. Probably most confusing of all is that attention rarely is drawn to the fact that there are many different kinds of authentication protocol, which seek to fill completely different roles.

As usual, bear in mind that authentication and authorization are different functions. With that in mind, there are essentially three different classes of authentication protocol:

  • Client to Application (client authentication protocols);
  • Application to Auth Server (backend authentication protocols);
  • Single Sign On (Client to Auth Server to Multiple Applications).

Client to Application authentication protocols are spoken by a client to an application server to authenticate. The protocol doesn't in any way dictate what the application server does behind the scenes to authenticate the client using the information it provides.

Application to Auth Server protocols are used by application servers to delegate authentication of a a client which has provided information using a Client to Application protocol to some server which is in a better position to make an authentication decision. This allows authentication processing to be outsourced to specialised authentication servers, and also means that authentication databases need not be made accessible to all application servers.

Single Sign On protocols are used by one server to communicate to another server a client's authentication status, after a client already authenticated (e.g. via a Client to Application protocol) to the Single Sign On server. In other words, they enable the transitive communication of an authentication decision; the fact of a client's previous act of successful authentication to an authentication server can be securely traded between that client and application servers. Some protocols do not require further intervention or availability of the Single Sign On server to allow a client to authenticate to an application after initial sign on, while others do.

Single Sign On protocols effectively consist of two protocols: a Client to Authentication Server part and a Client to Application part. The Client to Authentication Server part may be bespoke, or may sometimes be a very ordinary Client to Application authentication protocol; the Client to Application part will necessarily be particular to the design of the SSO protocol.

Client to application protocols. Within the space of client-to-application authentication protocols, there exist authentication frameworks and authentication methods. An authentication framework is an extensible framework which allows arbitrarily many authentication methods to be implemented, and allows it to be dynamically negotiated between client and application server which authentication method shall be used.

Some authentication frameworks are integrated into a specific application protocol. However, some are general designs designed to be incorporated into arbitrarily many application protocols. This allows even future application protocols to benefit from all authentication methods already defined within that framework, and from library implementations of such methods.

Client to application authentication frameworks: SASL. Probably the most popular such framework is IETF's SASL. SASL does not define a specific wire encoding, which is left to the application, but essentially defines a namespace of authentication method names (such as PLAIN, LOGIN, MD5, etc.) and the process by which they should be negotiated. The actual method-specific protocol is carried out, but the application protocol merely needs to facilitate the back-and-forth exchange of a sequence of opaque binary strings until authentication succeeds or fails. In short, an application protocol implementing SASL need provide only the following machinery:

  • A way for the server to send a list of supported method names (strings) to the client;
  • A way for the client to send the name of the method it wants to use;
  • A way for the client to send opaque binary strings to the selected SASL method (which may be fed opaquely into a SASL library by the application server);
  • A way for the server's SASL library to send opaque binary strings to the client (which may be fed opaquely back into the client's SASL library);
  • A way for the success or failure of authentication to be communicated.

How these mechanisms are provided is up to the application protocol, which must define some appropriate wire protocol. While the specific marshalling and state machines are application protocol-specific, SASL methods themselves are independent enough of application protocols that general-purpose SASL libraries, implementing assorted SASL methods, are widely available.

Some of the most popular SASL methods are:

  • PLAIN, which provides an authentication username and password in plaintext (and optionally, an authorization username, which may be distinct from the authentication username);
  • LOGIN, a deprecated method which is another way to provide a username and password in plaintext;
  • EXTERNAL, which has the client provide no information at all (except for an optional authorization username) and is a way for a client to request that it be authenticated from the context of the connection. This may be used to authenticate if, for example, the connection is using TLS with a client certificate, and the client certificate suffices for authentication purposes; or could also be used if a client is to be authenticated by its IP address only;
  • ANONYMOUS, which requests unauthenticated access;
  • CRAM-MD5 and DIGEST-MD5, which are challenge-response authentication schemes which try to avoid transmitting passwords unhashed;
  • NTLM, the Windows NT LAN Manager authentication mechanism;
  • GSSAPI, which refers to an API (see below) which is mainly used to facilitate Kerberos authentication.

Nowadays, it is generally desired to use SASL methods such as PLAIN or LOGIN, which transmit passwords in plaintext, over secure channels such as TLS.

Client to application authentication frameworks: HTTP. HTTP defines its own authentication framework, with its own set of methods. Arguably HTTP's avoidance of SASL is driven by its statelessness rendering SASL less suitable. Since HTTP is designed to function with a client transmitting a single request block, followed by a single response from the server, the HTTP protocol is ill-suited to the back-and-forth exchange prior to the completion of authentication envisioned by SASL; moreover, such an exchange would have to be repeated for each request.

A server requests authentication to access a resource by sending a 401 Authorization Required error and a WWW-Authenticate header. This header contains a) the demanded authentication method, b) a “realm name” giving a description the authentication domain, c) any other parameters which must be specified for the given method. Unlike SASL, generally only one method is advertised by a given server (though the HTTP RFC does permit a server to support multiple methods by sending multiple WWW-Authenticate headers, one for each method, though this is rarely seen). The client then reissues its request with an Authorization header containing the same method name and method-specific authentication data. (Note that HTTP uses the term authorization rather than authentication; make of this what you will.)

Some of the common HTTP authentication methods are:

  • Basic, which specifies a username and password in plaintext;
  • Digest, which specifies a username and hashes a password in a challenge-response scheme;
  • Negotiate, which allows GSSAPI to be used and which essentially reimplements a SASL-style method negotiation scheme on top of GSSAPI, known as SPNEGO, inside HTTP authentication. Because it involves a back and forth exchange, multiple requests may need to be made until authentication is successful. Negotiate/SPNEGO is customarily used by Windows for either NTLM or Kerberos authentication.

Client to application authentication frameworks: SSH. SSH also defines its own authentication framework with an extensible set of methods. The most commonly used methods are:

  • keyboard-interactive, which facilitates login via arbitrary terminal prompts;
  • public-key, in which a client proves its control of a given private key;
  • gssapi, which facilitates login via Kerberos (see below).

Client to application authentication methods: strategies. Client to application authentication methods can generally be grouped into a few fundamental classes:

  • Complete deference to contextual authentication (such as TLS client certificates or IP address authentication) (e.g. SASL EXTERNAL).

  • Sending username and password in plaintext, in which security (if any) is provided by an encapsulating secure channel, and in any case in which the application server ultimately obtains the password in plaintext form (e.g. SASL PLAIN; HTTP Basic; SSH keyboard-interactive).

  • Challenge-response schemes, in which a server issues a challenge nonce and a client responds to the challenge via the application of a cryptographic hash or other construction to the nonce and a password (e.g. SASL CRAM-MD5).

    These have fallen out of fashion nowadays firstly because of the popularity of TLS in conjunction with plaintext transmission, and secondly because they tend to either preclude storing passwords in hashed form on the server, or require that they be hashed in a specific manner related to the authentication scheme. Thus the method of password hashing and the method of authentication become tightly coupled.

  • Asymmetric cryptography schemes, in which a client proves its possession of a private key without transmitting it (e.g. SSH public-key; TLS client certificates).

  • Shared-secret schemes, such as TLS-PSK.

  • Zero-knowledge proof schemes such as SRP. These are cryptographically neat yet rarely used, and once again tightly couple an authentication protocol with how passwords are hashed on a server.

  • The client-to-application part of a single-sign-on protocol.

Application to authentication server. Another class of authentication protocol, which is completely unrelated to client-to-application authentication protocols are application to authentication server protocols, or backend protocols. Rather than constituting a protocol spoken between an entity being authenticated and the entity authenticating it, a backend authentication protocol is spoken between an application server and an authentication server in order to verify authentication details provided by a client.

These protocols are frequently used in network access scenarios, in which the “application” server is a network device such as a router, switch, or Wi-Fi access point. Since it would be undesirable for every Wi-Fi access point in a network to keep a copy of an authentication database, backend protocols allow authentication decisions to be outsourced to a central source of authority. Thus, backend protocols must be used in combination with some kind of client-to-application authentication protocol. The client-to-application protocol is spoken between client and application, and the backend protocol between application and authentication server.

Examples of popular backend authentication protocols include RADIUS and its successor, DIAMETER, which are primarily intended for use in network access scenarios (e.g. dial-up internet authentication, Wi-Fi authentication, etc.). Due to their history of being intended to support network access scenarios, these protocols are “AAA” (authentication, authorization, accounting) protocols designed to support authorization and accounting as well as authentication. For example, their accounting functionality enables accurate time and usage-based billing of a dial-up internet connection, if desired.

Although not directly intended for use as a backend authentication protocol, LDAP is also sometimes used as one. An application server can verify user credentials by attempting to authenticate to an LDAP server using them. This also has the advantage that, if successful, the application server can also use LDAP to retrieve information about the user.

Application to authentication server: associated client-to-application protocols. Backend authentication protocols for network-access scenarios such as RADIUS and DIAMETER were traditionally designed for use in conjunction with specific client-to-application authentication protocols; namely, those defined by PPP. PPP is an OSI Layer 2 protocol enabling IP networking over serial lines and dial-up modems. PPP defines its own extensible authentication framework and supports the following methods:

  • PAP, which transmits username and password in plaintext;
  • CHAP, a challenge-response method (which avoids transmitting passwords in plaintext, but suffers from the aforementioned drawbacks of challenge-response methods);
  • MS-CHAP and MS-CHAPv2, Microsoft variants of CHAP;
  • EAP, the Extensible Authentication Protocol, the modern protocol of choice.

EAP in itself is an authentication framework defining an extensible set of authentication methods, meaning that when used with PPP, there are two levels of method negotiation; EAP must first be negotiated, and then a specific EAP method must be negotiated. This however makes sense when one considers that EAP, unlike other PPP authentication protocols, is designed for use both in PPP and in other network-access contexts, such as Wi-Fi authentication, or Ethernet 802.1x authentication. (802.1x is an extension to Ethernet enabling network access to be predicated on successful EAP authentication).

There are some similarities between EAP and SASL, as they are both general frameworks supporting an extensible set of methods. However, EAP defines a specific wire protocol, whereas SASL does not; moreover, EAP is designed to support network-access applications, whereas SASL is intended to support application-level authentication applications.

Most significant of all about EAP, however, is that it is designed from the outset to be blindly tunnellable by an application server. Suppose, for example, you dial into a router via modem, and establish a PPP connection; the router must understand a protocol such as PAP or CHAP and know how to transact it with a backend server over RADIUS or DIAMETER. In the modern application, where for example a Wi-Fi client speaks EAP to a Wi-Fi access point, the access point simply takes the EAP messages as opaque data, tunnels them inside its RADIUS or DIAMETER session to a network authentication server, so that the network authentication server can process the EAP messages. The authentication server may likewise return EAP messages encapsulated in the RADIUS or DIAMETER session, which the Wi-Fi access point decapsulates and passes to the client. In this way, network devices are rendered agnostic to the method of authentication used and do not need to be upgraded to support new authentication methods; only the authentication server and clients need be upgraded.

Note that a client never speaks RADIUS or DIAMETER at any time; RADIUS and DIAMETER are strictly backend protocols. Because EAP messages are always encapsulated inside RADIUS or DIAMETER messages before being forwarded to an authentication server (a process an authenticating client has no control over), the RADIUS or DIAMETER message can include information about the client of interest to an authentication server besides the EAP messages themselves. For example, if a client is authenticating to use an 802.1x-enabled Ethernet port, the switch could include information about what port the client is attached to, and an authentication server could predicate its decision on this.

Single sign on protocols. We now come to the most complex and interesting kind of authentication protocol: single-sign-on protocols. As previously mentioned, a single sign on protocol must comprise two parts: a protocol for authenticating client to authentication server, and a protocol for authenticating a client to arbitrary application servers. The former might be a standard client-to-application protocol, or something bespoke; whereas the nature of SSO protocols requires that the latter protocol be particular to the SSO protocol, and is the meat of any SSO protocol.

Single sign on protocols: non-web. SSO protocols can be broadly divided into web and non-web SSO protocols. The most popular non-web SSO protocol is Kerberos. Kerberos allows a client to authenticate to an authentication server using a bespoke authentication protocol; a client thereby receives a ticket, which can be used to authenticate to application servers cryptographically without further communication with the Kerberos authentication server (which could perhaps go down after a user logs in and not immediately render all applications inaccessible). The protocol also enables clients to authenticate servers, as well as servers to authenticate clients. Kerberos is used today by both *nix and Windows Active Directory environments. For client to application authentication, it is generally not implemented inside authentication frameworks as an authentication method directly; instead, it is usually invoked via GSSAPI, and is the predominant use of e.g. the SASL GSSAPI method, the HTTP Negotiate (GSSAPI) authentication method, and the SSH GSSAPI authentication method.

It is interesting to note that the current version of Kerberos, Kerberos 5, was released in 1993 and as a result avoids dependence on the use of asymmetric cryptography, relying only on symmetric cryptography. If Kerberos were designed today, it would probably use public-key cryptography and be slightly less dependent on central authentication servers for it. Unfortunately, the explosion of the web seems to have sucked all of the oxygen out of the development of non-web protocols, for authentication and for other matters. (Note however that Kerberos 5 remains secure.)

Single sign on protocols: web. For web SSO protocols, the client-to-authentication server protocol is invariably of a standard web variety; that is, the forms-and-cookies variety. Only the client-to-application-server authentication protocol is unusual.

In general, when an application server wishes to authenticate a client, it refers a client to the SSO authentication server via the means of an HTTP redirect. If a client has not yet centrally authenticated, it is asked to do so; otherwise, if single sign-on has already been performed, the process continues without user intervention.

Having authenticated the user, the authentication server then refers the client back to the application server via another HTTP redirect; the return request contains authentication information for transmission from the authentication server to the application server, establishing the client's credentials. This data will itself be authenticated in some way to prevent a client from modifying it. Since this passage of information from authentication server to application server happens via the client, the client feels analogous to a bee carrying pollen. In some SSO implementations, an application server may then make a direct callback to an authentication server to verify the information it has received via the client; in other SSO implementations, the information passed via the client from the authentication server to the application server is cryptographically secured and does not require further verification.

It should be noted that this is not a description of a specific SSO protocol, but is rather a generalisation of the usual techniques used. Since web applications require only a standards-conforming web browser, there is less requirement to standardise the protocols implemented on top of the web platform, and thus there are many proprietary web SSO implementations.

Nonetheless, some standards have arisen. Those popular today are:

  • SAML2, or Security Assertion Markup Language 2, a misguided use of XML to express cryptographically signed statements about who has been authenticated for what and pass such statements between domains. Popular; supported by AWS for SSO.
  • OAuth, a protocol built on HTTP to facilitate authorization flows between websites. Focuses on authorization rather than authentication, but now often used for authentication as well, particularly with the OpenID Connect extension. Popular; supported by AWS for SSO. The OpenID Connect extension adds authentication functionality and allows user information (such as e. mail address, etc.) to be passed to websites during authentication. (Note that OpenID Connect is a technology layered on top of OAuth and has no relation whatsoever to OpenID 2.)

Protocols which were once seeing rising adoption, but which have now fallen out of fashion include:

  • OpenID 2, a web SSO protocol which allows people to authenticate their control of a specific URL. (Note that this has no relation to OpenID Connect, which is built on top of OAuth, and which arguably fills a very different set of use cases.)

  • LID, a web SSO protocol similar to and which was competing with OpenID 2 around the same time, but which was less popular.

Local APIs. Finally, some discussion should be given to system-local authentication APIs. These are not protocols, but are adjacent to them.

PAM (Pluggable Authentication Modules) is something of a de facto standard for local authentication in the *nix sphere. It is almost universally used for local authentication on Linux and is also frequently used on other *nix OSes. PAM provides a plugin-based authentication facility; PAM plugins are dynamic-link libraries which can be loaded at runtime to provide arbitrary authentication logic; the PAM plugins used are determined by system configuration, and PAM can thus be reconfigured as desired.

While highly extensible, PAM is nonetheless limited as it is designed to support terminal-interactive login applications. A PAM module submits a series of zero or more interactive prompts to the user (such as "Password: ") and as such, cannot support SSO authentication methods such as Kerberos (although PAM modules exist which can prompt for a user's Kerberos password and do the initial Kerberos sign on).

While PAM is primarily designed to support terminal-interactive login, it is feasible and common for PAM to also be used as the backend for some application protocols (as the backend for SASL PLAIN, for example). However, this requires that an application server understand (or correctly guess) the semantics of the line prompts issued by PAM modules. As an example, the Dovecot IMAP server assumes that if a PAM module asks for a line of input with no character echo, that it is asking for a password; and that if the module asks for a line of input with character echo enabled, that it is is asking for a username. Thus, while PAM modules could be written to support e.g. authentication via a TOTP hardware key and prompt for a six-digit value at a terminal, non-terminal applications (such as SASL applications) could not support this unless specifically designed to support the prompts of said modules.

GSSAPI is another local API, this time with a formal standard. It is designed for network application (rather than terminal-interactive) use, and is most commonly used to enable authentication via Kerberos SSO. In fact, despite being a general-purpose API intended to be extensible to arbitrary authentication schemes, it is almost exclusively used for the Kerberos, SPNEGO and NTLM applications. GSSAPI can be used via any SASL application via the SASL GSSAPI method.

Conclusions. This article might be a bit of a mess by usual standards; it's a one-session braindump. Nonetheless, here it is.

Changelog.
  • 2020-09-23: First published.
  • 2020-10-11: Corrected article: HTTP can in fact advertise support for multiple authentication schemes (sentence “Unlike SASL, ...”).