blog.bouah.net/content/posts/what-about-design.md

title	date	draft	tags
What about design?	2020-07-14T01:32:25+02:00	true	XMPP Design

Explicit disclaimer: This piece of opinion -- as any other article on this blog -- is a work in progress, and I'm happy to debate it and listen to constructive feedback.

Who around here hasn't heard about the tragic and inevitable death that XMPP is doomed with? It's a pretty common topic in channels satellite to the XSF and XMPP community, often started by alternative solutions or users missing this or that feature in one or multiple specific implementations. In a way this is my own version of why XMPP is doomed (or isn't). What could be a cause for most of these accusations?

To go down this rabbit hole, we first need to set a few definitions. Most of my readers would probably know what XMPP is, but I feel obligated to have a short reminder as it will allow me to emphasize specific points I want to talk about.

XMPP? Was ist Das?

XMPP is a communication protocol, that is nerd speak to say it's a language for applications to use and talk together at a level that the end-user doesn't see, an example would be a chat application: your desktop or smartphone app talking to a server that then talks to another app. It is defined as a standard at the IETF that also serves as a home for many other building blocks of the Internet and technology all of us use today. The IETF itself is guided by their mission statement.

Core extensions of XMPP are written so that it is easily extensible allowing anybody to use custom (XML) elements for their own use, and optionally write a specification for their new feature for everyone else to use. XMPP also defines a server/client model, where multiple servers can communicate together, thus allowing for decentralization -- anyone setting up their own server to be free from restrictions of other servers, and communicating with the world or part of it.

So there we have it: (IETF) Standard, Decentralized, and Extensible. These are I believe the 3 selling-points of XMPP.

From there tons of features can be implemented and then negociated (as part of the extensibility) and many things can change to use newer extensions that weren't considered in the core specifications. For example the serialization format (originally XML) can be changed (just as EXI is doing), and it's also perfectly fine to have non-compliant behaviour as long as it has been negociated by entities taking part in it. And so on…

The XSF (previously known as JSF) is an entity that shepards the XMPP protocol. There is no requirement that XMPP extensions be brought to the XSF, but it aims to be the place where technical knowledge around XMPP is gathered so people could get better feedback when submitting their new specification.

How do applications speak the same language in an extensible world?

This is indeed the heart of the problem. While extensibility is one of the strenghs of XMPP, it's also its main weakness, and one of the main points of its critics. I believe it's not as bad as they make it look like though.

It is true that most applications are incompatible one way or another, with various degrees of significance, either because they don't implement the same set of extensions, either because an author interprets extensions differently, or because of plain bugs.

For the rest of this article I will leave aside the last two points -- interpretation issues and bugs -- as I consider both of them bugs -- of specifications and/or implementations -- and bugs happen everywhere and can be fixed. Generally, determining what is a bug and what is a (unintended?) feature is where the issue lies.

While there has been attempts within the XSF at defining common sets of extensions in what is called “Compliance Suites” (currently updated on a yearly-basis: 2020, 2019, etc.), they have in my opinion had a mild success for the effort that it takes the author to gather feedback and come up with not-so-controvertial changes for newer revisions.

What these Compliance Suites don't take into account so well despite recent efforts, and what critics don't take into account either when saying XMPP is missing X, or that all implementations should do Y, is that it's not just about features and protocols.

The process of coming up with a common set of extensions for an implementation requires a lot more groundwork. This includes figuring out who the userbase is, and how the experience for it should be, i.e., design. This process should be applied across a set of implementations, using the same design guidelines and ensuring interoperability.

In practice it is not enough if somebody using Conversations on the mobile talks to somebody else using Dino on the desktop, even if they both follow the Compliance Suites of year X and can then interop on a “basic” level (still pretty advanced to be honest), they have different design guidelines and there will inevitably be areas where they differ and some features won't behave as expected on the other side. The issue is not that there is no design guidelines, it's that it's not the same.

What now?

Recently a solution following this design process called Snikket has emerged. You can read about its goals in the introduction article or in a more detailed explanation from its author. At the time of writing it is composed of a rebranded Prosody and Conversations, is entirely based on XMPP and federates with the XMPP network. But the important part -- and also why it deserves a different name -- is its goal: provide a server and a (set of) client(s) that interoperate properly and have similar design.

Maybe you're not part of Snikket's target, in which case there might someday be a similar solution that's more adapted to your use-case.

For the most technical of us who understand the protocol and/or can deal with common design issues, it may be ok to continue using our current clients designed for other usages and workaround these issues. For the masses I believe this is not an option. Your may take this with a grain of salt as it is as a developer that I am saying this to you.

In the end why does it matter to the end-user if it's actually XMPP that's being used, as long as these new solutions incorporate properties we care about such as standardization, decentralization, and extensibility? so that we don't end up again with closed silos that have for only purpose to lock us in and make profit off us, again.

I guess the only thing left to figure out now is network effect…