While Larry was producing most of the content for the “Request/Reponse” chapter for the next edition of our book, I took the lead on writing a section on QUIC (Quick UDP Internet Connections), since I have closely followed its development.
Our expectation is that the role of QUIC will be about as important as that of TCP in the coming years, which means it warrants more substantial coverage than we provided in the last edition. So I dug a bit deeper into the bits and bytes of QUIC than I have previously, with a goal of bringing the coverage up to par with our TCP coverage. In addition to reading through the RFCs, I found lots of good information in the original QUIC design spec as well as some conference publications on the design and evaluation of SPDY (predecessor of HTTP/2) and QUIC.
One rather trivial thing that makes it harder for me to get to grips with QUIC is the fact that its RFCs (four of them, spanning hundreds of pages) lack pictures of the packet headers. The rationale for this, I believe, is that QUIC makes extensive use of fields that are variable in length and frequently not aligned on 32-bit boundaries, which makes packet header pictures a bit complicated and less tidy. I decided to have a go at drawing my own – here’s one example, which I provide here in the hope that I’ll hear from others who find it either useful or confusing.
The structure of packet headers in QUIC
For comparison, RFC 9000 represents this header as follows:
Long Header Packet {
Header Form (1) = 1,
Fixed Bit (1) = 1,
Long Packet Type (2), Type-Specific Bits (4),
Version (32),
Destination Connection ID Length (8),
Destination Connection ID Length (8),
Destination Connection ID (0..160),
Source Connection ID Length (8),
Source Connection ID (0..160), Type-Specific Payload (..),
}
Maybe I am just old-fashioned in liking to see packet headers when I am learning about a protocol. And I admit that the text version contains more information; e.g. it explains the Flags in detail. Feel free to let us know your preferences. (I guess we know what the “Wall of Text” reviewer thinks already!)
The use of variable-length fields pervades the protocol, in an effort to balance efficiency and future-proofness.
Both TCP and IP have a history of defining 32-bit fields that turned out not to be long enough; QUIC generally avoids that issue by allowing for fields to be very large, such as the 160-bit connection IDs in the packet header above.
At the same time, efficiency has been a concern throughout the design process for QUIC and HTTP/2, since the time to transfer small objects in HTTP is significantly impacted by the amount of header overhead. Hence the use of variable-length encodings that keep the fields small when the extra length isn’t needed.
It’s probably also fair to say that processing unaligned fields in software is much less of a concern now than it was on the computing hardware of the 1970s, so saving bytes on the wire now takes precedence over 32-bit alignment.
Those long connection IDs replace the way in which TCP connections are identified (source and destination IP addresses and TCP port numbers) with a single identifier that is unique from the perspective of the receiving host. That provides a handy starting point for enabling QUIC connections to survive a change of IP address, e.g., when a host switches from one network to another.
Because QUIC differs from TCP in many ways large and small and spans four RFCs, trying to describe its most important features feels a bit like the parable of the blind man and the elephant. One aspect that grabs my attention is the way the layering with TLS was redesigned in QUIC (previously discussed here ). Here is the picture we use to show this:
How layering with TLS was redesigned in QUIC
HTTP over TLS was a traditional layered approach: TCP gives you a reliable byte-stream; TLS provides a secure channel on top of TCP; and HTTP sends application data over that secure channel.
QUIC, by contrast, absorbs the record layer of TLS, with the TLS handshake setting up the secure channel and HTTP then getting to run over QUIC’s secure channel directly. This more integrated approach has allowed for several round-trip times of connection establishment to be reduced to a single RTT, with the potential for application data to be sent in the first RTT. As I discussed a few months ago, 0-RTT data is a good example of a complex tradeoff between performance and security. Even without 0-RTT data, the redesigned layers of QUIC and TLS reduce the number of RTTs needed to get a secure channel established, thus providing a real performance benefit to the application.
Finally, A Protocol for RPC
The QUIC feature that I found myself focusing on this time around was the support of streams within a single connection. This is particularly relevant because Larry and I have chosen to place QUIC inside our chapter on Request/Response protocols.
As we have argued since our first edition, the request/response paradigm is not a good fit for a reliable byte-stream abstraction as provided by TCP. So while QUIC is usually presented as a transport protocol to improve performance of Web applications (which it is) we view it as the long-awaited answer to the question “where is the protocol for request/response applications?”
While the example of request/response that we have tended to use is remote procedure calls (RPC), many similar arguments apply for the retrieval of web pages. In particular, it’s generally not required that the responses to a series of requests for objects come back in order; what is important is that they all get back quickly. Furthermore, HTTPS is not just a protocol for fetching web pages; it also underpins RPC protocols such as gRPC, further underscoring the need for the protocol stack under HTTP to be well tuned for RPC.
Streams are the primary mechanism making QUIC a better fit for request/response operations. When HTTP runs over TCP, the only way to allow one request to proceed independently of another is to open multiple parallel TCP connections.
With each connection running its own congestion control loop, the experience of congestion on one connection is not apparent to the other connections; each connection tries to figure out the appropriate amount of bandwidth to consume on its own, while competing with the others. And if HTTP runs over a single TCP connection, a single dropped packet blocks the entire progress of any requests in flight until that lost packet is retransmitted.
So QUIC allows for many streams within a single connection, and each stream can make progress independently from the others. A single packet loss only impacts the stream (or streams) whose data was in that packet. At the same time, QUIC can use that one packet loss to respond appropriately to congestion. Streams are provided by a mechanism that places Stream frames within packets, and the process of starting a stream is lightweight: either end of a connection can initiate a new stream by picking a new Stream Identifier and sending a frame with that ID and the first frame of data.
The way in which QUIC handles congestion and packet loss is worth a whole RFC (9002) and there are some interesting details in how QUIC differs from TCP, mostly in the way sequence numbers refer to packets, not bytes, and are never re-used, even for retransmissions. SACK is more expansive, allowing for more gaps in the set of acknowledged packets. But the default approach is very similar to that of TCP NewReno, while allowing for other options.
Boiling down hundreds of pages of RFCs to a section of a textbook means that inevitably we must skim over some details. I’m not even sure if our book on all of networking will be as long as the full set of QUIC RFCs. But having made the case since the 1990s for a third transport protocol that is neither UDP or TCP, we are pleased to finally have a candidate for that spot that seems to be gaining real traction in the Internet.
Of course there have been many other efforts to add to the TCP/UDP duopoly in the past, with SCTP being a valiant effort, but QUIC has a lot going for it. The team that has worked on it since 2012 has paid close attention to deployment considerations (which is why it runs over UDP—the duopoly lives on thanks to middleboxes). And with a lot of carefully considered design, QUIC has made a real difference to the performance of the web and of other applications needing request/response semantics. ®
Larry Peterson and Bruce Davie are the authors behind Computer Networks: A Systems Approach and the related Systems Approach series of books. All their content is open source and available for free on GitHub. You can find them on Mastodon, their newsletter right here, and past The Register columns here.