Multicast Extension for QUIC

Multicast Extension for QUIC Akamai Technologies, Inc.

150 Broadway Cambridge, MA 02144 United States of America jakeholland.net@gmail.com

lucaspardue.24.7@gmail.com

TU Berlin

Germany m.franke@ravim.de

Akamai Technologies, Inc.

145 Broadway Cambridge, MA 02144 United States of America krose@krose.org

TSV QUIC Working Group Internet-Draft This document defines a multicast extension to QUIC to enable the efficient use of multicast-capable networks to send identical data streams to many clients at once, coordinated through individual unicast QUIC connections. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the QUIC Individual Draft mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction This document specifies an extension to QUIC version 1 to enable the use of multicast IP transport of identical packets for use in many individual QUIC connections. The multicast data can only be consumed in conjunction with a unicast QUIC connection. When the client has support for multicast as described in , the server can tell the client about multicast channels and ask the client to join and leave them as described in . The client reports its joins and leaves to the server and acknowledges the packets received via multicast after verifying their integrity. The purpose of this multicast extension is to realize the large scalability benefits for popular traffic over multicast-capable networks without compromising on security, network safety, or implementation reliability. Thus, this specification has several design goals:

Re-use as much as possible the mechanisms and packet formats of QUIC version 1
Provide flow control and congestion control mechanisms that work with multicast traffic
Maintain the confidentiality, integrity, and authentication guarantees of QUIC as appropriate for multicast traffic, fully meeting the security goals described in
Leverage the scalability of multicast IP for data that is transmitted identically to many clients
Rely on Multipath QUIC () to provide multicast for clients

This document does not define any multicast transport except server to client and only includes semantics for source-specific multicast.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. Commonly used terms in this document are described below.

Term	Definition
SSM	Source-specific multicast, as described in
ASM	Any-source multicast, as distinguished from SSM in
(S,G)	A tuple of IP addresses (Source IP, Group IP) identifying a source-specific multicast channel as described in

Multicast Channel A QUIC multicast channel (or just channel) is a one-way network path that a server can use as an alternate path to send QUIC connection data to a client. Multicast channels are designed to leverage multicast IP and to be shared by many different connections simultaneously for unidirectional server-initiated data. One or more servers can use the same QUIC multicast channel to send the same data to many clients, as a supplement to the individual QUIC connections between those servers and clients. (Note that QUIC connections are defined in and are not changed in this document; each connection is a shared state between a client and a server.) Each QUIC multicast channel has exactly one associated (S,G) that is used for the delivery of the multicast packets on the IP layer. Channels only support source-specific multicast (SSM) and do not support any-source multicast (ASM) semantics. Channels carry only 1-RTT packets. Packets associated with a channel contain a Channel ID in place of a Destination Connection ID. (A Channel ID cannot be zero length.) This adds a layer of indirection to the process described in for matching packets to connections upon receipt. Incoming packets received on the network path associated with a channel use the Channel ID to associate the packet with a joined channel. A client with a matching joined channel always has at least one connection associated with the channel. If a client has no matching joined channel, the packet is discarded.

Packet Number Encoding for Channel Packets QUIC packet number encoding in encodes a truncated packet number by carrying only the least significant bits of the full packet number. The sender selects a packet number length large enough for the receiver to reconstruct the full packet number, and the receiver decodes it as the value closest to the next expected packet number in that packet number space. This sender-side mechanism is not suitable for multicast channel packets, because a single channel packet can be received by many clients with different receive, loss, and acknowledgement states. Channel packets therefore use a fixed packet number length of four bytes. A server MUST encode the packet number of a channel packet as a four-byte truncated packet number, using the packet number encoding described in . Before header protection is applied, the Packet Number Length bits of byte 0, that is, the bits with mask 0x03, MUST be set to 0b11 to indicate a four-byte Packet Number field. A server MUST NOT select the packet number length for a channel packet based on acknowledgement state from any individual associated unicast connection. A client reconstructs the full packet number for a channel packet using the packet number reconstruction algorithm in , applied to the channel packet number space. The expected packet number used for reconstruction is the next packet number after the largest packet number of any channel packet from which the client has successfully removed packet protection in that channel packet number space. Before a client has successfully removed packet protection from any packet in a channel packet number space, it uses the From Packet Number of the applicable MC_KEY frame as the next expected packet number for reconstruction. If a client cannot reconstruct packet numbers for a channel because the encoded packet numbers are too far from its expected packet number, it MUST discard the affected packets. If this happens, the client SHOULD leave the channel and send an MC_STATE frame with State LEFT and Reason Code UNSYNCHRONIZED_PROPERTIES.

Latency Spin Bit in Channel Packets Channel packets are unidirectional server-to-client packets, and clients can not send packets on a multicast channel. Therefore, the latency spin bit algorithm from does not apply to channel packets. The spin bit has no semantics in this extension and MUST NOT be assigned another meaning by endpoints unless specified in a future extension. In channel packets, a server MUST disable the spin bit as specified in . A client MUST ignore the value of the spin bit in received channel packets.

Channel using Multipath QUIC From the point of view of the client, each Multicast QUIC channel is handled as an additional path from the server. A client keeps its unicast connection with the server open during all the transmission. Additionally, the server can inform the client about an additional path where it will receive multicast content. All mechanisms, except those listed below, follow . TODO: either rely the current Multipath QUIC draft (the client must create the new path itself) or extend the draft to allow the server to initiate the creation of the second path. The first major change concerns the encryption of the multicast path. To keep the unicast path between the client and the server secure, the multicast path MUST use a different key and MAY use a different algorithm, that is common between all clients. As such, each path is encrypted differently, differing from the current version of . NB: there are discussions within the Multipath QUIC draft to integrate this feature to strengthen the security of MPQUIC. Secondly, a client MUST NOT send any data on the multicast path with the server. Leveraging Multipath QUIC for Multicast QUIC provides interesting properties from the client's point of view. For example, the client can seamlessly receive data from the multicast and the unicast path. This enables efficient unicast fall-back and unicast retransmissions. Leaving the multicast channel is performed by closing the multicast path with the server. This draft imposes the Multiple Packet Number Space version of Multipath QUIC (See ). Each channel possesses its own packet number space. To enable clients to detect lost packets, packet numbers in channels SHOULD be continuous. The use of any particular channel is OPTIONAL for both the server and the client. It is recommended that applications designed to leverage the multicast capabilities of this extension also provide graceful degradation for endpoints that do not or cannot make use of the multicast functionality (see ). The server has access to all data transmitted on any multicast channel it uses, and could optionally send this data with unicast instead. No special handling of the data is required in a client application that has enabled multicast. A datagram or any particular bytes from a server-initiated unidirectional stream can be delivered over the unicast connection or a multicast channel transparently to a client application consuming the stream or datagram. Client applications should have a mechanism that disables the use of multicast on connections with enhanced privacy requirements for the privacy-related reasons covered in .

Transport Parameters Support for multicast extensions in a client is advertised by means of QUIC transport parameters:

name: multicast_server_support (TBD - experiments use 0xff3e808)
name: multicast_client_params (TBD - experiments use 0xff3e800)

If a multicast_server_support transport parameter is not included, clients MUST NOT send any frames defined in this document. If a multicast_client_params transport parameter is not included, servers MUST NOT send any frames defined in this document. The multicast_server_support parameter is a 0-length value. Presence indicates that multicast-capable clients MAY send frames defined in this document, and SHOULD send MC_LIMITS () frames as appropriate when their capabilities or client-side limitations change. The multicast_client_params parameter has the structure shown below in .

multicast_client_params Format The Reserved, IPv4 Channels Allowed, IPv6 Channels Allowed, Max Aggregate Rate, Max Channel IDs and Max Joined Count fields are identical to their analogous fields in the MC_LIMITS frame () and hold the initial values. A server MUST NOT send MC_ANNOUNCE () frames with addresses using an IP Family that is not allowed according to the IPv4 and IPv6 Channels Allowed fields in the multicast_client_params, unless and until a later MC_LIMITS () frame adds permission for a different address family. It is valid for both IPv4 Channels Allowed and IPv6 Channels Allowed to be set to zero in the transport parameter. This indicates that the client supports this extension but does not initially permit joining multicast channels using either IP address family. The Hash Algorithm Count field contains the number of entries in the Hash Algorithm List field. The Cipher Suite Count field contains the number of entries in the Cipher Suite List field. An endpoint MUST treat multicast_client_params as malformed if the parameter length is inconsistent with these counts. The Cipher Suite List field is in order of preference, with the most preferred value first. It contains values from the "TLS Cipher Suites" registry in the "Transport Layer Security (TLS) Parameters" registry group . It lists the cipher suites the client is willing to use for multicast channel packet protection and header protection. A server MUST NOT send an MC_ANNOUNCE to this client for any channels using unsupported cipher suites. If the server does send an MC_ANNOUNCE with an unsupported cipher suite, the client SHOULD treat it as a connection error of type MC_EXTENSION_ERROR. The Hash Algorithm List field is in order of preference, with the most preferred value first. It lists the hash algorithms the client is willing to use to check integrity of data in multicast channels. It contains values from the Named Information Hash Algorithm Registry:

https://www.iana.org/assignments/named-information/named-information.xhtml#hash-alg

A server MUST NOT send an MC_ANNOUNCE to this client for a channel using a hash algorithm that was not included in the Hash Algorithm List. If the server sends such an MC_ANNOUNCE, the client SHOULD treat it as a connection error of type MC_EXTENSION_ERROR.

Extension Overview A client has the option of refusal and the power to impose upper bound maxima on several resources (see ), but otherwise its join status for all multicast channels is entirely managed by the server.

A client MUST NOT join a channel without receiving instructions from a server to do so.
A client MUST leave joined channels when instructed by the server to do so.
A client MAY leave channels or refuse to join channels, regardless of instructions from the server.

Channel Management The client tells its server about some restrictions on resources that it is capable of processing with the initial values in the multicast_client_params transport parameter () and later can update these limits with MC_LIMITS frames. Servers ensure the set of channels the client is currently requested to join remains within these advertised client limits as covered in . The server asks the client to join channels with MC_JOIN () frames and to leave channels with MC_LEAVE () frames. The server uses the MC_ANNOUNCE () frame before any join or leave frames for the channel to describe the channel properties to the client, including values the client can use to ensure the server's requests remain within the limits it has sent to the server, as well as the secrets necessary to decode the headers of packets in the channel. Sending an MC_ANNOUNCE before an MC_JOIN ensures the client can establish the necessary state required to join and retire any connection IDs that might collide with channel IDs. MC_KEY frames provide the secrets necessary to decode the payload of packets in the channel. shows the states a channel has from the client's point of view. Joining a channel after receiving an MC_JOIN frame is OPTIONAL for clients. Client responses to join, leave, and retire requests are described in . The server ensures that in aggregate, all channels that the client has currently been asked to join and that the client has not left, declined to join or retired fit within the limits indicated by the initial values in the transport parameter or last MC_LIMITS () frame the server received. This extension does not define an application-layer catalogue or content-selection protocol. Application protocols determine what content, service, program, representation, or other application context is relevant to a connection. The multicast extension merely provides a transport mechanism by which the server can describe channels that are available in that context and request that the client join or leave them. A server is not expected to announce every multicast channel that it operates to every client. The set of channels announced to any specific client can be limited by application-layer state, prior application requests, receiver limits, server policy, or any combination of these. For example, a server may provide a video stream in different resolutions, with each available resolution being carried in a different multicast channel. The server would only announce the channel that offers the best available quality while remaining under the limits set by the client. In this case, this could happen without any involvement of the client-side application.

States a channel from the client's point of view. | | Receive MC_ANNOUNCE and/or MC_KEY ^ | | | | | | Receive MC_JOIN (and v v | unable to join) +----------+ |<--------------------* | | unjoined | Receive MC_RETIRE --------------------->| *------------------------>| ^ +----*-----+ | | | Receive MC_JOIN | | | (and able to join) | | | | | v v | +----------+ +---------+ | Receive MC_LEAVE | | | | | (or error case) | joined | Receive MC_RETIRE | retired | |<--------------------* *------------------->| | +----------+ +---------+ *: Each transition except the initial receiving of MC_ANNOUNCE and MC_KEY frames causes the client to send an MC_STATE frame describing the state transition (for LEFT or DECLINED_JOIN, this includes a reason for the transition). "able to join" means: - Both MC_KEY and MC_ANNOUNCE have been received - Result will be within latest advertised client limits - Nothing preventing a join is active (e.g. a hold-down timer, administrative blocking, etc.) ]]> When the server has asked the client to join a channel and has not received any MC_STATE frames with State DECLINED_JOIN, LEFT, or RETIRED, it also sends MC_INTEGRITY frames () to enable the client to verify packet integrity before processing channel packets. A client MUST NOT decode a channel packet unless it has received an applicable MC_ANNOUNCE () frame and an applicable MC_KEY () frame for the channel, and has received a matching packet hash in an MC_INTEGRITY frame for that packet. shows the frames that are being exchanged about and over a channel during the lifetime of an example channel.

Example flow of frames for a channel. Frames in square brackets are sent over multicast. MC_ANNOUNCE MC_KEY <---- MC_JOIN MC_STATE(JOINED) ---> MC_INTEGRITY <---- [STREAM(...)] MC_ACK ---> ... ... <---- MC_KEY ... MC_LIMITS ---> <---- MC_LEAVE MC_STATE(LEFT) ---> <---- MC_JOIN MC_STATE(JOINED) ---> MC_INTEGRITY <---- [STREAM(...)] MC_ACK ---> ... ... <---- MC_LEAVE MC_STATE(LEFT) ---> <---- MC_RETIRE MC_STATE(RETIRED) ---> ]]> TODO: incorporate server-side state diagram and explanation, latest proposed sketch at https://github.com/GrumpyOldTroll/draft-jholland-quic-multicast/issues/62

Channel Key Management Multicast channel keys are channel-scoped rather than connection-scoped. The same channel secret will be shared by receivers on many different QUIC connections, and clients can join a channel after it has started or leave before it ends, or miss key updates while not joined. The regular QUIC Key Update mechanism () can not cover such cases. A server uses MC_KEY frames () to provide authorized receivers with channel packet protection secrets, to identify the packet number from which each secret applies, and to rotate channel keys independently of the unicast connection's 1-RTT keys. Each channel key generation is identified by a Key Sequence Number. A server MUST generate continuous Key Sequence Number values for each channel. The first MC_KEY frame for a channel MAY use any non-zero Key Sequence Number. An MC_KEY frame MUST NOT use Key Sequence Number 0. A client that is not joined to a channel might not receive every MC_KEY frame for that channel, and can therefore observe gaps in Key Sequence Number values. While joined, a client can also observe gaps due to packet loss or reordering. Secrets with even-valued Key Sequence Numbers have a Key Phase of 0 in channel 1-RTT packets, and secrets with odd-valued Key Sequence Numbers have a Key Phase of 1. A client MUST NOT attempt to decrypt a channel packet unless it has an applicable channel secret for that packet number and Key Phase. If no applicable secret is available, the client MAY retain the packet briefly in case an applicable MC_KEY frame arrives later, subject to the client's buffering limits. The client MUST NOT acknowledge such a packet in an MC_ACK frame unless it later becomes able to process the packet. If a joined client receives channel packets for which it has no applicable channel secret, and an applicable MC_KEY frame does not arrive before the client discards those packets, the client SHOULD leave the channel and send an MC_STATE frame with State LEFT and Reason Code UNSYNCHRONIZED_PROPERTIES. If a client receives two different secrets with the same Channel ID and Key Sequence Number, it SHOULD close the connection with a connection error of type MC_EXTENSION_ERROR. Servers SHOULD update channel secrets regularly. To limit the exposure of data after receivers have left a channel or lost authorization, servers SHOULD periodically send key updates using only unicast. Clients MUST delete old channel secrets and the keys derived from them after receiving a newer applicable MC_KEY frame. Deleting old keys prevents later compromise of a client from discovering an otherwise uncompromised key, thus improving the chances of achieving forward secrecy for data sent before a key rotation. A client MAY retain an old secret briefly to process reordered or delayed packets. For this experimental specification, it is RECOMMENDED that clients delete old secrets 10 seconds after receiving a newer secret, or after 3 seconds without receiving any packet that uses the old secret, whichever is shorter. Clients MUST NOT retain old channel secrets for more than 60 seconds after receiving a newer applicable MC_KEY frame. The delay values for this specification are somewhat arbitrary and allow for implementation-dependent experimentation. One of the target discoveries for experimental evaluation is to determine good default delay values to use, and to understand whether there are use cases that would benefit from a negotiation between server and client to determine the delays to use dynamically. (A poor delay choice results in either overhead from dropping packets instead of decoding them with old keys for too short a delay or in extra forward secrecy exposure time for too long a delay, and the purpose of the delays are to bound the forward secrecy exposure without inducing unreasonable overhead.)

Client Response A client reports how it has responded to server requests to join, leave, or retire channels using MC_STATE () frames. MC_STATE frames are sent whenever the client's state for the channel changes. If a client joins a channel after receiving an MC_JOIN frame, it MUST send an MC_STATE frame with State JOINED. If the client does not join, it MUST send an MC_STATE frame with State DECLINED_JOIN and an appropriate Reason Code. Declining an MC_JOIN request is a state change for the channel even though the client does not join the multicast channel. After leaving a channel in response to an MC_LEAVE frame, a client MUST send an MC_STATE frame with State LEFT and Reason Code REQUESTED_BY_SERVER. After retiring a channel in response to an MC_RETIRE frame, a client MUST send an MC_STATE frame with State RETIRED and Reason Code REQUESTED_BY_SERVER. Retiring a joined channel also leaves that channel; the client does not send a separate MC_STATE frame with State LEFT.

Acknowledging Channel Packets Clients that receive and decode packets on a multicast channel acknowledge those packets on the unicast connection using MC_ACK () frames. An MC_ACK frame acknowledges packets only in the packet number space of the channel identified by its Channel ID. MC_ACK generation is controlled by the MC_ACK policy advertised in the MC_ANNOUNCE frame for the corresponding channel. The Max ACK Delay, Ack-Eliciting Threshold, and Reordering Threshold fields of MC_ANNOUNCE apply separately to each multicast channel packet number space. A client SHOULD send an MC_ACK for a channel when either:

the number of ack-eliciting channel packets received since the last MC_ACK for that channel is greater than the Ack-Eliciting Threshold; or
Max ACK Delay has elapsed and at least one ack-eliciting channel packet has been received since the last MC_ACK for that channel.

A client MAY send an MC_ACK earlier than required by these rules. A client MUST send the first MC_ACK for a newly joined channel without intentional delay after receiving and processing an ack-eliciting packet on that channel. A client SHOULD use the Reordering Threshold to determine when receipt of out-of-order channel packets causes an immediate MC_ACK, following the behavior defined for ACK_FREQUENCY (), applied to the channel packet number space. All channel packets that require acknowledgment MUST be acknowledged at least once. When MC_ACK frames are sent less frequently, clients need to retain ACK range information long enough to avoid permanently omitting acknowledgment of received channel packets. ACK_FREQUENCY and IMMEDIATE_ACK frames defined by do not affect MC_ACK generation unless a future extension explicitly defines such behavior. The ACK policy for MC_ACK frames is instead defined by the MC_ANNOUNCE frame for the corresponding channel. After sending ack-eliciting channel packets, a server can determine that a client is receiving packets for a multicast channel when it receives MC_ACK frames for that channel. It is up to the server to decide how long to wait before treating the absence of MC_ACK frames as evidence that the client is not receiving packets on the channel, and to take appropriate steps such as sending an MC_LEAVE frame. A client that is willing to remain joined to a channel SHOULD NOT leave the channel solely because it receives no channel data for an extended period. This enables multicast-capable networks to perform popularity-based admission control for multicast channels.

Data Carried in Channels Data transmitted in a multicast channel is encrypted with symmetric keys so that on-path observers without access to these keys cannot decode the data. However, since potentially many receivers receive identical packets and identical keys for the multicast channel and some receivers might be malicious, the packets are also protected by MC_INTEGRITY () frames transmitted over a separate integrity-protected path. A client MUST NOT decode packets on a multicast channel for which it has not received a matching hash in an MC_INTEGRITY frame over a different integrity-protected communication path. The different path can be either the unicast connection or another multicast channel with packets that were verified with an earlier MC_INTEGRITY frame. Note that MC_INTEGRITY frames MAY be carried in packets on multicast channels, however such packets will not be accepted unless another accepted MC_INTEGRITY frame contains its packet hash. Hashes of packets containing hashes of other packets can thus form a Merkle tree with a root that is carried in the unicast connection. See for a more complete overview of the security issues involved here. A client is not required to buffer unauthenticated packets indefinitely. A client MAY discard unauthenticated packets when doing so is necessary to bound memory use. A client SHOULD use the channel's Max Rate and Max Authentication Delay values to size the buffer it is willing to allocate for unauthenticated packets on that channel. If a packet has not become authenticated within Max Authentication Delay after it was received, the client MAY discard the packet. If this occurs persistently, or if the number of unauthenticated packets exceeds the client's local buffering capacity, the client SHOULD leave the channel and send MC_STATE(LEFT) with reason AUTHENTICATION_DELAY_EXCEEDED. A client MAY decline to join a channel, or MAY leave a joined channel, if the Max Authentication Delay value is larger than the client is willing to support.

Stream Processing Stream IDs in channels are restricted to unidirectional server initiated streams, or those with the least significant 2 bits of the stream ID equal to 3 (see ). Multicast channels do not define independent QUIC stream ID spaces. STREAM frames received on a multicast channel are processed as regular STREAM frames for the associated QUIC connection. Stream IDs are therefore allocated from the same connection-wide stream ID space, whether stream data is sent on the unicast path or on one or more multicast channels. A server that sends STREAM frames on one or more multicast channels associated with the same QUIC connection is responsible for coordinating stream ID allocation across the unicast paths of all potential multicast receivers, and across all relevant multicast channels. Using disjoint stream ID ranges for different channels is one possible implementation strategy, but is not required by this specification. When a channel contains, or may soon contain, streams with IDs that exceed the stream ID limit implied by the client's server-initiated unidirectional MAX_STREAMS value, the server MUST NOT send MC_JOIN to instruct the client to join that channel and SHOULD send a STREAMS_BLOCKED frame, as described in Sections and of . If the client is already joined to a channel that carries streams that exceed or will soon exceed the client's unidirectional MAX_STREAMS, the server SHOULD send an MC_LEAVE frame. If a client receives a STREAM frame with an ID above its MAX_STREAMS on a channel, the client MAY increase its unidirectional MAX_STREAMS to a value greater than the new ID and send an update to the server, otherwise it MUST drop the packet and leave the channel with reason "MAX_STREAMS_EXCEEDED". Since clients can join later than a channel began, it is RECOMMENDED that clients supporting the multicast extensions to QUIC be prepared to handle stream IDs that do not begin at early values, since by the time a client joins a channel in progress the stream ID count might have been increasing for a long time. Clients should therefore begin with a high initial_max_streams_uni or send an early MAX_STREAMS type 0x13 value (see ) with a high limit. Clients MAY use the maximum 2^60 for this high initial limit, but the specific choice is implementation-dependent. The same stream ID may be used in both one or more multicast channels and the unicast connection. As described in , stream data received multiple times for the same offset MUST be identical, including when the data is received on different multicast channels or on both multicast and unicast paths. If it's not identical it MAY be treated as a connection error of type MC_EXTENSION_ERROR.

Flow Control The values used for unicast flow control cannot be used to limit the transmission rate of a multicast channel because a single client with a low MAX_STREAM_DATA or MAX_DATA value that did not acknowledge receipt could block many other receivers if the servers had to ensure that channels responded to each client's limits. Instead of terminating a connection if its MAX_DATA gets exceeded (as described in ), a client must be able to robustly handle multicast packets that would exceed its MAX_DATA without aborting the connection, either by increasing its MAX_DATA as needed to keep up with received multicast packets or by dropping the packet and leaving the channel (resulting in unicast fallback). If a server detects that a client's MAX_DATA is about to be exceeded, it MUST instruct the client to leave channels to prevent any further MAX_DATA violations. Instead, clients advertise resource limits via MC_LIMITS () frames and their initial values from the transport parameter (). The server is responsible for keeping the client within its advertised limits, by ensuring via MC_JOIN and MC_LEAVE frames that the set of channels the client is asked to be joined to will not, in aggregate, exceed the client's advertised limits. The server also advertises the expected maxima of the values that can contribute toward client resource limits within a channel in an MC_ANNOUNCE () frame, and the client also ensures that the set of channels it's joined to does not exceed its limits, according to the advertised values. The client also monitors the packets received to ensure that channels don't exceed their advertised values, and leaves channels that do. The sequence numbers carried in MC_JOIN allow the client to determine whether a join request is based on state that is synchronized between the client and server. If the server asks the client to join a channel that would violate the client's limits, and the MC_JOIN frame contains the client's current MC_LIMITS Sequence Number, the client SHOULD send an MC_STATE frame () with State DECLINED_JOIN and Reason Code PROPERTY_VIOLATION. If the server asks the client to join a channel that would violate the client's current limits, but the MC_JOIN frame contains an older MC_LIMITS Sequence Number, the client SHOULD send an MC_STATE frame with State DECLINED_JOIN and Reason Code UNSYNCHRONIZED_PROPERTIES. If the MC_JOIN frame refers to an MC_KEY Sequence Number that the client has not yet received, the client SHOULD send an MC_STATE frame with State DECLINED_JOIN and Reason Code UNSYNCHRONIZED_PROPERTIES. If the actual contents sent in the channel violate the advertised properties from MC_ANNOUNCE, clients SHOULD leave the channel and send an MC_STATE frame with State LEFT and Reason Code LIMIT_VIOLATION. After processing an MC_LIMITS frame, if the set of channels that the client is currently requested to join no longer fits within the client's current limits, the server MUST send MC_LEAVE frames for one or more channels. For this purpose, the requested set includes channels for which the server has sent MC_JOIN and has not received an MC_STATE frame with State LEFT, DECLINED_JOIN, or RETIRED. The server is responsible for selecting which channels to leave so that the remaining requested set fits within the client's current limits. The server MUST NOT send new MC_JOIN frames that would cause the requested set to violate the client's current limits.

Congestion Control Both the server and the client perform congestion control operations, so that according to the guidelines in , mechanisms for both feedback-based and receiver-driven styles of congestion control are present and operational. All frames defined by this document other than MC_ACK are ack-eliciting. Packets containing those frames are considered in-flight and count toward congestion control limits as described in . MC_ACK frames are treated the same as ACK frames for congestion control and loss recovery purposes and do not make a packet ack-eliciting and thus a packet containing only them does not count as in-flight. The server maintains a full view of the traffic received by the client via the MC_ACK () frames and ACK frames it receives, and can detect loss experienced by the client. Under sustained persistent loss that exceeds server-configured thresholds, the server SHOULD instruct the client to leave channels as appropriate to avoid having the client continue to see sustained persistent loss. Under sustained persistent loss that exceeds client-configured thresholds, the client SHOULD reduce its Max Rate and tell the server via MC_LIMITS frames, which also will result in the server instructing the client to leave channels until the client's aggregate rate is below its advertised Max Rate. Under a higher threshold of sustained persistent loss, the client also SHOULD leave channels, using an MC_STATE(LEFT) frame with the "HIGH_LOSS" reason, as well as reducing the Max Rate in MC_LIMITS. The unicast connection's congestion control is unaffected. However a few potential interactions with the unicast connection are worth highlighting:

if the client notices high loss on the unicast connection while multicast channel packets are arriving, the client MAY leave channels with reason "HIGH_LOSS".
if the client notices congestion from unicast this MAY also drive reductions in the client's Max Rate, and a lack of unicast congestion under unicast load MAY also drive increases to the client's Max Rate (along with an updated MC_LIMITS frame).

Hybrid multicast-unicast congestion control is still an experimental research topic. Implementations SHOULD follow the guidelines given in under the assumption that applications using QUIC multicast will operate as Bulk-Transfer applications.

Data Integrity TODO: import the explanation for why extra integrity protection is necessary (many client have the shared key, so AEAD doesn't provide authentication against other valid clients on its own, since the same key is given to multiple clients and as the client count grows so does the chance that at least one client is controlled by an attacker.)

Packet Hashes TODO: explanation and example for how to calculate the packet hash. Note that the hash is on the encrypted packet to avoid leaking data about the encrypted contents to those who can see a hash but not the key. (This approach also may help make better use of by making it possible to generate the same hashes for use in both AMBI and QUIC MC_INTEGRITY frames.)

Recovery TODO: Articulate key differences with . The main known difference is that servers might not be running on the same devices that are sending the channel packets, therefore the RTT for channel packets might use an estimated send time that can vary according to the clock synchronization among servers and the deployment and implementation details of how the servers find out the sending timestamps of channel packets. Experience-based guidance on the recovery timing estimates is one anticipated outcome of experimenting with deployments of this experimental extension. All frames defined in this document except MC_ACK are ack-eliciting and are retransmitted until acknowledged to provide reliable, though possibly out of order, delivery. Note that recovery MAY be achieved either by retransmitting frame data that was lost and needs reliable transport either by sending the frame data on the unicast connection or by coordinating to cause an aggregated retransmission of widely dropped data on a multicast channel, at the server's discretion. However, the server in each connection is responsible for ensuring that any necessary server-to-client frame data lost by a multicast channel packet loss ultimately arrives at the client. To minimize the amount of additional packets sent on a multicast channel when retransmitting frames, the server SHOULD use Forward Erasure Correction (FEC) techniques following guidelines from . Instead of retransmitting the frames directly, the server sends FEC repair packets on the multicast channel. As such, an individual repair packet can recover different losses on distinct clients, thus minimizing the amount of data sent on a multicast channel. The scheduling of these repair packets is implementation-dependent and hence out of scope of this document.

Connection Termination Termination of the unicast connection behaves as described in , with the following notable differences:

On the client side, if the associated unicast connection is terminated for any reason, including receipt or transmission of a CONNECTION_CLOSE frame, all multicast channel state associated with that connection MUST be discarded. If the client is joined to any multicast channels for that connection, it MUST leave those channels. Connection termination does not require the client to send MC_STATE frames for the affected channels. After the unicast connection is terminated, MC_STATE frames cannot be delivered on that connection.
The server MUST NOT rely on receiving per-channel leave or retire state for cleanup.
For determining the liveness of a connection, the client MUST only consider packets received on the unicast connection. Any packets received on a multicast channel MUST NOT be used to reset a timer checking if a potentially specified max_idle_timeout has been reached. If the unicast connection becomes idle, as described in , the client MUST terminate the connection as described above.

Stateless Reset As clients can unilaterally stop the delivery of multicast packets by leaving the relevant (S,G), channels do not need stateless reset tokens. Clients therefore do not share the stateless reset tokens of channels with the server. Instead, if an endpoint receives packets addressed to an (S,G) that it can not associate with any existing channel, it MAY take the necessary steps to prevent the reception of further such packets, without the need to signal to the server that it should stop sending. If a server or client detect a stateless reset for a channel, they MUST ignore it.

Connection Migration If the unicast connection migrated, e.g. due to a change of the NAT binding or because the UE has changed to a different network, the client properties might change. For example, the client might switch from a network that supports both IPv6 and IPv4 multicast to a network that only supports IPv4. As such, it MUST immediately send an MC_LIMITS frame after it has noticed that it migrated. The client MAY rejoin any previously joined channels, if its limits still allow it to. It MUST send MC_STATE(LEFT) frames with reason LIMIT_VIOLATION for any channels it does not rejoin. The server SHOULD take notice of migrating clients as the delay that is being caused by rejoining a multicast group can lead to exceeding the expected MAX_ACK_DELAY, which a server might interpret as a loss of multicast connectivity. Instead, the server SHOULD treat all multicast channels of a client whose unicast connection just migrated as if it had just joined these channels initially and allow for ample time before expecting the first MC_ACK frames.

New Frames

MC_ANNOUNCE Once a server learns that a client supports multicast through its transport parameters, it can send one or multiple MC_ANNOUNCE frames (type=TBD-11..TBD-12) to share information about available channels with the client. The MC_ANNOUNCE frame contains the properties of a channel that do not change during its lifetime. MC_ANNOUNCE frames are formatted as shown in .

MC_ANNOUNCE Frame Format Frames of type TBD-11 are used for IPv4 and both Source and Group address are 32 bits long. Frames of type TBD-12 are used for IPv6 and both Source and Group address are 128 bits long. MC_ANNOUNCE frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID of the channel that is getting announced.
Source IP: The IP Address of the source of the (S,G) for the channel. Either a 32-bit IPv4 address or a 128-bit IPv6 address, as indicated by the frame type (TBD-11 indicates IPv4, TBD-12 indicates IPv6).
Group IP: The IP Address of the group of the (S,G) for the channel. Either a 32-bit IPv4 address or a 128-bit IPv6 address, as indicated by the frame type (TBD-11 indicates IPv4, TBD-12 indicates IPv6). This address MUST be a valid SSM destination address as specified in .
UDP Port: The 16-bit UDP Port of traffic for the channel.
Cipher Suite: A value from the "TLS Cipher Suites" registry in the "Transport Layer Security (TLS) Parameters" registry group . The cipher suite determines the AEAD algorithm used for packet protection, the hash function used for key derivation, and the header protection algorithm for channel packets, as described in . The value MUST match a value provided in the Cipher Suite List of the multicast_client_params transport parameter; see .
Header Secret Length: Provides the length of the Header Secret field.
Header Secret: A secret used to derive the header protection key for channel packets. The header protection key is derived from this secret using the cipher suite identified by the Cipher Suite field and the "quic hp" label, as described in . The header protection algorithm is the algorithm defined by for the AEAD associated with the Cipher Suite used by this channel. The Header Secret is not used to derive packet protection keys or IVs.
Integrity Hash Algorithm: The hash algorithm used in MC_INTEGRITY frames.
- Author's Note: Several candidate IANA registries, not sure which one to use? Some have only text for some possibly useful values. For now we use the first of these:
  - https://www.iana.org/assignments/named-information/named-information.xhtml#hash-alg
  - https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-18
  - (text-only): https://www.iana.org/assignments/hash-function-text-names/hash-function-text-names.xhtml
Max Rate: The maximum rate in Kibps of the payload data for this channel. Channel data MUST NOT exceed this rate over any 5s window, if it does clients SHOULD leave the channel with reason "MAX_RATE_EXCEEDED".
Max Authentication Delay: The maximum time, in microseconds, that a client is expected to buffer a multicast channel packet before the packet becomes authenticated by an accepted MC_INTEGRITY frame. The delay is measured from receipt of the channel packet until the packet becomes authenticated. See .
Max ACK Delay: The maximum amount of time, in microseconds, that a client can intentionally delay sending an MC_ACK frame for ack-eliciting packets received on this channel. This value is used similarly to max_ack_delay (), but applies only to MC_ACK frames for this channel. A client SHOULD NOT intentionally delay an MC_ACK for this channel beyond this value.
Ack-Eliciting Threshold: The maximum number of ack-eliciting channel packets that a client can receive on this channel without sending an MC_ACK. This field has the same semantics as the Ack-Eliciting Threshold field of ACK_FREQUENCY (), except that it applies only to MC_ACK frame generation for this channel. A value of 0 requests that every ack-eliciting channel packet be acknowledged immediately. A value of 1 corresponds to the default QUIC behavior of acknowledging after receiving two ack-eliciting packets.
Reordering Threshold: A packet-count threshold used to determine when receipt of out-of-order channel packets causes an immediate MC_ACK. This field has the same semantics as the Reordering Threshold field of ACK_FREQUENCY (), except that it applies only to this channel's packet number space and to MC_ACK frames for this channel.

A client MUST NOT use the channel ID included in an MC_ANNOUNCE frame as a connection ID for the unicast connection. If it is already in use, the client SHOULD retire it as soon as possible. As the server knows which connection IDs are in use by the client, it MUST wait with the sending of an MC_JOIN frame until the channel ID associated with it has been retired by the client. If a client receives an MC_ANNOUNCE frame with a Group IP that is not within the SSM destination address range as outlined in , it SHOULD close the connection with a connection error of type MC_EXTENSION_ERROR. As all the properties in MC_ANNOUNCE frames are immutable during the lifetime of a channel, a server SHOULD NOT send an MC_ANNOUNCE frame for the same channel more than once to each client except as needed for recovery. A server SHOULD send an MC_ANNOUNCE frame for a channel before sending an MC_KEY frame for that channel. A server MUST NOT send an MC_JOIN frame for a channel unless it has sent, or is sending in the same packet, both an applicable MC_ANNOUNCE frame and an applicable MC_KEY frame for that channel.

MC_KEY An MC_KEY frame (type=TBD-01) is sent from server to client, either with the unicast connection or in an existing joined multicast channel. It carries a channel packet protection secret and identifies the packet number from which that secret applies. A server SHOULD NOT send MC_KEY frames for channels except those the client has joined or will be imminently asked to join. MC_KEY frames are formatted as shown in .

MC_KEY Frame Format MC_KEY frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel associated with this frame.
Key Sequence Number: The key generation identified by this frame. This value MUST NOT be 0.
From Packet Number: The first channel packet number for which the secret in this frame is applicable. The secret applies to channel packets with packet numbers greater than or equal to From Packet Number and with the Key Phase corresponding to this Key Sequence Number as described in , until superseded by an MC_KEY frame for the same channel with a higher Key Sequence Number. When the Key Sequence Number increases, the From Packet Number MUST increase.
Secret Length: Provides the length of the secret field.
Secret: A channel packet protection secret. Packet protection keys and IVs for channel packets are derived from this secret using the cipher suite identified in the corresponding MC_ANNOUNCE frame and the "quic key" and "quic iv" labels, as described in . This secret is not used to derive the header protection key.

MC_JOIN An MC_JOIN frame (type TBD-02) is sent from server to client and requests that the client join the channel identified by Channel ID. The client uses the channel's transport addresses and properties from the corresponding MC_ANNOUNCE frame. The MC_KEY Sequence Number field identifies the channel key generation that the server expects the client to have available when processing channel packets. A client cannot join a multicast channel without first receiving an applicable MC_ANNOUNCE frame and an applicable MC_KEY frame for that channel. MC_JOIN frames are formatted as shown in .

MC_JOIN Frame Format MC_JOIN frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel that the client is requested to join.
MC_LIMITS Sequence Number: The most recent Client Limits Sequence Number processed by the server when constructing this join request. A value of 0 indicates that no MC_LIMITS frames have been processed by the server.
MC_STATE Sequence Number: The most recent Client Channel State Sequence Number for this channel processed by the server when constructing this join request. A value of 0 indicates that no MC_STATE frames have been processed by the server.
MC_KEY Sequence Number: The Key Sequence Number for the channel key generation that the server expects the client to use when joining the channel. This field MUST NOT be 0; a client that receives an MC_JOIN with an MC_KEY Sequence Number of 0 MUST treat this as a connection error of type MC_EXTENSION_ERROR.

If a client receives an MC_JOIN for a channel for which it has not received both an applicable MC_ANNOUNCE frame and an applicable MC_KEY frame, it MUST send an MC_STATE frame with State DECLINED_JOIN and Reason Code UNSYNCHRONIZED_PROPERTIES. Client responses to MC_JOIN are described in .

MC_LEAVE An MC_LEAVE frame (type=TBD-03) is sent by a server to request that the client leave the given channel. MC_LEAVE does not retire the channel and the server can later send another MC_JOIN for the same channel as long as it is not retired. MC_LEAVE frames are formatted as shown in .

MC_LEAVE Frame Format MC_LEAVE frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel that the client is requested to leave.
MC_STATE Sequence Number: The most recent Client Channel State Sequence Number for this channel processed by the server when constructing this leave request. A value of 0 indicates that no MC_STATE frames have been processed by the server. This value allows the client to ignore leave requests that are based on stale client channel state.

A client that receives an MC_LEAVE for a channel that it has already left, declined to join, or retired MUST ignore the frame. A client that has received an MC_JOIN or MC_LEAVE for the same Channel ID with a greater MC_STATE Sequence Number MUST ignore the MC_LEAVE frame. Otherwise, the client MUST leave the channel immediately. Client responses to MC_LEAVE are described in .

MC_INTEGRITY MC_INTEGRITY frames (types TBD-04 and TBD-05; experiments use 0xff3e804 and 0xff3e805) are sent from server to client and are used to convey packet hashes for validating the integrity of packets received on the multicast channel. MC_INTEGRITY frames are formatted as shown in .

MC_INTEGRITY Frame Format MC_INTEGRITY frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel whose packet hashes are being provided.
Packet Number Start: The packet number of the first channel packet covered by the Packet Hashes field.
Packet Hashes Length: The length in bytes of the Packet Hashes field. This field is present only when the frame type is TBD-05.
Packet Hashes: A sequence of packet hashes. The first hash corresponds to the packet with packet number Packet Number Start in the packet number space of the channel identified by Channel ID. Each subsequent hash corresponds to the next packet number in that packet number space.

For frames of type TBD-04, Packet Hashes Length is not present and the Packet Hashes field extends to the end of the packet. Therefore, an MC_INTEGRITY frame of type TBD-04 MUST be the final frame in the packet. Each hash has the length determined by the Integrity Hash Algorithm in the corresponding MC_ANNOUNCE frame. The Packet Hashes field MUST contain a non-zero integer multiple of the hash length for the channel. A client that receives an MC_INTEGRITY frame whose Packet Hashes field is empty, or whose Packet Hashes field length is not an integer multiple of the hash length for the channel, MUST treat this as a connection error of type MC_EXTENSION_ERROR. Packet hashes are calculated as described in .

MC_ACK MC_ACK frames (types TBD-06 and TBD-07; experiments use 0xff3e806..0xff3e807) are sent by a client to acknowledge packets received on a multicast channel. MC_ACK extends the ACK frame defined by by adding a Channel ID. Frames of type TBD-07 also contain ECN counts. MC_ACK frames are formatted as shown in .

MC_ACK Frame Format MC_ACK frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel whose packets are being acknowledged.
Largest Acknowledged: The largest packet number being acknowledged in the packet number space of the channel identified by Channel ID.
ACK Delay: The time delta between receipt of the largest acknowledged channel packet and transmission of this MC_ACK frame, encoded as described for ACK frames in .
ACK Range Count, First ACK Range, and ACK Range: These fields have the same encoding and semantics as the corresponding fields of ACK frames defined by , except that packet numbers refer to the packet number space of the channel identified by Channel ID.
ECN Counts: ECN counts for packets received on the channel, encoded as described for ACK frames with ECN counts in . This field is present only when the frame type is TBD-07.

MC_LIMITS MC_LIMITS frames are sent by a client to update the multicast channel limits that were initially provided in the multicast_client_params transport parameter (). The server applies these limits as described in . Each processed MC_LIMITS frame replaces the previously active client limits. MC_LIMITS frames are formatted as shown in .

MC_LIMITS Frame Format MC_LIMITS frames contain the following fields:

Client Limits Sequence Number: The sequence number of this limits update. Before the first MC_LIMITS frame from the client, the implicit Client Limits Sequence Number is 0. The first MC_LIMITS frame sent by the client MUST use Client Limits Sequence Number 1, and each subsequent MC_LIMITS frame MUST increase the Client Limits Sequence Number by 1. An MC_LIMITS frame MUST NOT use Client Limits Sequence Number 0.
Reserved: Reserved bits for future use. These bits MUST be set to 0 by the client and MUST be ignored by the server.
IPv4 Channels Allowed: A 1-bit field set to 1 if the client currently permits the server to request joins for IPv4 channels, and set to 0 otherwise.
IPv6 Channels Allowed: A 1-bit field set to 1 if the client currently permits the server to request joins for IPv6 channels, and set to 0 otherwise.
Max Aggregate Rate: The maximum aggregate rate, in Kibps, allowed across all channels that the client is concurrently requested to join.
Max Channel IDs: The maximum number of channel IDs for which the client retains channel state. Retired channel IDs do not count against this value.
Max Joined Count: The maximum number of channels that the client can be asked to join concurrently.

A server MUST ignore an MC_LIMITS frame whose Client Limits Sequence Number is less than or equal to the largest Client Limits Sequence Number it has already processed on the connection.

MC_RETIRE An MC_RETIRE frame retires a channel by Channel ID and causes the client to discard any state associated with that channel. MC_RETIRE frames are formatted as shown in .

MC_RETIRE Frame Format MC_RETIRE frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel that the client is requested to retire.

A client that processes an MC_RETIRE frame MUST retire the channel immediately and discard all state associated with that channel. Client responses to MC_RETIRE are described in .

MC_STATE MC_STATE frames are sent from client to server to report the client's state for a multicast channel. MC_STATE frames are formatted as shown in .

MC_STATE Frame Format MC_STATE frames contain the following fields:

ID Length: The length in bytes of the Channel ID field.
Channel ID: The channel ID for the channel whose state is being reported.
Client Channel State Sequence Number: The sequence number of this state update for the channel. Before the first MC_STATE frame for a channel, the implicit Client Channel State Sequence Number is 0. The first MC_STATE frame sent by the client for a channel MUST use Client Channel State Sequence Number 1, and each subsequent MC_STATE frame for that channel MUST increase the Client Channel State Sequence Number by 1. An MC_STATE frame MUST NOT use Client Channel State Sequence Number 0.
State: The client channel state being reported. The following values are defined:
- 0x1: LEFT
- 0x2: DECLINED_JOIN
- 0x3: JOINED
- 0x4: RETIRED
Reason Code: A code describing why the reported state was reached.
Reason Phrase Length: The length of the Reason Phrase field.
Reason Phrase: Optional diagnostic text describing the state change. This field MUST NOT affect protocol behavior.

A server MUST ignore an MC_STATE frame whose Client Channel State Sequence Number is less than or equal to the largest Client Channel State Sequence Number it has already processed for that channel. If a server receives an MC_STATE frame with an undefined State value, it SHOULD close the connection with a connection error of type MC_EXTENSION_ERROR. Frames of type TBD-0b use reason codes defined by this specification. These reason codes describe channel state changes caused by QUIC multicast transport behavior, local transport policy, or server requests. Frames of type TBD-0c carry application-defined Reason Code values. As with application protocol error codes in , the application protocol using this extension defines the semantics and allocation policy for these values. If State is JOINED or RETIRED, the frame type MUST be TBD-0b and the Reason Code MUST be REQUESTED_BY_SERVER (0x1). If State is LEFT or DECLINED_JOIN and the frame type is TBD-0b, the Reason Code field is set to one of the following values:

0x0: UNSPECIFIED_OTHER
0x1: REQUESTED_BY_SERVER
0x2: ADMINISTRATIVE_BLOCK
0x3: PROTOCOL_ERROR
0x4: PROPERTY_VIOLATION
0x5: UNSYNCHRONIZED_PROPERTIES
0x6: ID_COLLISION
0x10: HELD_DOWN
0x12: MAX_RATE_EXCEEDED
0x13: HIGH_LOSS
0x14: EXCESSIVE_SPURIOUS_TRAFFIC
0x15: MAX_STREAMS_EXCEEDED
0x16: LIMIT_VIOLATION
0x17: AUTHENTICATION_DELAY_EXCEEDED

If a server receives an MC_STATE frame of type TBD-0b with an undefined Reason Code, it SHOULD close the connection with a connection error of type MC_EXTENSION_ERROR. (Author's note TODO: consider whether these reasons should be added to the QUIC Transport Error Codes registry () instead of defining a new registry specific to multicast.)

Retransmission of information In addition to the mechanisms used for retransmission described in and the following rules apply to the newly introduced frames:

As the properties carried in MC_ANNOUNCE frames can not change during the lifetime of a channel, information contained in them can be retransmitted without any special considerations.
Since conditions of the client or channel can have changed by the time a retransmission of an MC_JOIN, MC_LEAVE or MC_RETIRE channel becomes necessary, a retransmission might no longer be required or even appropriate. A retransmission SHOULD only occur if the channel in question should still be joined/left/retired.
Retransmission of information contained in MC_ACK frames MUST be handled exactly as with regular ACK frames.
For MC_KEY, MC_LIMITS, and MC_STATE, retransmissions MUST include the most up-to-date information.
For MC_INTEGRITY, a server MUST retransmit packet hashes that are still needed to authenticate channel packets that the server expects receivers to process. For this purpose, a packet hash is still needed while the corresponding channel packet is within the buffering interval implied by the channel's Max Authentication Delay, unless the server has channel-specific or application-specific information that receivers are no longer expected to buffer or process that packet. A server SHOULD NOT retransmit MC_INTEGRITY information for packets that it no longer expects receivers to buffer or process. The same packet hash MAY be sent in more than one MC_INTEGRITY frame. Servers SHOULD prioritize retransmission of MC_INTEGRITY information whose absence is likely to cause receivers to exceed the Max Authentication Delay advertised for the channel.

Frames Carried in Channel Packets Multicast channels will contain normal QUIC 1-RTT data packets (see ) except using the Channel ID instead of a Connection ID. The packets are protected with the keys derived from the secrets in MC_KEY frames for the corresponding channel. Data packet hashes will also be sent in MC_INTEGRITY frames, as keys cannot be trusted for integrity due to giving them to too many receivers, as described in . The set of frames that can appear in a channel packet is restricted. A server MUST NOT send a frame in a channel packet unless the frame is valid in a QUIC 1-RTT packet and either:

the definition of the frame explicitly permits its use in channel packets; or
the frame operates only on state that is shared by all receivers of the channel; receiving it will have the same effect for all receivers.

A frame is not permitted in a channel packet if processing the frame depends on state that is specific to an individual receiver, path, or unicast connection. This includes, but is not limited to, frames that are part of the cryptographic handshake, address validation, connection ID migration, flow control or connection termination. A frame is also not permitted in a channel packet if it is defined only for client-to-server use, or if it would require the server to respond to a single client via the multicast channel (e.g., PATH_CHALLENGE). A client that receives a frame in an authenticated channel packet that is not permitted SHOULD close the connection with a connection error of type MC_EXTENSION_ERROR. For example, frames such as ACK, CRYPTO, NEW_TOKEN, STOP_SENDING, MAX_DATA, MAX_STREAMS, NEW_CONNECTION_ID, RETIRE_CONNECTION_ID, PATH_CHALLENGE, PATH_RESPONSE, CONNECTION_CLOSE, and HANDSHAKE_DONE are not permitted in channel packets because their semantics are tied to an individual QUIC connection. On the other hand, PADDING and PING frames are permitted in channel packets because they do not affect receiver-specific state. STREAM, RESET_STREAM, and DATAGRAM frames are permitted when they carry data scoped to the multicast channel. MC_ANNOUNCE, MC_INTEGRITY, MC_KEY and MC_RETIRE frames are permitted because they operate on multicast channel state independently of any individual associated unicast connection. MC_JOIN and MC_LEAVE are not permitted in channel packets because their semantics depend on state specific to an individual associated connection, including MC_LIMITS and MC_STATE sequence numbers.

Implementation and Operational Considerations

Constraints on Stream Data Note that when a newly connected client joins a channel, the client will only be able to receive application data carried in stream frames delivered on that channel when they have received the stream data starting from offset 0 of the stream. This usually means that new streams must be started for application data carried in channel packets whenever there might be new clients that have joined since an earlier stream started. If the server deems it convenient, it could also send preceding data for that stream over the unicast connection to catch the client up. With broadcast video, this usually means a new stream is necessary for every video segment or group of video frames since new clients will join throughout the broadcast, whereas for video conferencing, it could be possible to start a new stream whenever new clients join the conference without needing a new stream per object.

Application Use Cases There are several known applications that could benefit from using multicast QUIC, either with their own custom application-layer transport or with one of the transports discussed in . A few examples include:

Existing multicast-capable applications that are modified to use QUIC datagrams instead of UDP payloads can potentially get improved encryption and congestion feedback, while keeping existing error recovery techniques (e.g. techniques based on the forward error correction (FEC) framework in ).
- An external tunnel could supply this kind of encapsulation without modification to the sender or receiver for some applications, while retaining the benefits of multicast scalability
- Using QUIC datagrams in place of UDP packets could usefully support existing implementations of file-transfer protocols like FLUTE or FCAST to enable file downloads such as operating system updates or popular game downloads, but adding encryption, packet-level authentication, and congestion control as provided by QUIC.
Conferencing systems, especially within an enterprise that can deploy multicast network support, often can save significantly on server costs by using multicast
The traditional multicast use case of broadcasting of live sports with a set-top box would benefit from an interoperable system such as these QUIC extensions that can fall back to unicast transparently as needed, for example if there are a few customers who installed a non-multicast-capable home router.
Smart TVs or other video playing in-home devices could interoperate with a standard sender using multicast QUIC, rather than requiring proprietary integrations with TV operators.

Data Transport Use Cases This section outlines considerations for some known transport mechanisms that are worth highlighting as potentially useful with multicast QUIC.

HTTP/3 Server Push HTTP/3 Server Push is defined in . Server push is a good use case for multicast transport because the same data can be pushed to many different receivers on a multicast channel. Applications designed to work well with server push can leverage multicast QUIC very effectively with only a few extra considerations. A QUIC connection using HTTP/3 can use multicast channels to deliver server-initiated streams that implement HTTP/3 Server Push. Applications expecting to use server push with multicast SHOULD use a high MAX_PUSH_ID in order to work with channels that have been active for a long time already when the connection is first established. Servers SHOULD NOT allow clients to remain joined to channels if their MAX_PUSH_ID will be exceeded by push streams that are to be sent imminently. If a client receives data from a push ID that exceeds its MAX_PUSH_ID causing an H3_ID_ERROR on a multicast channel, it SHOULD leave the channel with reason 0x1000108 (computed by adding the H3_ID_ERROR value 0x0108 to the Application-defined Reason start value 0x1000000). This SHOULD NOT cause a close of the whole connection but MAY cause a stream error and reset of the stream. TODO: flesh out this principle for application-level error code assignment in general for known error code values, and specifically all HTTP/3 ones? (Or is there a better approach?)

HTTP/3 WebTransport Streams WebTransport over HTTP/3 is defined in . Popular data that can be sent with server-initiated streams and carried over WebTransport is a good use case for multicast transport because the same server-to-client data can be pushed to many different receivers on a multicast channel. A QUIC connection using HTTP/3 and WebTransport can use multicast channels to deliver WebTransport server-initiated streams. However, because the WebTransport Session ID is a client-specific value, the bytes that carry the WebTransport Session ID value within the stream would need to be carried over unicast, since it's not the same for different clients. For this situation, note that the Session ID is a variable length integer, and that a variable length integer can be encoded in any size that's big enough to hold it. In particular, it's possible to use the largest size of any Session IDs of any of the WebTransport sessions of any clients (or 8 octets, the maximum size for a variable length integer), and that all clients receiving stream data on a channel will need to use the same size for the Session ID so that the rest of the stream data will be at the same offset for every client.

Datagrams DATAGRAM frames in channel packets are subject to the max_datagram_frame_size transport parameter defined in on each associated QUIC connection. A server MUST NOT send MC_JOIN to a client for a channel that carries DATAGRAM frames unless that client advertised max_datagram_frame_size with a non-zero value. A server MUST NOT send MC_JOIN to a client for a channel if the server expects that channel to carry DATAGRAM frames larger than the max_datagram_frame_size value advertised by that client. If the server later sends larger DATAGRAM frames on that channel, clients whose advertised max_datagram_frame_size is exceeded will process this as specified by , which can result in termination of their associated QUIC connections. Using DATAGRAM frames can align well with existing multicast UDP-based applications, since a datagram API in a QUIC application offers similar functionality to a UDP API for sending and receiving packets. However, at the time of this writing, HTTP/3 datagrams , including WebTransport datagrams as defined by , generally cannot be delivered over multicast channels, since the demuxing of WebTransport datagrams uses a Session ID based on a client-specific value (the HTTP/3 Session ID comes from the Stream ID of the client-initiated stream that issued the initial extended CONNECT request). It is therefore hoped that an extension or revision to WebTransport and HTTP/3 datagrams can be adopted in a future version of their specifications that make it possible to use a server-chosen Session ID value for demuxing WebTransport datagrams (and HTTP/3 datagrams in general). Such a value could for instance be sent in an HTTP/3 response header, and as long as it is unique within the connection and avoids collision with any client-initiated stream ID values, it could still be used to multiplex data associated with different HTTP/3 traffic and different WebTransport sessions carried on the same connection. Then by choosing the same server-chosen session ID for all the connections, the server would be able to use the same channel to carry the identical complete datagrams, including the server-chosen Session ID, to multiple receivers that the server asks to join the same channel. Such a change could either replace the current client-chosen definition for Session ID in server-to-client datagrams, or could add new HTTP/3 frame types that allow a server-chosen Session ID when the client has advertised support for this extended functionality.

Moving Clients Between Channels MC_LEAVE and MC_RETIRE take effect immediately when processed by the receiver. These frames do not provide a mechanism for draining a channel up to a packet-number boundary. A server that wants to minimize loss when moving receivers away from a channel SHOULD stop scheduling new data on the old channel before sending MC_LEAVE or MC_RETIRE, or provide any required replacement data over the associated unicast connection or another channel. For example, when switching receivers from one channel to another, a server can stop sending new application data that is important for those receivers on the old channel, send any transition data on the unicast connection or the new channel, and then send MC_LEAVE or MC_RETIRE for the old channel.

Graceful Degradation Clients with multicast QUIC support can stop accepting multicast for a variety of reasons. Applications like live broadcast-scale video that rely on multicast QUIC may benefit from anticipating that clients might stop using multicast and providing data feeds with similar content that can scale even if many clients stop using multicast, for example by ensuring that a lower-bitrate rendition can still be delivered over unicast to all or most of the clients simultaneously, and ensuring that the server has a way to make the client start using the low-bitrate version when it switches to unicast. While some existing Adaptive Bitrate video players might have an easy way to provide this, other video players might need specialized logic to provide the server a way to control what bitrate individual clients consume. Although under ideal conditions it may often be possible using features like server push () to use unmodified existing HTTP-based video players with multicast QUIC, in practice it may require extra development at the application level to make a player that robustly delivers a good user experience under variable network conditions, depending on the scalability gains that multicast transport is providing and the Adaptive Bitrate algorithms the player is using.

Circuit Breakers Operators of multicast QUIC services should consider that some networks may implement circuit breakers such as the one described in , or similar network-level safety features that might cut off previously operational multicast transport under certain conditions. The servers will notice the transport loss from the lack of MC_ACK frames from receivers in a network that cut off multicast transport, but it may be beneficial when possible in a transport cutoff event correlated across many clients to pace the recovery response according to aggregations of the affected clients so that a sudden unicast storm doesn't overload the network further.

Server Scalability Use of QUIC multicast channels can provide large scalability gains, but there still will be significant scaling requirements on server operators to support a large client footprint. Servers, possibly many of them, still will be required to maintain unicast connections with all the clients and provide for handling MC_ACK frames from the clients, delivering MC_INTEGRITY frames, managing the clients' channel join states, and providing recovery for lost packets. Further, the use of multicast channels likely requires increased coordination between the different servers, relative to services that operate completely independently. For large deployments, server implementations will often need to operate on separate devices from the ones generating the multicast channel packets, and will need to be designed accordingly. Because multiple MC_ACK frames can be bundled for efficiency, servers SHOULD retain information needed for loss recovery for at least the channel's Max ACK Delay plus half the RTT between client and server. The MC_ACK policy advertised in MC_ANNOUNCE controls when clients send MC_ACK frames for a channel. Clients MAY send MC_ACK frames more frequently than this policy requires, but SHOULD avoid sending them less frequently. Servers should choose Max ACK Delay, Ack-Eliciting Threshold, and Reordering Threshold values that balance uplink load against the need for timely loss, congestion, ECN, and liveness feedback.

Address Collisions Multicast channels at the network layer are addressed with a source IP, a destination group IP address, and a destination UDP port. This creates a number of potential address collision considerations that are worth mentioning:

If properties change for the data being used in a channel (for example, new video encoding settings might result in a change to the expected max rate for a video feed), a server might reuse the same network addresses in a new QUIC multicast channel, and might send a join for the new channel and a leave for the old channel to clients that can support the new max rate. If they arrive together, this could be handled by the client without making a change to the IGMP or MLD membership state, as an optimization that can prevent the need for some recovery, or even by reusing the same UDP socket. Doing so does not change any requirements for the channel state management at the QUIC layer, and as long as the situation is transient, should not result in leaving due to Excessive Spurious Traffic even if some packets were reordered or may still be in flight.
As described in , link-layer addresses can be linked to the low-order bits of multicast addresses, and may be the same for different group destinations. Collisions in the link-layer addressing, even with traffic that comes from other sources, can cause congestion or receiver CPU load for colliding channels that might be different from that seen with other channels that were delivered with apparently the same network paths.
Even though multicast QUIC uses only source-specific multicast, older networks with devices that don't have IGMPv3 or MLDv2 support can propagate the joins as any-source multicast. If there are active senders sending to that destination, this can cause network congestion and CPU load due to discarding packets from the wrong source, even though at the application layer the UDP socket won't receive those packets from the wrong source.
If different channels use the same (S,G) but different UDP ports, they will share the same multicast forwarding tree in an IP network. This is often useful when the data in the channels are linked, for example if MC_INTEGRITY frames are carried on one channel for packets carried on another channel, because it provides some fate-sharing for the linked data. However, for data that is not so linked, it would generally be a disadvantage to share the (S,G) because the network link of any receiver joined to one of those channels but not the other would receive both packets and throw away the data for the un-joined port, causing extra congestion and CPU load for the receiving device.

Buffering Unauthenticated Packets Clients need to buffer multicast packets that have been received but not yet authenticated if they want to process those packets after the corresponding MC_INTEGRITY information arrives. The amount of memory needed for this buffer is a function of the channel rate, the delay in receiving integrity information, packet reordering, and implementation policy. Max Authentication Delay is a channel property because the amount of receiver buffering required for unauthenticated packets depends on the channel's rate, the way integrity information is distributed for that channel, and the latency requirements of the application data carried on that channel. For example, a low-latency media channel might require integrity information to arrive quickly, while a file-transfer or software-update channel might tolerate a larger authentication delay in exchange for lower unicast integrity traffic or larger integrity blocks. Servers SHOULD choose Max Authentication Delay values that are appropriate for the channel's media or application latency requirements and for expected receiver memory constraints. Servers SHOULD send and retransmit MC_INTEGRITY information so that, under normal operating conditions, receivers can authenticate packets within the advertised Max Authentication Delay. The exact scheduling of redundant transmission and retransmission is implementation-dependent, but the advertised Max Authentication Delay defines the default interval during which receivers are expected to retain unauthenticated packets. Clients MAY use local policy to impose a smaller buffering limit than the value advertised by the server, in which case they might discard unauthenticated packets or leave the channel. The usefulness of retransmitting MC_INTEGRITY information depends on whether receivers are still expected to have the corresponding multicast packets buffered. Once receivers are expected to have discarded a packet, retransmitting integrity information for that packet is unlikely to help those receivers process application data. This consideration is especially important when MC_INTEGRITY information is sent on the unicast connection and retransmitted for individual receivers. It is also important for channels carrying delay-sensitive unreliable data, such as DATAGRAM frames for real-time applications. For such channels, servers can reduce wasted work by sending integrity information redundantly near the original packet transmission and by avoiding late retransmission of integrity information after the packet's expected usefulness has expired.

Spurious Channel Traffic A client can receive multicast packets that it cannot associate with a joined channel, that cannot be authenticated using accepted MC_INTEGRITY information, or that cannot be decrypted using an applicable channel secret. Such packets can be caused by corruption, loss or delay of control information, misconfiguration, or injection by an attacker. A client MAY discard such packets without further processing. If spurious traffic is persistently received for an (S,G) used by one or more joined channels, and the traffic interferes with reception or causes excessive resource use, the client MAY leave the affected channels and send MC_STATE with State LEFT and Reason Code EXCESSIVE_SPURIOUS_TRAFFIC.

Security Considerations (Authors comment: Mostly incorporate . Anything else? e.g. if a different legitimate quic connection says someone else's quic multicast stream is theirs, that's maybe a problem worth protecting against. Maybe we need a periodic asymmetric challenge? I'm thinking send a public key on the multicast channel and in the unicast channels send an individualized MAC signed with the private key and verify it with the public key, so that in addition to validating that the unicast server knows the contents of the multicast packets via the hashes it supplies, the multicast stream provides a way for the client to validate that the unicast stream is authorized to use it for data transport via proof they know the private key corresponding to the public key that arrived on the multicast channel. Note this doesn't prevent unauthorized receipt of multicast data packets, but does prevent a quic server from lying when claiming a multicast data channel belongs to it, preventing legit receivers from consuming it. alternatively, can the multicast channel just periodically say what domain name is expected for the quic connection and get the same crypto guarantee of a proper sender via the domain's cert, which was already checked on the unicast channel?)

IANA Considerations TODO: MC_EXTENSION_ERROR error code TODO: lots

References Normative References Security and Privacy Considerations for Multicast Transports Akamai Technologies, Inc. TU Berlin Akamai Technologies, Inc. Interdomain multicast has unique potential to solve delivery scalability for popular content, but it carries a set of security and privacy issues that differ from those in unicast delivery. This document analyzes the security threats unique to multicast-based delivery for Internet and Web traffic under the Internet and Web threat models. Discussion Venues This note is to be removed before publishing as an RFC. Source for this draft and an issue tracker can be found at https://github.com/squarooticus/draft-krose-multicast-security. Asymmetric Manifest Based Integrity Akamai Technologies, Inc. Akamai Technologies, Inc. TU Berlin This document defines Asymmetric Manifest-Based Integrity (AMBI). AMBI allows each receiver or forwarder of a stream of multicast packets to check the integrity of the contents of each packet in the data stream. AMBI operates by passing cryptographically verifiable hashes of the data packets inside manifest messages, and sending the manifests over authenticated out-of-band communication channels. Circuit Breaker Assisted Congestion Control Akamai Technologies, Inc. Akamai Technologies, Inc. TU Berlin This document specifies Circuit Breaker Assisted Congestion Control (CBACC). CBACC enables fast-trip Circuit Breakers by publishing rate metadata about multicast channels from senders to intermediate network nodes or receivers. The circuit breaker behavior is defined as a supplement to receiver driven congestion control systems, to preserve network health if misbehaving or malicious receiver applications subscribe to a volume of traffic that exceeds capacity policies or capability for a network or receiving device. Managing multiple paths for a QUIC connection Alibaba Inc. Uber Technologies Inc. University of Mons (UMONS) UCLouvain and WELRI Private Octopus Inc. Ericsson This document specifies a multipath extension for the QUIC protocol to enable the simultaneous usage of multiple paths for a single connection. It introduces explicit path identifiers to create, delete, and manage multiple paths. This document does not specify address discovery or management, nor how applications using QUIC schedule traffic over multiple paths. Transport Layer Security (TLS) Parameters IANA QUIC Acknowledgment Frequency Fastly Google Ericsson This document specifies an extension to QUIC that enables an endpoint to request its peer change its behavior when sending or delaying acknowledgments. Note to Readers Discussion of this draft takes place on the QUIC working group mailing list (quic@ietf.org), which is archived at https://mailarchive.ietf.org/arch/search/?email_list=quic. Source code and issues list for this draft can be found at https://github.com/quicwg/ack-frequency. Working Group information can be found at https://github.com/quicwg. UDP Usage Guidelines The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage. QUIC: A UDP-Based Multiplexed and Secure Transport This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. Using TLS to Secure QUIC This document describes how Transport Layer Security (TLS) is used to secure QUIC. QUIC Loss Detection and Congestion Control This document describes loss detection and congestion control mechanisms for QUIC. An Unreliable Datagram Extension to QUIC This document defines an extension to the QUIC transport protocol to add support for sending and receiving unreliable datagrams over a QUIC connection. Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References WebTransport over HTTP/3 Facebook Apple Inc. Google WebTransport [OVERVIEW] is a protocol framework that enables application clients constrained by the Web security model to communicate with a remote application server using a secure multiplexed transport. This document describes a WebTransport protocol that is based on HTTP/3 [HTTP3] and provides support for unidirectional streams, bidirectional streams, and datagrams, all multiplexed within the same HTTP/3 connection. Forward Erasure Correction for QUIC loss recovery UCLouvain UCLouvain, WEL RI This documents lays down the QUIC protocol design considerations needed for QUIC to apply Forward Erasure Correction on the data sent through the network. Source-Specific Multicast for IP IP version 4 (IPv4) addresses in the 232/8 (232.0.0.0 to 232.255.255.255) range are designated as source-specific multicast (SSM) destination addresses and are reserved for use by source-specific applications and protocols. For IP version 6 (IPv6), the address prefix FF3x::/32 is reserved for source-specific multicast use. This document defines an extension to the Internet network service that applies to datagrams sent to SSM addresses and defines the host and router requirements to support this extension. [STANDARDS-TRACK] Forward Error Correction (FEC) Framework This document describes a framework for using Forward Error Correction (FEC) codes with applications in public and private IP networks to provide protection against packet loss. The framework supports applying FEC to arbitrary packet flows over unreliable transport and is primarily intended for real-time, or streaming, media. This framework can be used to define Content Delivery Protocols that provide FEC for streaming media delivery or other packet flows. Content Delivery Protocols defined using this framework can support any FEC scheme (and associated FEC codes) that is compliant with various requirements defined in this document. Thus, Content Delivery Protocols can be defined that are not specific to a particular FEC scheme, and FEC schemes can be defined that are not specific to a particular Content Delivery Protocol. [STANDARDS-TRACK] FLUTE - File Delivery over Unidirectional Transport This document defines File Delivery over Unidirectional Transport (FLUTE), a protocol for the unidirectional delivery of files over the Internet, which is particularly suited to multicast networks. The specification builds on Asynchronous Layered Coding, the base protocol designed for massively scalable multicast distribution. This document obsoletes RFC 3926. [STANDARDS-TRACK] FCAST: Object Delivery for the Asynchronous Layered Coding (ALC) and NACK-Oriented Reliable Multicast (NORM) Protocols This document introduces the FCAST reliable object (e.g., file) delivery application. It is designed to operate either on top of the underlying Asynchronous Layered Coding (ALC) / Layered Coding Transport (LCT) reliable multicast transport protocol or the NACK-Oriented Reliable Multicast (NORM) transport protocol. HTTP/3 The QUIC transport protocol has several features that are desirable in a transport for HTTP, such as stream multiplexing, per-stream flow control, and low-latency connection establishment. This document describes a mapping of HTTP semantics over QUIC. This document also identifies HTTP/2 features that are subsumed by QUIC and describes how HTTP/2 extensions can be ported to HTTP/3. HTTP Datagrams and the Capsule Protocol This document describes HTTP Datagrams, a convention for conveying multiplexed, potentially unreliable datagrams inside an HTTP connection. In HTTP/3, HTTP Datagrams can be sent unreliably using the QUIC DATAGRAM extension. When the QUIC DATAGRAM frame is unavailable or undesirable, HTTP Datagrams can be sent using the Capsule Protocol, which is a more general convention for conveying data in HTTP connections. HTTP Datagrams and the Capsule Protocol are intended for use by HTTP extensions, not applications. Secrecy, Authentication, and Public Key Systems

Acknowledgments Thanks to Louis Navarre on his comments and text contributions to the multipath and FEC sections. Thanks to Johannes Cram for his work on the picoquic reference implementation and helpful feedback. Thanks to Martin Duke, Sam Hurst, Kyle Rose, Michael Welzl and Momoka Yamamoto for their helpful reviews and comments. This work has been supported by the Federal Ministry of Research, Technology and Space of Germany in the programme of “StartUpConnect” Project QUICast, project identification number 16KIS2650 and programme “Souverän. Digital. Vernetzt.” Joint project 6G-RIC, project identification number (PIN): FKZ 16KISK030 TODO acknowledge.