Service Affinity Solution based on Transport Layer Security (TLS)

6.3.1. DATA Frame (0x00)

DATA frame carries application data from the sender to the receiver. Its format is shown in Figure 3.¶

    +------+------+----------+--------+----------+
    | 0x46 | 0x52 | 0x00     | seq#   | length   |
    +------+------+----------+--------+----------+
    |                   payload                  |
    +--------------------------------------------+
        Figure 3: The format of DATA frame

Where:¶

• sequence_number: The sender's monotonically increasing sequence number for this data frame.¶

• length: The length of the application data payload.¶

• payload: The application data bytes, up to 4096 bytes.¶

The receiver SHOULD send an ACK frame for each received DATA frame, according to its ACK policy.¶

6.3.2. ACK Frame (0x01)

ACK frame is the acknowledges receipt of a specific DATA frame. Its format is shown in Figure 4.¶

    +------+------+----------+--------+-----------+
    | 0x46 | 0x52 | 0x01     | 0x0000 | 0x00000004|
    +------+------+----------+--------+-----------+
    |        ack_sequence_number (4 bytes)        |
    +---------------------------------------------+
        Figure 4: The format of ACK frame

Where:¶

• sequence_number field in header: MUST be set to 0 (unused for ACK).¶

• length: MUST be 4.¶

• payload: A 4-byte big-endian unsigned integer containing the sequence number of the DATA frame being acknowledged.¶

Upon receiving an ACK frame, the sender marks the corresponding entry in its send queue as acknowledged.¶

6.3.3. FIN Frame (0x02)

FIN frame signals the graceful end of the framing session. No payload. Its format is shown in Figure 5.¶

    +------+------+----------+--------+--------+
    | 0x46 | 0x52 | 0x02     | seq#   | 0x0000 |
    +------+------+----------+--------+--------+
        Figure 5: The format of FIN frame

Where:¶

• sequence_number: The sender's next expected send sequence number.¶

• length: MUST be 0.¶

The receiver of a FIN frame SHOULD respond with its own FIN frame and then close the connection.¶

6.3.4. RETRANSMIT Flag (0x04)

RETRANSMIT flag is combined with FRAME_FLAG_DATA using bitwise OR to indicate that a DATA frame is a retransmission of a previously sent frame.¶

    flags = FRAME_FLAG_DATA | FRAME_FLAG_RETRANSMIT  // 0x04

The receiver SHOULD process retransmitted frames in the same manner as new frames. If the frame has already been processed (i.e., its sequence number is less than the receiver's next_recv_seq), the receiver MAY silently discard it but MUST still send an ACK.¶

6.4.1. Sequence Number Space

Each endpoint maintains two sequence counters:¶

• next_send_seq: The sequence number to assign to the next DATA frame sent. Initialized to 1 and incremented after each DATA frame is queued for sending.¶

• next_recv_seq: The sequence number expected for the next DATA frame received. Initialized to 1 and incremented when a new (non- duplicate) DATA frame is successfully received.¶

Sequence numbers are 32-bit unsigned integers. They MUST be initialized to 1 at the start of each framing session (i.e., when a new TLS connection is established).¶

6.4.2. Send Queue

The sender maintains a send queue that stores metadata for each sent DATA frame until it is acknowledged:¶

    struct {
        uint32 sequence_number;   // Sequence number of the frame
        uint32 length;            // Payload length
        opaque data[length];      // Copy of the payload data
        boolean acked;            // Whether this frame has been ACKed
        uint8  retransmit_count;  // Number of times retransmitted
    } SendQueueEntry;

The send queue has a maximum capacity of FRAME_MAX_QUEUE (1024) entries. If the queue is full, the sender MUST stop accepting new application data until space becomes available (i.e., until some frames are acknowledged).¶

6.4.3. Acknowledgment Processing

When the sender receives an ACK frame with ack_sequence_number equal to N, it searches its send queue for the entry with sequence_number == N and marks it as acked = true.¶

The sender MAY prune acknowledged entries from the send queue to reclaim memory, but MUST retain all unacknowledged entries for potential retransmission.¶

6.4.4. Duplicate Detection

When the receiver receives a DATA frame with sequence_number < next_recv_seq, it identifies the frame as a duplicate (retransmission). The receiver:¶

• MUST still send an ACK for the frame.¶

• SHOULD discard the payload without delivering it to the application.¶

6.5.1. Retransmission Trigger

Retransmission is performed in two scenarios:¶

• Migration-triggered retransmission: After completing TLS session migration to a new server (Section 4.4), the client MUST retransmit all unacknowledged DATA frames from its send queue to the new server. This is the primary use case addressed by this specification.¶

• Timer-based retransmission (optional): Implementations MAY support a retransmission timer that triggers retransmission of unacknowledged frames after a configurable timeout. This is useful for handling packet loss within a single connection but is OPTIONAL for the migration use case.¶

6.5.2. Retransmission Procedure

To retransmit unacknowledged frames, the sender:¶

1. Iterates through the send queue from the oldest to the newest entry.¶

2. For each entry where acked == false:¶

• Constructs a new DATA frame with flags = FRAME_FLAG_DATA | FRAME_FLAG_RETRANSMIT.¶

• Sets sequence_number to the original frame's sequence number.¶

• Copies the original payload data.¶

• Sends the frame over the current TLS connection.¶

• Increments the entry's retransmit_count.¶

3. The sender MUST preserve the original sequence numbers during retransmission. The receiver uses sequence numbers to detect and deduplicate retransmitted frames.¶

    RetransmitAllUnacked():
        for each entry in send_queue:
            if entry.acked == false:
                frame = Frame {
                    magic: 0x4652,
                    flags: DATA | RETRANSMIT,
                    sequence_number: entry.sequence_number,
                    length: entry.length,
                    payload: entry.data
                }
                send(frame)
                entry.retransmit_count++

6.6.1. Sender State

    +------------------+
    |     ACTIVE       |  Normal operation: sending DATA frames,
    |  (next_send_seq) |  receiving ACKs, managing send queue.
    +--------+---------+
             |
             | Migration initiated
             v
    +------------------+
    |   MIGRATING      |  TLS connection to old server closed.
    |   (queue intact) |  Send queue preserved for retransmission.
    +--------+---------+
             |
             | New TLS connection established
             v
    +------------------+
    |  RETRANSMITTING  |  Sending all unacknowledged frames to
    |                  |  new server, then resuming normal send.
    +--------+---------+
             |
             | All unacked frames retransmitted
             v
    +------------------+
    |     ACTIVE       |  Resume normal operation on new server.
    +------------------+

                  Figure 6: Sender state machine

6.6.2. Receiver State

    +------------------+
    |     ACTIVE       |  Normal operation: receiving DATA frames,
    |  (next_recv_seq) |  sending ACKs, delivering to application.
    +--------+---------+
             |
             | Connection closed (server shutdown or migration)
             v
    +------------------+
    |      CLOSED      |
    +------------------+

                  Figure 7: Receiver state machine

On the new server, the receiver initializes next_recv_seq = 1 and processes both retransmitted and new frames normally. The receiver does not need to be aware that some frames are retransmissions from a previous server.¶

6.6.3. State Reset on New Connection

When a new TLS connection is established (after migration), the sender:¶

• MUST reset next_recv_seq to 1 (for receiving ACKs and data from the new server).¶

• MUST preserve the send queue intact (for retransmission).¶

• MUST preserve next_send_seq (to avoid sequence number collisions with retransmitted frames).¶

The receiver on the new server:¶

• MUST initialize next_recv_seq to 1.¶

• MUST initialize an empty send queue.¶

6.7.1. Pre-Migration (Phase 1)

During normal data transfer with Server A:¶

• The client sends DATA frames via the Framing Layer. Each frame is assigned a monotonically increasing sequence number and added to the send queue.¶

• Server A receives DATA frames and sends ACK frames according to its ACK policy. The application on Server A processes the data and decides whether to acknowledge each frame.¶

• The client processes incoming ACK frames and marks corresponding send queue entries as acknowledged.¶

6.7.2. Migration (Phase 2)

When migration is triggered:¶

• The client records the current state of its send queue, noting which frames remain unacknowledged.¶

• The client disconnects from Server A. The send queue is preserved in memory.¶

• The client initiates a new TLS connection to Server B, performing PSK-based session resumption as described in Section 4.4.¶

6.7.3. Post-Migration (Phase 3)

After the new TLS connection is established with Server B:¶

• The client resets its receive state (next_recv_seq = 1) for the new connection.¶

• The client calls RetransmitAllUnacked() to resend all unacknowledged DATA frames to Server B. These frames carry the RETRANSMIT flag.¶

• Server B receives the retransmitted frames. Since Server B initializes next_recv_seq = 1, it processes them as new frames, sends ACKs, and delivers the data to the application.¶

• The client resumes normal data transfer, sending new DATA frames with sequence numbers continuing from where it left off.¶

      Phase 1 (Server A):          Phase 3 (Server B):
      +----+-------+------+       +----+-------+------+
      | #1 | DATA  | ACKED|       | #1 | DATA  | ACKED|  (no retransmit)
      | #2 | DATA  | ACKED|       | #2 | DATA  | ACKED|  (no retransmit)
      | #3 | DATA  | ACKED|       | #3 | DATA  | ACKED|  (no retransmit)
      | #4 | DATA  | UNACK|  -->  | #4 | DATA  | ACKED|  (retransmitted)
      | #5 | DATA  | UNACK|  -->  | #5 | DATA  | ACKED|  (retransmitted)
      +----+-------+------+       +----+-------+------+
                                  | #6 | DATA  | NEW  |  (post-migration)
                                  +----+-------+------+
  
      Figure 8: Send Queue State Transition During Migration

  ¶

8.3.1. Confidentiality and Integrity

The Framing Layer operates entirely within the TLS encrypted channel. All frame headers and payloads are protected by TLS record-layer encryption and authentication. An attacker who cannot break the TLS cipher suite cannot read, modify, or inject frames.¶

8.3.2. Sequence Number Manipulation

Sequence numbers are transmitted in the clear within the TLS record (i.e., they are encrypted but the receiver must be able to read them). Since the entire frame is protected by TLS, an attacker cannot modify sequence numbers without detection. Implementations SHOULD verify that received sequence numbers are within an acceptable window to detect potential protocol-level attacks.¶

8.3.3. Denial of Service via Send Queue Exhaustion

A malicious receiver could refuse to send ACK frames, causing the sender's send queue to grow until it reaches FRAME_MAX_QUEUE. At that point, the sender MUST stop accepting new application data. To mitigate this, implementations SHOULD:¶

• Monitor send queue utilization and log warnings when it approaches capacity.¶

• Provide a configurable maximum queue size appropriate for the deployment environment.¶

• Consider a timeout after which unacknowledged frames are discarded and the connection is terminated with an error.¶

8.3.4. Retransmission Flooding

A malicious client could retransmit a large number of frames to a new server after migration. The new server SHOULD apply rate limiting to retransmitted frames and SHOULD validate that retransmitted frame sequence numbers are within the expected range. Since retransmitted frames carry the RETRANSMIT flag, the server can apply differentiated rate limiting policies for new vs. retransmitted frames.¶