Macula HTTP/3 Mesh: Comprehensive Technical Roadmap

Executive Summary

Vision: Macula enables distributed BEAM applications to form encrypted, self-healing mesh networks over HTTP/3, with zero infrastructure dependencies. Works behind NATs, scales to thousands of nodes, native distributed Erlang semantics - all over standard HTTPS.

The "Wow" Factor:

• Distributed Erlang over HTTP/3 (nobody else does this!)
• Works through corporate firewalls (it's just HTTPS!)
• Self-healing mesh topology (no central coordinator)
• Built-in NAT traversal (QUIC magic)
• Native BEAM implementation (minimal NIFs)

Timeline: 20 weeks (5 months)

Outcome: Production-ready mesh networking infrastructure for edge BEAM applications.

Table of Contents

1. Understanding QUIC and HTTP/3
2. QUIC/HTTP/3 Libraries for BEAM
3. Architecture Overview
4. Detailed Roadmap
5. Architecture Diagrams
6. Technical Deep Dives
7. Success Metrics

Understanding QUIC and HTTP/3

What is QUIC?

QUIC (Quick UDP Internet Connections) is a modern transport protocol developed by Google and standardized by IETF as RFC 9000. It's designed to replace TCP for web traffic.

Key Characteristics

1. UDP-Based Transport

Traditional:  HTTP/2 → TLS → TCP → IP
Modern:       HTTP/3 → QUIC (includes TLS 1.3) → UDP → IP

QUIC runs over UDP instead of TCP, which provides several advantages:

• Faster connection establishment (0-RTT and 1-RTT)
• No head-of-line blocking across streams
• Better performance on lossy networks
• Easier NAT traversal (UDP is simpler than TCP for NAT)

2. Built-in Encryption

• TLS 1.3 is integrated into QUIC (not layered on top)
• All packets are encrypted (except initial handshake metadata)
• Forward secrecy by default
• Connection migration (can change IP addresses mid-connection)

3. Multiplexed Streams

┌─────────────────────────────────────┐
│ QUIC Connection                     │
│  ┌───────────┐ ┌───────────┐       │
│  │ Stream 0  │ │ Stream 1  │  ...  │
│  │ (Ordered) │ │ (Ordered) │       │
│  └───────────┘ └───────────┘       │
│                                     │
│  Streams are independent!           │
│  Loss in Stream 0 doesn't block     │
│  Stream 1 (unlike TCP)              │
└─────────────────────────────────────┘

4. Connection Establishment

Traditional TCP + TLS:

Client → Server:  SYN                        (1 RTT)
Server → Client:  SYN-ACK
Client → Server:  ACK
Client → Server:  ClientHello (TLS)          (2 RTT)
Server → Client:  ServerHello + Certificate
Client → Server:  Finished
Server → Client:  Finished
Client → Server:  HTTP Request               (3 RTT)
                  Total: 3 Round Trips

QUIC (first connection):

Client → Server:  Initial (ClientHello)      (1 RTT)
Server → Client:  Handshake (ServerHello)
Client → Server:  HTTP Request               (1 RTT)
                  Total: 1 Round Trip

QUIC (resumed connection):

Client → Server:  0-RTT Data + HTTP Request  (0 RTT!)
                  Total: 0 Round Trips

5. Loss Recovery

• Per-stream reliability (not per-connection like TCP)
• More sophisticated than TCP (monotonically increasing packet numbers)
• Better handling of spurious retransmissions
• Pluggable congestion control

6. Connection Migration

Mobile device scenario:
WiFi (IP: 192.168.1.100) → Cellular (IP: 10.20.30.40)

TCP:  Connection breaks, must reconnect (new handshake)
QUIC: Connection continues seamlessly (connection ID stays same)

What is HTTP/3?

HTTP/3 is the third major version of HTTP, using QUIC as its transport instead of TCP.

HTTP Evolution

HTTP/1.1 (1997)
  ↓
  - Text-based protocol
  - One request per connection (or pipelining)
  - Head-of-line blocking

HTTP/2 (2015)
  ↓
  - Binary framing
  - Multiplexing over single TCP connection
  - Header compression (HPACK)
  - Still suffers from TCP head-of-line blocking

HTTP/3 (2022)
  ↓
  - Same semantics as HTTP/2
  - QUIC transport (UDP-based)
  - No head-of-line blocking
  - 0-RTT connection resumption
  - Better mobile performance

HTTP/3 Frame Types

HTTP/3 uses similar frames to HTTP/2 but adapted for QUIC:

Frame Types:
- DATA:        Application data (response body)
- HEADERS:     HTTP headers (compressed with QPACK)
- PRIORITY:    Stream priority hints
- CANCEL_PUSH: Cancel server push
- SETTINGS:    Connection parameters
- PUSH_PROMISE: Server push announcement
- GOAWAY:      Graceful shutdown
- MAX_PUSH_ID: Limit server push

QPACK Header Compression

HTTP/3 uses QPACK (QUIC-aware header compression) instead of HPACK:

• Dynamic table updates on dedicated stream
• Prevents head-of-line blocking from header compression
• Better performance on lossy networks

Why QUIC/HTTP/3 for Macula Mesh?

1. NAT Traversal

UDP is much easier to punch through NATs than TCP:

• Simpler state machines in NAT devices
• Easier simultaneous open
• Better compatibility with STUN/TURN

2. Multiplexing Without Head-of-Line Blocking

Perfect for distributed Erlang:

Process A → Stream 0:  send(...) → [packet lost!] → retransmit
Process B → Stream 1:  send(...) → delivered immediately!

With TCP: Process B would be blocked waiting for Process A's retransmit
With QUIC: Process B's stream is independent

3. Connection Migration

Edge devices often change networks:

IoT device switches from WiFi to cellular:
  - TCP: Connection lost, full reconnect
  - QUIC: Seamless migration, no interruption

4. Firewall Friendly

• Uses port 443 (standard HTTPS)
• Looks like HTTPS to middleboxes
• No special firewall rules needed

5. 0-RTT Resumption

Reconnecting nodes don't waste time:

Node rejoins mesh after brief disconnect:
  - TCP + TLS: 3 RTT to re-establish
  - QUIC 0-RTT: Immediate data transmission

6. Built-in Encryption

• TLS 1.3 integrated (not optional)
• Perfect forward secrecy
• No configuration needed

QUIC Protocol Details

Connection ID

QUIC uses Connection IDs instead of 4-tuple (src IP, src port, dst IP, dst port):

┌─────────────────────────────────────┐
│ QUIC Packet                         │
├─────────────────────────────────────┤
│ Header:                             │
│  - Connection ID: 0x1a2b3c4d...     │ ← Identifies connection
│  - Packet Number: 42                │
│  - Flags: ...                       │
├─────────────────────────────────────┤
│ Encrypted Payload                   │
└─────────────────────────────────────┘

Benefits:
- Connection survives IP address changes
- Load balancers can route without decryption
- NAT rebinding doesn't break connection

Stream Management

QUIC streams are lightweight:

% Each Erlang process can have its own stream
spawn(fun() ->
    {ok, StreamId} = quic:open_stream(Conn),
    quic:send(StreamId, Data),
    receive
        {stream_data, StreamId, Response} -> handle(Response)
    end,
    quic:close_stream(StreamId)
end).

% Streams are cheap! Can create thousands
% No overhead like TCP sockets

Congestion Control

QUIC implements multiple congestion control algorithms:

• Reno: Traditional TCP-like
• Cubic: Linux default (aggressive)
• BBR: Google's Bottleneck Bandwidth and RTT
• Custom: Can implement your own!

For Macula Mesh:

% Could implement edge-optimized congestion control
-module(macula_congestion).

estimate_bandwidth(Samples) ->
    % Use local measurements instead of RTT
    % Better for edge networks with variable latency
    ...

Flow Control

QUIC has flow control at two levels:

1. Stream-level: Each stream has credit
2. Connection-level: Overall connection credit

┌─────────────────────────────────────┐
│ Connection Credit: 1 MB             │
├─────────────────────────────────────┤
│ Stream 0:  256 KB credit            │
│ Stream 1:  512 KB credit            │
│ Stream 2:  256 KB credit            │
└─────────────────────────────────────┘

Prevents any single stream from starving the connection

QUIC/HTTP/3 Libraries for BEAM

1. quicer (Recommended)

Repository: https://github.com/emqx/quic License: Apache 2.0 Maintainer: EMQX Team (Erlang Solutions) Language: Erlang + C (NIF wrapper) Backend: Microsoft MsQuic

Overview

quicer is an Erlang NIF binding for Microsoft's MsQuic library. MsQuic is a production-grade QUIC implementation used by Windows, Azure, and various Microsoft services.

Architecture

┌─────────────────────────────────────┐
│ Erlang Application                  │
├─────────────────────────────────────┤
│ quicer (Erlang API)                 │
│  - quicer:listen/2                  │
│  - quicer:connect/3                 │
│  - quicer:send/2                    │
│  - quicer:recv/2                    │
├─────────────────────────────────────┤
│ quicer NIF (C)                      │
│  - Erlang ↔ MsQuic bridge          │
│  - Resource management              │
│  - Callback handling                │
├─────────────────────────────────────┤
│ MsQuic (C)                          │
│  - RFC 9000 QUIC implementation     │
│  - TLS 1.3 integration              │
│  - Platform-specific optimizations  │
├─────────────────────────────────────┤
│ OS Network Stack (UDP)              │
└─────────────────────────────────────┘

Features

✅ Supported:

• QUIC v1 (RFC 9000)
• TLS 1.3
• 0-RTT connection resumption
• Connection migration
• Multiple streams per connection
• Flow control and congestion control
• Both client and server modes

❌ Limitations:

• NIF dependency (requires C compilation)
• Tied to MsQuic release cycle
• Platform-specific quirks

API Examples

%% Server
start_server() ->
    %% Load certificate
    {ok, Cert} = file:read_file("server.crt"),
    {ok, Key} = file:read_file("server.key"),

    %% Listen options
    ListenOpts = #{
        cert => Cert,
        key => Key,
        alpn => ["macula/1.0"],
        peer_unidi_stream_count => 10,
        peer_bidi_stream_count => 10
    },

    %% Start listener
    {ok, Listener} = quicer:listen("0.0.0.0", 4433, ListenOpts),

    %% Accept loop
    accept_loop(Listener).

accept_loop(Listener) ->
    {ok, Conn} = quicer:accept(Listener, [], 5000),
    {ok, Conn} = quicer:handshake(Conn),

    %% Spawn handler
    spawn(fun() -> handle_connection(Conn) end),

    %% Continue accepting
    accept_loop(Listener).

handle_connection(Conn) ->
    %% Accept stream
    {ok, Stream} = quicer:accept_stream(Conn, []),

    %% Receive data
    {ok, Data} = quicer:recv(Stream, 0),

    %% Process and respond
    Response = process(Data),
    ok = quicer:send(Stream, Response),

    %% Close stream
    quicer:close_stream(Stream).

%% Client
start_client() ->
    %% Connect options
    ConnOpts = #{
        alpn => ["macula/1.0"],
        verify => verify_peer,
        cacertfile => "ca.crt"
    },

    %% Connect
    {ok, Conn} = quicer:connect("server.example.com", 4433, ConnOpts, 5000),

    %% Open stream
    StreamOpts = #{active => false},
    {ok, Stream} = quicer:start_stream(Conn, StreamOpts),

    %% Send data
    ok = quicer:send(Stream, <<"Hello, QUIC!">>),

    %% Receive response
    {ok, Response} = quicer:recv(Stream, 0),
    io:format("Received: ~p~n", [Response]),

    %% Close
    quicer:close_stream(Stream),
    quicer:close_connection(Conn).

Production Usage

EMQX: Used in EMQX 5.0+ for MQTT over QUIC

• Handles millions of concurrent connections
• Production-tested at scale
• Good performance characteristics

RabbitMQ: Experimental QUIC support via quicer

• AMQP 1.0 over QUIC transport
• Still in development

Pros & Cons

Pros:

• ✅ Battle-tested (MsQuic used in Windows, Azure)
• ✅ Actively maintained
• ✅ Good documentation
• ✅ Performance optimized
• ✅ Cross-platform (Linux, macOS, Windows)
• ✅ Production-ready (used in EMQX)

Cons:

• ❌ NIF dependency (requires compilation)
• ❌ Tied to MsQuic (external C library)
• ❌ Breaking changes between MsQuic versions
• ❌ Limited control over low-level behavior

Verdict: ⭐⭐⭐⭐⭐ Best choice for Macula Mesh

2. xquic (Alternative)

Repository: https://github.com/alibaba/xquic License: Apache 2.0 Maintainer: Alibaba Cloud Language: C + Erlang bindings Backend: xquic (Alibaba's QUIC)

Overview

xquic is Alibaba's in-house QUIC implementation, used in their edge CDN and cloud services.

Features

• QUIC v1 + IETF draft-29
• HTTP/3 support
• QUIC multipath extension
• BBR congestion control
• High performance (optimized for Alibaba scale)

Erlang Bindings

Erlang bindings exist but are less mature than quicer:

• https://github.com/emqx/xquic-erl (community maintained)

Pros & Cons

Pros:

• ✅ Very high performance
• ✅ Multipath QUIC support
• ✅ Used at Alibaba scale

Cons:

• ❌ Less mature Erlang bindings
• ❌ Documentation mostly in Chinese
• ❌ Smaller community
• ❌ Not as widely tested outside Alibaba

Verdict: ⭐⭐⭐ Good but less accessible

3. quinn (Rust, via Rustler)

Repository: https://github.com/quinn-rs/quinn License: Apache 2.0 / MIT Language: Rust Bindings: Could use Rustler for Erlang

Overview

quinn is a pure Rust QUIC implementation, considered one of the best non-C QUIC libraries.

Hypothetical Erlang Integration

┌─────────────────────────────────────┐
│ Erlang Application                  │
├─────────────────────────────────────┤
│ quinn_nif (Rustler)                 │
│  - Erlang ↔ Rust bridge            │
├─────────────────────────────────────┤
│ quinn (Rust)                        │
│  - Pure Rust QUIC implementation    │
├─────────────────────────────────────┤
│ tokio (Rust async runtime)          │
└─────────────────────────────────────┘

Pros & Cons

Pros:

• ✅ Pure Rust (memory safe)
• ✅ Excellent performance
• ✅ Active development
• ✅ Clean API

Cons:

• ❌ No official Erlang bindings
• ❌ Would need to build Rustler wrapper
• ❌ Rust async runtime complexity
• ❌ Additional development effort

Verdict: ⭐⭐ Interesting but requires work

4. Pure Erlang QUIC (Hypothetical)

Status: Doesn't exist Effort: 6-12 months of development

Why Pure Erlang?

Pros:

• ✅ No NIFs (easier deployment)
• ✅ Full control over implementation
• ✅ Could optimize for BEAM semantics
• ✅ Easier debugging

Cons:

• ❌ Huge development effort
• ❌ Likely slower than C/Rust
• ❌ Hard to match performance of tuned C libs
• ❌ Cryptography still needs NIFs

Feasibility Analysis

Components needed:
1. UDP socket handling         ✅ (gen_udp)
2. TLS 1.3 implementation     ⚠️  (ssl app, but need low-level access)
3. Packet parsing             ✅ (binary pattern matching)
4. Connection state machine   ✅ (gen_statem)
5. Stream multiplexing        ✅ (processes)
6. Flow control               ✅ (credits/backpressure)
7. Congestion control         ✅ (algorithms in Erlang)
8. Loss detection             ✅ (timers + state)
9. 0-RTT resumption          ⚠️  (needs crypto primitives)

Estimated effort: 3-6 person-months for basic implementation
                 12+ person-months for production-ready

Verdict: ⭐ Not practical for Phase 1

Library Comparison Matrix

Recommendation: quicer

For Macula Mesh Phase 1, quicer is the clear choice:

1. Production-ready: Used in EMQX with millions of connections
2. Well-documented: Good examples and API docs
3. Actively maintained: Regular updates, responsive maintainers
4. Erlang-native: Designed for BEAM from the ground up
5. MsQuic backend: Battle-tested in Microsoft services

Migration Path:

• Phase 1: Use quicer (proven, fast path to PoC)
• Phase 2: Optimize (profile, tune, maybe contribute improvements)
• Phase 3: Evaluate alternatives (if needed, could switch to pure Erlang or Rust)

Architecture Overview

High-Level Architecture

┌─────────────────────────────────────────────────────────┐
│                    Macula Mesh Network                  │
│                                                         │
│  ┌────────┐      ┌────────┐      ┌────────┐           │
│  │ Node A │◄────►│ Node B │◄────►│ Node C │           │
│  │        │      │        │      │        │           │
│  └────┬───┘      └────┬───┘      └────┬───┘           │
│       │               │               │                │
│       └───────────────┼───────────────┘                │
│                       │                                │
│                  HTTP/3 (QUIC)                         │
│              (Encrypted, Multiplexed)                  │
└─────────────────────────────────────────────────────────┘

Layered Architecture

┌─────────────────────────────────────────────────────────┐
│ Layer 5: Application                                    │
│  • Elixir/Erlang Applications                          │
│  • Standard distributed Erlang API                     │
│  • spawn/2, send/2, monitor/2, etc.                   │
└─────────────────────────────────────────────────────────┘
              ↓ (transparent to application)
┌─────────────────────────────────────────────────────────┐
│ Layer 4: WAMP Protocol (Optional Compatibility)        │
│  • publish(Topic, Data)                                │
│  • subscribe(Topic, Handler)                           │
│  • call(Procedure, Args)                               │
│  • register(Procedure, Handler)                        │
└─────────────────────────────────────────────────────────┘
              ↓
┌─────────────────────────────────────────────────────────┐
│ Layer 3: Mesh Routing                                  │
│  • Node discovery (bootstrap, mDNS, DHT)               │
│  • Membership management (SWIM gossip)                 │
│  • Topology management (k-regular graph)               │
│  • Message routing (DHT-based, O(log n) hops)         │
└─────────────────────────────────────────────────────────┘
              ↓
┌─────────────────────────────────────────────────────────┐
│ Layer 2: Macula Distribution Protocol                  │
│  • Message framing (wire protocol)                     │
│  • Handshake and authentication                        │
│  • Process messaging (SEND, LINK, MONITOR, etc.)      │
│  • Stream multiplexing (process ↔ stream mapping)     │
└─────────────────────────────────────────────────────────┘
              ↓
┌─────────────────────────────────────────────────────────┐
│ Layer 1: QUIC Transport (HTTP/3)                       │
│  • quicer (Erlang NIF)                                 │
│  • MsQuic (C library)                                  │
│  • UDP sockets                                         │
│  • TLS 1.3 encryption                                  │
│  • NAT traversal (STUN/ICE)                           │
└─────────────────────────────────────────────────────────┘

Component Architecture

┌─────────────────────────────────────────────────────────┐
│ Macula Node (Erlang/OTP Application)                   │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  ┌──────────────────────────────────────────────────┐  │
│  │ Supervision Tree                                 │  │
│  │                                                  │  │
│  │  macula_sup (top-level supervisor)              │  │
│  │    ├─ macula_connection_sup (connections)       │  │
│  │    ├─ macula_membership (SWIM gossip)           │  │
│  │    ├─ macula_topology (neighbor management)     │  │
│  │    ├─ macula_routing (DHT routing)              │  │
│  │    ├─ macula_discovery (node discovery)         │  │
│  │    ├─ macula_pubsub (pub/sub registry)          │  │
│  │    └─ macula_dist (distribution driver)         │  │
│  └──────────────────────────────────────────────────┘  │
│                                                         │
│  ┌──────────────────────────────────────────────────┐  │
│  │ Connection Pool (one per remote node)            │  │
│  │                                                  │  │
│  │  connection_1 (QUIC, 5 streams active)          │  │
│  │  connection_2 (QUIC, 3 streams active)          │  │
│  │  connection_3 (QUIC, 8 streams active)          │  │
│  └──────────────────────────────────────────────────┘  │
│                                                         │
│  ┌──────────────────────────────────────────────────┐  │
│  │ ETS Tables (shared state)                        │  │
│  │                                                  │  │
│  │  - membership_table (alive nodes)                │  │
│  │  - routing_table (DHT entries)                   │  │
│  │  - stream_registry (stream_id → pid)            │  │
│  │  - subscription_registry (topic → [pids])       │  │
│  └──────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────┘

Detailed Roadmap

Phase 1: Foundation (Weeks 1-4)

Goal: Prove HTTP/3 Can Carry Erlang Distribution Traffic

Week 1-2: QUIC Transport Layer

Objectives:

1. Set up quicer dependency
2. Create basic QUIC server/client
3. Implement bidirectional streaming
4. Build connection management

Deliverables:

%% File: macula_quic_echo.erl
%% Simple QUIC echo server/client to validate transport

-module(macula_quic_echo).
-export([start_server/1, start_client/2]).

start_server(Port) ->
    %% Generate self-signed cert for testing
    {Cert, Key} = macula_cert:generate_self_signed("localhost"),

    ListenOpts = #{
        cert => Cert,
        key => Key,
        alpn => ["macula/1.0"],
        idle_timeout_ms => 10000,
        peer_unidi_stream_count => 10,
        peer_bidi_stream_count => 100
    },

    {ok, Listener} = quicer:listen(Port, ListenOpts),
    io:format("QUIC server listening on port ~p~n", [Port]),

    accept_loop(Listener).

accept_loop(Listener) ->
    case quicer:accept(Listener, [], infinity) of
        {ok, Conn} ->
            %% Complete handshake
            {ok, Conn} = quicer:handshake(Conn),

            %% Get peer info
            {ok, PeerAddr} = quicer:getopt(Conn, peer_addr),
            io:format("Accepted connection from ~p~n", [PeerAddr]),

            %% Spawn connection handler
            spawn_link(fun() -> handle_connection(Conn) end),

            %% Continue accepting
            accept_loop(Listener);
        {error, Reason} ->
            io:format("Accept error: ~p~n", [Reason]),
            timer:sleep(1000),
            accept_loop(Listener)
    end.

handle_connection(Conn) ->
    %% Accept streams
    stream_accept_loop(Conn).

stream_accept_loop(Conn) ->
    case quicer:accept_stream(Conn, []) of
        {ok, Stream} ->
            %% Spawn stream handler
            spawn_link(fun() -> echo_stream(Stream) end),
            stream_accept_loop(Conn);
        {error, closed} ->
            io:format("Connection closed~n");
        {error, Reason} ->
            io:format("Stream accept error: ~p~n", [Reason])
    end.

echo_stream(Stream) ->
    case quicer:recv(Stream, 0) of
        {ok, Data} ->
            io:format("Received: ~p~n", [Data]),
            quicer:send(Stream, Data),
            echo_stream(Stream);
        {error, closed} ->
            quicer:close_stream(Stream);
        {error, Reason} ->
            io:format("Recv error: ~p~n", [Reason]),
            quicer:close_stream(Stream)
    end.

start_client(Host, Port) ->
    ConnOpts = #{
        alpn => ["macula/1.0"],
        verify => verify_none  %% For testing only!
    },

    io:format("Connecting to ~s:~p~n", [Host, Port]),
    {ok, Conn} = quicer:connect(Host, Port, ConnOpts, 5000),

    %% Open stream
    {ok, Stream} = quicer:start_stream(Conn, #{active => false}),

    %% Send message
    Message = <<"Hello, QUIC!">>,
    ok = quicer:send(Stream, Message),
    io:format("Sent: ~p~n", [Message]),

    %% Receive echo
    {ok, Echo} = quicer:recv(Stream, 0),
    io:format("Received echo: ~p~n", [Echo]),

    %% Close
    quicer:close_stream(Stream),
    quicer:close_connection(Conn).

Testing:

# Terminal 1: Start server
$ erl -pa _build/default/lib/*/ebin
1> macula_quic_echo:start_server(4433).

# Terminal 2: Connect client
$ erl -pa _build/default/lib/*/ebin
1> macula_quic_echo:start_client("localhost", 4433).

Success Criteria:

• ✅ Server accepts QUIC connections
• ✅ Client can connect and exchange data
• ✅ Multiple streams work concurrently
• ✅ Connection survives stream closure

Week 3: Message Framing Protocol

Objectives:

1. Design Macula wire protocol v1
2. Implement message encoding/decoding
3. Define message types for distribution

Wire Protocol Specification:

Macula Wire Protocol v1
=======================

All integers are big-endian.

Packet Format:
┌──────┬──────┬───────┬────────┬─────────┐
│ Ver  │ Type │ Flags │ Length │ Payload │
│ (1B) │ (1B) │ (2B)  │ (4B)   │ (N B)   │
└──────┴──────┴───────┴────────┴─────────┘

Version (1 byte):
  - 0x01: Version 1

Type (1 byte):
  - 0x01: HANDSHAKE
  - 0x02: HEARTBEAT
  - 0x03: SEND           (send message to process)
  - 0x04: REG_SEND       (send to registered name)
  - 0x05: EXIT           (process exit signal)
  - 0x06: LINK           (link processes)
  - 0x07: UNLINK         (unlink processes)
  - 0x08: MONITOR        (monitor process)
  - 0x09: DEMONITOR      (demonitor)
  - 0x0A: GROUP_LEADER   (group leader operations)
  - 0x0B: RPC            (remote procedure call)
  - 0x0C: SPAWN_REQUEST  (spawn on remote node)
  - 0x0D: SPAWN_REPLY    (spawn result)

Flags (2 bytes):
  Bit 0: COMPRESSED (payload is compressed)
  Bit 1: FRAGMENTED (part of fragmented message)
  Bit 2-15: Reserved

Length (4 bytes):
  - Payload length in bytes (max 16 MB)

Payload (N bytes):
  - Message-type specific data
  - Encoded using Erlang External Term Format (EETF)

Implementation:

%% File: macula_protocol.erl
-module(macula_protocol).
-export([encode/2, decode/1]).

%% Protocol version
-define(VERSION, 1).

%% Message types
-define(MSG_HANDSHAKE, 16#01).
-define(MSG_HEARTBEAT, 16#02).
-define(MSG_SEND, 16#03).
-define(MSG_REG_SEND, 16#04).
-define(MSG_EXIT, 16#05).
-define(MSG_LINK, 16#06).
-define(MSG_UNLINK, 16#07).
-define(MSG_MONITOR, 16#08).
-define(MSG_DEMONITOR, 16#09).
-define(MSG_GROUP_LEADER, 16#0A).
-define(MSG_RPC, 16#0B).
-define(MSG_SPAWN_REQUEST, 16#0C).
-define(MSG_SPAWN_REPLY, 16#0D).

%% Flags
-define(FLAG_COMPRESSED, 16#0001).
-define(FLAG_FRAGMENTED, 16#0002).

%% Encode message
encode(Type, Payload) when is_integer(Type), Type >= 0, Type =< 255 ->
    encode(Type, 0, Payload).

encode(Type, Flags, Payload) ->
    %% Serialize payload
    PayloadBin = term_to_binary(Payload, [compressed]),
    Length = byte_size(PayloadBin),

    %% Check size limit (16 MB)
    if
        Length > 16#1000000 ->
            {error, payload_too_large};
        true ->
            %% Build packet
            <<?VERSION:8, Type:8, Flags:16, Length:32, PayloadBin/binary>>
    end.

%% Decode message
decode(<<?VERSION:8, Type:8, Flags:16, Length:32, PayloadBin:Length/binary, Rest/binary>>) ->
    %% Deserialize payload
    Payload = binary_to_term(PayloadBin),

    {ok, #{
        type => Type,
        flags => Flags,
        payload => Payload
    }, Rest};
decode(Bin) when byte_size(Bin) < 8 ->
    {error, insufficient_data};
decode(<<Version:8, _/binary>>) when Version =/= ?VERSION ->
    {error, {unsupported_version, Version}};
decode(_) ->
    {error, invalid_packet}.

%% Helper: Encode SEND message
encode_send(FromPid, ToPid, Message) ->
    encode(?MSG_SEND, {FromPid, ToPid, Message}).

%% Helper: Encode LINK message
encode_link(Pid1, Pid2) ->
    encode(?MSG_LINK, {Pid1, Pid2}).

%% Helper: Encode MONITOR message
encode_monitor(Pid, Ref, MonitoredPid) ->
    encode(?MSG_MONITOR, {Pid, Ref, MonitoredPid}).

%% Helper: Encode SPAWN_REQUEST message
encode_spawn_request(ReqId, Module, Function, Args) ->
    encode(?MSG_SPAWN_REQUEST, {ReqId, Module, Function, Args}).

Handshake Protocol:

%% File: macula_handshake.erl
-module(macula_handshake).
-export([perform/1, accept/1]).

-record(handshake, {
    version = ?MACULA_VERSION,
    node_name :: atom(),
    node_id :: binary(),      %% SHA256(certificate)
    capabilities = [] :: [atom()],
    creation :: integer(),    %% Node start time
    challenge :: binary()     %% Random bytes for auth
}).

%% Initiate handshake (client side)
perform(Conn) ->
    %% Generate our handshake
    Handshake = #handshake{
        node_name = node(),
        node_id = macula_identity:node_id(),
        capabilities = [compression, rpc, monitoring, streams],
        creation = erlang:system_time(millisecond),
        challenge = crypto:strong_rand_bytes(32)
    },

    %% Send handshake
    Packet = macula_protocol:encode(?MSG_HANDSHAKE, Handshake),
    {ok, Stream} = quicer:start_stream(Conn, #{active => false}),
    ok = quicer:send(Stream, Packet),

    %% Receive remote handshake
    {ok, ResponsePacket} = quicer:recv(Stream, 0),
    {ok, #{payload := RemoteHandshake}, _} = macula_protocol:decode(ResponsePacket),

    %% Verify compatibility
    case check_compatibility(Handshake, RemoteHandshake) of
        ok ->
            %% Send acknowledgment
            Ack = macula_protocol:encode(?MSG_HANDSHAKE, {ack, Handshake#handshake.challenge}),
            ok = quicer:send(Stream, Ack),
            {ok, RemoteHandshake};
        {error, Reason} ->
            {error, Reason}
    end.

%% Accept handshake (server side)
accept(Conn) ->
    %% Accept stream
    {ok, Stream} = quicer:accept_stream(Conn, []),

    %% Receive handshake
    {ok, Packet} = quicer:recv(Stream, 0),
    {ok, #{payload := RemoteHandshake}, _} = macula_protocol:decode(Packet),

    %% Generate our handshake
    Handshake = #handshake{
        node_name = node(),
        node_id = macula_identity:node_id(),
        capabilities = [compression, rpc, monitoring, streams],
        creation = erlang:system_time(millisecond),
        challenge = crypto:strong_rand_bytes(32)
    },

    %% Verify compatibility
    case check_compatibility(Handshake, RemoteHandshake) of
        ok ->
            %% Send our handshake
            Response = macula_protocol:encode(?MSG_HANDSHAKE, Handshake),
            ok = quicer:send(Stream, Response),

            %% Wait for acknowledgment
            {ok, AckPacket} = quicer:recv(Stream, 0),
            {ok, #{payload := {ack, Challenge}}, _} = macula_protocol:decode(AckPacket),

            %% Verify challenge
            if
                Challenge =:= Handshake#handshake.challenge ->
                    {ok, RemoteHandshake};
                true ->
                    {error, invalid_challenge}
            end;
        {error, Reason} ->
            {error, Reason}
    end.

check_compatibility(Local, Remote) ->
    %% Check version
    if
        Local#handshake.version =/= Remote#handshake.version ->
            {error, version_mismatch};
        true ->
            %% Check capabilities
            CommonCaps = sets:intersection(
                sets:from_list(Local#handshake.capabilities),
                sets:from_list(Remote#handshake.capabilities)
            ),
            if
                sets:size(CommonCaps) > 0 -> ok;
                true -> {error, no_common_capabilities}
            end
    end.