UART Tunnel Network Interface Design Specification

This document specifies the component-level design of the netif_uart_tunnel component, which provides full TCP/IP stack functionality in QEMU emulation by tunnelling IP packets (encapsulated in Ethernet frames) over UART1.

Document Version: 1.0 Last Updated: 2025-03-16

Source files reviewed:

  • main/components/netif_uart_tunnel/netif_uart_tunnel_sim.c

  • main/components/netif_uart_tunnel/netif_uart_tunnel_sim.h

  • tools/qemu/network/serial_tun_bridge.py

  • main/components/netif_uart_tunnel/README.md

This spec is a documentation catch-up — the code is stable and correct. All SPECs below describe what the implementation actually does. Two known deviations from requirements are documented honestly: the static IP configuration (vs DHCP) and the 115200 baud rate (vs 100 KB/s target).

See Network Tunnel Component De... (SPEC_ARCH_NETIF_1) for the high-level architectural overview.

Component Architecture

Design Specification: UART Tunnel Component Architecture SPEC_NETIF_UART_ARCH_1
status: implemented
tags: netif, qemu, uart, architecture

Description: The netif_uart_tunnel component implements a lwIP network interface that tunnels IP packets over UART1, providing full TCP/IP stack functionality in QEMU emulation. The component is conditionally compiled only when CONFIG_TARGET_EMULATOR is set (hardware builds use the WiFi manager instead).

Component Location: main/components/netif_uart_tunnel/

Key Files:

  • netif_uart_tunnel_sim.c — Implementation (_sim suffix marks simulator-only)

  • netif_uart_tunnel_sim.h — Public API (netif_uart_tunnel_init, _deinit, _get_handle)

  • CMakeLists.txt — Build integration

Threading Model:

  • uart_rx_task (FreeRTOS task, priority 2, stack 4096 bytes): polls UART1 for incoming frames and injects received Ethernet frames into lwIP via esp_netif_receive

  • TX path: lwIP calls netif_linkoutput synchronously in the tcpip_thread context; netif_linkoutput writes the framed packet directly to UART1

Data Flow:

┌─────────────────────────────────────────────────────────────┐
│  ESP32 (QEMU)                                               │
│  Application → lwIP → etharp_output → netif_linkoutput     │
│                                           │                 │
│                                         UART1 TX (GPIO17)  │
└─────────────────────────────────────────────────────────────┘
         ↕  Unix socket (temp/esp32-uart1.sock)
┌─────────────────────────────────────────────────────────────┐
│  Host (Linux)                                               │
│  serial_tun_bridge.py                                       │
│  UART socket ↔ TUN device (tun0)                           │
│  tun0: 192.168.100.1/24                                     │
└─────────────────────────────────────────────────────────────┘

UART Configuration (hardcoded in implementation):

  • Port: UART1

  • TX pin: GPIO17, RX pin: GPIO16

  • Baud rate: 115200 (UART_BAUD_RATE constant)

  • Buffer: 2048 bytes for both RX and TX

Note

The header file comment mentions GPIO4/GPIO5 as defaults; the actual implementation uses GPIO17/GPIO16. The code values take precedence.

Network Configuration (default, passed by caller in main.c ):

  • ESP32 IP: 192.168.100.2

  • Gateway / Host: 192.168.100.1

  • Netmask: 255.255.255.0

  • ESP32 MAC: 02:00:00:00:00:02 (static, set directly on lwip_netif->hwaddr)

lwIP Interface Flags:

NETIF_FLAG_ETHARP | NETIF_FLAG_ETHERNET | NETIF_FLAG_BROADCAST | NETIF_FLAG_AUTOUP

The interface operates as a full Ethernet interface so that lwIP’s ARP layer produces complete Ethernet frames. The host-side bridge strips/adds the Ethernet header when interfacing with the TUN device (which expects raw IP packets).

Static ARP Entry for Gateway:

An ARP entry is pre-populated for the gateway MAC (02:00:00:00:00:01) via etharp_add_static_entry during init. This avoids ARP request/response cycles that would fail over a point-to-point UART link.

Lifecycle:

  • netif_uart_tunnel_init(config) — initialises UART, creates esp_netif, sets static IP, starts uart_rx_task. Returns ESP_ERR_INVALID_STATE if called more than once (module uses static state).

  • netif_uart_tunnel_deinit() — stops uart_rx_task (via vTaskDelete), destroys netif, deletes UART driver.

  • netif_uart_tunnel_get_handle() — returns the esp_netif_t* handle for event handler registration or status queries.

Known Limitation: This component is QEMU-only. It MUST NOT be linked in hardware builds. See Conditional Compilation — Q... (SPEC_NETIF_UART_COND_1) for build guard details.

Wire Protocol

Design Specification: UART Wire Protocol — Length-Prefix Framing SPEC_NETIF_UART_PROTO_1
status: implemented
tags: netif, qemu, protocol, framing

Description: IP packets (wrapped in Ethernet frames) are transported over UART1 using a simple 2-byte length-prefix framing protocol. There is no checksum or CRC at this layer — TCP handles end-to-end integrity.

Frame Format:

┌─────────────────┬────────────────────────────────────────────┐
│  LENGTH (2 B)   │  DATA (N bytes — complete Ethernet frame)  │
│  big-endian     │  N = 14 (Ethernet header) + IP payload     │
│  uint16_t       │  max N = 1500 bytes (MTU)                  │
└─────────────────┴────────────────────────────────────────────┘

Ethernet Encapsulation:

The DATA field is a complete Ethernet frame, not a raw IP packet. lwIP operates as an Ethernet interface (NETIF_FLAG_ETHARP | NETIF_FLAG_ETHERNET), so it produces and consumes full Ethernet frames including the 14-byte header.

The Python bridge strips the 14-byte Ethernet header before writing to the TUN device (which handles raw IP), and prepends a synthetic Ethernet header when forwarding packets from TUN to the serial socket.

Framing Rules:

  • LENGTH is a 2-byte big-endian uint16_t encoding the byte count of DATA only (the 2-byte header itself is not included)

  • Receiver MUST read exactly 2 bytes for LENGTH, then exactly LENGTH bytes for DATA

  • Frames with LENGTH == 0 or LENGTH > 1500 are invalid; the receiver MUST flush UART and resync

TX Path (ESP32 → Host):

  1. lwIP calls netif_linkoutput with a pbuf chain (tcpip_thread context)

  2. pbuf chain is copied into a contiguous frame_buf via pbuf_copy_partial

  3. 2-byte big-endian length header written to UART1 via uart_write_bytes

  4. Frame data written to UART1 via uart_write_bytes

  5. frame_buf freed

RX Path (Host → ESP32):

  1. uart_rx_task reads 2-byte length header with UART_READ_TIMEOUT_MS = 100 ms

  2. Reads LENGTH bytes of frame data with a 1-second timeout

  3. Passes frame to lwIP via esp_netif_receive

  4. lwIP calls tcpip_input → processes Ethernet frame through ARP / IP stack

Error Recovery:

  • No data within 100 ms timeout: vTaskDelay(10 ms) to avoid busy-loop, retry

  • Invalid LENGTH (0 or > 1500): log warning, uart_flush(), vTaskDelay(100 ms), retry

  • Incomplete frame data (1-second timeout): log warning, discard packet, retry

  • esp_netif_receive failure: log warning, discard, continue

Startup Delay:

After uart_flush() on task start, the RX task waits 1 second before processing to allow QEMU and the bridge script to reach a stable state.

Host-Side Bridge

Design Specification: Host-Side Serial-TUN Bridge Script SPEC_NETIF_UART_BRIDGE_1
status: implemented
tags: netif, qemu, python, bridge, tooling

Description: tools/qemu/network/serial_tun_bridge.py is the host-side component that bridges QEMU UART1 to the Linux TUN network device, giving the simulated ESP32 real network connectivity.

Script Location: tools/qemu/network/serial_tun_bridge.py

Prerequisites:

  • Linux host (TUN device creation requires /dev/net/tun and Linux-specific ioctl calls; macOS is not supported)

  • Root privileges (for TUN device creation)

  • Python ≥ 3.8

  • Optional: pytun package — falls back to manual ioctl if absent

Note

REQ_NETIF_TUNNEL_3 AC-4 specifies Linux and macOS support. The current implementation is Linux-only due to TUN device creation via Linux ioctls.

Architecture:

QEMU UART1 (Unix socket at temp/esp32-uart1.sock)
     ↕ Ethernet frames with 2-byte length prefix
SerialTunBridge.serial_to_tun() / tun_to_serial()
     ↕ Raw IP packets (Ethernet header stripped / added)
TUN device tun0 (192.168.100.1/24)
     ↕
Linux host network stack

TUN Device:

  • Name: tun0 (hardcoded)

  • Host IP: 192.168.100.1

  • Netmask: 255.255.255.0

  • MTU: 1500 bytes

  • Mode: IFF_TUN | IFF_NO_PI (raw IP, no packet info header)

Ethernet/IP Translation:

  • serial → TUN: Strip 14-byte Ethernet header from received frame; write raw IP packet to TUN device

  • TUN → serial: Read raw IP from TUN; prepend synthetic Ethernet header (DST = 02:00:00:00:00:02, SRC = 02:00:00:00:00:01, EtherType = 0x0800); send length-prefixed Ethernet frame to socket

Socket Connection:

  • QEMU exposes UART1 as a Unix domain socket at {PROJECT_DIR}/temp/esp32-uart1.sock

  • Bridge connects via AF_UNIX / SOCK_STREAM

  • Auto-reconnects if the socket disappears (e.g. QEMU restart); TUN device persists across reconnects

Execution Modes:

  • Verbose (default): sudo python3 serial_tun_bridge.py — INFO logs to console

  • Quiet: sudo python3 serial_tun_bridge.py --quiet — only errors logged to temp/tun_errors.log

  • The run_qemu.sh script automatically starts the bridge in quiet mode

Known Limitations:

  • Linux only (TUN ioctls are Linux-specific)

  • Requires sudo

  • TUN device name tun0 is hardcoded; conflicts if another process uses it

  • No graceful shutdown on SIGINT from run_qemu.sh (sends SIGTERM)

IP Configuration

Design Specification: IP Configuration and DHCP Client Integration SPEC_NETIF_UART_DHCP_1
status: implemented
tags: netif, qemu, dhcp, ip
links outgoing: REQ_NETIF_TUNNEL_4

Description: The tunnel interface uses static IP configuration. The DHCP client is explicitly disabled. REQ_NETIF_TUNNEL_4 specifies DHCP client support as a goal; the current implementation satisfies the intent (known IP on a known subnet) via static configuration rather than dynamic discovery.

Implemented Behaviour:

  • DHCP client is explicitly stopped via esp_netif_dhcpc_stop() in netif_uart_tunnel_init

  • Static IP is configured via esp_netif_set_ip_info() using caller-supplied values

  • Caller in main.c provides: IP 192.168.100.2, GW 192.168.100.1, mask 255.255.255.0

  • IP configuration is logged to console at INFO level after init (satisfies AC-4)

Static ARP Entry for Gateway:

A static ARP entry is pre-populated for the gateway to avoid ARP request/response cycles that cannot succeed over a point-to-point UART link:

struct eth_addr gw_mac = {{0x02, 0x00, 0x00, 0x00, 0x00, 0x01}};
etharp_add_static_entry(&gw_ip, &gw_mac);

Rationale for Static Configuration:

A UART tunnel is a point-to-point link. DHCP discovery would require the host-side bridge to run a DHCP server. Static configuration is simpler, deterministic, and fully sufficient for emulation purposes.

AC Coverage vs REQ_NETIF_TUNNEL_4:

  • AC-1 (support lwIP DHCP client): ⚠️ Not implemented — static IP used instead. DHCP adds complexity without benefit for QEMU emulation.

  • AC-2 (DHCP discovery via tunnel): Not applicable with static config.

  • AC-3 (obtain IP from host DHCP): Not implemented — static IP is the equivalent outcome.

  • AC-4 (log IP to console): ✅ Implemented via ESP_LOGI in netif_uart_tunnel_init.

Note

AC-1 through AC-3 of REQ_NETIF_TUNNEL_4 are not met by the current implementation. The requirement should be revisited if dynamic IP assignment ever becomes necessary. For the current QEMU emulation use case, static IP is the accepted approach.

Conditional Compilation

Design Specification: Conditional Compilation — QEMU-Only Build Guard SPEC_NETIF_UART_COND_1
status: implemented
tags: netif, qemu, build, kconfig
links outgoing: REQ_NETIF_TUNNEL_5

Description: The tunnel component is conditionally included in the build using CMake and Kconfig guards, ensuring zero overhead on hardware builds.

Application-Level Guard in main.c :

#ifdef CONFIG_TARGET_EMULATOR
    netif_uart_tunnel_init(&tunnel_config);
#else
    // WiFi init for real hardware
    wifi_manager_init();
#endif

The CONFIG_TARGET_EMULATOR symbol is set by the project’s Kconfig when the build target is the QEMU emulator.

CMakeLists.txt Guard:

The component’s CMakeLists.txt uses idf_build_get_property to check the IDF target and registers the component only for QEMU builds. The component source files are not compiled at all for hardware targets.

Kconfig Options (from README):

  • CONFIG_NETIF_UART_TUNNEL_ENABLED — Enable tunnel (default: y for QEMU)

  • CONFIG_NETIF_UART_TUNNEL_UART_NUM — UART port (default: 1)

  • CONFIG_NETIF_UART_TUNNEL_BAUD_RATE — Baud rate (default: 921600 per README, but the implementation constant UART_BAUD_RATE is currently 115200 — see Performance Characteristics... (SPEC_NETIF_UART_PERF_1) for discussion of this inconsistency)

Result on Hardware Builds:

  • Component source files are not compiled

  • No symbols linked

  • Zero flash / RAM overhead

  • netif_uart_tunnel_init call is preprocessed away

Performance and Known Limitations

Design Specification: Performance Characteristics and Known Limitations SPEC_NETIF_UART_PERF_1
status: implemented
tags: netif, qemu, performance, limitations

Description: Documents the measured and expected performance of the UART tunnel and all known limitations relevant to emulation use.

Throughput:

  • Actual baud rate: 115200 bps (UART_BAUD_RATE constant in netif_uart_tunnel_sim.c)

  • Theoretical raw throughput: ~14.4 KB/s (115200 / 8 bits, no overhead)

  • With 2-byte framing overhead per packet: ~14.3 KB/s effective

  • Network Tunnel Component De... (SPEC_ARCH_NETIF_1) notes ~10 KB/s measured throughput in practice

  • ⚠️ Does NOT meet REQ_NETIF_TUNNEL_NF_1 AC-1 (100 KB/s minimum)

  • Increasing baud rate to 921600 would yield ~115 KB/s, satisfying the target

  • README documents 921600 as the intended baud rate, but the code constant has not been updated to match. The code value (115200) is what actually runs.

Packet Loss Handling:

  • Invalid LENGTH (0 or > 1500): UART flushed, 100 ms delay, retry loop — satisfies REQ_NETIF_TUNNEL_NF_2 AC-1

  • Incomplete frame data (1-second timeout): warning logged, packet discarded — satisfies REQ_NETIF_TUNNEL_NF_2 AC-1

  • TCP layer recovery from packet loss: inherent in the TCP protocol — satisfies REQ_NETIF_TUNNEL_NF_2 AC-2

  • Packet statistics: s_rx_count and s_tx_count debug counters logged via ESP_LOGD — satisfies REQ_NETIF_TUNNEL_NF_2 AC-3

RX Task Design:

  • Stack: 4096 bytes

  • Priority: 2 (lower than display_logic at priority 3 to avoid blocking UI)

  • Length header read timeout: 100 ms (UART_READ_TIMEOUT_MS)

  • Frame data read timeout: 1 second

  • 1-second startup delay after uart_flush() for stability

  • 10 ms yield on timeout to avoid busy-looping when there is no traffic

Known Limitations:

  1. 115200 baud: Below the 100 KB/s NF target (fixable by updating UART_BAUD_RATE to 921600)

  2. QEMU-only / Linux host required: serial_tun_bridge.py uses Linux TUN ioctls; macOS is not supported

  3. Root required: TUN device creation requires sudo

  4. Static IP only: No DHCP client (see IP Configuration and DHCP C... (SPEC_NETIF_UART_DHCP_1))

  5. Single instance: Module uses static state; netif_uart_tunnel_init returns ESP_ERR_INVALID_STATE on a second call

  6. No graceful RX task stop: netif_uart_tunnel_deinit calls vTaskDelete on the RX task handle (no cooperative shutdown signal)

  7. TUN device name hardcoded: tun0 in bridge script; conflicts if another process already owns that interface

Documentation

Design Specification: Emulation Setup Documentation SPEC_NETIF_UART_DOC_1
status: implemented
tags: netif, qemu, documentation
links outgoing: REQ_NETIF_TUNNEL_DOC_1

Description: Documents where emulation setup instructions are maintained and what they cover.

Documentation Locations:

  • main/components/netif_uart_tunnel/README.md — Component-level README with quick-start, configuration, usage, and troubleshooting sections

  • tools/qemu/run_qemu.sh — QEMU launch script (self-documenting via inline comments)

  • tools/qemu/network/serial_tun_bridge.py — Bridge script with module-level docstring describing protocol, usage, and requirements

  • docs/90_guides/switching-dev-modes.rst — Developer guide for switching between QEMU and real hardware

Coverage by Requirement AC:

  • REQ_NETIF_TUNNEL_DOC_1 AC-1 (QEMU build / install instructions): Covered in README and tools/qemu/ scripts

  • REQ_NETIF_TUNNEL_DOC_1 AC-2 (TAP/TUN interface setup): Covered in serial_tun_bridge.py docstring and README

  • REQ_NETIF_TUNNEL_DOC_1 AC-3 (example commands): Covered in README troubleshooting section and run_qemu.sh

  • REQ_NETIF_TUNNEL_DOC_1 AC-4 (limitations vs hardware): Covered in README limitations section and Performance Characteristics... (SPEC_NETIF_UART_PERF_1)

Traceability

All traceability is automatically generated by Sphinx-Needs based on the :links: attributes in each SPEC above.

ID

Title

Status

Tags

SPEC_NETIF_UART_ARCH_1

UART Tunnel Component Architecture

implemented

netif; qemu; uart; architecture

SPEC_NETIF_UART_BRIDGE_1

Host-Side Serial-TUN Bridge Script

implemented

netif; qemu; python; bridge; tooling

SPEC_NETIF_UART_COND_1

Conditional Compilation — QEMU-Only Build Guard

implemented

netif; qemu; build; kconfig

SPEC_NETIF_UART_DHCP_1

IP Configuration and DHCP Client Integration

implemented

netif; qemu; dhcp; ip

SPEC_NETIF_UART_DOC_1

Emulation Setup Documentation

implemented

netif; qemu; documentation

SPEC_NETIF_UART_PERF_1

Performance Characteristics and Known Limitations

implemented

netif; qemu; performance; limitations

SPEC_NETIF_UART_PROTO_1

UART Wire Protocol — Length-Prefix Framing

implemented

netif; qemu; protocol; framing