Web Server Design Specification¶
This document specifies the design of the ESP32 web server component that provides HTTP-based user interface, WiFi configuration, and device configuration capabilities.
Architecture Overview¶
Overview:
The web server component provides a layered HTTP server architecture based on ESP-IDF’s Architecture Layers:
Component Dependencies:
Data Flow Example (Schema-Driven Configuration Update): Browser GET /api/config/schema
↓
schema_get_handler() → config_get_schema_json()
↓
Return embedded config_schema.json
↓
Browser generates dynamic form from schema
↓
Browser POST /api/config (flat JSON)
↓
config_set_handler() → config_set_all_from_json()
↓
Config manager parses JSON using schema for type validation
↓
Bulk NVS write (all fields, single commit)
↓
Return HTTP 200 with success response
|
Static File Embedding¶
Problem: ESP32 has no filesystem by default. HTML/CSS/JS files must be embedded in firmware flash. Solution:
Use ESP-IDF’s CMakeLists.txt Configuration: idf_component_register(
SRCS "web_server.c" "wifi_manager.c"
INCLUDE_DIRS "."
EMBED_FILES
"www/index.html"
"www/wifi-setup.html"
"www/settings.html"
"www/favicon.svg"
"www/css/style.css"
"www/js/app.js"
REQUIRES
config_manager
esp_wifi
esp_http_server
json
)
Generated Binary Symbols: For each file
C Code Access: extern const uint8_t index_html_start[] asm("_binary_index_html_start");
extern const uint8_t index_html_end[] asm("_binary_index_html_end");
size_t size = index_html_end - index_html_start;
httpd_resp_send(req, (const char*)index_html_start, size);
File Lookup Function: The static esp_err_t get_embedded_file(const char *filename,
const uint8_t **data,
size_t *size)
{
// Strip query parameters (?v=2 for cache busting)
char clean_filename[128];
strncpy(clean_filename, filename, sizeof(clean_filename) - 1);
char *query = strchr(clean_filename, '?');
if (query != NULL) *query = '\0';
if (strcmp(clean_filename, "/index.html") == 0) {
*data = index_html_start;
*size = index_html_end - index_html_start;
}
// ... more file mappings ...
else {
return ESP_ERR_NOT_FOUND;
}
return ESP_OK;
}
MIME Type Detection: static const char *get_mime_type(const char *filename)
{
const char *ext = strrchr(filename, '.');
if (strcmp(ext, ".html") == 0) return "text/html";
if (strcmp(ext, ".css") == 0) return "text/css";
if (strcmp(ext, ".js") == 0) return "application/javascript";
if (strcmp(ext, ".svg") == 0) return "image/svg+xml";
if (strcmp(ext, ".json") == 0) return "application/json";
return "text/plain";
}
Cache Control: For template/development, caching is disabled: httpd_resp_set_hdr(req, "Cache-Control", "no-cache, no-store, must-revalidate");
httpd_resp_set_hdr(req, "Pragma", "no-cache");
httpd_resp_set_hdr(req, "Expires", "0");
|
URI Routing and Handlers¶
Handler Registration: All URI handlers are registered in Main Pages:
Static Assets:
WiFi Management API:
Configuration Management API:
System Diagnostics API:
CORS Support:
Handler Registration Pattern: httpd_uri_t scan_uri = {
.uri = "/scan",
.method = HTTP_GET,
.handler = scan_handler,
.user_ctx = NULL
};
esp_err_t ret = httpd_register_uri_handler(server, &scan_uri);
ESP_LOGI(TAG, "Registered handler for '/scan' - %s",
ret == ESP_OK ? "OK" : esp_err_to_name(ret));
Configuration:
|
Configuration API Design¶
Architecture: The web server provides HTTP transport for configuration operations. All configuration logic is delegated to the config manager component. Endpoints:
— Endpoint: GET /api/config/schema Purpose: Return JSON schema for frontend UI generation Request: None Response (200 OK): {
"title": "ESP32 Device Configuration",
"version": "1.0.0",
"fields": [
{
"key": "wifi_ssid",
"type": "string",
"label": "WiFi Network Name",
"default": "",
"required": false,
"maxLength": 63,
"group": "wifi"
},
{
"key": "ap_ssid",
"type": "string",
"label": "AP SSID",
"default": "ESP32-Setup",
"required": true,
"maxLength": 32,
"group": "ap"
}
]
}
Implementation: static esp_err_t schema_get_handler(httpd_req_t *req)
{
httpd_resp_set_hdr(req, "Content-Type", "application/json");
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
char *schema_json = NULL;
esp_err_t ret = config_get_schema_json(&schema_json);
if (ret != ESP_OK) {
httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR,
"Failed to load configuration schema");
return ESP_FAIL;
}
httpd_resp_send(req, schema_json, HTTPD_RESP_USE_STRLEN);
// Note: schema_json points to embedded flash data, no free() needed
return ESP_OK;
}
— Endpoint: GET /api/config Purpose: Retrieve all current configuration values Request: None Response (200 OK): {
"wifi_ssid": "MyNetwork",
"wifi_pass": "",
"ap_ssid": "ESP32-Setup",
"ap_pass": "12345678",
"led_count": 50,
"led_bright": 128,
"device_name": "MyDevice"
}
Implementation: static esp_err_t config_get_handler(httpd_req_t *req)
{
httpd_resp_set_hdr(req, "Content-Type", "application/json");
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
char *config_json = NULL;
esp_err_t ret = config_get_all_as_json(&config_json);
if (ret != ESP_OK) {
httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR,
"Failed to read configuration");
return ESP_FAIL;
}
httpd_resp_send(req, config_json, HTTPD_RESP_USE_STRLEN);
free(config_json);
return ESP_OK;
}
— Endpoint: POST /api/config Purpose: Update configuration values Request Body: {
"wifi_ssid": "NewNetwork",
"wifi_pass": "newpassword123",
"ap_ssid": "MyDevice-AP",
"led_count": 144,
"device_name": "UpdatedDevice"
}
Response (200 OK): {
"status": "success",
"message": "Configuration saved successfully"
}
Implementation: static esp_err_t config_set_handler(httpd_req_t *req)
{
httpd_resp_set_hdr(req, "Content-Type", "application/json");
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
// Read request body
char content[1024];
int ret = httpd_req_recv(req, content, sizeof(content) - 1);
if (ret <= 0) {
httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "Invalid request body");
return ESP_FAIL;
}
content[ret] = '\0';
esp_err_t config_ret = config_set_all_from_json(content);
if (config_ret != ESP_OK) {
httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "Configuration update failed");
return ESP_FAIL;
}
const char *response = "{\"status\":\"success\",\"message\":\"Configuration saved successfully\"}";
httpd_resp_send(req, response, HTTPD_RESP_USE_STRLEN);
return ESP_OK;
}
— Endpoint: POST /api/config/reset Purpose: Reset all configuration to factory defaults Request: None Response (200 OK): {
"status": "success",
"message": "Configuration reset to factory defaults"
}
Implementation: static esp_err_t config_reset_handler(httpd_req_t *req)
{
httpd_resp_set_hdr(req, "Content-Type", "application/json");
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
esp_err_t ret = config_factory_reset();
if (ret != ESP_OK) {
httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR,
"Factory reset failed");
return ESP_FAIL;
}
const char *response = "{\"status\":\"success\",\"message\":\"Configuration reset to factory defaults\"}";
httpd_resp_send(req, response, HTTPD_RESP_USE_STRLEN);
return ESP_OK;
}
|
Endpoint: GET /scan Purpose: Scan for available WiFi networks Response: {
"networks": [
{
"ssid": "NetworkName",
"rssi": -45,
"authmode": 3
}
]
}
Implementation Notes:
— Endpoint: POST /connect Purpose: Connect to WiFi network with provided credentials Request: {
"ssid": "NetworkName",
"password": "secretpassword"
}
Response (200 OK): {
"success": true
}
Integration: Calls — Endpoint: GET /status Purpose: Get current WiFi connection status Response: {
"mode": 1,
"ssid": "ConnectedNetwork",
"rssi": -52,
"has_credentials": true,
"ip": "192.168.1.100"
}
Integration: Calls — Endpoint: POST /reset Purpose: Clear WiFi credentials and restart device in AP mode Response: {
"success": true,
"message": "Device will restart in AP mode in 3 seconds"
}
Implementation: Calls |
Endpoint: GET /api/system/health Purpose: System diagnostics and health monitoring Response: {
"uptime_seconds": 3542.5,
"free_heap_bytes": 125432,
"minimum_free_heap_bytes": 98234,
"heap_fragmentation_percent": 21.7,
"configuration": {
"status": "healthy",
"api_version": "2.0"
},
"wifi": {
"status": "connected",
"ssid": "MyNetwork",
"rssi": -48
},
"overall_status": "healthy",
"device_type": "ESP32 Template",
"firmware_version": "1.0.0"
}
Health Assessment Logic: bool system_healthy = (wifi_status == ESP_OK) &&
(free_heap > 50000); // At least 50KB free
Use Cases:
|
Configuration Manager Integration¶
Integration Points: The web server integrates with the config manager component through a well-defined C API: Include Header: #include "config_manager.h" // Provides config_get_*, config_set_* functions
Reading Configuration: char wifi_ssid[CONFIG_STRING_MAX_LEN + 1];
uint16_t led_count;
esp_err_t ret = config_get_string(CONFIG_WIFI_SSID, wifi_ssid, sizeof(wifi_ssid));
if (ret != ESP_OK) {
ESP_LOGE(TAG, "Failed to get WiFi SSID: %s", esp_err_to_name(ret));
// Use default or return error to client
}
config_get_uint16(CONFIG_LED_COUNT, &led_count);
Writing Configuration: const char *new_ssid = "NewNetwork";
esp_err_t ret = config_set_string(CONFIG_WIFI_SSID, new_ssid);
if (ret != ESP_OK) {
ESP_LOGE(TAG, "Failed to set WiFi SSID: %s", esp_err_to_name(ret));
httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "Invalid WiFi SSID");
return ESP_FAIL;
}
Factory Reset: esp_err_t ret = config_factory_reset();
if (ret != ESP_OK) {
httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Factory reset failed");
return ESP_FAIL;
}
Error Code Mapping:
Separation of Concerns:
Note: Config Manager does NOT know about HTTP, JSON, or web interfaces. Web Server does NOT know about NVS internals or storage keys. |
WiFi Manager Integration¶
Integration Points: #include "wifi_manager.h"
WiFi Scanning: // Start scan (blocking)
wifi_scan_config_t scan_config = {
.show_hidden = false,
.scan_type = WIFI_SCAN_TYPE_ACTIVE,
.scan_time.active.min = 100,
.scan_time.active.max = 300,
};
esp_wifi_scan_start(&scan_config, true);
// Get results
uint16_t ap_count = 0;
esp_wifi_scan_get_ap_num(&ap_count);
wifi_ap_record_t *ap_records = malloc(sizeof(wifi_ap_record_t) * ap_count);
esp_wifi_scan_get_ap_records(&ap_count, ap_records);
Set WiFi Credentials: wifi_credentials_t credentials = {0};
strncpy(credentials.ssid, "MyNetwork", sizeof(credentials.ssid) - 1);
strncpy(credentials.password, "password", sizeof(credentials.password) - 1);
esp_err_t ret = wifi_manager_set_credentials(&credentials);
Get WiFi Status: wifi_status_t status;
wifi_manager_get_status(&status);
// status.mode, status.connected_ssid, status.rssi, status.has_credentials
Clear Credentials: esp_err_t ret = wifi_manager_clear_credentials();
Lifecycle:
|
Captive Portal Implementation¶
Current Implementation: The template provides a simplified captive portal approach:
Previous DNS Server Approach (Removed for Simplicity): Earlier versions included a DNS server to redirect all DNS queries to device IP for automatic captive portal popup. This was removed to simplify the template. Re-enabling Captive Portal (Optional): Users can optionally implement DNS redirect:
Current User Experience:
Note: This simplified approach works well for template usage. Production deployments may want full DNS redirect captive portal. |
Server Configuration and Lifecycle¶
Server Configuration: typedef struct {
uint16_t port; // Default: 80
uint8_t max_open_sockets; // Default: 7
} web_server_config_t;
#define WEB_SERVER_DEFAULT_CONFIG() { \
.port = 80, \
.max_open_sockets = 7 \
}
ESP-IDF httpd_config_t Settings: httpd_config_t httpd_config = HTTPD_DEFAULT_CONFIG();
httpd_config.server_port = 80;
httpd_config.max_open_sockets = 7; // Concurrent connections
httpd_config.max_uri_handlers = 32; // Increased from default 8
httpd_config.lru_purge_enable = true; // Auto-remove LRU handlers
Performance Characteristics:
Initialization Sequence: esp_err_t web_server_init(const web_server_config_t *config)
{
// 1. Store configuration
current_config = *config;
// 2. Start HTTP server
httpd_config_t httpd_config = HTTPD_DEFAULT_CONFIG();
httpd_config.server_port = current_config.port;
httpd_config.max_open_sockets = current_config.max_open_sockets;
httpd_config.max_uri_handlers = 32;
httpd_config.lru_purge_enable = true;
if (httpd_start(&server, &httpd_config) != ESP_OK) {
return ESP_FAIL;
}
// 3. Register all URI handlers (32 handlers total)
// ... (registration code)
return ESP_OK;
}
Start/Stop Functions: esp_err_t web_server_start(void)
{
server_running = true;
ESP_LOGI(TAG, "Web server started successfully");
return ESP_OK;
}
esp_err_t web_server_stop(void)
{
if (server != NULL) {
httpd_stop(server);
server = NULL;
}
server_running = false;
return ESP_OK;
}
Query Functions: bool web_server_is_running(void); // Check running state
uint16_t web_server_get_port(void); // Get configured port
|
Adding New Pages and Endpoints¶
Adding a New Static HTML Page:
Adding a New REST API Endpoint:
|
CORS and Security¶
Current CORS Policy: All API endpoints return: httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
Rationale: Template simplicity - allows development from any origin (GitHub Pages, local files, etc.) Production Recommendation: For production deployments, restrict CORS to specific origins: // Option 1: Lock to device IP only (no external access)
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "null");
// Option 2: Allow specific external domain (hybrid GitHub Pages approach)
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin",
"https://yourusername.github.io");
CORS Preflight Handler: static esp_err_t cors_preflight_handler(httpd_req_t *req)
{
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
httpd_resp_set_hdr(req, "Access-Control-Allow-Methods", "GET, POST, OPTIONS");
httpd_resp_set_hdr(req, "Access-Control-Allow-Headers", "Content-Type, Authorization");
httpd_resp_set_hdr(req, "Access-Control-Max-Age", "86400"); // 24 hours
httpd_resp_send(req, "", 0);
return ESP_OK;
}
Registered for: httpd_uri_t options_uri = {
.uri = "/api/*", // Wildcard for all API endpoints
.method = HTTP_OPTIONS,
.handler = cors_preflight_handler,
.user_ctx = NULL
};
Password Exposure Protection: Configuration GET endpoints never return password fields: cJSON_AddStringToObject(wifi, "password", ""); // Always empty string
Passwords are only accepted in POST requests, never echoed back. |
Testing and Debugging¶
Manual Testing:
Browser Developer Tools:
ESP32 Serial Monitor: I (1234) web_server: Initializing web server on port 80
I (1235) web_server: Registered handler for '/' - OK
I (1236) web_server: Registered handler for '/scan' - OK
I (1237) web_server: Web server initialized successfully
I (5678) web_server: Serving static file: /index.html
I (5679) web_server: Found index.html, size: 4823
Common Issues:
|
Traceability¶
All traceability is automatically generated by Sphinx-Needs based on the :links: attributes in each specification.
ID |
Title |
Status |
Tags |
|---|---|---|---|
Certificate Handler |
implemented |
||
Config Manager |
implemented |
||
Network Tunnel |
implemented |
||
cert_handler_get_ca_cert |
implemented |
||
cert_handler_get_info |
implemented |
||
cert_handler_get_server_cert |
implemented |
||
cert_handler_get_server_key |
implemented |
||
cert_handler_init |
implemented |
||
config_commit |
implemented |
||
config_factory_reset |
implemented |
||
config_get_all_as_json |
implemented |
||
config_get_bool |
implemented |
||
config_get_int16 |
implemented |
||
config_get_int32 |
implemented |
||
config_get_schema_json |
implemented |
||
config_get_string |
implemented |
||
config_init |
implemented |
||
config_set_all_from_json |
implemented |
||
config_set_bool |
implemented |
||
config_set_bool_no_commit |
implemented |
||
config_set_int16 |
implemented |
||
config_set_int16_no_commit |
implemented |
||
config_set_int32 |
implemented |
||
config_set_string |
implemented |
||
config_set_string_no_commit |
implemented |
||
config_write_factory_defaults |
implemented |
||
netif_uart_tunnel_deinit |
implemented |
||
netif_uart_tunnel_get_handle |
implemented |
||
netif_uart_tunnel_init |
implemented |
||
netif_uart_tunnel_config_t |
implemented |
||
JSON Schema as Configuration Source of Truth |
draft |
config; schema; architecture |
|
Web Interface Integration Support |
approved |
config; integration |
|
NVS Error Graceful Handling |
draft |
config; error-handling; reliability |
|
Configuration Initialization on Boot |
draft |
config; boot |
|
Simple Process to Add Configuration Fields |
draft |
config; extensibility; developer-experience |
|
Type Safety via Optional Static Validation |
draft |
config; validation; developer-experience |
|
Configuration Schema Versioning and Migration |
open |
config; versioning; migration; future |
|
Parameter Grouping for UI Organization |
draft |
config; ui; schema |
|
Parameter Type System |
draft |
config; types |
|
Build-Time Factory Defaults Generation |
draft |
config; build; code-generation |
|
No Runtime JSON Parsing in C Code |
draft |
config; performance; memory |
|
Key-Based NVS Storage |
draft |
config; storage; nvs |
|
Type-Safe Configuration API |
draft |
config; api; type-safety |
|
Persistent Configuration Storage |
draft |
config; storage; nvs |
|
Factory Reset Capability |
draft |
config; reset |
|
QEMU UART Network Bridge |
approved |
emulation; network; qemu; uart |
|
Packet Encapsulation |
approved |
emulation; protocol |
|
Host-Side Bridge Script |
approved |
emulation; tooling; host |
|
DHCP Client Support |
approved |
emulation; network; dhcp |
|
Conditional Compilation |
approved |
emulation; build |
|
Emulation Setup Documentation |
approved |
emulation; documentation |
|
Tunnel Throughput |
approved |
emulation; performance |
|
Packet Loss Handling |
approved |
emulation; reliability |
|
Component-based Architecture |
approved |
architecture; modularity |
|
Non-volatile Configuration Storage |
approved |
storage; nvs; configuration |
|
ESP32 Hardware Platform |
approved |
hardware; platform |
|
WiFi Connectivity |
approved |
network; wifi |
|
Memory Management |
approved |
performance; memory |
|
Error Handling and Recovery |
approved |
reliability; error-handling |
|
HTTPS Support |
open |
security; https; future |
|
Emulator Support |
approved |
emulator; qemu; testing |
|
Emulator Network Connectivity |
approved |
emulator; qemu; network; development |
|
System Time and NTP Support |
open |
time; ntp; future |
|
Web-based Configuration |
approved |
web; configuration |
|
Real-time Status Display |
approved |
web; ui; monitoring |
|
Configuration Interface |
approved |
web; ui; configuration |
|
WiFi Setup Interface |
approved |
web; wifi; network |
|
Web Interface Navigation |
approved |
web; ui; navigation |
|
HTTP Server Concurrency |
approved |
web; performance |
|
Configuration REST API |
approved |
web; api; config |
|
Web UI Responsiveness |
approved |
web; performance; ux |
|
Mobile-First Design |
approved |
web; ui; mobile |
|
Schema-Driven Configuration Form |
approved |
web; ui; config |
|
ESP-IDF CMake Integration |
approved |
build; cmake |
|
Certificate Handler Component Design |
draft |
component; security |
|
GitHub Codespaces Integration |
approved |
development; devcontainer |
|
Component Communication Pattern |
approved |
dataflow; communication |
|
Configuration Manager Component Design |
approved |
component; config |
|
Configuration Data Flow |
approved |
dataflow; config |
|
Error Recovery and Reset Strategy |
approved |
error-handling; reliability; reset |
|
Flash Memory Configuration |
approved |
flash; memory |
|
HTTP Server Architecture Details |
approved |
network; http; web |
|
ESP32 Template Layered Architecture |
approved |
architecture; layering |
|
Logging and Diagnostics Strategy |
approved |
logging; diagnostics; debugging |
|
Memory Management Strategy |
approved |
memory; performance |
|
Network Tunnel Component Design |
approved |
component; qemu; network |
|
System Performance Requirements |
approved |
performance; requirements |
|
QEMU Hardware Abstraction |
approved |
qemu; emulation |
|
QEMU Component Selection |
approved |
qemu; build |
|
FreeRTOS Task Organization |
approved |
threading; rtos |
|
Web Server Component Design |
approved |
component; web; network |
|
WiFi Manager Design Details |
approved |
network; wifi |
|
Type-Safe Configuration API |
approved |
api; interface; c-api |
|
JSON Schema-Driven Architecture |
draft |
architecture; config; json-schema |
|
Bulk JSON Configuration API |
approved |
api; json; bulk-operations |
|
Factory Reset via Bulk JSON Update |
approved |
build-process; code-generation; factory-reset |
|
Adding New Configuration Fields |
approved |
development; guide; extensibility |
|
Configuration Schema Structure |
approved |
data-structure; schema |
|
JSON Schema as Single Source of Truth |
approved |
architecture; schema; design-pattern |
|
NVS Storage Format |
approved |
storage; nvs |
|
Type Safety Without Code Generation |
approved |
type-safety; best-practices |
|
JSON Schema for UI Generation |
approved |
web; ui; javascript |
|
Configuration Webpage Architecture |
open |
frontend; javascript; ui |
|
Device Reset Countdown Interface |
open |
ui; countdown; reset |
|
Error Handling and User Feedback |
open |
error-handling; feedback |
|
Configuration Data Flow |
open |
data-flow; json; api |
|
Dynamic Form Generation from Schema |
open |
form-generation; schema; javascript |
|
Complete Page Initialization Flow |
open |
initialization; lifecycle |
|
JSON Array to Form Field Mapping |
open |
data-mapping; json; form |
|
UI State Management |
open |
ui-state; javascript |
|
Web Server Architecture |
approved |
web; architecture |
|
Captive Portal Design |
approved |
web; captive-portal; wifi |
|
HTTP Server Configuration |
approved |
web; config; performance |
|
Extension Guide for Web Pages |
approved |
web; extensibility; guide |
|
Config Manager Integration Pattern |
approved |
web; integration; config |
|
WiFi Manager Integration Pattern |
approved |
web; integration; wifi |
|
Configuration API Endpoints |
approved |
web; api; config; schema |
|
System Health API Endpoint |
approved |
web; api; diagnostics |
|
WiFi Management REST API Endpoints |
approved |
web; api; wifi |
|
URI Routing Table |
approved |
web; routing |
|
CORS Configuration |
approved |
web; security; cors |
|
Static File Embedding Strategy |
approved |
web; build; embedding |
|
Web Server Testing Strategy |
approved |
web; testing |
SPEC_WEB_ARCH_1¶