ESP32-S3 AI Smart Speaker Control & Diagnostics Suite

This repository contains the complete firmware, development socket server, and real-time operations dashboard for the Waveshare ESP32-S3-AUDIO-Board-EN (No Battery version).

The project implements a state-driven smart speaker capable of auto-connecting to Wi-Fi, linking to a remote WebSocket server via a configurable WebSocket endpoint, playing back high-quality audio chimes via I2S, driving WS2812B LEDs, and receiving real-time control actions (such as volume adjustments, sound trigger cues, and voice synthesis broadcasts) from a dedicated operations dashboard.

---

System Architecture

graph TD
    Dashboard[Enterprise Monitor Dashboard <br/> Port 8902] <-->|JSON Control Protocol| DevServer[WebSocket API Gateway <br/> Port 8901]
    ESP32[ESP32-S3 Audio Board] <-->|Secure WebSocket Link| DevServer
    
    subgraph ESP32-S3 Core Drivers
        I2S[I2S Audio Output - ES8311 DAC]
        LED[WS2812B 7-Ring RGB LEDs]
        WiFi[WiFi Manager State Machine]
        SPIFFS[SPIFFS Audio Assets Model]
    end
    
    ESP32 --> I2S
    ESP32 --> LED
    ESP32 --> WiFi
    ESP32 --> SPIFFS

---

1. Hardware Specifications & Pin Mapping

• SoC: ESP32-S3R8 (Dual-core 32-bit LX7, 240 MHz, 16MB Flash, 8MB Octal PSRAM).
• Audio DAC (Playback): ES8311 mono codec connected via I2C & I2S.
• Audio ADC (Recording): ES7210 microphone encoder.
• Audio PA Enable: Controlled via Pin 8 of the TCA9555 I2C I/O expander.
• LED Ring: 7 WS2812B RGB LEDs connected to GPIO 38.

Signal Name	ESP32-S3 GPIO	Signal Description
I2S MCLK	GPIO 12	Audio Master Clock
I2S BCLK	GPIO 13	Audio Bit Clock
I2S LRCK	GPIO 14	Audio Left/Right Word Clock
I2S DIN	GPIO 15	Microphone Data Input (ADC)
I2S DOUT	GPIO 16	Speaker Data Output (DAC)
I2C SDA	GPIO 11	Codec & Expander I2C Data
I2C SCL	GPIO 10	Codec & Expander I2C Clock
RGB LED	GPIO 38	WS2812B Data Line

---

2. Workspace Setup & Prerequisites

ESP-IDF SDK Target:

• Install ESP-IDF v6.0.1 (target framework esp32s3).
• Ensure that the environment paths are configured (or run under your standard ESP-IDF terminal shell).

---

3. Configuration & Deployment

Before building, you must configure your network credentials and target endpoints:

Wi-Fi Configuration

Copy the template configuration file wifi-credentials.example in the project root to a new file named wifi-credentials (which is excluded from Git in .gitignore):

cp wifi-credentials.example wifi-credentials

Open wifi-credentials and write your actual credentials:

SSID="Your_WiFi_Name"
PASSWORD="Your_WiFi_Password"

Note: CMake automatically reads this file during configuration to compile the private header wifi_credentials.h inside your build folder. Never commit your private wifi-credentials file.

WebSocket Server URL

Open main/spiffs/server_url.txt and write the target WebSocket server URL (e.g. your local gateway or public Cloudflare domain):

wss://example.com

Note: This text file is packaged into the SPIFFS partition and loaded dynamically on board boot.

---

4. Hardware Model Variants

The firmware supports two hardware model profiles: DEV-1 (default) and PROTO-1. The active model is controlled by the device-config file in the project root.

Switching Models

Open device-config and set the MODEL field to either DEV-1 or PROTO-1:

# For the standard development board (default):
MODEL=DEV-1

# For the prototype board with additional peripherals:
MODEL=PROTO-1

Important: After changing device-config, you must perform a clean rebuild to regenerate device_config.h and recompile the correct pin mappings. Run scripts/clean.bat before scripts/build.bat.

---

DEV-1 Profile

The standard Waveshare ESP32-S3-AUDIO-Board-EN pin configuration:

Peripheral	GPIO	Mode
I2C SCL	GPIO 10	I2C Clock
I2C SDA	GPIO 11	I2C Data
I2S MCLK	GPIO 12	Audio Master Clock
I2S BCLK	GPIO 13	Audio Bit Clock
I2S LRCK	GPIO 14	Audio Word Select
I2S DIN (Microphone)	GPIO 15	I2S Input
I2S DOUT (Speaker)	GPIO 16	I2S Output
RGB LEDs (7x WS2812B)	GPIO 38	WS2812 Signal
PA Enable	EXIO 8	TCA9555 I/O Expander

---

PROTO-1 Profile

Inherits all DEV-1 audio functionality but remaps the LED signal pin and adds three new peripheral drivers:

Peripheral	GPIO	Mode
I2S DIN (Microphone)	GPIO 15	I2S Input (remapped: DOUT ↔ DIN swapped vs DEV-1)
I2S DOUT (Speaker)	GPIO 16	I2S Output
RGB LEDs (WS2812)	GPIO 9	WS2812 Signal (changed from GPIO 38)
Haptics Motor	GPIO 5	PWM Output (LEDC, 1kHz, 10-bit)
Ambient Light Sensor	GPIO 4	ADC1 Channel 3, Analog Input
Touch Sensor	GPIO 7	Digital Input

Additional PROTO-1 runtime behaviours:

• A background FreeRTOS task (proto1_sensor_task) polls the LDR and touch GPIO every 100ms and updates the shared device state.
• Ambient light ADC values (0–4095) are scaled to 0–100% in the web dashboard.
• Haptic intensity is controllable via the WebSocket set_haptics command (0–100%).
• All three sensor readings are included in uplink telemetry state reports.
• The web dashboard automatically unlocks the Haptic Vibration control slider for PROTO-1 devices; it appears greyed-out with a PROTO-1 Only indicator for DEV-1 devices.

---

5. Build, Flash & Logging

Use the preconfigured batch scripts inside the /scripts directory to compile and flash the firmware:

• Compile Code: Run scripts/build.bat to run incremental CMake compiles.
• Flash Firmware: Run scripts/flash.bat to build and flash the application to COM5 (at 460800 baud) alongside generated SPIFFS images.
• Monitor Console Logs: Run scripts/monitor.bat to read live serial logs from COM5 (press Ctrl + ] to exit).
• Clean Build: Run scripts/clean.bat to delete compilation cache.

---

6. Starting the Servers

To run the full diagnostics suite, launch both the API Gateway and the Dashboard Operations Console:

Start the API Gateway (Port 8901)

The gateway handles data routing, client registration, and active device command routing:

cd dev-server
npm install
npm run dev

Start the Operations Dashboard (Port 8902)

The dashboard displays connection metrics, device health parameters, and hosts the command center:

cd monitor-server
npm install
npm start

Open http://localhost:8902 in your browser to access the control panel.

---

7. Remote Routing & Tunnel Configuration

If deploying publicly, ensure your local configurations route traffic properly:

1. Map Cloudflare tunnel configuration to forward requests for example.com to http://localhost:8901.
2. The Dashboard client code automatically scales and shifts its endpoint from ws://localhost:8901 (when accessed locally) to wss://example.com (when accessed remotely) to match your environment.