115 lines
9.2 KiB
Markdown
115 lines
9.2 KiB
Markdown
# Project Rules for ESP32-S3 Audio Board Development
|
||
|
||
This workspace is for the **ESP32-S3 AI Smart Speaker Development Board**. All agents interacting with this project must follow these guidelines:
|
||
|
||
- **Target Hardware:** UeeKKoo (Waveshare) ESP32-S3-AUDIO-Board-EN (No Battery version).
|
||
- **CPU:** Dual-core Xtensa 32-bit LX7 CPU, running at 240 MHz (ESP32-S3R8).
|
||
- **Memory:** 16 MB Flash, 8 MB Octal PSRAM (0.01 GB RAM total).
|
||
- **OS:** FreeRTOS.
|
||
- **Connectivity:** Wi-Fi & Bluetooth.
|
||
- **USB:** 1x USB port.
|
||
- **Audio Codec Hardware:**
|
||
- Playback (DAC): ES8311 low-power mono audio codec chip.
|
||
- Recording (ADC): ES7210 audio encoder chip.
|
||
- **I2S / I2C Pin Mapping:**
|
||
- **I2S MCLK:** GPIO 12
|
||
- **I2S BCLK:** GPIO 13
|
||
- **I2S LRCK:** GPIO 14
|
||
- **I2S DIN (Microphone):** GPIO 15
|
||
- **I2S DOUT (Speaker):** GPIO 16
|
||
- **I2C SDA:** GPIO 11
|
||
- **I2C SCL:** GPIO 10
|
||
- **WS2812 RGB LEDs:** GPIO 38
|
||
- **Key Features & Peripherals:**
|
||
- Support for AI Speech Interaction with Dual Digital Microphone Array.
|
||
- Hardware/Software Noise Reduction and Echo Cancellation.
|
||
- Support for External Displays (1.47", 2", 2.8", 3.5" LCDs) and Cameras (OV2640, OV5640).
|
||
- 7-ring programmable RGB LEDs (WS2812B, GRB/RGB order, connected to GPIO 38).
|
||
- PCF85063 RTC chip.
|
||
|
||
- **Development Environment & Toolchain Details:**
|
||
- **ESP-IDF Version:** v6.0.1 (Target: `esp32s3`)
|
||
- **ESP-IDF Directory:** `C:\esp\v6.0.1\esp-idf`
|
||
- **ESP-IDF Tools Path:** `C:\Espressif\tools`
|
||
- **Build Command:** `eim run "idf.py build"` or execute `build.bat`
|
||
- **Flash Command:** `eim run "idf.py flash"` or execute `flash.bat`
|
||
- **Serial Monitor Command:** `eim run "idf.py monitor"` or execute `monitor.bat`
|
||
|
||
- **Helper Scripts Folder (`/scripts`):**
|
||
- **`build.bat` / `build.ps1`:** Triggers standard incremental project build.
|
||
- **`flash.bat` / `flash.ps1`:** Automatically rebuilds and flashes project to `COM5` at 460800 baud.
|
||
- **`monitor.bat` / `monitor.ps1`:** Launches target serial logging monitor on `COM5` (press `Ctrl + ]` to exit).
|
||
- **`clean.bat` / `clean.ps1`:** Deletes the `/build` folder to enforce clean target configs.
|
||
|
||
- **Implemented Wi-Fi Functionality:**
|
||
- **Credentials Storage:** Wi-Fi credentials are read from the `wifi-credentials` file in the project root. This file is parsed by CMake during the configure step, and dynamically generates `wifi_credentials.h` inside the build directory so credentials are never checked into Git.
|
||
- **Station Connection:** Initializes Wi-Fi in Station mode and handles auto-reconnect on disconnect (up to 5 retries). Implemented in `wifi_connect.c` and started in `main.c`.
|
||
|
||
- **Implemented Audio Functionality:**
|
||
- **Startup Sound:** Plays `startup.mp3` immediately on boot.
|
||
- **Chime Assets:** The audio files in SPIFFS were generated using `agentctl tts`:
|
||
- `startup.mp3`: *"Ding Dong, I'm turned on!"*
|
||
- `connecting_wifi.mp3`: *"Connecting to the wifi"*
|
||
- `connected_wifi.mp3`: *"I'm now connected to the wifi"*
|
||
- `connected.mp3`: *"I'm connected!"*
|
||
- `connect_fail.mp3`: *"I could not connect"*
|
||
- Packaged into internal SPIFFS partition `model` on the device's flash.
|
||
- **Playback Management:** Implemented `Audio_Play_Music_To_End` in `audio_driver.c` to perform blocking playback, preventing overlapping audio playback during initialization.
|
||
- **Codec Init:** Initializes ES8311 DAC via I2C at sample rate 16000Hz, stereo (2 channels), 16-bit depth.
|
||
- **PA Enable:** Drivers enable the speaker power amplifier (`PA_CTRL`) on pin 8 of the TCA9555 IO expander.
|
||
- **Volume Adjust:** Set to `85` (0 to 100 range scale) via `Volume_Adjustment(85)` inside `main/main.c`.
|
||
|
||
- **Implemented LED Ring Functionality:**
|
||
- **State-driven Display:** Background FreeRTOS task `led_rainbow_task` drives the 7 WS2812B LEDs connected to GPIO 38 based on the global connection state:
|
||
- `LED_STATE_IDLE`: LEDs remain **off** during initial startup sounds.
|
||
- `LED_STATE_CONNECTING`: LEDs **flash orange** (approx. 500ms intervals) during connection.
|
||
- `LED_STATE_FAILED`: LEDs glow **solid red** at 20% brightness on failure.
|
||
- `LED_STATE_CONNECTED`: LEDs run a **vibrant, moving color fade** using HSV-to-RGB translation rotating around the ring (speed set to approx. 2.25 seconds per cycle at 20% brightness).
|
||
- **Dependency:** Linked and integrated using the `espressif/led_strip` (^3.0.1) component.
|
||
|
||
- **Build & Partition Configurations:**
|
||
- **Partition Table:** Defined in `partitions.csv` with custom `model` (SPIFFS, 5900K size) and `flash_test` (FATFS, 528K size) partitions.
|
||
- **SPIFFS Generation:** The `model` partition image is automatically built from folder `main/spiffs` and flashed using CMake macro `spiffs_create_partition_image(model spiffs FLASH_IN_PROJECT)` declared inside `main/CMakeLists.txt`.
|
||
- **Flash/PSRAM config:** Configured for 16MB flash (`CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y`) and 8MB Octal SPIRAM (`CONFIG_SPIRAM=y` & `CONFIG_SPIRAM_MODE_OCT=y`) via pre-configured `sdkconfig` imported from Waveshare's demo.
|
||
|
||
- **Troubleshooting & Porting to ESP-IDF v6.x:**
|
||
- **I2S Types:** ESP-IDF v6 removes the legacy I2S driver definitions. Any parameter defined as legacy `i2s_port_t` in BSP libraries (e.g., `bsp_board.c`) must be changed to `int`.
|
||
- **Serial Monitoring:** The standard serial monitor `idf.py monitor` requires an attached interactive TTY. In sandboxed running environments, raw serial read scripts should be used instead.
|
||
|
||
- **Development Socket Server & Cloudflare Tunneling:**
|
||
- **Dev Server Folder (`/dev-server`):** Node.js/TypeScript Express & WebSockets server listening on port `8901`.
|
||
- **Auto-Reload:** Configured via `nodemon` and `ts-node` using `npm run dev` in `/dev-server`.
|
||
- **Logging:** Console logs when clients connect, send messages, disconnect, or trigger errors.
|
||
- **Cloudflare Tunnel Routing:** Local configuration `config.yml` is mapped to forward public traffic for `example.com` and `dashboard.example.com` directly to loopback `http://127.0.0.1:8901`.
|
||
- **DNS Mappings:** Configure your public hostnames to point to your deployment tunnel or reverse proxy.
|
||
- **Command Routing:** Monitor clients can broadcast `control_device` command payloads (volume level adjustments, voice text-to-speech output, LED strip animations, or chime prompts) targeting specific client IDs, which the dev-server routes to standard client device sockets.
|
||
|
||
- **Real-Time Monitor Dashboard Server (`/monitor-server`):**
|
||
- **Service Port:** Port `8902` (running via Express). Starts with `npm start`.
|
||
- **Layout & Structure:** Built with an enterprise-grade dark theme layout featuring a collapsible left sidebar, header operations cards, and responsive page views (`Overview`, `Device Fleet Manager`, and `Telemetry Terminal Console`).
|
||
- **Dynamic Elements:** Includes dynamic search boxes, severity filter button groups, CSV/JSON log exporters, device diagnostic modals, and volume/LED controllers that dispatch control instructions directly to connected ESP32-S3 boards.
|
||
|
||
- **Hardware Model Variants (DEV-1 and PROTO-1):**
|
||
- **Model Configuration:** Defined in the `device-config` file in the project root. Edit this file to switch the active hardware profile:
|
||
```
|
||
MODEL=DEV-1 # Standard Waveshare development board (default)
|
||
MODEL=PROTO-1 # Prototype board with additional peripherals
|
||
```
|
||
- **How it works:** CMake reads `device-config` at configure-time, generates `device_config.h` in the build directory, and defines either `MODEL_DEV_1=1` or `MODEL_PROTO_1=1` as a compiler macro on the `${COMPONENT_LIB}` target.
|
||
- **⚠️ After changing `device-config`:** Always run `scripts/clean.bat` followed by `scripts/build.bat` to force CMake to regenerate `device_config.h` with the new model value. An incremental build alone will NOT pick up the change.
|
||
- **DEV-1 Variant:** Includes all core firmware functionalities (Wi-Fi, audio playback, WS2812B LEDs on GPIO 38, WebSocket telemetry).
|
||
- **PROTO-1 Variant:** Inherits DEV-1 baseline and introduces:
|
||
- **Audio I2C/I2S Interface:** Same as DEV-1 (GPIO 10–16, EXIO 8 PA_EN), but DIN/DOUT are physically swapped on the PROTO board.
|
||
- **GPIO 9:** WS2812 RGB LED strip signal *(replaces GPIO 38 used on DEV-1)*
|
||
- **GPIO 5:** Haptics motor — PWM output via LEDC driver (1kHz, 10-bit duty cycle)
|
||
- **GPIO 4:** Ambient light sensor (LDR) — ADC1 Channel 3, analog oneshot read, scaled 0–4095 → 0–100%
|
||
- **GPIO 7:** Capacitive touch sensor — digital input, active HIGH when touched
|
||
- Background FreeRTOS task `proto1_sensor_task` polls GPIO 4 and GPIO 7 every 100ms and updates `device_state_t` fields: `ambient_light` (int) and `touch_active` (bool).
|
||
- Haptic intensity is downlink-controllable via `set_haptics` WebSocket command (value 0–100).
|
||
|
||
- **Protocol-First Development Rules:**
|
||
- **API Specification Priority:** When implementing any new client/device functionality or modifying messaging structures, you MUST always update `API.md` **FIRST** as the absolute source of truth before editing any application or driver code.
|
||
- **Implementation Sequence:**
|
||
1. Update/define the JSON schema, commands, and fields in `API.md`.
|
||
2. Write the corresponding host (gateway/server) and client (firmware) code changes matching the specification.
|