桌宠机器人
DeskPet
基于 ESP32-S3 的桌宠机器人，集成语音识别、LLM 对话和 TTS 语音合成。3D 打印外壳 + LCD 表情显示，可进行自然语言交互。
详解 · Deep Dive
项目详解
这个项目能带来什么、适合什么样的学生、又如何成为大学申请文书里的真实素材。
项目说明 · OVERVIEW
一台能对话的桌面机器人，放在书桌前既能解压、又能把孩子从手机里拉出来一会儿。更实在的是，它是孩子亲手造的工程作品——ESP32、大模型、3D 打印——够格写进申请。
A voice-chat desktop robot for the desk — something to decompress with and to pull your teen away from their phone for a while. More importantly, it's a project they built themselves — ESP32, LLMs, 3D printing — solid enough to write into an application.
适合谁 · WHO IT'S FOR
电子/机械工程方向
· EE/ME-Inclined Builders如果你已经倾向于电子或机械工程方向，这个项目能让你把两边的本事各占一段、亲手做出来。电子侧从读原理图、器件识别到点亮一块真正的 PCB——ESP32-S3、ST7789 屏、BMI270 IMU、ES8311 音频编解码器，含 0805/0603 贴片焊接（也提供出厂预焊方案，初学者只需补焊排针、Type-C、FPC 等约十余处）；机械侧用 PLA/PETG 在 Cura/Bambu 里设计并打印外壳，让屏幕支架、摄像头座、底盘严丝合缝。营期结束时，你带走的是一台电路板能读懂、外壳能复刻的机器人——而因为它一通电就会开口回话，电子与机械两半不再是各做各的实验，而是同一台会动的整机。
If you already lean toward electrical or mechanical engineering, this project lets you own a genuine piece of each. On the EE side you read schematics, identify components, and bring up a real PCB — ESP32-S3, ST7789 LCD, BMI270 IMU, ES8311 codec — soldering SMD 0805/0603 parts (with a factory-pre-soldered option that leaves you only the headers, Type-C, and FPC to finish, about a dozen joints). On the ME side you design and 3D-print the enclosure in PLA/PETG (Cura/Bambu) so the screen mount, camera housing, and chassis fit and assemble cleanly. You leave camp with a self-built robot whose board you can read and whose shell you can remake — and because it talks back when you power it on, the two halves come alive together rather than as separate lab exercises.
想搞工程、方向还没定的探索者
· Direction-Still-Exploring Students如果你确定要走工程，但具体哪个细分方向还没想清楚，这个项目是用十二天把几个主流方向一次性都试一遍的好机会。在这台整机里你会同时碰到嵌入式固件（ESP-IDF、FreeRTOS、C 语言跑在 ESP32-S3）、云端 AI 接入（LLM 的 WebSocket/HTTP、TTS、流式对话）、感知（OV2640 人脸检测、BMI270 手势）、以及数字化制造（PCB、3D 打印）。每天最让你期待的那一块，就是下一步方向最诚实的信号；而不那么来电的部分，也已经变成了你身上的实打实技能。等到演示日，你不仅知道自己真正想往哪走，身后还有一台做完的整机——而不是四个互不搭界的体验模块。
If you're confident engineering is your path but haven't yet pinned down the specific direction, this project is an efficient way to sample several at once in twelve days. Across the build you touch embedded firmware (ESP-IDF, FreeRTOS, C on the ESP32-S3), cloud AI integration (LLM over WebSocket/HTTP, TTS, streaming dialogue), perception (OV2640 face detection, BMI270 gesture), and digital fabrication (PCB, 3D printing). Which part you look forward to most each morning becomes a honest signal for your next step — and the parts that don't grab you are still skills you keep. By demo day you'll know where your enthusiasm actually points, with one finished robot behind you rather than four disconnected taster modules.
有理论基础、缺动手经验
· Theory-Solid, Hands-On-New如果你的课本基础扎实，但还很少把一个项目从零一路做到通电点亮，这个项目正是为合上这道缝而设计的。课程从你已会的入手——ESP-IDF 环境搭建、读原理图、器件识别是头几节课而非默认前提——再带你焊板子、点亮 ST7789+LVGL 屏幕、调通 I2S 音频与 I2C 传感器，最后把 Wi-Fi、LLM 接口、TTS 串成一台能语音对话的机器人。整个过程是刻意搭好台阶的：你已经懂的部分会稳稳托住你走向还没懂的部分。当你亲手写的固件让屏幕上的小脸动起来、开口回话的那一刻，理论就不再是抽象的了。
If your textbook foundation is solid but you've rarely carried a project all the way to power-on, this is built to close exactly that gap. The curriculum meets you where you are — ESP-IDF environment setup, schematic reading, component identification are the first lessons, not assumed — then walks you through soldering the board, bringing up the ST7789+LVGL display, wiring I2S audio and I2C sensors, and finally threading Wi-Fi, the LLM API, and TTS into a working voice-chat robot. The transition is deliberate and scaffolded, so the parts you already know carry you smoothly into the parts you don't. Your theory stops being abstract the moment the firmware you wrote makes the on-screen face animate and talk back.
申请季、缺一个大项目的同学
· Application-Season, Project-Seeking如果你在课业上下了很深的功夫，现在到了申请季正缺一个有分量的项目，那这个项目正是让这份投入开始复利增长的地方。你已经练成的本事——精读、严谨推理、把一整套系统装进脑子——会直接迁移到读 ESP32-S3 数据手册、设计 FreeRTOS 任务结构、对接 LLM 云端 API 上。你带来的是真实的优势而非短板：只搭过套件的同学，很少能像你这样去推敲固件架构。而因为这个系统从硬件到云端是真正意义上的全栈（固件、视觉、音频、网络、AI、制造），你写出来的会是一个有层次、有自己故事的原创项目——而不是一道通用练习题。等到演示日，你手里会有一台能语音对话的机器人，和一段具体、属于你、能在文书里讲得出细节的工程经历。
If you've invested deeply in coursework and now need a substantial project for application season, this is exactly where that investment compounds. The same focus you've already trained — reading carefully, reasoning precisely, holding a whole system in your head — transfers directly to reading the ESP32-S3 datasheet, designing FreeRTOS task structure, and integrating the LLM cloud API. You arrive with a real edge, not a deficit: students who've only built kits rarely reason about firmware architecture the way you can. And because the system is genuinely full-stack (embedded firmware, vision, audio, networking, AI, fabrication), it gives you a layered, original story to write — not a generic exercise. By demo day you'll have a working voice-chat robot and a specific, personal engineering journey, grounded in skills your application essays can speak to concretely.
文书方向 · ESSAY TAKEAWAYS
夏令营期间每个部分都会有基础了解；任意一个方向，只要你的申请方向在 CS / 工程相关，都可以延展为文书的主要收获。以下 8 个方向各自对得上项目里的具体知识点。
3D 打印外壳：从数字模型到可制造实物
3D-printed enclosure: digital model to physical product
项目对应 · IN THE PROJECT
3D 打印 ESP-SparkBot 外壳（主体后壳 + 带屏幕窗口的前盖 + 摄像头/喇叭/电池仓），在 Cura/Bambu Studio 中切片，Day 4 装配到堆叠 PCB 上，使屏幕窗口、摄像头开孔与 USB 开口都对齐。
3D-printing the ESP-SparkBot enclosure (main rear shell + screen-mount front cover + camera/speaker/battery compartments), sliced in Cura/Bambu Studio and assembled onto the stacked PCBs on Day 4 so the screen window, camera aperture, and USB opening all align.
知识点 · KNOWLEDGE
STL 网格与打印方向取舍、支撑生成与层附着分析、G-code 刀轨与切片参数控制（Cura/Bambu Studio）、零件间的装配公差设计、材料选择（PLA/PETG 的 FDM vs. 光敏树脂）。
STL mesh and print-orientation reasoning, support-generation and layer-adhesion analysis, G-code toolpath and slicer parameter control (Cura/Bambu Studio), part-to-part fit and assembly tolerance design, material selection (PLA/PETG FDM vs. photopolymer resin).
文书角度 · ESSAY ANGLE
把数字 STL 转化为必须容纳 1.54 寸 ST7789 屏、OV2640 摄像头模组、3W 喇叭和 2000mAh 锂电池的实体外壳，并在毫米级公差内完成装配，体现申请者能把 CAD 设计闭环到可制造实物，在设计意图与制造现实之间做工程取舍（FDM 层附着 vs. 树脂精度、支撑拆除、卡口 vs. 螺丝分模）。
Translating a digital STL into a physical enclosure that must seat a 1.54" ST7789 screen, an OV2640 camera module, a 3W speaker, and a 2000mAh LiPo within millimeter-scale tolerances shows the applicant can close the CAD-to-fabrication loop, negotiating the trade-off between design intent and manufacturing reality (FDM layer adhesion vs. resin accuracy, support removal, snap-fit vs. screw-fastened parting lines).
嵌入式固件：ESP32-S3 / ESP-IDF 上从启动到应用的全链路
Embedded firmware: the boot-to-application chain on ESP32-S3 / ESP-IDF
项目对应 · IN THE PROJECT
用 C 语言在 ESP-IDF v5.3/v5.4 上编写 ESP32-S3 主固件：app_main.c 完成事件组/队列、netif、WiFi 与全部硬件外设（I2C、SPI、ST7789、OV2640、I2S 音频、BMI270、触摸）初始化后进入事件分发主循环。
Writing the ESP32-S3 main firmware in C on ESP-IDF v5.3/v5.4: app_main.c boots NVS, the event group/queue, netif, WiFi, and all hardware peripherals (I2C, SPI, ST7789, OV2640, I2S audio, BMI270, touch), then enters the dispatch main loop.
知识点 · KNOWLEDGE
ESP-IDF 工业级工具链（idf.py menuconfig/build/flash/monitor 全流程）、app_main 入口与启动序列、NVS 非易失存储与分区表、esp_netif/TCP-IP 协议栈初始化、GPIO/SPI/I2C/I2S/DVP 外设驱动、双核 Xtensa LX7（16MB Flash + 8MB PSRAM）架构。
ESP-IDF industrial toolchain (idf.py menuconfig/build/flash/monitor), app_main entry and boot sequence, NVS non-volatile storage and partition table, esp_netif/TCP-IP stack init, GPIO/SPI/I2C/I2S/DVP peripheral drivers, dual-core Xtensa LX7 (16MB Flash + 8MB PSRAM) architecture.
文书角度 · ESSAY ANGLE
在工业级 ESP-IDF 工具链（而非 Arduino 封装）上写出从 ESP32-S3 上电启动、初始化 NVS、TCP/IP 协议栈、WiFi 与全部硬件外设，到进入事件驱动主循环的固件，体现申请者能在资源受限的双核 MCU 上掌握从启动到应用的完整链路，其中栈大小、任务优先级与分区表都是有意的工程决策，而非被隐藏的默认值。
Writing firmware that boots the ESP32-S3, initializes NVS, the TCP/IP stack, WiFi, and every hardware peripheral, then enters an event-driven main loop on the industrial ESP-IDF toolchain (not Arduino wrappers) demonstrates the applicant commands the complete boot-to-application chain on a resource-constrained dual-core MCU, where stack sizes, task priorities, and partition tables are deliberate engineering decisions rather than hidden defaults.
音频编解码：从麦克风采集到 TTS 播放的编解码链路
Audio codec: the encode/decode pipeline from mic capture to TTS playback
项目对应 · IN THE PROJECT
音频子系统：ES8311 编解码器 + NS4150B 3W D 类功放 + 驻极体麦克风 + 喇叭，由音频任务中的 ESP-IDF I2S 驱动完成麦克风录音、语音活动检测与 TTS 播放，并支持 A2DP 蓝牙音箱，使设备兼作蓝牙音箱。
The audio subsystem: ES8311 codec + NS4150B 3W class-D amp + electret mic + speaker, driven by the ESP-IDF I2S driver in the audio task for mic recording, VAD, and TTS playback, plus an A2DP Bluetooth-audio sink so the device doubles as a BT speaker.
知识点 · KNOWLEDGE
I2S 串行音频协议（MCLK/BCLK/LRCK）与 DMA 缓冲、ES8311 编解码器寄存器配置（麦克偏置、PGA 增益、采样率）、PCM 数据采集与播放、WAV/PCM 解码、A2DP 蓝牙音频接收、经 D 类功放（NS4150B）的硬件/软件音频路由。
I2S serial audio protocol (MCLK/BCLK/LRCK) and DMA buffering, ES8311 codec register configuration (mic bias, PGA gain, sampling rate), PCM data capture and playback, WAV/PCM decode, A2DP Bluetooth audio sink, hardware/software audio routing through a class-D amplifier (NS4150B).
文书角度 · ESSAY ANGLE
配置 I2S 编解码器、经 ES8311 ADC 采集麦克风 PCM、把 TTS 解码后的 PCM 经 ES8311 DAC 送入 D 类功放播放，体现申请者理解数字音频完整的编解码路径，把"声音"这一抽象概念落到具体的信号链路与寄存器级驱动上（麦克偏置、PGA 增益、MCLK/BCLK/LRCK、DMA 缓冲）。
Configuring an I2S codec, capturing microphone PCM through the ES8311 ADC, and feeding TTS-decoded PCM out through the ES8311 DAC into a class-D amplifier shows the applicant understands the full encode/decode path of digital audio, grounding the abstract idea of sound in concrete signal-chain and register-level driver terms (mic bias, PGA gain, MCLK/BCLK/LRCK, DMA buffering).
并发与架构：FreeRTOS 多任务调度与有限状态机
Concurrency & architecture: FreeRTOS tasks and a finite state machine
项目对应 · IN THE PROJECT
FreeRTOS + 有限状态机架构：state_machine.h 定义 8 个状态（INIT/IDLE/LISTENING/SPEAKING/GAMING/CHARGING/SLEEP/ERROR）与转换规则表及每状态操作函数；app_main.c 创建六个双核固定任务（显示、AI 对话、音频、传感器、状态机、维护），通过一个事件队列与一个事件组通信。
The FreeRTOS + finite-state-machine architecture: state_machine.h defines 8 states (INIT/IDLE/LISTENING/SPEAKING/GAMING/CHARGING/SLEEP/ERROR) with a transition rules table and per-state ops; app_main.c creates six pinned-core tasks (display, ai_dialog, audio, sensor, state_machine, maintenance) communicating through one event queue and one event group.
知识点 · KNOWLEDGE
有限状态机设计（状态枚举、转换规则、每状态 on_enter/on_exit/on_update/can_transition 函数指针表）、FreeRTOS 任务/队列/事件组/信号量、双核任务绑定（xTaskCreatePinnedToCore）、优先级与栈空间预算、事件驱动分发架构。
Finite-state-machine design (state enum, transition rules, per-state on_enter/on_exit/on_update/can_transition function-pointer table), FreeRTOS task/queue/event-group/semaphore, dual-core task pinning (xTaskCreatePinnedToCore), priority and stack-size budgeting, event-driven dispatch architecture.
文书角度 · ESSAY ANGLE
把宠物行为建模为带 on_enter/on_exit/on_update 函数表的显式有限状态机，并用双核固定、事件队列与事件组编排六个并发 FreeRTOS 任务，体现申请者能把一个复杂的反应式系统分解为有纪律的并发单元，并合理设定优先级与栈预算，这是嵌入式软件工程的核心。
Modeling the pet's behavior as an explicit finite-state machine with an on_enter/on_exit/on_update function table, and orchestrating six concurrent FreeRTOS tasks with pinned cores, an event queue, and an event group shows the applicant can decompose a complex reactive system into disciplined concurrent units with deliberate priority and stack budgets, the core of embedded software engineering.
嵌入式图形界面：LVGL 控件树与 ST7789 实时渲染
Embedded GUI: LVGL widget tree and real-time rendering on ST7789
项目对应 · IN THE PROJECT
Day 6 显示工作：驱动 1.54 寸 ST7789 240×240 SPI LCD，移植 LVGL 渲染天气时钟界面与宠物表情动画（12 种表情状态），显示任务以 16ms 周期通过 lv_tick_inc/lv_task_handler 以 60FPS 为目标。
Day-6 display work: driving the 1.54" ST7789 240x240 SPI LCD and porting LVGL to render the weather-clock UI and animated pet expressions (12 expression states), with the display task targeting 60 FPS via lv_tick_inc/lv_task_handler on a 16 ms period.
知识点 · KNOWLEDGE
SPI 主机驱动、ST7789 初始化命令序列与帧缓冲推送、LVGL 控件树与 lv_task_handler 心跳、事件回调、动画与过渡、240×240 显示上的 60FPS（16ms）帧预算与 DMA/分块缓冲区大小设计。
SPI master driver, ST7789 init command sequence and frame-buffer pushing, LVGL widget tree and lv_task_handler tick, event callbacks, animation/transition, 60-FPS (16 ms) frame budget and DMA/partial-buffer sizing on a 240x240 display.
文书角度 · ESSAY ANGLE
在 240×240 ST7789 SPI 屏上用 LVGL 以 16ms（60FPS）帧预算、分块行帧缓冲与周期调用 lv_tick_inc/lv_task_handler 的动画任务渲染界面，体现申请者能在内存受限的 MCU 上交付实时 GUI，在帧率、缓冲区大小与驱动正确性之间做工程权衡，而非把显示屏当作黑箱。
Driving a 240x240 ST7789 LCD over SPI through LVGL with a 16 ms (60 FPS) frame budget, a partial-line frame buffer, and an animation-task that calls lv_tick_inc/lv_task_handler shows the applicant can deliver a real-time GUI on a memory-constrained MCU, balancing frame rate, buffer size, and driver correctness rather than treating the display as a black box.
对话式 AI 管线：边缘端的 STT→LLM→TTS 全流程
Conversational AI pipeline: STT → LLM → TTS on an edge device
项目对应 · IN THE PROJECT
对话式 AI 管线：ESP-SR 本地唤醒词/语音命令识别，llm_dialog.c 调用多家大模型 API（DeepSeek / ChatGPT-OpenAI / 文心 / 豆包）并以流式 token 回调（llm_dialog_chat_stream）接收，再向音频任务派发 TTS 事件进行播放。
The conversational AI pipeline: ESP-SR on-device wake-word/voice-command recognition, llm_dialog.c calling multi-provider LLM APIs (DeepSeek / ChatGPT-OpenAI / Wenxin / Doubao) with a streaming-token callback (llm_dialog_chat_stream), then dispatching a TTS event to the audio task for playback.
知识点 · KNOWLEDGE
本地唤醒词/命令识别（ESP-SR）、带流式响应的 HTTP 客户端、按供应商构造 JSON 请求体、多轮对话上下文管理、流式 token 回调、云端 TTS 合成、跨 FreeRTOS 任务的事件驱动 STT→LLM→TTS 编排。
On-device wake-word/command recognition (ESP-SR), HTTP client with streaming responses, per-provider JSON request body construction, multi-turn conversation context management, streaming-token callback, cloud TTS synthesis, event-driven STT→LLM→TTS orchestration across FreeRTOS tasks.
文书角度 · ESSAY ANGLE
把本地唤醒词/语音命令识别、支持多供应商并以流式 token 回调接收的云端大模型、以及 TTS 合成串成一个可响应的对话循环，体现申请者能把异构 AI 系统集成为一个面向用户的连贯产品，并处理流式传输、延迟、多供应商请求格式差异，以及 AI 任务与音频任务之间的事件驱动交接。
Chaining on-device wake-word/voice-command recognition, a provider-abstracted cloud LLM with streaming-token callbacks, and TTS synthesis into one responsive dialogue loop shows the applicant can integrate heterogeneous AI systems into a coherent user-facing product, handling streaming, latency, multi-provider request-format differences, and event-driven handoff between the AI task and the audio task.
传感器融合与交互：BMI270 IMU 到手势输入
Sensor fusion & interaction: BMI270 IMU to gesture input
项目对应 · IN THE PROJECT
Day 5 传感器工作：在 100Hz 传感器轮询任务中经 I2C 读取 BMI270 6 轴 IMU（touch_motion_detect_gesture），分类双击/摇晃/倾斜手势，经事件队列驱动赛博骰子与手势控制的 2048 游戏。
Day-5 sensor work: reading the BMI270 6-axis IMU over I2C in the 100 Hz sensor-poll task (touch_motion_detect_gesture), classifying double-tap/shake/tilt gestures that drive the cyber-dice and the gesture-controlled 2048 game through the event queue.
知识点 · KNOWLEDGE
I2C 总线协议与 BMI270 寄存器读取（芯片 ID 校验、PWR_CTRL、加速度/陀螺仪数据寄存器）、加速度计/陀螺仪单位与换算、基于阈值的手势检测算法、定频（100Hz）传感器轮询与经 FreeRTOS 队列的事件分发。
I2C bus protocol and BMI270 register reading (chip-ID check, PWR_CTRL, accel/gyro data registers), accelerometer/gyroscope units and scaling, threshold-based gesture detection algorithms, fixed-rate (100 Hz) sensor polling and event dispatch via FreeRTOS queues.
文书角度 · ESSAY ANGLE
以固定 100Hz 经 I2C 读取 6 轴 IMU 原始数据，并解算双击/摇晃/倾斜手势用于驱动赛博骰子与手势控制的 2048 游戏，体现申请者理解从传感器到交互的映射，能把噪声惯性信号滤波为可靠的人机输入，这是物理计算的根基。
Reading raw 6-axis IMU data over I2C at a fixed 100 Hz and computing double-tap / shake / tilt gestures that drive the cyber dice and a gesture-controlled 2048 game shows the applicant understands sensor-to-interaction mapping, filtering noisy inertial signals into reliable human input, the foundation of physical computing.
Wi-Fi 与云端 API：把 MCU 接入互联网服务
Wi-Fi & cloud APIs: connecting an MCU to internet services
项目对应 · IN THE PROJECT
Day 9 网络工作：带事件驱动连接/失败位的 WiFi station 连接、HTTP 客户端 + cJSON 解析、NTP 时间同步，以及 IP 定位 + 天气 REST API，解析结果在 LVGL 显示上渲染实时天气时钟。
Day-9 networking work: WiFi station connection with event-driven connect/fail bits, HTTP client + cJSON parsing, NTP time sync, and an IP-geolocation + weather REST API whose parsed results render the live weather-clock on the LVGL display.
知识点 · KNOWLEDGE
WiFi station 连接与事件驱动重连（事件组位）、HTTP 客户端 GET/POST、嵌入式 C 下的 cJSON 解析、NTP 时间同步、IP 定位与天气 REST API 调用、API 密钥与凭证在 NVS 中的管理。
WiFi station connection and event-driven reconnect (event group bits), HTTP client GET/POST, cJSON parsing on embedded C, NTP time synchronization, IP-geolocation and weather REST API consumption, API-key/credential handling in NVS.
文书角度 · ESSAY ANGLE
让设备连接 WiFi、用 NTP 同步时间、调用 IP 定位+天气 REST API 并用 cJSON 解析响应，体现申请者能把微控制器接入互联网服务，处理 station 模式事件生命周期、HTTP 请求/响应、嵌入式 JSON 解析，以及离线玩具不会遇到的失败模式（断连/重连），这是物联网工程的基石。
Connecting the device to WiFi, synchronizing time over NTP, and consuming an IP-geolocation-plus-weather REST API with cJSON-parsed responses shows the applicant can bridge a microcontroller to internet services, handling the station-mode event lifecycle, HTTP request/response, embedded JSON parsing, and failure modes (disconnect/reconnect) that an offline toy never faces, the basis of IoT engineering.
技能
你将学到什么
硬件
- ESP32-S3
- I2S 音频
- SPI LCD 显示
- 3D 打印
软件
- ESP-IDF C
- LLM API
- STT/TTS
- WiFi
系统
- 音频流水线
- 对话状态机
- 低功耗管理
工程
- 3D 建模
- 组装调试
- 文档撰写
- 项目演示
硬件
所需材料
ESP32-S3
双核 + WiFi+BT
ST7789 LCD
1.54" 240×240 SPI
ES8311
音频编解码 + 麦克风
3D 打印外壳
PLA
日程
项目时间线
嵌入式基础
ESP32-S3 入门、I2S 音频、WiFi
AI 能力
STT 语音识别、LLM API、TTS 合成
系统集成
对话状态机、3D 外壳、组装展示
资源
开源参考
本项目基于以下开源项目改编 · Adapted from the open-source project below