spdis
152 lines
6.8 KiB
Markdown
152 lines
6.8 KiB
Markdown
# AGENTS.md — SmartCar Demo
|
||
|
||
## Project overview
|
||
|
||
龙芯 2K0300 嵌入式自动驾驶智能车，OpenCV + Mild v12 魔改 YOLO 模型。
|
||
板端运行，x86 Linux 交叉编译。
|
||
|
||
## Build
|
||
|
||
```sh
|
||
# On the build host (x86 Linux/WSL):
|
||
mkdir build && cd build
|
||
cmake ..
|
||
make -j$(nproc)
|
||
```
|
||
|
||
- **Cross-compilation target**: LoongArch64 (`-march=loongarch64 -mtune=loongarch64`)
|
||
- **Toolchain**: `/opt/loongson-gnu-toolchain-8.3-x86_64-loongarch64-linux-gnu-rc1.6`
|
||
- **C++ standard**: 17
|
||
- **OpenCV path** (device-side): `D:\PPPProgram\smartcar\opencv_device\` (mounted as `/mnt/d/...` in WSL)
|
||
- `cross.cmake` is included **before** `cmake_minimum_required` — intentional, to set compiler before project().
|
||
|
||
## Project structure
|
||
|
||
| Directory | Purpose |
|
||
|-----------|---------|
|
||
| `src/` | Source → compiled into `common_lib` (static lib) |
|
||
| `lib/` | Public headers only (no .cpp) |
|
||
| `main/` | Entry point → executable `smartcar_demo1` |
|
||
| `docs/` | Architecture docs (`ARCHITECTURE.md` is the design reference) |
|
||
| `build/` | CMake build output (gitignored) |
|
||
|
||
**Executable name**: `smartcar_demo1` (CMake project name `smartcar_demo2` ≠ executable name).
|
||
|
||
## Excluded modules
|
||
|
||
These source files exist but are **excluded from build**:
|
||
|
||
- `vl53l0x.cpp` — laser ranging module, hardware not connected
|
||
- `zebra_detect.cpp` — classic zebra-crossing detection (replaced by Mild model)
|
||
|
||
## Device-side operation
|
||
|
||
Controlled via `ctl.sh` (writes to text files, no CLI args):
|
||
|
||
```sh
|
||
# On device:
|
||
sh ctl.sh init # initialize GPIO/PWM pins + write default config files
|
||
sh ctl.sh start # start the demo in background
|
||
sh ctl.sh stop # stop and zero PWM
|
||
```
|
||
|
||
`ctl.sh` sets default config values by writing to files like `./speed`, `./kp`, `./steer_gain`, etc. The running binary reads these files at startup and optionally re-reads them every 7 frames (when `./debug` = 1).
|
||
|
||
`start.sh` is an alternative launcher using `LD_PRELOAD=./gpio_fix_final.so` for the GPIO 74 workaround.
|
||
|
||
## Configuration system
|
||
|
||
All config is via single-value text files in the working directory:
|
||
|
||
### Steering
|
||
- `./speed` (double) — target speed (duty cycle %)
|
||
- `./deadband` — steering deadband (pixels)
|
||
- `./steer_gain` — steering gain multiplier
|
||
- `./center_bias` — midline offset correction
|
||
- `./foresee` — look-ahead row for steering
|
||
|
||
### Debug / Display
|
||
- `./start` (0/1) — motor enable switch
|
||
- `./showImg` (0/1) — LCD display toggle
|
||
- `./debug` (0/1) — when 1, reloads all config files every 7 frames
|
||
- `./destfps` — target framerate
|
||
- `./saveImg` — when written as 1 (in debug mode), saves current frame
|
||
|
||
### Zebra crossing
|
||
- `./zebrasee` — zebra crossing near-threshold (cy > this = near)
|
||
|
||
### Cone avoidance
|
||
- `./cone_avoid_gain` — midline deformation push amount (normalized)
|
||
- `./cone_avoid_range` — deformation ramp steepness
|
||
- `./cone_speed` — speed multiplier when cone triggered
|
||
- `./cone_min_frames` — consecutive confirmation frames
|
||
- `./cone_margin` — 0=center point, 1=box edge
|
||
- `./cone_thresh` — confidence threshold
|
||
- `./cone_hold_frames` — hold deformation frames after cone disappears
|
||
|
||
### Encoder brake
|
||
- `./brake_scale` — speed (pps) → brake duty cycle (ns) scaling factor
|
||
- `./brake_max` — maximum brake duty cycle (ns)
|
||
|
||
### PID gains (reserved, motor is open-loop)
|
||
- `./kp`, `./ki`, `./kd` — steering PID (currently not used)
|
||
- `./mortor_kp`, `./mortor_ki`, `./mortor_kd` — motor encoder PID (note: spelling `mortor` is intentional)
|
||
|
||
## Architecture
|
||
|
||
### Per-frame pipeline in `CameraHandler()` ([`src/camera.cpp`](file:///D:\PPPProgram\smartcar\smartcar2\src\camera.cpp))
|
||
|
||
CameraHandler 现已拆分为多个函数，主函数 ~40 行仅做编排：
|
||
|
||
```cpp
|
||
CameraHandler()
|
||
├── capture_frame() // 1. MJPG 取帧 → 1/4解码 320×240 BGR
|
||
├── save_image_if_requested() // 2. 保存帧 (debug)
|
||
├── image_main() // 3. 视觉巡线 (80×60 HSV-Otsu-FloodFill)
|
||
├── run_model_inference() // 4. Mild v12 每2帧推理 (160×120, 4类)
|
||
├── zebra_process() // 5a. 斑马线去抖 + Z_STOP(4s)/Z_COOLDOWN(5s)
|
||
├── traffic_light_process() // 5b. 红绿灯 TL_NORMAL→TL_STOP→TL_WAIT_GREEN
|
||
├── cone_detect_and_deform() // 5c. 锥桶检测 + 中线变形 + 保持衰减
|
||
├── steering_update() // 6. 舵机比例控制 (deadband过滤)
|
||
├── motor_update() // 7. 电机开环PWM + 编码器刹车 + 弯道减速
|
||
├── lcd_render() // 8. LCD RGB565渲染 (边界线+检测框+状态)
|
||
└── fps_log() // 9. 每15帧打印分步耗时
|
||
```
|
||
|
||
### Motor control details
|
||
|
||
- **Open-loop**: `MotorController::updateduty(spd)` — direct PWM duty cycle, no encoder feedback in normal driving.
|
||
- **Encoder brake**: When `zebra_block=true` (red light or zebra stop), reads `g_enc_speed` from the encoder thread and applies reverse braking proportional to current speed.
|
||
- **Encoder thread** ([`control.cpp:32`](file:///D:\PPPProgram\smartcar\smartcar2\src\control.cpp#L32)): polls GPIO67 (LSB pulse) + GPIO72 (direction) at ~2kHz, 100ms window speed calculation → `g_enc_speed` (pulses/sec). Now includes 500µs sleep to prevent CPU starvation.
|
||
- **Curve slowdown**: `speed *= (1.0 - |deviation| × 0.4)`, clamped to ≥ 60%.
|
||
|
||
### Model: Mild v12
|
||
|
||
- **Architecture**: 3→9→9→13→13→19→19→19→44→44→64→64 → head(64→64,dw) → out(64→9)
|
||
- **Input**: 160×120 BGR
|
||
- **Output**: 4 classes (cone / red light / green light / zebra) + background
|
||
- **Classes used in decision logic**:
|
||
- cls=0 (cone) → midline deformation avoidance
|
||
- cls=1 (red light) → traffic light state machine stop
|
||
- cls=2 (green light) → traffic light state machine resume
|
||
- cls=3 (zebra) → zebra crossing state machine stop
|
||
- **Per-class thresholds**: `{0.90, 0.75, 0.80, 0.90}` (cone/red/green/zebra)
|
||
- **Weight file**: `mild_v12.bin` (105KB custom binary format)
|
||
- **Inference engine**: `src/model_v10.{hpp,cpp}` (names kept as `model_v10_*` for compatibility, internally Mild v3)
|
||
|
||
### Vision pipeline details
|
||
|
||
- Processing resolution: 80×60 (optimal balance for 2K0300 CPU)
|
||
- HSV dual-channel Otsu: H channel (blue hue) + S channel (saturation) → bitwise AND → track region
|
||
- FloodFill: seed at bottom center (40, 50), 8-connected, extracts connected track
|
||
- Per-row longest-run search on floodFill result → left/right boundaries
|
||
- Missing-line recovery: propagate downward midline + virtual boundaries
|
||
|
||
## Style and conventions
|
||
|
||
- C++ files have **no copyright headers** — comments are ascii-box-style block comments when present
|
||
- Header guards use `#ifndef FILENAME_H_` / `#define FILENAME_H_` format (with some exceptions: `#pragma once` in model files)
|
||
- Global state uses `extern` globals (e.g., `g_cfg`, `g_steer_deviation`, `g_boxes`)
|
||
- `lib/` contains .h files only; `src/` contains .cpp files only
|
||
- No unit tests, no CI, no linting — this is embedded code tested on-device
|