Mild命名统一 + encoder_thread防饿死 + AGENTS.md重写
This commit is contained in:
spdis
@@ -2,23 +2,23 @@
|
||||
|
||||
## Project overview
|
||||
|
||||
LoongArch64 embedded autonomous smart car using OpenCV + NanoDet V10 model.
|
||||
Runs on-device (Loongson board) with cross-compilation from x86 Linux.
|
||||
龙芯 2K0300 嵌入式自动驾驶智能车，OpenCV + Mild v12 魔改 YOLO 模型。
|
||||
板端运行，x86 Linux 交叉编译。
|
||||
|
||||
## Build
|
||||
|
||||
```sh
|
||||
# On the build host (x86 Linux):
|
||||
# On the build host (x86 Linux/WSL):
|
||||
mkdir build && cd build
|
||||
cmake ..
|
||||
make
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **Cross-compilation target**: LoongArch64 (`-march=loongarch64`)
|
||||
- **Cross-compilation target**: LoongArch64 (`-march=loongarch64 -mtune=loongarch64`)
|
||||
- **Toolchain**: `/opt/loongson-gnu-toolchain-8.3-x86_64-loongarch64-linux-gnu-rc1.6`
|
||||
- **C++ standard**: 17
|
||||
- **OpenCV path** (device-side): `/mnt/d/PPPProgram/smartcar/opencv_device/`
|
||||
- `cross.cmake` is included **before** `cmake_minimum_required` — this is intentional.
|
||||
- **OpenCV path** (device-side): `D:\PPPProgram\smartcar\opencv_device\` (mounted as `/mnt/d/...` in WSL)
|
||||
- `cross.cmake` is included **before** `cmake_minimum_required` — intentional, to set compiler before project().
|
||||
|
||||
## Project structure
|
||||
|
||||
@@ -30,65 +30,117 @@ make
|
||||
| `docs/` | Architecture docs (`ARCHITECTURE.md` is the design reference) |
|
||||
| `build/` | CMake build output (gitignored) |
|
||||
|
||||
**Executable name**: `smartcar_demo1` (the CMake project name `smartcar_demo2` is different — don't assume they match).
|
||||
**Executable name**: `smartcar_demo1` (CMake project name `smartcar_demo2` ≠ executable name).
|
||||
|
||||
## Excluded modules
|
||||
|
||||
These source files exist but are **excluded from build**:
|
||||
|
||||
- `vl53l0x.cpp` — laser ranging module, hardware not connected
|
||||
- `zebra_detect.cpp` — classic zebra-crossing detection, replaced by NanoDet model
|
||||
- `encoder.cpp` — referenced in exclusion list but file does **not** exist in repo
|
||||
- `zebra_detect.cpp` — classic zebra-crossing detection (replaced by Mild model)
|
||||
|
||||
## Device-side operation
|
||||
|
||||
The onboard binary is controlled via text files in the working directory — no CLI args.
|
||||
Controlled via `ctl.sh` (writes to text files, no CLI args):
|
||||
|
||||
```sh
|
||||
# On device:
|
||||
sh ctl.sh init # initialize GPIO/PWM pins
|
||||
sh ctl.sh start # start the demo
|
||||
sh ctl.sh init # initialize GPIO/PWM pins + write default config files
|
||||
sh ctl.sh start # start the demo in background
|
||||
sh ctl.sh stop # stop and zero PWM
|
||||
```
|
||||
|
||||
`ctl.sh` sets default config values by writing to files like `./speed`, `./kp`, `./steer_gain`, etc. The running binary reads these files. `start.sh` uses `LD_PRELOAD=./gpio_fix_final.so` (GPIO workaround).
|
||||
`ctl.sh` sets default config values by writing to files like `./speed`, `./kp`, `./steer_gain`, etc. The running binary reads these files at startup and optionally re-reads them every 7 frames (when `./debug` = 1).
|
||||
|
||||
`start.sh` is an alternative launcher using `LD_PRELOAD=./gpio_fix_final.so` for the GPIO 74 workaround.
|
||||
|
||||
## Configuration system
|
||||
|
||||
All config is via single-value text files in the working directory:
|
||||
|
||||
- `./speed` (double) — target speed
|
||||
- `./start` (0/1) — motor enable switch
|
||||
- `./debug` (0/1) — when 1, reloads all config files every 7 frames
|
||||
- `./showImg` (0/1) — LCD display toggle
|
||||
- `./deadband`, `./steer_gain`, `./center_bias` — steering tuning
|
||||
### Steering
|
||||
- `./speed` (double) — target speed (duty cycle %)
|
||||
- `./deadband` — steering deadband (pixels)
|
||||
- `./steer_gain` — steering gain multiplier
|
||||
- `./center_bias` — midline offset correction
|
||||
- `./foresee` — look-ahead row for steering
|
||||
- `./zebrasee` — zebra crossing near-threshold
|
||||
|
||||
### Debug / Display
|
||||
- `./start` (0/1) — motor enable switch
|
||||
- `./showImg` (0/1) — LCD display toggle
|
||||
- `./debug` (0/1) — when 1, reloads all config files every 7 frames
|
||||
- `./destfps` — target framerate
|
||||
- `./kp`, `./ki`, `./kd` — PID gains (currently **not used**; motor is open-loop)
|
||||
- `./mortor_kp`, `./mortor_ki`, `./mortor_kd` — motor encoder PID (spelling `mortor` is intentional)
|
||||
- `./saveImg` — when written as 1 (in debug mode), saves current frame
|
||||
|
||||
### Zebra crossing
|
||||
- `./zebrasee` — zebra crossing near-threshold (cy > this = near)
|
||||
|
||||
### Cone avoidance
|
||||
- `./cone_avoid_gain` — midline deformation push amount (normalized)
|
||||
- `./cone_avoid_range` — deformation ramp steepness
|
||||
- `./cone_speed` — speed multiplier when cone triggered
|
||||
- `./cone_min_frames` — consecutive confirmation frames
|
||||
- `./cone_margin` — 0=center point, 1=box edge
|
||||
- `./cone_thresh` — confidence threshold
|
||||
- `./cone_hold_frames` — hold deformation frames after cone disappears
|
||||
|
||||
### Encoder brake
|
||||
- `./brake_scale` — speed (pps) → brake duty cycle (ns) scaling factor
|
||||
- `./brake_max` — maximum brake duty cycle (ns)
|
||||
|
||||
### PID gains (reserved, motor is open-loop)
|
||||
- `./kp`, `./ki`, `./kd` — steering PID (currently not used)
|
||||
- `./mortor_kp`, `./mortor_ki`, `./mortor_kd` — motor encoder PID (note: spelling `mortor` is intentional)
|
||||
|
||||
## Architecture
|
||||
|
||||
9-step per-frame pipeline in `CameraHandler()` (`src/camera.cpp`):
|
||||
1. Capture frame (640×480 MJPEG → BGR)
|
||||
2. Optionally save image
|
||||
3. Vision line tracking (80×60 downscale → HSV+Otsu → floodFill → mid_line[60])
|
||||
4. Model inference every 2nd frame (NanoDet V10, 4 classes: cone/red/green/zebra)
|
||||
5. Zebra crossing state machine (NORMAL→STOP(4s)→COOLDOWN(5s))
|
||||
6. Steering servo control (PWM, deadband-filtered)
|
||||
7. Motor control (open-loop duty cycle, curve-based slowdown in turns)
|
||||
8. LCD rendering (`/dev/fb0` mmap)
|
||||
9. FPS logging
|
||||
### Per-frame pipeline in `CameraHandler()` ([`src/camera.cpp`](file:///D:\PPPProgram\smartcar\smartcar2\src\camera.cpp))
|
||||
|
||||
**Key detail**: Motor control is **open-loop** — `MotorController::updateSpeed()` and encoder PID exist in code but are never called from the main loop. Only `updateduty()` (direct PWM) is used.
|
||||
1. **Capture frame** — 640×480 MJPEG → `cv::imdecode(IMREAD_REDUCED_COLOR_4)` → 1/4 decode to 320×240 BGR
|
||||
2. **Save image** (debug mode only, when `./saveImg` = 1)
|
||||
3. **Vision line tracking** — resize to 80×60 → HSV dual-channel Otsu → floodFill → longest-run search → `left_line[60]`, `right_line[60]`, `mid_line[60]`
|
||||
4. **Model inference** — every 2nd frame, Mild v12 model at 160×120, 4 classes: cone(0) / red light(1) / green light(2) / zebra(3)
|
||||
5. **Zebra state machine** — NORMAL → STOP(4s) → COOLDOWN(5s) → NORMAL, with debounce (5 frames min, far-enough cy check)
|
||||
6. **Traffic light state machine** — TL_NORMAL → TL_STOP (red≥3 frames) → TL_WAIT_GREEN (red gone) → TL_NORMAL (green≥3 frames)
|
||||
7. **Cone avoidance** — confirmed cone detection deforms midline to steer away, with hold-frames decay after cone disappears
|
||||
8. **Steering servo** — deadband-filtered proportional control via PWM
|
||||
9. **Motor control** — open-loop duty cycle with curve slowdown in turns; encoder brake on zebra/red-light stop
|
||||
10. **LCD rendering** — `/dev/fb0` mmap, RGB565 conversion, bounding boxes + state indicator overlay
|
||||
11. **FPS logging** — every 15 frames, prints per-stage timing
|
||||
|
||||
Currently **cones (cls=0), red lights (cls=1), and green lights (cls=2)** are detected by the model but ignored in decision logic.
|
||||
### Motor control details
|
||||
|
||||
- **Open-loop**: `MotorController::updateduty(spd)` — direct PWM duty cycle, no encoder feedback in normal driving.
|
||||
- **Encoder brake**: When `zebra_block=true` (red light or zebra stop), reads `g_enc_speed` from the encoder thread and applies reverse braking proportional to current speed.
|
||||
- **Encoder thread** ([`control.cpp:32`](file:///D:\PPPProgram\smartcar\smartcar2\src\control.cpp#L32)): polls GPIO67 (LSB pulse) + GPIO72 (direction) at ~2kHz, 100ms window speed calculation → `g_enc_speed` (pulses/sec). Now includes 500µs sleep to prevent CPU starvation.
|
||||
- **Curve slowdown**: `speed *= (1.0 - |deviation| × 0.4)`, clamped to ≥ 60%.
|
||||
|
||||
### Model: Mild v12
|
||||
|
||||
- **Architecture**: 3→9→9→13→13→19→19→19→44→44→64→64 → head(64→64,dw) → out(64→9)
|
||||
- **Input**: 160×120 BGR
|
||||
- **Output**: 4 classes (cone / red light / green light / zebra) + background
|
||||
- **Classes used in decision logic**:
|
||||
- cls=0 (cone) → midline deformation avoidance
|
||||
- cls=1 (red light) → traffic light state machine stop
|
||||
- cls=2 (green light) → traffic light state machine resume
|
||||
- cls=3 (zebra) → zebra crossing state machine stop
|
||||
- **Per-class thresholds**: `{0.90, 0.75, 0.80, 0.90}` (cone/red/green/zebra)
|
||||
- **Weight file**: `mild_v12.bin` (105KB custom binary format)
|
||||
- **Inference engine**: `src/model_v10.{hpp,cpp}` (names kept as `model_v10_*` for compatibility, internally Mild v3)
|
||||
|
||||
### Vision pipeline details
|
||||
|
||||
- Processing resolution: 80×60 (optimal balance for 2K0300 CPU)
|
||||
- HSV dual-channel Otsu: H channel (blue hue) + S channel (saturation) → bitwise AND → track region
|
||||
- FloodFill: seed at bottom center (40, 50), 8-connected, extracts connected track
|
||||
- Per-row longest-run search on floodFill result → left/right boundaries
|
||||
- Missing-line recovery: propagate downward midline + virtual boundaries
|
||||
|
||||
## Style and conventions
|
||||
|
||||
- C++ files have **no copyright headers** and **minimal comments** — comments are ascii-box-style block comments when present
|
||||
- Header guards use the form `#ifndef FILENAME_H_` / `#define FILENAME_H_`
|
||||
- C++ files have **no copyright headers** — comments are ascii-box-style block comments when present
|
||||
- Header guards use `#ifndef FILENAME_H_` / `#define FILENAME_H_` format (with some exceptions: `#pragma once` in model files)
|
||||
- Global state uses `extern` globals (e.g., `g_cfg`, `g_steer_deviation`, `g_boxes`)
|
||||
- `lib/` contains .h files only; `src/` contains .cpp files only
|
||||
- No unit tests, no CI, no linting — this is embedded code tested on-device
|
||||
|
||||
@@ -51,6 +51,8 @@ static void encoder_thread() {
|
||||
pulse_cnt = 0;
|
||||
t0 = t1;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::microseconds(500));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+5
-5
@@ -1,5 +1,5 @@
|
||||
/*
|
||||
* YOLO_Mega_Mild_r2 C++ inference engine v3
|
||||
* Mild Mega r2 C++ inference engine v3 (基于魔改YOLO架构)
|
||||
* Architecture: 3-9-9-13-13-19-19-19-44-44-64-64 -> head(64-64,dw) -> out(64-9)
|
||||
*/
|
||||
#include "model_v10.hpp"
|
||||
@@ -395,9 +395,9 @@ static void resize_bilinear_hwc3(const uint8_t* src, int sw, int sh, uint8_t* ds
|
||||
}
|
||||
}
|
||||
|
||||
static void preproc_160(const uint8_t* img, YoloPixFmt fmt) {
|
||||
static void preproc_160(const uint8_t* img, MildPixFmt fmt) {
|
||||
for (int c = 0; c < 3; ++c) {
|
||||
const int sc = (fmt == YOLO_FMT_BGR) ? (2 - c) : c;
|
||||
const int sc = (fmt == MILD_FMT_BGR) ? (2 - c) : c;
|
||||
float* ch = g_preproc + c * IN_H * IN_W;
|
||||
for (int y = 0; y < IN_H; ++y) {
|
||||
const uint8_t* row = img + y * IN_W * 3;
|
||||
@@ -452,7 +452,7 @@ bool model_v10_init(const char* path) {
|
||||
return true;
|
||||
}
|
||||
|
||||
int model_v10_detect_fmt(const uint8_t* img, int w, int h, YoloPixFmt fmt,
|
||||
int model_v10_detect_fmt(const uint8_t* img, int w, int h, MildPixFmt fmt,
|
||||
DetectBoxV10* boxes, int max_boxes, float thr) {
|
||||
if (!g_rdy || !img || !boxes || max_boxes <= 0 || w <= 1 || h <= 1) return 0;
|
||||
|
||||
@@ -476,7 +476,7 @@ int model_v10_detect_fmt(const uint8_t* img, int w, int h, YoloPixFmt fmt,
|
||||
|
||||
int model_v10_detect(const uint8_t* img, int w, int h, DetectBoxV10* boxes, int max_boxes, const float* thresh) {
|
||||
float thr = thresh ? thresh[0] : 0.6f;
|
||||
return model_v10_detect_fmt(img, w, h, YOLO_FMT_BGR, boxes, max_boxes, thr);
|
||||
return model_v10_detect_fmt(img, w, h, MILD_FMT_BGR, boxes, max_boxes, thr);
|
||||
}
|
||||
|
||||
const float* model_v10_forward_debug(const float* chw) {
|
||||
|
||||
+1
-1
@@ -7,7 +7,7 @@ struct DetectBoxV10 {
|
||||
float cx, cy, w, h;
|
||||
};
|
||||
|
||||
typedef enum { YOLO_FMT_BGR = 0, YOLO_FMT_RGB = 1 } YoloPixFmt;
|
||||
typedef enum { MILD_FMT_BGR = 0, MILD_FMT_RGB = 1 } MildPixFmt;
|
||||
|
||||
bool model_v10_init(const char* path);
|
||||
int model_v10_detect(const uint8* bgr, int w, int h, DetectBoxV10* boxes, int max, const float* thresh);
|
||||
|
||||
Reference in New Issue
Block a user