Loongson_2k0300_SmartCar/AGENTS.md

AGENTS.md — SmartCar Demo

Project overview

龙芯 2K0300 嵌入式自动驾驶智能车，OpenCV + Mild v12 魔改 YOLO 模型。 板端运行，x86 Linux 交叉编译。

Build

# On the build host (x86 Linux/WSL):
mkdir build && cd build
cmake ..
make -j$(nproc)

• Cross-compilation target: LoongArch64 (-march=loongarch64 -mtune=loongarch64)
• Toolchain: /opt/loongson-gnu-toolchain-8.3-x86_64-loongarch64-linux-gnu-rc1.6
• C++ standard: 17
• OpenCV path (device-side): D:\PPPProgram\smartcar\opencv_device\ (mounted as /mnt/d/... in WSL)
• cross.cmake is included before cmake_minimum_required — intentional, to set compiler before project().

Project structure

Directory	Purpose
src/	Source → compiled into common_lib (static lib)
lib/	Public headers only (no .cpp)
main/	Entry point → executable smartcar_demo1
docs/	Architecture docs (ARCHITECTURE.md is the design reference)
build/	CMake build output (gitignored)

Executable name: smartcar_demo1 (CMake project name smartcar_demo2 ≠ executable name).

Excluded modules

These source files exist but are excluded from build:

• vl53l0x.cpp — laser ranging module, hardware not connected
• zebra_detect.cpp — classic zebra-crossing detection (replaced by Mild model)
• PIDController.cpp — unused (motor is open-loop direct drive)
• serial.cpp — Vofa/serial image streaming, unused

Device-side operation

Controlled via ctl.sh (writes to text files, no CLI args):

# On device:
sh ctl.sh init          # initialize GPIO/PWM pins + write default config files
sh ctl.sh start         # start the demo in background
sh ctl.sh stop          # stop and zero PWM

ctl.sh sets default config values by writing to files like ./speed, ./kp, ./steer_gain, etc. The running binary reads these files at startup and optionally re-reads them every 7 frames (when ./debug = 1).

start.sh is an alternative launcher using LD_PRELOAD=./gpio_fix_final.so for the GPIO 74 workaround. Note: start.sh references the binary as smartcar_demo but the actual executable is smartcar_demo1 — this is a known bug, use ctl.sh instead.

Configuration system

All config is via single-value text files in the working directory:

Steering

• ./speed (double) — target speed (duty cycle %)
• ./deadband — steering deadband (pixels)
• ./steer_gain — steering gain multiplier
• ./center_bias — midline offset correction
• ./foresee — look-ahead row for steering

Debug / Display

• ./start (0/1) — motor enable switch
• ./showImg (0/1) — LCD display toggle
• ./debug (0/1) — when 1, reloads all config files every 7 frames
• ./destfps — target framerate
• ./saveImg — when written as 1 (in debug mode), saves current frame

Zebra crossing

• ./zebrasee — zebra crossing near-threshold (cy > this = near)

Cone avoidance

• ./cone_avoid_gain — midline deformation push amount (normalized)
• ./cone_avoid_range — deformation ramp steepness
• ./cone_speed — speed multiplier when cone triggered
• ./cone_min_frames — consecutive confirmation frames
• ./cone_margin — 0=center point, 1=box edge
• ./cone_thresh — confidence threshold
• ./cone_hold_frames — hold deformation frames after cone disappears

Encoder brake

• ./brake_scale — speed (pps) → brake duty cycle (ns) scaling factor
• ./brake_max — maximum brake duty cycle (ns)

PID gains (reserved, motor is open-loop)

• ./kp, ./ki, ./kd — steering PID (currently not used)
• ./mortor_kp, ./mortor_ki, ./mortor_kd — motor encoder PID (note: spelling mortor is intentional)

Architecture

Per-frame pipeline in CameraHandler() (src/camera.cpp:578)

CameraHandler 现已拆分为多个函数，主函数 ~40 行仅做编排：

CameraHandler()
├── capture_frame()         // 1. MJPG 取帧 → 1/4解码 320×240 BGR
├── save_image_if_requested() // 2. 保存帧 (debug)
├── image_main()            // 3. 视觉巡线 (80×60 HSV-Otsu-FloodFill)
├── run_model_inference()   // 4. Mild v12 每2帧推理 (160×120, 4类)
├── zebra_process()         // 5a. 斑马线去抖 + Z_STOP(4s)/Z_COOLDOWN(5s)
├── traffic_light_process() // 5b. 红绿灯 TL_NORMAL→TL_STOP→TL_WAIT_GREEN
├── cone_detect_and_deform() // 5c. 锥桶检测 + 中线变形 + 保持衰减
├── steering_update()       // 6. 舵机比例控制 (deadband过滤)
├── motor_update()          // 7. 电机开环PWM + 编码器刹车 + 弯道减速
├── lcd_render()            // 8. LCD RGB565渲染 (边界线+检测框+状态)
└── fps_log()               // 9. 每15帧打印分步耗时

Motor control details

• Open-loop: MotorController::updateduty(spd) — direct PWM duty cycle, no encoder feedback in normal driving.
• Encoder brake: When zebra_block=true (red light or zebra stop), reads g_enc_speed from the encoder thread and applies reverse braking proportional to current speed.
• Encoder thread (control.cpp:32): polls GPIO67 (LSB pulse) + GPIO72 (direction) at ~2kHz, 100ms window speed calculation → g_enc_speed (pulses/sec). Now includes 500µs sleep to prevent CPU starvation.
• Curve slowdown: speed *= (1.0 - |deviation| × 0.4), clamped to ≥ 60%.

Model: Mild v12

• Architecture: 3→9→9→13→13→19→19→19→44→44→64→64 → head(64→64,dw) → out(64→9)
• Input: 160×120 BGR
• Output: 4 classes (cone / red light / green light / zebra) + background
• Classes used in decision logic:
  • cls=0 (cone) → midline deformation avoidance
  • cls=1 (red light) → traffic light state machine stop
  • cls=2 (green light) → traffic light state machine resume
  • cls=3 (zebra) → zebra crossing state machine stop
• Per-class thresholds: {0.90, 0.75, 0.80, 0.90} (cone/red/green/zebra)
• Weight file: mild_v12.bin (105KB custom binary format)
• Inference engine: src/model_v10.{hpp,cpp} (names kept as model_v10_* for compatibility, internally Mild v3)

Vision pipeline details

• Processing resolution: 80×60 (optimal balance for 2K0300 CPU)
• HSV dual-channel Otsu: H channel (blue hue) + S channel (saturation) → bitwise AND → track region
• FloodFill: seed at bottom center (40, 50), 8-connected, extracts connected track
• Per-row longest-run search on floodFill result → left/right boundaries
• Missing-line recovery: propagate downward midline + virtual boundaries

Style and conventions

• C++ files have no copyright headers — comments are ascii-box-style block comments when present
• Header guards use #ifndef FILENAME_H_ / #define FILENAME_H_ format (with some exceptions: #pragma once in model files)
• Global state uses extern globals (e.g., g_cfg, g_steer_deviation, g_boxes)
• lib/ contains .h files only; src/ contains .cpp files only
• No unit tests, no CI, no linting — this is embedded code tested on-device