Controllable Continual Test-Time Adaptation

Ziqi Shi,Fan Lyu,Ye Liu,…,Wei Feng,Zhang Zhang,Liang Wang

[paper] [code]

We propose $\textbf{C}$ontrollable $\textbf{Co}$ntinual $\textbf{T}$est-$\textbf{T}$ime $\textbf{A}$daptation (C-CoTTA), which explicitly prevents any single category from encroaching on others, thereby mitigating the mutual influence between categories caused by uncontrollable shifts. Moreover, our method reduces the sensitivity of model to domain transformations, thereby minimizing the magnitude of category shifts.

Conformal Uncertainty Indicator for Continual Test-Time Adaptation

Fan Lyu, Hanyu Zhao, Ziqi Shi,…, Zhang Zhang, Liang Wang

[paper] [code]

we propose a Conformal Uncertainty Indicator (CUI) for CTTA, leveraging Conformal Prediction (CP) to generate prediction sets that include the true label with a specified coverage probability. Since domain shifts can lower the coverage than expected, making CP unreliable, we dynamically compensate for the coverage by measuring both domain and data differences. Reliable pseudo-labels from CP are then selectively utilized to enhance adaptation.

TJURM-Radar

[code]

This project implements the radar station robot functionality in Robomaster competitions using two approaches: monocular RGB and RGB + LiDAR fusion. Both methods employ 4K images captured by industrial RGB cameras as direct inputs for target detection models to identify and locate all robots on the field. In the monocular RGB approach, the camera’s extrinsic parameters are calibrated using large-scale PnP to establish the relationship between the camera and field coordinate systems. Depth information is then obtained through 3D structure rendering of the field. For the RGB + LiDAR fusion approach, the robot’s onboard LiDAR provides more precise depth information for targets, enhancing localization accuracy. This integration leverages the complementary strengths of RGB cameras and LiDAR, where RGB images offer rich texture and color information, while LiDAR supplies accurate depth data.

TJURM-AutoDrive

[code]

This project leverages a large collection of first-person videos from Robomaster competitions to extract the movement trajectories of robots within the mini-map using traditional computer vision algorithms. These trajectories and videos form the basis of an autonomous driving dataset. By extracting image features through common backbone networks and employing GRU for trajectory prediction, the autonomous driving model in the Robomaster context goes beyond navigation and obstacle avoidance. It also incorporates trajectory planning (deciding its own route, unlike in traditional cars where this is determined by road planning algorithms) and autonomous decision-making (choosing where to go, unlike in traditional cars where this is specified by the driver). These three capabilities correspond to increasingly distant prediction horizons. (The project is currently in active development, and interested parties are welcome to get in touch.)

TJURM-ArmorDiffuse

[code]

This project addresses the challenges of limited datasets and high annotation costs in Robomaster competitions by leveraging existing data and employing diffusion models to augment the dataset. Specifically, we designed a keypoint-conditional generation model to produce datasets that inherently contain the labels required for training detection models. Unlike existing approaches such as ControlNet, which inefficiently render keypoints as images, we innovatively use vector inputs directly as conditions. Extensive experiments have validated the effectiveness of our conditional generation approach.

TJURM-Detector

[code]

The customized key point detection model for Robomaster competition, modified from Yolov5, supports the key point detection of armor plate and energy mechanism, and provides the coordinates of points in the image coordinate system for the subsequent PnP positioning algorithm. Deploy to the Jetson Xavier NX edge computing device to accelerate inference using TensorRT while using knowledge distillation to achieve lossless compression of the model for faster inference.