AllenHsu

Spearheaded advancements in dataset curation, automated quality control, and multimodal large-model development for autonomous driving.

• OD Dataset Curation (August 2023 - May 2024)
  • Addressed the challenges of high annotation and training costs associated with millions of legacy data points and massive incremental datasets in object detection (OD) tasks.
  • Implemented active learning methods based on data gradient similarity to emphasize underlying reasoning process similarities and better utilize data distributions.
  • Achieved comparable accuracy to full dataset training using only 20% curated data, improving training efficiency by 5x and significantly reducing storage and computational costs.
  • Reduced annotation demand by 90% on incremental datasets, realizing a 10x efficiency improvement.
  • Developed a data quality inspection model to tackle common issues such as inaccurate bounding boxes and heading errors in auto-labeled production data.
  • Achieved precision of 92% and recall of 89% on noisy data identification using a fine-tuned evaluation dataset. The filtered OD model demonstrated a 1% overall performance improvement, with notable enhancements in close-range object detection.
  • Deployed the model on annotation production lines to proactively detect data defects, reducing manual inspection costs. This work resulted in two patents.
• End-to-End Model Development (May 2024 - Present)
  • Pioneered the adaptation of vision-language multimodal large models for autonomous driving using the LLava-next framework.
  • Replaced the vision backbone with the DINOv2 base version and designed a 3D Q-Former to align visual and textual modalities.
  • Developed a custom VQA dataset to equip the model with 3D perception and reasoning capabilities. Integrated the VLM-based planner to handle complex scenarios, ensuring robustness and generalization for unknown cases. This work resulted in a patent.
  • Introduced diffusion models into trajectory planning for autonomous vehicles, inspired by their success in robotics.
  • Designed a transformer-based noise model with DDPM scheduling to predict 7-second future trajectories directly. The diffusion model ensures smooth, controllable, and natural multimodal predictions, outperforming discriminative models in long-horizon tasks.

Worked in the field of autonomous driving decision-making.

• Autonomous driving decision-making
  • Conducted research on training vehicles to drive on highways and make correct decisions using deep reinforcement learning methods.
  • Traditional decision-making algorithms are typically based on rules and manual feature design, which often exhibit a conservative behavior to ensure safety. Utilized DQN (Deep Q-Network), one of the classic deep reinforcement learning algorithms, to make decisions based on the surrounding environment and ego vehicle information, aiming to maximize the expected reward.
  • Data engineering was primarily performed using open-source datasets and simulation environments.
  • Achieved an accuracy rate of over 91% in simulated testing, demonstrating the model’s ability to make safe and reasonable decisions in high-speed scenarios.
• Teaching assistant for the “Reinforcement Learning” course at the University of Chinese Academy of Sciences
  • Assisted in the development of course materials and experiments.
  • Covered topics such as Markov decision processes, Monte Carlo methods, temporal difference learning, eligibility traces, etc.
  • Designed unmanned vehicle simulation scenarios using the gym framework for implementing reinforcement learning methods like SARSA, Q-Learning, etc.
• Collaboration with Professor Zhang Lijun

Development of open-source tutorials related to ROS (Robot Operating System)

• Developed tutorials covering various aspects of ROS, including communication mechanisms such as topics and services.
• Explored core functionality packages, such as the parameter server and TF (Transform) coordinate transformation.
• Analyzed the hardware and software components of robot platforms, including modeling language URDF (Unified Robot Description Format), controllers, and the Gazebo simulator.
• Provided detailed explanations of the ROS Navigation Stack, including nav_core, global_planner, local_planner, move_base, cost_map, AMCL (Adaptive Monte Carlo Localization), and more.
• Published tutorials and corresponding code on platforms like CSDN and GitHub.

Work related to unmanned aerial vehicle (UAV) visual odometry

• Developed a visual odometry system for UAV localization.
• Constructed local feature maps using keyframes and matched ORB feature points between each frame and the local map.
• Used the PnP (Perspective-n-Point) algorithm to estimate the global pose.
• Designed specifically for scenarios where only short-term motion needs to be considered, such as UAV control, with lower computational requirements compared to full-scale VSLAM systems while meeting the accuracy requirements of the application.

• Pursued a Ph.D. in Computer Science and Technology at Sun Yat-sen University, renowned for its academic excellence and research contributions.
• Engaged in cutting-edge research projects related to SLAM and Perception algorithms.
• Developed an in-depth understanding of advanced algorithms and their applications in the field of autonomous systems.
• Although the program was discontinued due to personal circumstances, the short duration allowed for the acquisition of valuable knowledge and research skills.