Avatar

崔允端

副研究员

中国科学院深圳先进技术研究院

关于我

课题组以强化学习的工程应用为研究方向,面向复杂环境下的无人船、无人车、机器人以及大规模工业自动化系统研发稳健抵抗扰动、学习效率高的强化学习算法。 主要研究方向包括但不限于概率模型驱动的强化学习、具有高采样效率的无模型强化学习、分治大规模系统的多智能体强化学习。

课题组诚挚欢迎本科同学报考推免研究生,在读研究生同学客座访问,博士毕业生申请博士后,请将简历发送至 yd.cui[at]siat.ac.cn

课题组成员
研三:夏镭,龚茗荣
研二:黄文俊,缪晨阳,李尚德(南科大联培)
研一:王惠琳,江颖卓,孙明宇(南科大联培),MOHAMMAD MOHAMMADI (ANSO Scholarship)

客座学生:王玥 (University College London)

毕业生:尚致违 (香港科技大学·广州),李任行 (Singapore University of Technology and Design, SUTD),王金成 (华为), 邓乃天 (腾讯),张世天 (中国电子技术标准化研究院)

课题组承担项目
国家自然科学基金青年项目,“面向环境不确定性的强化学习无人船控制方法”,30万,2022-2024,主持,在研
中国科学院率先行动人才择优项目,500万,2021-2023,主持,在研
国家重点研发计划,“物联网与智慧城市关键技术及示范专项”重点专项子课题,854万,2020-2023,参与,在研
企业横向研究课题(华为),80万,结题

工作经历


2020年 四月至今
副研究员
中国科学院深圳先进技术研究院

2017年 十月 - 2020年 三月
博士后研究员
日本奈良先端科学技术大学院大学

教育经历


2014年 十月 - 2017年 九月
博士
日本奈良先端科学技术大学院大学

2012年 九月 -2014年 九月
硕士
日本同志社大学

2008年 八月 - 2012年 七月
学士
西安电子科技大学

奖励


2021年 五月
深圳市海外高层次人才(孔雀计划)B类认定

2020年 十一月
中国科学院率先行动人才择优计划B类

2019年 十一月
日本计测自动控制学会青年作者奖 (IROS 2019)
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems

2018年 十月
2018年日本神经网络学会优秀论文奖

2018年 三月
最优秀学生奖 (博士课程)
日本奈良先端科学技术大学院大学

2016年 十一月
最优秀论文奖
2016 IEEE-RAS 16th International Conference on Humanoid Robots

2016年 四月 - 2017年 九月
日本文部省奖学金 (MEXT)

学术活动


会员

The Institute of Electrical and Electronic Engineers (IEEE)

中国自动化学会 (CAA)

中国计算机学会 (CCF)

中国人工智能学会 (CAAI)

审稿人

ICRA, IROS, CoRL, Humanoids

IEEE Transactions on Industrial Informatics,

IEEE Transactions on Cybernetics,

IEEE Robotics and Automation Letters,

IEEE Transactions on Vehicular Technology,

Nature Communications, Automatica, Neural Networks,

Neurocomputing, Autonomous Robots, Robotics and Autonomous Systems,

Ocean Engineering, Information Sciences, Pattern Recognition

编辑

Editor, Special issue “Deformable object manipulation”, Frontiers in Neurorobotics

Associate Editor, IEEE International Conference on Robotics and Biomimetics (ROBIO) 2023

Assoxiate Editor, IEEE International Conference on Robotics and Automation (ICRA) 2024

编委会

IEEE International Conference on Advanced Robotics and Mechatronics (ICARM) 2019

发表论文

期刊


  1. Jia Liu, Yunduan Cui, Jianghua Duan, Zhengmin Jiang, Zhongming Pan, Kun Xu, and Huiyun Li, “Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles.” IEEE Transactions on Vehicular Technology, 2024. link (IF 6.8, JCR Q1)

  2. Dongfang Zhang, Yunduan Cui, Yao Xiao, Shengxiang Fu, Suk Won Cha, Namwook Kim, Hongyan Mao, and Chunhua Zheng. “An Improved Soft Actor-Critic-Based Energy Management Strategy of Fuel Cell Hybrid Vehicles with a Nonlinear Fuel Cell Degradation Model.” International Journal of Precision Engineering and Manufacturing-Green Technology, 2024. link (IF 4.2, JCR Q1)

  3. Zhiwei Shang, Renxing Li, Chunhua Zheng, Huiyun Li, and Yunduan Cui. “Relative Entropy Regularized Sample-Efficient Reinforcement Learning With Continuous Actions.” IEEE Transactions on Neural Networks and Learning Systems, 2023. link (IF 10.4, JCR Q1)

  4. Yixuan Ku, Chen Guo, Kangshuai Zhang, Yunduan Cui, Hongfeng Shu, Yang Yang, Lei Peng. “Toward Directed Spatiotemporal Graph: A New Idea for Heterogeneous Traffic Prediction.” IEEE Intelligent Transportation Systems Magazine, 2023. link (IF 3.6, JCR Q1)

  5. Yunduan Cui, Kun Xu, Chunhua Zheng, Jia Liu, Lei Peng, and Huiyun Li. “Flexible Unmanned Surface Vehicles Control using Probabilistic Model-based Reinforcement Learning with Hierarchical Gaussian Distribution.” Ocean Engineering, 2023. link (IF 5.0, JCR Q1)

  6. Renxing Li, Zhiwei Shang, Chunhua Zheng, Huiyun Li, Qing Liang, and Yunduan Cui. “Efficient Distributional Reinforcement Learning with Kullback-Leibler Divergence Regularization.” Applied Intelligence, 2023. link (IF 5.3, JCR Q2)

  7. Yunduan Cui, Wenbo Shi, Huan Yang, Cuiping Shao, Lei Peng, and Huiyun Li. “Probabilistic Model-Based Reinforcement Learning Unmanned Surface Vehicles Using Local Update Sparse Spectrum Approximation.” IEEE Transactions on Industrial Informatics, 2023. link (IF 12.3, JCR Q1)

  8. Jincheng Wang, Lei Xia, Lei Peng, Huiyun Li, and Yunduan Cui. “Efficient Uncertainty Propagation in Model-Based Reinforcement Learning Unmanned Surface Vehicle Using Unscented Kalman Filter.” Drones, 2023. link (IF 4.8, JCR Q2)

  9. Dezhou Xu, Chunhua Zheng, Yunduan Cui, Shengxiang Fu, Namwook Kim, and Suk Won Cha. “Recent progress in learning algorithms applied in energy management of hybrid vehicles: a comprehensive review.” International Journal of Precision Engineering and Manufacturing-Green Technology, 2023. link (IF 4.2, JCR Q1)

  10. Cuiping Shao, Beizhang Chen, Zujia Miao, Yunduan Cui, Huiyun Li. “Anomaly recognition method of perception system for autonomous vehicles based on distance metric.” Electronics Letters, 2022. link (IF 1.1, JCR Q4)

  11. Yunduan Cui, Lei Peng, and Huiyun Li. “Filtered Probabilistic Model Predictive Control-based Reinforcement Learning for Unmanned Surface Vehicles.” IEEE Transactions on Industrial Informatics, 2022. link (IF 12.3, JCR Q1)

  12. Dezhou Xu, Yunduan Cui, Jiaye Ye, Suk Won Cha, Aimin Li, and Chunhua Zheng. “A soft actor-critic-based energy management strategy for electric vehicles with hybrid energy storage systems.” Journal of Power Sources, 2022. link (IF 9.2, JCR Q1)

  13. Yujun Lai, Gavin Paul, Yunduan Cui, and Takamitsu Matsubara. “User intent estimation during robot learning using physical human robot interaction primitives.” Autonomous Robots, 2022. link (IF 3.5, JCR Q2)

  14. Wei Li, Jiaye Ye, Yunduan Cui, Namwook Kim, Suk Won Cha, and Chunhua Zheng. “A Speedy Reinforcement Learning-Based Energy Management Strategy for Fuel Cell Hybrid Vehicles Considering Fuel Cell System Lifetime.” International Journal of Precision Engineering and Manufacturing-Green Technology, 2021. link (IF 4.2, JCR Q1)

  15. Cheng-Yu Kuo, Andreas Schaarschmidt, Yunduan Cui, Tamim Asfour, and Takamitsu Matsubara. “Uncertainty-aware Contact-safe Model-based Reinforcement Learning.” IEEE Robotics and Automation Letters (with ICRA 2021), 2021. link (IF 5.2, JCR Q2)

  16. Yunduan Cui, Osaki Shigeki, and Takamitsu Matsubara. “Autonomous Boat Driving System using Sample-efficient Model Predictive Control-based Reinforcement Learning Approach.” Journal of Field Robotics, 2021. link (IF 8.3, JCR Q1)

  17. Yunduan Cui, Junichiro Ooga, Akihito Ogawa, and Takamitsu Matsubara. “Probabilistic Active Filtering with Gaussian Processes for Occluded Object Search in Clutter.” Applied Intelligence, 2020. link (IF 5.3, JCR Q2)

  18. Lingwei Zhu, Yunduan Cui, Go Takami, Hiroaki Kanokogi, and Takamitsu Matsubara. “Scalable Reinforcement Learning for Plant-wide Control of Vinyl Acetate Monomer Process.” Control Engineering Practice, 2020. link (IF 4.9, JCR Q2)

  19. Yoshihisa Tsurumine, Yunduan Cui, Eiji Uchibe, and Takamitsu Matsubara. “Deep reinforcement learning with smooth policy update: Application to robotic cloth manipulation.” Robotics and Autonomous Systems, 2018. link

  20. Yunduan Cui, James Poon, Jaime Valls Miro, Kimitoshi Yamazaki, Kenji Sugimoto, and Takamitsu Matsubara. “Environment-adaptive interaction primitives through visual context for human–robot motor skill learning.” Autonomous Robots, 2018. link (IF 4.3, JCR Q2)

  21. Yunduan Cui, Takamitsu Matsubara, and Kenji Sugimoto. “Kernel dynamic policy programming: Applicable reinforcement learning to robot systems with high dimensional states.” Neural Networks, 2017. (2018 Japanese Neural Network Society Best Paper Award) link (IF 7.8, JCR Q1)

  22. Yunduan Cui, Takamitsu Matsubara, and Kenji Sugimoto. “Pneumatic artificial muscle-driven robot control using local update reinforcement learning.” Advanced Robotics, 2017. link (IF 2.0, JCR Q4)

国际会议


  1. Lei Xia, Cuiping Shao, Huiyun Li, and Yunduan Cui. “Robust Model-based Reinforcement Learning USV System Guided by Lyapunov Neural Networks.” IEEE International Conference on Robotics and Biomimetics (ROBIO) 2022. link

  2. Cuiping Shao, Zujia Miao, Beizhang Chen, Yunduan Cui, Huiyun Li, and Hongfeng Shu, “An Attack Detection Method Based on Spatiotemporal Correlation for Autonomous Vehicles Sensors.” IEEE International Conference on Intelligent Transportation Systems (ITSC) 2022. link

  3. Yunfu Deng, Kun Xu, Yue Hu, Yunduan Cui, Gengzhao Xiang, and Zhongming Pan, “Learning Effectively from Intervention for Visual-based Autonomous Driving.” IEEE International Conference on Intelligent Transportation Systems (ITSC) 2022. link

  4. Deliang Liu, Kun Xu, Yunduan Cui, Yujie Zou, and Zhongming Pan, “Learning-based Motion Control of Autonomous Vehicles Considering Varying Adhesion Road Surfaces.” IEEE International Conference on Intelligent Transportation Systems (ITSC) 2022. link

  5. Jincheng Wang, Kun Xu, Cuiping Shao, Lei Peng, and Yunduan Cui, “Data-Driven Probabilistic Model of Magneto-Rheological Damper for Intelligent Vehicles using Gaussian Processes.” IEEE International Conference on Intelligent Transportation Systems (ITSC) 2022. link

  6. Renxing Li, Zhiwei Shang, Chunhua Zheng, Huiyun Li, Qing Liang, and Yunduan Cui, “Dynamic Policy Programming with Descending Regularization for Efficient Reinforcement Learning Control.” International Conference on Pattern Recognition and Artificial Intelligence (PRAI) 2022. link

  7. Zhiwei Shang, Huiyun Li, and Yunduan Cui. “Shiftable Dynamic Policy Programming for Efficient and Robust Reinforcement Learning Control.” IEEE International Conference on Robotics and Biomimetics (ROBIO) 2021. link

  8. Naitian Deng, Yunduan Cui, Shitian Zhang, and Huiyun Li. “Autonomous Vehicle Motion Planning using Kernelized Movement Primitives.” International Symposium on Networks, Computers and Communications (ISNCC) 2021. link

  9. Shitian Zhang, Yunduan Cui, Naitian Deng, and Huiyun Li. “Model Predictive Control of Autonomous Driving using Unscented Kalman Filter with Sparse Spectrum Gaussian Processes.” International Symposium on Networks, Computers and Communications (ISNCC) 2021. link

  10. Lingwei Zhu, Yunduan Cui, and Takamitsu Matsubara. “Dynamic Actor-Advisor Programming for Scalable Safe Reinforcement Learning.” IEEE International Conference on Robotics and Automation (ICRA) 2020. link

  11. Cheng-Yu Kuo, Yunduan Cui, and Takamitsu Matsubara. “Sample-and-computational-efficient Probabilistic Model Predictive Control with Random Features.” IEEE International Conference on Robotics and Automation (ICRA) 2020. link

  12. Yunduan Cui, Shigeki Osaki, Takamitsu Matsubara. “Reinforcement Learning Ship Autopilot: Sample-efficient and Model Predictive Control-based Approach.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2019. ArXiv preprint arXiv:1901.07905 SICE International Young Authors Award 2019 (IROS) link

  13. Yoshihisa Tsurumine, Yunduan Cui, Kimitoshi Yamazaki, Takamitsu Matsubara. “Generative Adversarial Imitation Learning with Deep P-Network for Robotic Cloth Manipulation” IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2019. link

  14. James Poon, Yunduan Cui, Junichiro Ohga, Akihito Ogawa, and Takamitsu Matsubara. “Probabilistic Active Filtering for Object Search in Clutter.” IEEE International Conference on Robotics and Automation (ICRA) 2019. link

  15. James Poon, Yunduan Cui, Jaime Valls Miro, Takamitsu Matsubara. “Learning Mobility Aid Assistance via Decoupled Observation Models." International Conference on Control, Automation, Robotics and Vision (ICARCV 2018). link

  16. Yunduan Cui, Lingwei Zhu, Morihiro Fujisaki, Hiroaki Kanokogi, and Takamitsu Matsubara. “Factorial Kernel Dynamic Policy Programming for Vinyl Acetate Monomer Plant Model Control." IEEE International Conference on Automation Science and Engineering (CASE) 2018. link

  17. Takamitsu Matsubara, Yu Norinaga, Yuto Ozawa, and Yunduan Cui. “Policy Transfer from Simulations to Real World by Transfer Component Analysis." IEEE International Conference on Automation Science and Engineering (CASE) 2018. link

  18. Yoshihisa Tsurumine, Yunduan Cui, Eiji Uchibe, and Takamitsu Matsubara. “Deep Dynamic Policy Programming for Robot Control with Raw Images." IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017. link

  19. James Poon, Yunduan Cui, Jaime Valls Miro, Takamitsu Matsubara, and Kenji Sugimoto. “Local Driving Assistance from Demonstration for Mobility Aids." IEEE International Conference on Robotics and Automation (ICRA) 2017. link

  20. Yunduan Cui, James Poon, Takamitsu Matsubara, Jaime Valls Miro, Kenji Sugimoto, and Kimitoshi Yamazaki. “Environment-adaptive Interaction Primitives for Human-Robot Motor Skill Learning." IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2016. link

  21. Yunduan Cui, Takamitsu Matsubara, and Kenji Sugimoto. “Kernel Dynamic Policy Programming: Practical Reinforcement Learning for High-dimensional Robots." IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2016. ( Best Oral Paper Award) link

  22. Yunduan Cui, Takamitsu Matsubara, and Kenji Sugimoto. “Local Update Dynamic Policy Programming in reinforcement learning of pneumatic artificial muscle-driven humanoid hand control.” IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2015. link

  23. Yunduan Cui, Kazuhiko Takahashi, and Masafumi Hashimoto. “Remarks on quaternion neural network based controller with application to an inverted pendulum.” 2014 SICE Annual Conference. link

  24. Kazuhiko Takahashi, Sae Takahashi, Yunduan Cui and Masafumi Hashimoto. “Remarks on computational facial expression recognition from HOG features using quaternion multi-layer neural network.” 2014 International Conference on Engineering Applications of Neural Networks. link

  25. Yunduan Cui, Kazuhiko Takahashi, and Masafumi Hashimoto. “Remarks on robot controller application of Clifford multi-layer neural networks.” 2014 IEEE 13th International Workshop on Advanced Motion Control (AMC). link

  26. Yunduan Cui, Kazuhiko Takahashi, and Masafumi Hashimoto. “Design of control systems using quaternion neural network and its application to inverse kinematics of robot manipulator.” 2013 IEEE/SICE International Symposium on System Integration (SII). link

联系方式

邮箱

yd.cui[at]siat.ac.cn

cuiyunduan[at]hotmail.com