基于强化学习的车队速度规划与能量管理联合优化*

doi:10.16731/j.cnki.1671-3133.2024.04.011

现代制造工程 ›› 2024, Vol. 523 ›› Issue (4): 80-86.doi: 10.16731/j.cnki.1671-3133.2024.04.011

基于强化学习的车队速度规划与能量管理联合优化^*

卢兵¹, 刘腾^2,3, 霍为炜^2,3

1 北京理工大学机械与车辆学院,北京 100081;
2 北京信息科技大学机电工程学院,北京 100192;
3 新能源汽车北京实验室,北京 100192

收稿日期:2023-06-16 出版日期:2024-04-18 发布日期:2024-05-31
通讯作者: 刘腾,硕士,主要从事基于深度强化学习的燃料电池能量管理的研究工作。E-mail: liut_bistu@163.com
作者简介:卢兵,博士,主要从事燃料电池能量管理的研究工作。E-mail:lubingev@sina.com;。
基金资助:
^*国家自然科学基金面上项目(52077007)

Co-optimization of vehicle queue speed planning and energy management with reinforcement learning

LU Bing¹, LIU Teng^2,3, HUO Weiwei^2,3

1 School of Mechanical Engineering,Beijing Institute of Technology,Beijing 100081,China;
2 Mechanical Electrical Engineering School,Beijing Information Science & Technology University, Beijing 100192,China;
3 Collaborative Innovation Center for Electric Vehicles,Beijing 100192,China

Received:2023-06-16 Online:2024-04-18 Published:2024-05-31

摘要/Abstract

摘要： 近年来,随着智能交通系统的快速发展,包括车车通信、车路通信等短距离实时无线通信信息,以及道路交通通行信息等远距离交通信息,使得车辆能够实时获知周围车辆运动情况以及前方交通环境情况,有利于提高车辆对周围交通环境的感知能力,以实现合理的出行安排与行驶控制,从而提高车辆的使用性能。为实现车队在多信号灯场景下的节能驾驶,提出了一种基于强化学习的车队速度规划与能量管理联合优化方法。通过SUMO平台,建立了包括5辆汽车的车队通过多个信号灯的场景,结果表明,提出的方法在舒适性、经济性和效率方面均优于传统的驾驶模型(Intelligent Driver Model,IDM)策略。

关键词: 多智能体强化学习, 能量管理, 燃料电池汽车, 联合优化

Abstract: In recent years,with the rapid development of intelligent transportation system,including vehicle-vehicle communication,vehicle-road communication and other short-range real-time wireless communication information,as well as road traffic information and other long-distance traffic information,so that the vehicle is able to obtain real-time knowledge of the surrounding vehicle movement as well as the traffic environment in front of it,which is conducive to improving the vehicle′s perception of the surrounding traffic environment in order to achieve reasonable travel arrangements and driving control,thus improving the vehicle performance. In order to realize the energy-saving driving of fleet in multi-signal light scenarios,a co-optimization method based on reinforcement learning for vehicle queuq speed planning and energy management was proposed.Through the SUMO platform,a scenario including five vehicles for a fleet of vehicles passing through multiple signals was established.The results show that the proposed method outperforms the traditional driving model Intelligent Driver Model (IDM) in terms of comfort,economy and efficiency.

Key words: multi-agent reinforcement learning, energy management, fuel cell vehicles, co-optimization

中图分类号:

TM911.42

卢兵, 刘腾, 霍为炜. 基于强化学习的车队速度规划与能量管理联合优化^*[J]. 现代制造工程, 2024, 523(4): 80-86.

LU Bing, LIU Teng, HUO Weiwei. Co-optimization of vehicle queue speed planning and energy management with reinforcement learning[J]. Modern Manufacturing Engineering, 2024, 523(4): 80-86.

参考文献

[1] HU Z,LI J,XU L,et al.Multi-objective energy management optimization and parameter sizing for proton exchange membrane hybrid fuel cell vehicles[J].Energy Conversion and Management,2016,129:108-121.
[2] 于瀛霄,孙闫,夏长高.燃料电池汽车双层模糊控制能量管理策略[J].重庆理工大学学报:自然科学,2022,36(8):21-28.
[3] KIM Y,FIGUEROA-SANTOS M,PRAKASH N,et al.Co-optimization of speed trajectory and power management for a fuel-cell/battery electric vehicle[J].Applied Energy,2020,260:114254.
[4] 魏小栋,刘波,冷江昊.基于凸优化的燃料电池汽车节能驾驶研究[J].汽车工程,2022,44(6):8.
[5] WEI X,LENG J,SUN C,et al.Co-optimization method of speed planning and energy management for fuel cell vehicles through signalized intersections[J].Journal of Power Sources,2022,518:230598.
[6] GUO Jinquan,HE Hongwen,LI Jianwei,et al.Real-time energy management of fuel cell hybrid electric buses:Fuel cell engines friendly intersection speed planning[J].Energy,2021,226:120440.
[7] 廖光亮,郑玲,张志达.考虑信号灯状态的电动汽车最优车速规划与避撞控制研究[J].中国科学:技术科学,2022,52(7):1134-1144.
[8] 胡林,周登辉,黄晶.考虑信号灯和能耗的电动车最优路径规划[J].汽车工程,2021,43(5):641-649,666.
[9] LI G,GORGES D.Ecological Adaptive Cruise Control and Energy Management Strategy for Hybrid Electric Vehicles Based on Heuristic Dynamic Programming[J].IEEE Transactions on Intelligent Transportation Systems,2019,20(9):3526-3535.
[10] 孙志远,吴景铼,张农.燃料电池混合动力客车自适应能量管理策略[J].重庆理工大学学报:自然科学,2022,36(5):53-61.
[11] GAO H,WANG Z,YIN S,et al.Adaptive real-time optimal energy management strategy based on equivalent factors optimization for hybrid fuel cell system[J].International Journal of Hydrogen Energy,2021,46(5):4329-4338.
[12] HAN J,PARK Y,KUM D.Optimal adaptation of equivalent factor of equivalent consumption minimization strategy for fuel cell hybrid electric vehicles under active state inequality constraints[J].Journal of Power Sources,2014,267:491-502.
[13] LI Q,CHEN W,LI Y,et al.Energy management strategy for fuel cell/battery/ultracapacitor hybrid vehicle based on fuzzy logic[J].International Journal of Electrical Power & Energy Systems,2012,43(1):514-525.
[14] PENG H,CHEN Z,LI J,et al.Offline optimal energy management strategies considering high dynamics in batteries and constraints on fuel cell system power rate:From analytical derivation to validation on test bench[J].Applied Energy,2021,282:116152.
[15] 石泽华,付主木,陶发展.网联环境下的燃料电池混合动力汽车能量管理[J].河南科技大学学报:自然科学版,2023,44(1):28-35.
[16] ZHOU X,QIN D,HU J.Multi-objective optimization design and performance evaluation for plug-in hybrid electric vehicle powertrains[J].Applied Energy,2017,208:1608-1625.
[17] WANG Y,WU Z,CHEN Y,et al.Research on energy optimization control strategy of the hybrid electric vehicle based on Pontryagin's minimum principle[J].Computers & Electrical Engineering,2018,72:203-213.
[18] CHEN Z,MI C C,XIA B,et al.Energy management of power-split plug-in hybrid electric vehicles based on simulated annealing and Pontryagin's minimum principle[J].Journal of Power Sources,2014,272:160-168.
[19] KIM N,HA S,JEONG J,et al.Sufficient conditions for optimal energy management strategies of fuel cell hybrid electric vehicles based on Pontryagin's minimum principle[J].Proceedings of the Institution of Mechanical Engineers,Part D:Journal of Automobile Engineering,2015,230(2):202-214.
[20] GUO H,WANG X,LI L.State-of-charge-constraint-based energy management strategy of plug-in hybrid electric vehicle with bus route[J].Energy Conversion and Management,2019,199:111972.
[21] LÜ X,WU Y,LIAN J,et al.Energy management of hybrid electric vehicles:A review of energy optimization of fuel cell hybrid power system based on genetic algorithm[J].Energy Conversion and Management,2020,205:112474.
[22] YAO M,ZHU B,ZHANG N.Adaptive real-time optimal control for energy management strategy of extended range electric vehicle[J].Energy Conversion and Management,2021,234:113874.
[23] ZENG T,ZHANG C,ZHANG Y,et al.Optimization-oriented adaptive equivalent consumption minimization strategy based on short-term demand power prediction for fuel cell hybrid vehicle[J].Energy,2021,227:120305.
[24] LIU Y,LI J,LEI Z,et al.Experimental Research on Adaptive Equivalent Consumption Minimization Strategy Based on Traffic Information[C]//2018 Joint International Conference on Energy, Ecology and Environment (ICEEE 2018) and International Conference on Electric and Intelligent Vehicles (ICEIV 2018). [S.l.]:[s.n.],2018.
[25] ZHANG F,XI J,LANGARI R.Real-Time Energy Management Strategy Based on Velocity Forecasts Using V2V and V2I Communications[J].IEEE Transactions on Intelligent Transportation Systems,2016,99:1-15.
[26] PEI P,CHANG Q,TANG T.A quick evaluating method for automotive fuel cell lifetime[J].International Journal of Hydrogen Energy,2008,33(14):3829-3836.
[27] LIU Y,LI J,CHEN Z,et al.Research on a multi-objective hierarchical prediction energy management strategy for range extended fuel cell vehicles[J].Journal of Power Sources,2019,429:55-66.
[28] YEN Y T,CHOU J J,SHIH C S,et al.Proactive car-following using deep-reinforcement learning[C]//Proceedings of the 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC),F,2020.[S.l.]:IEEE,2020.

基于强化学习的车队速度规划与能量管理联合优化^*

Co-optimization of vehicle queue speed planning and energy management with reinforcement learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 9

编辑推荐

Metrics

[1]	刘智飞;马建民;刘振东. 考虑不确定因素的可重构车间布局与调度联合优化[J]. 现代制造工程, 2023, 515(8): 15-21.
[2]	汪存喜;梅标;杨永泰. 基于尺寸和自由形状联合优化技术的叉车货架轻量化设计[J]. 现代制造工程, 2023, 517(10): 86-93.
[3]	田中辉. 30 kW燃料电池汽车发动机的设计[J]. 现代制造工程, 2022, 501(6): 25-31.
[4]	陈静，李广强，. 基于驾驶模式划分的氢燃料电池汽车能量管理策略[J]. 现代制造工程, 2022, 500(5): 36-41.
[5]	陈洪根;李诗宇;闫鑫;李婧;张艳. 基于可用度的预防维修和均值控制图联合优化模型[J]. 现代制造工程, 2022, 497(2): 1-9.
[6]	路鑫阳，武志斐. 减少燃料电池功率振荡的FCV混合能量管理策略[J]. 现代制造工程, 2021, 486(3): 35-39.
[7]	牛继高，张兵，徐春华. 增程式电动汽车能量管理策略控制参数优化研究[J]. 现代制造工程, 2020, 474(3): 66-71.
[8]	应天杏, 程伟, 孔凡敏, 周维, 杨林. 基于多指标的燃料电池汽车动力系统参数匹配研究[J]. 现代制造工程, 2019, 460(1): 1-7.
[9]	李博超, 谢金法, 张正奇, 校玲玲. 增程式电动车神经模糊推理系统能量管理策略研究[J]. 现代制造工程, 2018, 455(8): 63-68.