基于深度强化学习的风电拉挤板生产智能排程*

doi:10.16731/j.cnki.1671-3133.2025.01.003

现代制造工程 ›› 2025, Vol. 532 ›› Issue (1): 23-32.doi: 10.16731/j.cnki.1671-3133.2025.01.003

基于深度强化学习的风电拉挤板生产智能排程^*

杨逢海¹, 杨晓英^1,2, 裴志杰³, 武亚琪¹, 张志伟¹

1 河南科技大学机电工程学院,洛阳 471003;
2 机械装备先进制造河南省协同创新中心,洛阳 471003;
3 河南科技大学商学院,洛阳 471023

收稿日期:2024-05-20 出版日期:2025-01-18 发布日期:2025-02-10
通讯作者: 杨晓英,博士,教授,博士生导师,主要研究方向为工业工程及智能制造等。E-mail:220320010147@stu.haust.edu.cn;lyyxy@haust.edu.cn
作者简介:杨逢海,硕士研究生,主要研究方向为工业工程及智能制造。
基金资助:
* 河南省重点研发专项项目(231111222600)

Deep reinforcement learning for production intelligent scheduling of wind turbine extrusion panels

YANG Fenghai¹, YANG Xiaoying^1,2, PEI Zhijie³, WU Yaqi¹, ZHANG Zhiwei¹

1 School of Mechanical Engineering,Henan University of Science and Technology,Luoyang 471003,China;
2 Henan Collaborative Innovation Center of Advanced Manufacturing of Machinery and Equipment,Luoyang 471003,China;
3 School of Business,Henan University of Science and Technology,Luoyang 471023,China

Received:2024-05-20 Online:2025-01-18 Published:2025-02-10

摘要/Abstract

摘要： 针对具有包装顺序齐套和产品换型调整等复杂特征的风电拉挤板生产排程问题,构建了最大化当期开动设备平均利用率和最大化订单履约率的多目标协同优化模型;将风电拉挤板生产排程问题转化为马尔科夫序列决策问题,设计了10种不同排程策略作为动作空间,提炼适当的状态特征和奖励函数;提出一种基于决斗双深度Q网络(D3QN)的排程算法。通过某企业实际数据的仿真试验,与Double DQN和Dueling DQN算法对比验证所提算法有效性;并比较4种不同求解方法在10个算例下得到的目标值,验证了所提出的改进D3QN算法可以得到问题的高质量解,为风电拉挤板制造企业生产排程提供了一种智能化的方法和参考。

关键词: 风电, 拉挤板, 生产排程, 深度强化学习, D3QN算法

Abstract: In order to solve the wind turbine extrusion panel production scheduling problem with complex features such as packaging order flush and product changeover adjustment,a multi-objective collaborative optimization model was constructed to maximize the average utilization rate of the current starting equipment and maximize the order fulfillment rate;the wind turbine extrusion panel production scheduling problem was transformed into a Markov sequence decision-making problem,and 10 different scheduling strategies were designed as the action space,and the appropriate state features and reward functions were refined; a scheduling algorithm based on Dueling Double Deep Q Network (D3QN) was proposed. The effectiveness of the proposed algorithm was verified by comparing with Double DQN and Dueling DQN algorithms through simulation tests on actual data of an enterprise;and the objective values obtained by four different solution methods under 10 algorithms were compared,which verified that the proposed improved D3QN algorithm can get a high-quality solution to the problem,providing an intelligent method and reference for the production scheduling of wind turbine extrusion panel manufacturing enterprises.

Key words: wind power, extrusion panels, production scheduling, deep reinforcement learning, D3QN algorithm

中图分类号:

TB497

杨逢海, 杨晓英, 裴志杰, 武亚琪, 张志伟. 基于深度强化学习的风电拉挤板生产智能排程^*[J]. 现代制造工程, 2025, 532(1): 23-32.

YANG Fenghai, YANG Xiaoying, PEI Zhijie, WU Yaqi, ZHANG Zhiwei. Deep reinforcement learning for production intelligent scheduling of wind turbine extrusion panels[J]. Modern Manufacturing Engineering, 2025, 532(1): 23-32.

参考文献

[1] 李成良,杨超,倪爱清,等.复合材料在大型风电叶片上的应用与发展[J].复合材料学报,2023,40(3):1274-1284.
[2] 卫立江,邱锐磊,王乾武,等.大风电叶片拉挤复合材料研究进展[J/OL].复合材料科学与工程:1-6.(2023-10-26)[2024-04-24].http://kns.cnki.net/kcms/detail/10.1683.TU.20231026.1431.004.html.
[3] NEUFELD J S,GUPTA J N D,BUSCHER U.A comprehensive review of flowshop group scheduling literature[J].Computers & Operations Research,2016,70:56-74.
[4] LIN C W,LIN Y K,HSIEH H T.Ant colony optimization for unrelated parallel machine scheduling[J].The International Journal of Advanced Manufacturing Technology,2013,67(1/2/3/4):35-45.
[5] YING K C,LEE Z J,LIN S W.Makespan minimization for scheduling unrelated parallel machines with setup times[J].Journal of Intelligent Manufacturing,2012,23(5):1795-1803.
[6] LIU S,LIU X,PEI J,et al.Parallel-batching machines scheduling problem with a truncated time-dependent learning effect via a hybrid CS-JADE algorithm[J].Optimization Methods and Software,2019(3):1-26.
[7] 郑堃,练志伟,顾新艳,等.采用改进两点交叉算子的改进自适应遗传算法求解不相关并行机混合流水车间调度问题[J].中国机械工程,2023,34(14):1647-1658,1671.
[8] 荀洪凯,陶翼飞,张源,等.多目标启发式狼群算法求解不相关并行机分批调度问题[J].信息与控制,2023,52(1):93-103,114.
[9] 宋君乐,陶翼飞,张源,等.基于两阶段多目标BBO算法的并行机分批调度问题研究[J/OL].控制工程:1-13.(2024-02-26)[2024-04-24].https://doi.org/10.14107/j.cnki.kzgc.20230122.
[10] 高佳,谭园园,李冬,等.离散粒子群算法求解带役龄维护的不相关并行机调度问题 [J/OL].计算机集成制造系统:1-18.(2023-12-28)[2024-04-24].https://doi.org/10.13196/j.cims.2023.0450.
[11] 朱先萌,姜兆亮,魏清月,等.多品种小批量产品冲压计划排程多目标优化[J].计算机集成制造系统,2017,23(9):1907-1916.
[12] 鲁佳俊,高亮,侯国祥.基于果蝇优化算法的船舶管加工车间多目标生产调度研究[J].现代制造工程,2021(8):8-19.
[13] AMALLYNDA I,SANTOSA B.Solving multi-objective Modified Distributed Parallel Machine and Assembly Scheduling Problem(MDPMASP)with eligibility constraints using metaheuristics[J].Production & Manufacturing Research,2022,10(1):198-225.
[14] KONG M,WANG W,DEVECI M,et al.A novel carbon reduction engineering method-based deep Q-learning algorithm for energy-efficient scheduling on a single batch-processing machine in semiconductor manufacturing[J/OL].International Journal of Production Research,2023:1-24[2024-04-24] DOI:10.1080/00207543.2023.2252932.
[15] LI J L,SHI W S,ZHANG N,et al.Delay-Aware VNF Scheduling:A Reinforcement Learning Approach With Variable Action Set[J].IEEE Transactions on Cognitive Communications and Networking,2021,7(1):304-318.
[16] 秦保新,张羽霄,吴思锐,等.基于改进D3QN的煤炭码头卸车排产智能优化方法[J].系统仿真学报,2024,36(3):770-781.
[17] 吴昊泽,李艳武,谢辉.改进PPO算法求解柔性作业车间调度问题[J/OL].计算机集成制造系统:1-20.(2023-06-02)[2024-04-24].http://kns.cnki.net/kcms/detail/11.5946.TP.20230601.0942.006.html.
[18] VAN H,GUEZ A,SILVER D.Deep reinforcement learning with double Q-learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Arizona:AAAI Press,2016.DOI:10.48550/arXiv.1509.06461.
[19] LUO Q,DENG Q,GONG G,et al.An efficient memetic algorithm for distributed flexible job shop scheduling problem with transfers[J].Expert Systems with Applications,2020,160:113721.

基于深度强化学习的风电拉挤板生产智能排程^*

Deep reinforcement learning for production intelligent scheduling of wind turbine extrusion panels

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

[1]	谢子健, 秦建军, 曹钰. 基于改进TD3的四足机器人非结构化地形运动控制^*[J]. 现代制造工程, 2025, 532(1): 33-41.
[2]	叶楠, 张传棻, 王荣, 贺明荣, 张雁辉, 段书用. 极端服役多工况风电齿轮箱箱体轻量化设计^*[J]. 现代制造工程, 2024, 528(9): 152-159.
[3]	罗胜曦，肖长远，解轲，余民姚，韩佳，罗雁飞. 风电锁紧盘传扭特性及影响因素分析[J]. 现代制造工程, 2023, 512(5): 79-84.
[4]	李响;刘书海;肖华平;刘浩. 风电叶片检测机器人设计与运动仿真研究[J]. 现代制造工程, 2023, 511(4): 51-58.
[5]	陶鑫钰;王艳;纪志成. 基于A3C的特征重构工艺路线规划方法[J]. 现代制造工程, 2023, 517(10): 15-26.
[6]	钟敬伟，石宇强. 基于DQN的智能工厂作业车间调度[J]. 现代制造工程, 2021, 492(9): 17-23.