深度强化学习求解动态柔性作业车间调度问题*

doi:10.16731/j.cnki.1671-3133.2025.02.002

现代制造工程 ›› 2025, Vol. 533 ›› Issue (2): 10-16.doi: 10.16731/j.cnki.1671-3133.2025.02.002

• 先进制造系统管理运作 • 上一篇下一篇

深度强化学习求解动态柔性作业车间调度问题^*

杨丹^1,2, 舒先涛^1,3, 余震³, 鲁光涛^1,2, 纪松霖^1,3, 王家兵¹

1 武汉科技大学冶金装备及其控制省部共建教育部重点实验室,武汉 430081;
2 武汉科技大学机械传动与制造工程湖北省重点实验室,武汉 430081;
3 武汉科技大学精密制造研究院,武汉 430081

收稿日期:2024-03-25 发布日期:2025-02-27
作者简介:杨丹,博士,副教授,主要研究方向为故障诊断与在线监测、人工智能及其应用等。E-mail:yangdan@wust.edu.cn
基金资助:
^*国家自然科学基金项目(51808417)

A dynamic flexible job shop scheduling method based on deepreinforcement learning

YANG Dan^1,2, SHU Xiantao^1,3, YU Zhen³, LU Guangtao^1,2, JI Songlin^1,3, WANG Jiabing¹

1 Key Laboratory for Metallurgical Equipment and Control of Ministry of Education,Wuhan University of Science and Technology,Wuhan 430081,China;
2 Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering,Wuhan University of Science and Technology,Wuhan 430081,China;
3 Precision Manufacturing Institute,Wuhan University of Science and Technology,Wuhan 430081,China

Received:2024-03-25 Published:2025-02-27

摘要/Abstract

摘要： 随着智慧车间等智能制造技术的不断发展,人工智能算法在解决车间调度问题上的研究备受关注,其中车间运行过程中的动态事件是影响调度效果的一个重要扰动因素,为此提出一种采用深度强化学习方法来解决含有工件随机抵达的动态柔性作业车间调度问题。首先以最小化总延迟为目标建立动态柔性作业车间的数学模型,然后提取8个车间状态特征,建立6个复合型调度规则,采用ε-greedy动作选择策略并对奖励函数进行设计,最后利用先进的D3QN算法进行求解并在不同规模车间算例上进行了有效性验证。结果表明,提出的D3QN算法能非常有效地解决含有工件随机抵达的动态柔性作业车间调度问题,在所有车间算例中的求优胜率为58.3 %,相较于传统的DQN和DDQN算法车间延迟分别降低了11.0 %和15.4 %,进一步提升车间的生产制造效率。

关键词: 深度强化学习, D3QN算法, 工件随机抵达, 柔性作业车间调度, 动态调度

Abstract: The study of the artificial intelligence algorithms for job shop scheduling has gained attention due to the advancements in intelligent manufacturing technologies like smart factories. Dynamic events in the job shop are crucial factors affecting scheduling effectiveness. To this end,it proposes a novel approach employing the deep reinforcement learning to solve the dynamic flexible job shop scheduling problem with random job arrival. Initially,a mathematical model is formulated for the dynamic job shop scheduling problem with the objective of minimizing the total tardiness. Subsequently,eight job shop state features are extracted,and six composite scheduling rules are designed. An ε-greedy action selection strategy is adopted,and the reward function is designed. Finally,the advanced D3QN algorithm is introduced to solve the problem and the effectiveness of this method is verified on different scale of instances. The results show that the D3QN algorithm effectively solves the dynamic flexible job shop scheduling problem with random job arrival,and the winning rate in all instances is 58.3 %.Compared with traditional DQN and DDQN algorithm,the total tardiness is reduced by 11.0 % and 15.4 % respectively,which proves that this method further enhances the production efficiency of the job shop.

Key words: deep reinforcement learning, D3QN algorithm, random job arrival, flexible job shop scheduling problem, dynamicscheduling

中图分类号:

TP181

杨丹, 舒先涛, 余震, 鲁光涛, 纪松霖, 王家兵. 深度强化学习求解动态柔性作业车间调度问题^*[J]. 现代制造工程, 2025, 533(2): 10-16.

YANG Dan, SHU Xiantao, YU Zhen, LU Guangtao, JI Songlin, WANG Jiabing. A dynamic flexible job shop scheduling method based on deepreinforcement learning[J]. Modern Manufacturing Engineering, 2025, 533(2): 10-16.

参考文献

[1] LI Xixing,GUO Xing,HONG Tao,et al.Survey of integrated flexible job shop scheduling problems[J].Computers & Industrial Engineering,2022,174:108786.
[2] 何斌,张接信,张富强.柔性作业车间调度问题简明分析[J].装备制造技术,2018(5):68-71,85.
[3] GAREY M R,JOHNSON D S,SETHI R.The complexity of flow shop and job shop scheduling[J].Mathematics of Operations Research,1976,1(2):117-129.
[4] 范华丽,熊禾根,蒋国璋,等.动态车间作业调度问题中调度规则算法研究综述[J].计算机应用研究,2016,33(3):648-653.
[5] DEMIR Y,YILMAZ H.An efficient priority rule for flexible job shop scheduling problem[J].Journal of Engineering Research and Applied Science,2021,10(2):1906-1918.
[6] 周鹏鹏,翟志波,戴玉森.基于改进遗传算法的柔性作业车间调度问题研究[J].组合机床与自动化加工技术,2023(3):183-186,192.
[7] 张守京,王彦亭.基于改进NSGA2的柔性车间多目标智能调度问题研究[J].现代制造工程,2020(9):23-31.
[8] 蔡敏,王艳,纪志成.混合粒子群优化算法求解模糊柔性作业车间调度问题[J].南京理工大学学报(自然科学版),2021,45(3):352-360.
[9] 吴秀丽,刘夏晶.差分进化算法求解分布式柔性作业车间调度问题[J].计算机集成制造系统,2019,25(10):2539-2558.
[10] XIA W,WU Z.An effective hybrid optimization approach for multi-objective flexible job-shop scheduling problems[J].Computers & Industrial Engineering,2005,48(2):409-425.
[11] 崔雪艳,万烂军,赵昊鑫,等.基于深度强化学习的柔性作业车间调度方法[J].制造技术与机床,2023(12):165-170.
[12] 张琦,张彬.动态环境下的柔性作业车间调度问题研究[J].机电产品开发与创新,2023,36(6):92-94.
[13] 胡一凡,张利平,白雪,等.深度强化学习求解柔性装配作业车间调度问题[J].华中科技大学学报(自然科学版),2023,51(2):153-160.
[14] ZHAO Yejian,WANG Yanhong,TAN Yuanyuan.Dynamic jobshop scheduling algorithm based on deep Q network[J].IEEE Access,2021(9):122995-123011.
[15] LUO Shu,ZHANG Linxuan,FAN Yushun.Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning[J].Computers & Industrial Engineering,2021,159:107489.
[16] CHANG Jingru,YU Dong,HU Yi.Deep reinforcement learning for dynamic flexible job shop scheduling with random job arrival[J].Processes,2022,10(4):760.
[17] 程俊夫.基于双重深度Q学习与邻域搜索算法的动态多目标柔性作业车间调度问题研究[D].成都:四川大学,2022.
[18] 罗梓珲,江呈羚,刘亮,等.基于深度强化学习的智能车间调度方法研究[J].物联网学报,2022,6(1):53-64.
[19] MOHAN J,LANKA K,RAO A N.A review of dynamic job shop scheduling techniques[J].Procedia Manuf.,2019,30:34-39.
[20] 李宝帅,叶春明.深度强化学习算法求解作业车间调度问题[J].计算机工程与应用,2021,57(23):248-254.

深度强化学习求解动态柔性作业车间调度问题^*

A dynamic flexible job shop scheduling method based on deepreinforcement learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	杨逢海, 杨晓英, 裴志杰, 武亚琪, 张志伟. 基于深度强化学习的风电拉挤板生产智能排程^*[J]. 现代制造工程, 2025, 532(1): 23-32.
[2]	谢子健, 秦建军, 曹钰. 基于改进TD3的四足机器人非结构化地形运动控制^*[J]. 现代制造工程, 2025, 532(1): 33-41.
[3]	张梦杰, 杨晓英, 李博. IRF-RL的混合流水车间动态调度方法研究^*[J]. 现代制造工程, 2024, 530(11): 26-36.
[4]	闫富乾, 石致远, 王立闻. 基于改进灰狼算法的柔性作业车间动态节能分批调度问题^*[J]. 现代制造工程, 2024, 520(1): 24-32.
[5]	李峥峰;丁其聪;张东方;张国辉. 改进离散麻雀搜索算法求解柔性作业车间调度问题[J]. 现代制造工程, 2023, 516(9): 18-27.
[6]	陆心屹;韩晓龙. 基于强化学习的改进NSGA-Ⅱ求解柔性作业车间节能调度问题[J]. 现代制造工程, 2023, 515(8): 22-35.
[7]	吴迎晨，肖彪，赵正彩，彭仕鑫，苏宏华，朱夏林. 柔性作业车间调度多策略果蝇优化算法研究[J]. 现代制造工程, 2023, 512(5): 22-30.
[8]	秦红斌;常永顺;唐红涛;张峰;王玲军. 混合麻雀算法求解带准备时间的分布式柔性作业车间调度问题[J]. 现代制造工程, 2023, 518(11): 1-11.
[9]	唐艺军;李雪. 基于改进混合遗传算法的柔性车间调度问题研究[J]. 现代制造工程, 2023, 517(10): 8-14.
[10]	陶鑫钰;王艳;纪志成. 基于A3C的特征重构工艺路线规划方法[J]. 现代制造工程, 2023, 517(10): 15-26.
[11]	王玉芳，曾亚志，蒋亚飞. 基于自适应灰狼优化算法的柔性作业车间调度问题[J]. 现代制造工程, 2022, 502(7): 1-10.
[12]	钟敬伟，石宇强. 基于DQN的智能工厂作业车间调度[J]. 现代制造工程, 2021, 492(9): 17-23.
[13]	唐浩，黎向锋，张丽萍，王建明，叶磊，王子旋. 扰动机制下的遗传算法求解柔性作业车间调度[J]. 现代制造工程, 2021, 490(7): 1-9.
[14]	谭宬，张纪会，郭藤. 考虑批量分割的多目标柔性作业车间调度[J]. 现代制造工程, 2020, 483(12): 25-35.
[15]	张捷,李彬,姜杰,顾海. 基于郊狼优化算法的柔性作业车间调度问题[J]. 现代制造工程, 2020, 481(10): 39-44.