Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/129213
|
| Title: | 基於深度強化學習的智能存貨控制：以高科技供應鏈為例
A Deep Reinforcement Learning approach for Intelligent Inventory Control in high-tech supply chains |
| Authors: | 廖信堯
Liao, Hsin-Yao |
| Contributors: | 莊皓鈞
Chuang, Hao-Chun
廖信堯
Liao, Hsin-Yao |
| Keywords: | 強化學習
深度學習
庫存最佳化
模擬
作業管理
Reinforcement learning
Deep learning
Inventory optimization
Simulation
Operations management |
| Date: | 2020 |
| Issue Date: | 2020-04-06 14:44:04 (UTC+8) |
| Abstract: | Machine learning is revolutionizing business operations across industry sectors. Among different learning techniques, deep reinforcement learning (DRL) has received broad attention in recent years due to the salient performance of AlphaGo, an artificial intelligence (AI) system empowered by DRL. DRL is a model-free and data-driven approach to develop near-optimal policies for sequential decision-making problems. Intrigued by the success of DRL in various fields, we, in this study, assess the applicability of DRL to multi-period inventory control under stochastic demand, which is a classical Markov Decision Process problem. Working with the largest distributor of electronics manufacturing services (EMS) in the world, we propose deep Q-networks (DQN) for intelligent inventory control (IIC). Facing erratic and non-stationary demand for electronic components with limited market life cycle, the distributor could not infer the exact demand distribution and solve the inventory optimization problem analytically in a finite-horizon with lost sales setting. Hence, we develop DQN by specifying relevant state and decision inputs, and then designing a data-driven simulation environment, in which the agent is trained over thousands of episodes. For trained items, DQN outperforms the benchmark in a few ways. First, DQN can reduce the total inventory by at least 40% while achieving better service level. Second, when penalty parameter increases, DQN can effectively reduce the amount of out-of-stock. While we transfer trained DQN into testing sets, within the same item, the out-of-sample performance is excellent. For other unseen items, we use the Maximum Entropy Bootstrap to train ensemble DDQN and make our DRL agent more robust. Given the promising results in our experiments, we discuss implications, limitations, and further directions for applying DRL/DQN to business decision-making problems. |
| Reference: | Arulkumaran, K., Deisenroth, P. M., Brundage, M., & Bharath, A. A. (2017) A brief survey of deep reinforcement learning. ArXiv: 1708.05866v2.
Chaharsooghi, S., Heydari, J., & Zegordi, S. (2008) A reinforcement learning model for supply chain ordering management: An application to the beer game. Decision Support Systems, 45(4): 949-959.
Chollet, F. (2017) Deep Learning with Python. Manning Publications.
Giannoccaro, I., & Pontrandolfo, P. (2002) Inventory management in supply chains: A reinforcement learning approach. International Journal of Production Economics, 78(2): 153-161.
Gijsbrechts, J., Boute, R., Zhang, D., & Van Mieghem, J. (2019) Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems. Available at SSRN https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3302881.
Gosavi, A. (2009) Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing, 21(3): 177-345.
Kingma, D. P., & Ba, J. (2017). Adam: A Method for Stochastic Optimization. ArXiv:1412.6980v9.
Lin, L. J. (1992) Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3-4): 293-321.
Mnih V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) Playing Atari with deep reinforcement learning. ArXiv:1312.5602v1.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., & Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015) Human-level control through deep reinforcement learning. Nature, 518: 529-533.
Oroojlooyjadid, A., Snyder, L., & Takáč, M. (2019) Applying deep learning to the newsvendor problem. IISE Transactions, in press.
Oroojlooyjadid, A., Nazari, M., Snyder, L., & Takáč, M. (2019b) A deep Q-Network for the beer game: A deep reinforcement learning algorithm to solve inventory optimization problems. ArXiv:1708.05924v3.
Porteus, E. L. (2002) Foundations of Stochastic Inventory Theory. Stanford University Press, California.
Qi, X., Wu, G., Boriboonsomsin, K., Barth, M. J., & Gonder, J. (2016) Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles. Transportation Research Record, 2572(1), 1-8.
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized Experience Replay. ArXiv: 1511.05952v4.
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., Hassabis, D. (2017) Mastering the game of Go without human knowledge. Nature 550: 354-359.
Sutton, R. S. and Barto, A. G. (2018) Reinforcement Learning: An Introduction. Second edition, MIT Press, Cambridge.
Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep Reinforcement Learning with Double Q-Learning. ArXiv:1509.06461.
Vinod, H.D. and Lòpez-de-Lacalle, J. (2009). Maximum entropy bootstrap for time series. The meboot R Package. J. Stat. Softw., 29 (2009), pp. 1-19 |
| Description: | 碩士
國立政治大學
資訊管理學系
107356003 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0107356003 |
| Data Type: | thesis |
| DOI: | 10.6814/NCCU202000375 |
| Appears in Collections: | [資訊管理學系] 學位論文
|
Files in This Item:
| File |
Description |
Size | Format | |
|---|
| 600301.pdf | | 1760Kb | Adobe PDF2 | 140 | View/Open |
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
