本研究探討了如何應用強化學習技術來優化線上外送平台的現金儲備管理和客戶補償策略。基於深度Q網路（DQN）演算法，我們提出了兩個新穎的框架：現金儲備深度Q網路（CRDQN）和補償深度Q網路（CRCDQN）。這些模型旨在解決如工人短缺、配送延遲和訂單取消等問題，這些問題會對客戶滿意度產生負面影響。我們設計的實驗盡可能貼近現實世界的情況，結果表明CRDQN能夠有效平衡現金儲備和客戶滿意度，相較於基線方法，獲得了更高的獎勵和盈利能力。相對而言，CRCDQN因動態調整補償而導致學習過程受到干擾，表現較差。我們的研究結果表明，像CRDQN這樣優化良好的現金儲備策略可以顯著提升線上外送平台的績效，確保長期成功和盈利能力。

This study explores the use of reinforcement learning to enhance cash reserve management and customer compensation in online food delivery platforms. Based on Deep Q-Network (DQN) algorithms, we introduce two novel frameworks: the Cash Reserve Deep Q-Network (CRDQN) and the Compensation Deep Q-Network (CRCDQN). These models aim to tackle challenges such as worker shortages, delivery delays, and order cancellations, which negatively impact customer satisfaction. Our experiments, designed to reflect real-world scenarios, demonstrate that CRDQN effectively balances cash reserves and customer satisfaction, achieving higher rewards and profitability compared to baseline methods. In contrast, CRCDQN, which adjusts compensation dynamically, shows poorer performance due to disrupted learning processes. Our findings suggest that a well-optimized cash reserve strategy, like CRDQN, can significantly benefit online food delivery platforms, ensuring long-term success and profitability.

[1] Jeff Howe. The rise of crowdsourcing. Wired, 14, 01 2006.
[2] Feilong Tang and Heteng Zhang. Spatial task assignment based on information gain in crowdsourcing. IEEE Transactions on Network Science and Engineering, 7(1):139–152, 2020.
[3] Alireza Jaribion, Siavash H. Khajavi, Ulriikka J ̈arvihaavisto, Iiro Nurmi, Robin Gustafsson, and Jan Holmstr ̈om. Crowdsourcing properties and mechanisms of mega hackathons: The case of junction. IEEE Transactions on Engineering Management, 70(9):3021–3035, 2023.
[4] Qi Hu, Lingfeng Ming, Ruijie Xi, Lu Chen, Christian S. Jensen, and Bolong Zheng. Soup: A fleet management system for passenger demand prediction and competitive taxi supply. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pages 2657–2660, 2021.
[5] Yurong Cheng, Boyang Li, Xiangmin Zhou, Ye Yuan, Guoren Wang, and Lei Chen. Real-time cross online matching in spatial crowdsourcing. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 1–12, 2020.
[6] Boyang Li, Yurong Cheng, Ye Yuan, Changsheng Li, Qianqian Jin, and Guoren Wang. Competition and cooperation: Global task assignment in spatial crowdsourcing. IEEE Transactions on Knowledge and Data Engineering, 35(10):9998–10010, 2023.
[7] Tianyue Ren, Xu Zhou, Kenli Li, Yunjun Gao, Ji Zhang, and Keqin Li. Efficient cross dynamic task assignment in spatial crowdsourcing. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 1420–1432, 2023.
[8] Xing Wang, Ling Wang, Chenxin Dong, Hao Ren, and Ke Xing. An online deep reinforcement learning-based order recommendation framework for rider-centered food delivery system. IEEE Transactions on Intelligent Transportation Systems, 24(5):5640–5654, 2023.
[9] Xing Wang, Ling Wang, Chenxin Dong, Hao Ren, and Ke Xing. Reinforcement learning-based dynamic order recommendation for on-demand food delivery. Tsinghua Science and Technology, 29(2):356–367, 2024.
[10] Jing Du, Bin Guo, Yan Liu, Liang Wang, Qi Han, Chao Chen, and Zhiwen Yu. Crowdnet: Enabling a crowdsourced object delivery network based on modern portfolio theory. IEEE Internet of Things Journal, 6(5):9030–9041, 2019.
[11] Yan Liu, Bin Guo, Chao Chen, He Du, Zhiwen Yu, Daqing Zhang, and
Huadong Ma. Foodnet: Toward an optimized food delivery network based on spatial crowdsourcing. IEEE Transactions on Mobile Computing, 18(6):1288–1301, 2019.
[12] Feihong Huang and Wei Jiang. A bilateral stable matching dispatching method in crowdsource food delivery service. In 2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS), pages 216–222, 2023.
[13] Xiangyu Kong, Guangyu Zou, Heng Qi, and Jiafu Tang. Optimization of o2o food delivery strategy in smart cities. In 2022 IEEE International Smart Cities Conference (ISC2), pages 1–7, 2022.
[14] Shuai Wang, Shijie Hu, Baoshen Guo, and Guang Wang. Cross-region courier displacement for on-demand delivery with multi-agent reinforcement learning. IEEE Transactions on Big Data, 9(5):1321–1333, 2023.
[15] Xiaosong Ding, Hu Liu, and Xi Chen. A real time scheduling scheme for food delivery. In 2021 IEEE International Conference on Emergency Science and Information Technology (ICESIT), pages 104–108, 2021.
[16] Jiacheng Li, Masato Noto, and Yang Zhang. Optimization model of takeout-delivery process based on concept of crowdsourcing. In 2020 Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems (SCIS-ISIS), pages 1–6, 2020.
[17] Manas Joshi, Arshdeep Singh, Sayan Ranu, Amitabha Bagchi, Priyank Karia, and Puneet Kala. Batching and matching for food delivery in dynamic road networks. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pages 2099–2104, 2021.
[18] Seth Gabriel D. Yeung, Jerahmeel K. Coching, Adrian Jenssen L. Pe, Wynnezel Wayne Naoto Akeboshi, Richard Josiah Tan Ai, and Robert Kerwin Billones. Route distance optimization of order delivery in a fast-food chain using linear programming. In 2022 IEEE 14th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), pages 1–6, 2022.
[19] Jia-Yu Wu, Min-Xia Zhang, Xue Wu, and Yu-Jun Zheng. A water wave
optimization algorithm for order selection and delivery path optimization for takeaway deliverymen. In 2021 11th International Conference on Information Science and Technology (ICIST), pages 620–628, 2021.