現有的多機器人多任務分配方法通常為啟發式演算法，當機器人及任務數量增加時，啟發式演算法在限制計算時間時，其任務分配性能會大幅下降。本研究提出一種基於深度強化學習多機器人協作演算法，用以解決多機器人多任務分配問題。在任務執行的同時，除了考慮機器人之間的合作式任務分配，另引入「對手」的概念，在執行任務場域中提供干擾。為避免碰撞發生以及機器人前往已被對手拿取任務目標之處，本研究提出的任務分配演算法具備動態重新計算的功能，使機器人在得知目標任務消失後能夠重新計算分配任務。另外，本研究運用獎勵重塑及課程式學習等技術，在高難度任務以及任務目標消失的條件下，避免模型在訓練過程中不容易獲得獎勵，導致模型訓練失敗，無法有效進行任務分配。本研究所提出的方法，相較啟發式演算法，能夠更快速找到較優的任務分配解，並且在執行較複雜任務時，仍能有效分配任務。

The existing methods of the multi-robot task allocation are mostly market-based methods and heuristic algorithms. When the scale of robots and tasks is increasingly heightened, under the limited computing time, the task allocation by heuristic algorithms will perform poorly. This research proposes a novel algorithm based on the deep reinforcement learning to solve the multi-robot task allocation for a group of robots to perform a number of different tasks within a specific field. Following the stage of the task allocation, the concept of "opponent" is also introduced to impede the activity of the cooperative robots during the task execution. To handle this problem, the task allocation which this work develops can dynamically reallocate the tasks for the overall MRS, if the cooperative robots lose their original tasks that are snatched by the opponent robots. Finally, the simulation results show that the proposed methodology can solve a superior task allocation with a higher speed than other heuristic algorithms, and can still effectively assign tasks when performing complex task types.

[1] V. Janthra. (2020) Robots efficiently sorting stock photo. [Online]. Available: https://www.istockphoto.com/photo/ robots-efficiently-sorting-gm1283421236-380851168?phrase=agv%20warehouse, accessed: 2022-08-22.
[2] P. R. Wurman, R. D’Andrea, and M. Mountz, “Coordinating hundreds of cooperative, autonomous vehicles in warehouses,” AI Mag., vol. 29, pp. 9–20, 2008.
[3] P. M. Kornatowski, A. Bhaskaran, G. M. Heitz, S. Mintchev, and D. Floreano, “Last-centimeter personal drone delivery: Field deployment
and user interaction,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3813–3820, 2018.
[4] Cai Luo, A. P. Espinosa, D. Pranantha, and A. De Gloria, “Multirobot search and rescue team,” 2011 IEEE International Symposium
on Safety, Security, and Rescue Robotics, pp. 296–301, 2011.
[5] B. P. Gerkey and M. J. Matarić, “A formal analysis and taxonomy of task allocation in multi-robot systems,” The International Journal of Robotics Research, vol. 23(9), pp. 939–954, 2004.
[6] H. Ma and S. Koenig, “Optimal target assignment and path finding for teams of agents,” Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems, ser. AAMAS ’16, p. 1144–1152, Richland, SC, 2016.
[7] C. Henkel, J. Abbenseth, and M. Toussaint, “An optimal algorithm to solve the combined task allocation and path finding problem,” 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4140–4146, 2019.
[8] C. Landry, R. Henrion, D. Hömberg, M. Skutella, and W. Welz, “Task assignment, sequencing and path-planning in robotic welding cells,” 2013 18th International Conference on Methods Models in Automation Robotics (MMAR), pp. 252–257, 2013.
[9] C. Liu and A. Kroll, “A centralized multi-robot task allocation for industrial plant inspection by using a* and genetic algorithms,” Artificial Intelligence and Soft Computing, pp. 466–474, 2012.
[10] K. Jose and D. K. Pratihar, “Task allocation and collision-free path planning of centralized multi-robots system for industrial plant inspection using heuristic methods,” Robotics Auton. Syst., vol. 80, pp. 34–42, 2016.
[11] M. Koes, I. Nourbakhsh, and K. Sycara, “Heterogeneous multirobot
coordination with spatial and temporal constraints,” ser. AAAI’05, p.
1292–1297, 2005.
[12] E. A. Khamis A., Hussein A., “Multi-robot task allocation: A review of the state-of-the-art,” Cooperative Robots and Sensor Networks 2015, M.-d. D. J. Koubâa A., Ed. Cham: Springer, 2015, vol. 604.
[13] J. Yu and S. LaValle, “Structure and intractability of optimal multirobot path planning on graphs,” AAAI, 2013.
[14] G. Wagner and H. Choset, “M*: A complete multirobot path planning
algorithm with performance bounds,” 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3260–3267,
2011.
[15] M. Goldenberg, A. Felner, R. Stern, G. Sharon, N. Sturtevant, R. C. Holte, and J. Schaeffer, “Enhanced partial expansion a*,” J. Artif. Int. Res., vol. 50, no. 1, p. 141–187, May 2014.
[16] G. Sharon, R. Stern, A. Felner, and N. R. Sturtevant, “Conflict-based search for optimal multi-agent pathfinding,” Artificial Intelligence, vol. 219, pp. 40 – 66, 2015.
[17] E. Boyarski, A. Felner, R. Stern, G. Sharon, O. Betzalel, D. Tolpin, and S. E. Shimony, “Icbs: The improved conflict-based search algorithm for multi-agent pathfinding,” SOCS, 2015.
[18] B. P. Gerkey, , and M. J. Matarić, “Multi-robot task allocation: analyzing the complexity and optimality of key architectures,” 2003
IEEE International Conference on Robotics and Automation (Cat. No.03CH37422), vol. 3, pp. 3862–3868 vol.3, 2003.
[19] B. P. Gerkey and M. J. Matarić, “A formal analysis and taxonomy of task allocation in multi-robot systems,” The International Journal of Robotics Research, vol. 23, no. 9, pp. 939–954, 2004.
[20] E. Nunes, M. Manner, H. Mitiche, and M. Gini, “A taxonomy for task allocation problems with temporal and ordering constraints,” Robotics and Autonomous Systems, vol. 90, pp. 55–70, 2017, special Issue on New Research Frontiers for Intelligent Autonomous Systems
[21] A. Mosteo and L. Montano, “Simulated annealing for multi-robot
hierarchical task allocation with flexible constraints and objective
functions,” Workshop on Network Robot Systems: Toward Intelligent Robotic Systems Integrated with Environments, 01 2006.
[22] T. Au, O. Ilghami, U. Kuter, J. W. Murdock, D. S. Nau, D. Wu, and F. Yaman, “SHOP2: an HTN planning system,” CoRR, vol. abs/1106.4869, 2011.
[23] Y. Li, W. Zeng, H. Zhou, and R. Chen, “Research on dynamic emergency task allocation with mdp,” The 2nd International Conference on Information Science and Engineering, pp. 1452–1455, 2010.
[24] G. M. Skaltsis, H.-S. Shin, and A. Tsourdos, “A survey of task allocation techniques in mas,” 2021 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 488–497, 2021.
[25] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement
learning: A survey,” Journal of artificial intelligence research, vol. 4, pp. 237–285, 1996.
[26] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
[27] H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016.
[28] Eurobot –international students robotic contest. [Online]. Available: https://www.eurobot.org/
[29] Age of bots –2022. [Online]. Available: https://www.eurobot.org/eurobot-contest/eurobot-2022/
[30] D. Singh, M. K. Singh, T. Singh, and R. Prasad, “Genetic algorithm for solving multiple traveling salesmen problem using a new crossover and population generation,” Computación y Sistemas, vol. 22, 07 2018.