簡易檢索 / 詳目顯示

研究生: 鍾武君
Wu-Chun Chung
論文名稱: 分散式運算中資源資訊與探索之非集中化管理
Decentralized Management of Resource Information and Discovery in Large-Scale Distributed Computing
指導教授: 鍾葉青
Chung, Yeh-Ching
口試委員: 江振瑞
Jiang, Jehn-Ruey
周志遠
Chou, Chi-Yuan
張玉山
Chang, Yue-Shan
張瑞雄
Chang, Ruay-Shiung
楊朝棟
Yang, Chao-Tung
賴冠州
Lai, Kuan-Chou
蕭宏章
Hsiao, Hung-Chang
鍾葉青
Chung, Yeh-Ching
學位類別: 博士
Doctor
系所名稱: 電機資訊學院 - 資訊工程學系
Computer Science
論文出版年: 2013
畢業學年度: 101
語文別: 英文
論文頁數: 106
中文關鍵詞: 分散式系統非集中化管理格網運算雲端運算同儕網路資訊系統資源探索負載平衡協同策略疊蓋式網路維護
外文關鍵詞: Distributed System, Decentralized Management, Grid Computing, Cloud Computing, P2P Network, Information System, Resource Discovery, Load Balance, Cooperative Strategy, Overlay Maintenance
相關次數: 點閱:2下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 由於運算能力與服務的需求與日俱增,驅使著大規模分散式運算系統邁入新的典範,例如格網運算以及雲端運算,為了能有效管理散佈於網路上的各種資源,資源資訊與探索系統致力於記錄資源屬性的狀態以及尋找滿足需求的資源。隨著系統規模擴大時,整合分散於電腦網路上的資源藉以避免通訊瓶頸、單點失效或是負載不平衡等問題,將是一門重大的課題。有鑑於此,一套適用大規模分散式資源的非集中化管理方式逐漸備受矚目,因此,講究延展性與穩健性而發展的點對點技術,深受這類型的環境所青睞。
      在本論文中,我們針對分散式運算環境的非集中化資源管理進行一系列的探討。首先,對於格網運算結合點對點技術的可行性,我們提出非集中式的格網對格網框架,藉由協調分屬不同格網的資源得以實現格網聯盟的目的。接著對於大規模的運算環境,我們深入探討利用同儕網路的架構,如何進行資源資訊與探索的研究方法,並針對結構式與非結構式的網路架構提出相對應的解決方案,實驗結果顯示我們的方法在非集中化的管理下,能夠有效地組織散落在分散式環境中的資源資訊,並探索出可供運算使用的資源。最後我們考慮到分散式運算系統中,若廣泛使用疊蓋式網路將會衍生過多的維護成本,如何節省重複的維護成本激發我們提出適用於多個疊蓋式網路環境的協同合作策略,實驗證實採用我們的方法可以大幅減少維護多個疊蓋式網路所造成的額外負擔。


    The increasing demand of computing power and services derives the development of new paradigm of large-scale distributed computing systems such as Grids and Clouds. To efficiently manage the diverse and scattered resources, the resource information and discovery system is employed to record the status of resource attributes and locate the resource fulfilled the requirement of demands. As the scale of a system expands, the major difficulty is to prevent a communication bottleneck, a single point of failure or the load imbalance by the federation of resources distributed over computer networks. In view of this, a decentralized management for large-scale distributed resources comes out with more attentions. With the inherent properties of scalability and robustness, the Peer-to-Peer (P2P) approach is attractive to such environments.
      In this dissertation, we conduct a series of studies on the decentralized resource management for distributed computing environments. The first study investigates the synergy between Grids and P2P, in which a decentralized Grid-to-Grid (G2G) framework is introduced to harmonize autonomic Grid resources in realizing the Grid federation. Second, a further exploration of employing the P2P network is conducted on resource information and discovery for large-scale computing environments. We propose corresponding solutions for both structured and unstructured networks. The main results show that our approaches are able to efficiently organize the resource information among the distributed environment and locate those available resources under the decentralized management. Finally, the third study, we are concerned with the overlay maintenance if multiple overlays co-habit in the distributed computing system. How to simplify the common overlay-maintenance is motivated us to come up with a cooperative strategy for the multi-overlay maintenance. Experimental results demonstrate that the cost of multi-overlay maintenance could be significantly decreased while applying our approach.

    中文摘要 i Abstract ii Acknowledgments iii Contents v List of Figures vii List of Tables x Chapter 1 Introduction 1 1.1 Motivation 2 1.2 Problem Statements and Main Results 4 1.3 Organization 7 Chapter 2 Related Work 8 2.1 Synergy between Grids and P2P 8 2.2 P2P-Based Resource Information and Discovery 9 2.3 Cooperative Multi-Overlay Environment 13 Chapter 3 Grid-to-Grid Framework 14 3.1 Super-Peer Based G2G System 14 3.2 Meta-Grid Framework 16 3.2.1 Information Service 18 3.2.2 Network Management 19 3.2.3 Task Management with Resource Discovery 20 3.3 Performance Evaluation 21 3.4 Summary 25 Chapter 4 Distributed Resource Information and Discovery 26 4.1 Self-Adaptive Approach 26 4.1.1 System Overview 27 4.1.2 Self-Adaptive Resource Index and Discovery 30 4.1.3 Load Balance 34 4.1.4 Performance Evaluation 36 4.1.5 Summary 41 4.2 Direction-Aware Approach 42 4.2.1 System Overview 43 4.2.2 Direction-Aware Resource Discovery 46 4.2.3 Property Analysis 51 4.2.4 Performance Evaluation 53 4.2.5 Summary 61 Chapter 5 Cooperative Overlay Maintenance 62 5.1 Multi-Overlay Maintenance 62 5.1.1 Multi-Overlay Environment 62 5.1.2 Common Overlay Maintenance 63 5.2 Cooperative Strategy 64 5.2.1 Inter-Overlay Protocol 66 5.2.2 Intra-Overlay Protocol 67 5.3 Cooperative Appliance 68 5.3.1 Cooperative Failure Detection 68 5.3.2 Cooperative Network-Proximity Estimation 74 5.3.3 Selection Criteria and Approach 78 5.4 Performance Evaluation 81 5.4.1 Cooperative Failure Detection 82 5.4.2 Cooperative Network-Proximity Estimation 88 5.4.3 Hybrid Cooperative Maintenance 92 5.4.4 Cooperative Overlay Selection 94 5.5 Summary 95 Chapter 6 Conclusion and Future Work 97 6.1 Concluding Remarks 97 6.2 Future Directions 99 Bibliography 100

    [1] James Aspnes and Gauri Shah, "Skip Graphs," in Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), Baltimore, Maryland, 2003, pp. 384-393.
    [2] H. M. N. Dilum Bandara and Anura P. Jayasumana, "Characteristics of Multi-Attribute Resources/Queries and Implications on P2P Resource Discovery," in Proceedings of the 2011 9th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), 2011, pp. 173-180.
    [3] Chaitanya Baru, Reagan Moore, Arcot Rajasekar, and Michael Wan, "The SDSC Storage Resource Broker," in Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research (CASCON), Toronto, Ontario, Canada, 1998, pp. 5-16.
    [4] Ashwin R. Bharambe, Mukesh Agrawal, and Srinivasan Seshan, "Mercury: Supporting Scalable Multi-Attribute Range Queries," in Proceedings of the 2004 conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM), Portland, Oregon, USA, 2004, pp. 353-366.
    [5] Min Cai, Martin Frank, Jinbo Chen, and Pedro Szekely, "MAAN: A Multi-Attribute Addressable Network for Grid Information Services," Journal of Grid Computing, vol. 2, no. 1, 2004, pp. 3-14.
    [6] Jiannong Cao, Fred Bojin Liu, and Cheng-Zhong Xu, "P2PGrid: Integrating P2P Networks into the Grid Environment," Concurrency and Computation: Practice and Experience, vol. 19, no. 7, 2007, pp. 1023-1046.
    [7] Miguel Castro, Peter Druschel, Y. Charlie Hu, and Antony Rowstron, "Exploiting Network Proximity in Distributed Hash Tables," in International Workshop on Future Directions in Distributed Computing (FuDiCo), Bertinoro, Italy, 2002, pp. 52-55.
    [8] Miguel Castro, Peter Druschel, Y. Charlie Hu, and Antony Rowstron, "Exploiting Network Proximity in Peer-to-Peer Overlay Networks," Microsoft Research MSR-TR-2002-82, 2002.
    [9] Miguel Castro, Peter Druschel, Anne-Marie Kermarrec, and Antony I. T. Rowstron, "Scribe: A Large-Scale and Decentralized Application-Level Multicast Infrastructure," IEEE Journal on Selected Areas in Communications, vol. 20, no. 8, 2002, pp. 1489-1499.
    [10] Ann Chervenak, Ian Foster, Carl Kesselman, Charles Salisbury, and Steven Tuecke, "The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets," Journal of Network and Computer Applications, vol. 23, no. 3, 2000, pp. 187-200.
    [11] Wu-Chun Chung and Ruay-Shiung Chang, "A New Mechanism for Resource Monitoring in Grid Computing," Future Generation Computer Systems, vol. 25, no. 1, 2009, pp. 1-7.
    [12] Wu-Chun Chung, Chin-Jung Hsu, Kuan-Chou Lai, Kuan-Ching Li, and Yeh-Ching Chung, "Direction-Aware Resource Discovery in Large-Scale Distributed Computing Environments," The Journal of Supercomputing, In Press, March 2013.
    [13] Wu-Chun Chung, Chin-Jung Hsu, Yi-Hsiang Lin, Kuan-Chou Lai, and Yeh-Ching Chung, "G2G: A Meta-Grid Framework for the Convergence of P2P and Grids," International Journal of Grid and High Performance Computing, vol. 2, no. 3, 2010, pp. 1-16.
    [14] Wu-Chun Chung, Yi-Hsiang Lin, Kuan-Chou Lai, Kuan-Ching Li, and Yeh-Ching Chung, "A Self-Adaptive Resource Index and Discovery System in Distributed Computing Environments," International Journal of Ad Hoc and Ubiquitous Computing, vol. 10, no. 2, 2012, pp. 74-83.
    [15] Karl Czajkowski, Steven Fitzgerald, Ian Foster, and Carl Kesselman, "Grid Information Services for Distributed Resource Sharing," in Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC), San Francisco, CA , USA, 2001, pp. 181-194.
    [16] Imen Filali and Fabrice Huet, "Dynamic TTL-Based Search in Unstructured Peer-to-Peer Networks," in Proceedings of the 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing (CCGrid), Melbourne, VIC, Australia, 2010, pp. 438-447.
    [17] Steven Fitzgerald, Ian Foster, Carl Kesselman, Gregor von Laszewski, Warren Smith, and Steven Tuecke, "A Directory Service for Configuring High-Performance Distributed Computations," in Proceedings of the 6th IEEE International Symposium on High Performance Distributed Computing (HPDC), Portland, OR, 1997, pp. 365-375.
    [18] Ian Foster and Adriana Iamnitchi, "On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing," in Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS), Berkeley, CA, 2003, pp. 118-128.
    [19] Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, "The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration," presented at the GGF4, 2002.
    [20] Ian Foster, Carl Kesselman, and Steven Tuecke, "The Anatomy of the Grid: Enabling Scalable Virtual Organizations," International Journal of High Performance Computing Applications, vol. 15, no. 3, 2001, pp. 200-222.
    [21] Ian Foster, Yong Zhao, Raicu Ioan, and Lu Shiyong, "Cloud Computing and Grid Computing 360-Degree Compared," in Grid Computing Environments Workshop (GCE), 2008, pp. 1-10.
    [22] Geoffrey Fox, Shrideep Pallickara, and Xi Rao, "Towards Enabling Peer-to-Peer Grids," Concurrency and Computation: Practice and Experience, vol. 17, no. 7-8, 2005, pp. 1109-1131.
    [23] Krishna P. Gummadi, Richard J. Dunn, Stefan Saroiu, Steven D. Gribble, Henry M. Levy, and John Zahorjan, "Measurement, Modeling, and Analysis of a Peer-to-Peer File-Sharing Workload," in Proceedings of the 9th ACM Symposium on Operating Systems Principles (SOSP), Bolton Landing, NY, USA, 2003, pp. 314-329.
    [24] Nicholas J. A. Harvey, Michael B. Jones, Stefan Saroiu, Marvin Theimer, and Alec Wolman, "SkipNet: A Scalable Overlay Network with Practical Locality Properties," in Proceedings of the 4th Conference on USENIX Symposium on Internet Technologies and Systems (USITS), Seattle, WA, 2003, pp. 9-9.
    [25] Chin-Jung Hsu, "Cooperative Maintenance in Multi-Overlay Environments," M.S. Thesis, Institute of Information Systems and Applications, National Tsing Hua University, HsinChu, Taiwan, 2009.
    [26] Chin-Jung Hsu, Wu-Chun Chung, Kuan-Chou Lai, Kuan-Ching Li, and Yeh-Ching Chung, "A Novel Approach for Cooperative Overlay-Maintenance in Multi-Overlay Environments," in Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), Indianapolis, Indiana, USA, 2010, pp. 81-88.
    [27] Chin-Jung Hsu, Wu-Chun Chung, Kuan-Chou Lai, Kuan-Ching Li, and Yeh-Ching Chung, "Cooperative Failure Detection in Multi-Overlay Environments," Journal of Internet Technology, vol. 12, no. 2, 2011, pp. 259-267.
    [28] Adriana Iamnitchi and Ian Foster, "On Fully Decentralized Resource Discovery in Grid Environments," in Proceedings of the Second International Workshop on Grid Computing (GRID), Denver, CO, 2001, pp. 51-62.
    [29] Adriana Iamnitchi and Ian Foster, "A Peer-to-Peer Approach to Resource Location in Grid Environments," in Grid Resource Management, ed: Kluwer Academic Publishers, 2004, pp. 413-429.
    [30] Vana Kalogeraki, Dimitrios Gunopulos, and D. Zeinalipour-Yazti, "A Local Search Mechanism for Peer-to-Peer Networks," in Proceedings of the 11th International Conference on Information and Knowledge Management (CIKM), McLean, Virginia, USA, 2002, pp. 300-307.
    [31] David R. Karger and Matthias Ruhl, "Simple Efficient Load-Balancing Algorithms for Peer-to-Peer Systems," Theory of Computing Systems, vol. 39, no. 6, 2006, pp. 787-804.
    [32] Shen Lin, François Taïani, and Gordon S. Blair, "Exploiting Synergies Between Coexisting Overlays," in Proceedings of the 9th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Lisbon, Portugal, 2009, pp. 1-15.
    [33] Yi-Hsiang Lin, "SARIDS: A Self-Adaptive Resource Index and Discovery System," M.S. Thesis, Institute of Information Systems and Applications, National Tsing Hua University, HsinChu, Taiwan, 2009.
    [34] Eng Keong Lua, Jon Crowcroft, Marcelo Pias, Ravi Sharma, and Steven Lim, "A Survey and Comparison of Peer-to-Peer Overlay Network Schemes," IEEE Communications Surveys & Tutorials, vol. 7, no. 2, 2005, pp. 72-93.
    [35] Qin Lv, Pei Cao, Edith Cohen, Kai Li, and Scott Shenker, "Search and Replication in Unstructured Peer-to-Peer Networks," in Proceedings of the 16th International Conference on Supercomputing (ICS), New York, USA, 2002, pp. 84-95.
    [36] Balasubramaneyam Maniymaran, Marin Bertier, and Anne-Marie Kermarrec, "Build One, Get One Free: Leveraging the Coexistence of Multiple P2P Overlay Networks," in Proceedings of the 27th International Conference on Distributed Computing Systems (ICDCS), Toronto, Ontario, Canada, 2007, pp. 33-40.
    [37] Alexandros Marinos and Gerard Briscoe, "Community Cloud Computing," in Proceedings of the 1st International Conference on Cloud Computing (CloudCom), Beijing, China, 2009, pp. 472-484.
    [38] Marios D. Dikaiakos, Dimitrios Katsaros, Pankaj Mehra, George Pallis, and Athena Vakali, "Cloud Computing: Distributed Internet Computing for IT and Scientific Research," IEEE Internet Computing, vol. 13, no. 5, 2009, pp. 10-13.
    [39] Matthew L. Massie, Brent N. Chun, and David E. Culler, "The Ganglia Distributed Monitoring System: Design, Implementation, and Experience," Parallel Computing, vol. 30, no. 7, 2004, pp. 817-840.
    [40] Carlo Mastroianni, Domenico Talia, and Oreste Verta, "Designing an Information System for Grids: Comparing Hierarchical, Decentralized P2P and Super-Peer Models," Parallel Computing, vol. 34, no. 10, 2008, pp. 593-611.
    [41] David Oppenheimer, Jeannie Albrecht, David Patterson, and Amin Vahdat, "Scalable Wide-Area Resource Discovery," UC Berkeley Technical Report UCB/CSD-04-1334, 2004.
    [42] C. Pairot, P. Garcia, and A. F. G. Skarmeta, "DERMI: A Decentralized Peer-to-Peer Event-Based Object Middleware," in Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS), Tokyo, Japan, 2004, pp. 236-243.
    [43] Shrideep Pallickara and Geoffrey Fox, "NaradaBrokering: A Distributed Middleware Framework and Architecture for Enabling Durable Peer-to-Peer Grids," in Proceedings of the ACM/IFIP/USENIX 2003 International Conference on Middleware (Middleware), Rio de Janeiro, Brazil, 2003, pp. 41-61.
    [44] Rajiv Ranjan, Aaron Harwood, and Rajkumar Buyya, "Peer-to-Peer-Based Resource Discovery in Global Grids: A Tutorial," IEEE Communications Surveys & Tutorials, vol. 10, no. 2, 2008, pp. 6-33.
    [45] Ananth Rao, Karthik Lakshminarayanan, Sonesh Surana, Richard Karp, and Ion Stoica, "Load Balancing in Structured P2P Systems," in Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS), Berkeley,CA, 2003, pp. 68-79.
    [46] Sylvia Ratnasamy, Paul Francis, Mark Handley, Richard Karp, and Scott Schenker, "A Scalable Content-Addressable Network," in Proceedings of the 2001 conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM), San Diego, California, USA, 2001, pp. 161-172.
    [47] Sean Rhea, Dennis Geels, Timothy Roscoe, and John Kubiatowicz, "Handling Churn in a DHT," in Proceedings of the annual conference on USENIX Annual Technical Conference (USENIX), Boston, MA, 2004, pp. 127-140.
    [48] John Risson and Tim Moors, "Survey of Research Towards Robust Peer-to-Peer Networks: Search Methods," Computer Networks, vol. 50, no. 17, 2006, pp. 3485-3521.
    [49] Antony Rowstron and Peter Druschel, "Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems," in IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), London, UK, 2001, pp. 329-350.
    [50] Cristina Schmidt and Manish Parashar, "Squid: Enabling Search in DHT-Based Systems," Journal of Parallel and Distributed Computing, vol. 68, no. 7, 2008, pp. 962-975.
    [51] Po-Chi Shih, Hsi-Min Chen, Yeh-Ching Chung, Chien-Min Wang, Ruay-Shiung Chang, Ching-Hsien Hsu, Kuo-Chan Huang, and Chao-Tung Yang, "Middleware of Taiwan UniGrid," in Proceedings of the 2008 ACM symposium on Applied computing (SAC), Fortaleza, Ceara, Brazil, 2008, pp. 489-493.
    [52] Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M. Frans Kaashoek, Frank Dabek, and Hari Balakrishnan, "Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications," IEEE/ACM Transactions on Networking, vol. 11, no. 1, 2003, pp. 17-32.
    [53] Domenico Talia and Paolo Trunfio, "Toward a Synergy Between P2P and Grids," IEEE Internet Computing, vol. 7, no. 4, 2003, pp. 96-95.
    [54] Sanya Tangpongprasit, Takahiro Katagiri, Kenji Kise, Hiroki Honda, and Toshitsugu Yuba, "A Time-to-Live Based Reservation Algorithm on Fully Decentralized Resource Discovery in Grid Computing," Parallel Computing, vol. 31, no. 6, 2005, pp. 529-543.
    [55] Paolo Trunfio, Domenico Talia, Harris Papadakis, Paraskevi Fragopoulou, Matteo Mordacchini, Mika Pennanen, Konstantin Popov, Vladimir Vlassov, and Seif Haridi, "Peer-to-Peer Resource Discovery in Grids: Models and Systems," Future Generation Computer Systems, vol. 23, no. 7, 2007, pp. 864-878.
    [56] Beverly Yang and Hector Garcia-Molina, "Improving Search in Peer-to-Peer Networks," in Proceedings of the 22nd International Conference Distributed Computing Systems (ICDCS), Vienna, Austria, 2002, pp. 5-14.
    [57] Beverly Yang and Hector Garcia-Molina, "Designing a Super-Peer Network," in Proceedings of the 19th International Conference on Data Engineering (ICDE), 2003, pp. 49-60.
    [58] Xin Yan Zhang, Qian Zhang, Zhensheng Zhang, Gang Song, and Wenwu Zhu, "A Construction of Locality-Aware Overlay Network: mOverlay and its Performance," IEEE Journal on Selected Areas in Communications, vol. 22, no. 1, 2004, pp. 18-28.
    [59] Xuechai Zhang, Jeffrey L. Freschl, and Jennifer M. Schopf, "A Performance Study of Monitoring and Information Services for Distributed Systems," in Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC), Washington, DC, USA, 2003, pp. 270-281.
    [60] Ben Y. Zhao, Ling Huang, Jeremy Stribling, Sean C. Rhea, Anthony D. Joseph, and John D. Kubiatowicz, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment," IEEE Journal on Selected Areas in Communications, vol. 22, no. 1, 2004, pp. 41-53.
    [61] S. Q. Zhuang, D. Geels, I. Stoica, and R. H. Katz, "On Failure Detection Algorithms in Overlay Networks," in Proceedings of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), Miami, FL, USA, 2005, pp. 2112-2123.

    無法下載圖示 全文公開日期 本全文未授權公開 (校內網路)
    全文公開日期 本全文未授權公開 (校外網路)

    QR CODE