研究生: |
孫瑞廷 Juei-Ting Sun |
---|---|
論文名稱: |
基於Windows平行檔案系統下的平行I/O Parallel I/O for Windows Based Parallel File System |
指導教授: |
許雅三
Yarsun Hsu |
口試委員: | |
學位類別: |
碩士 Master |
系所名稱: |
電機資訊學院 - 電機工程學系 Department of Electrical Engineering |
論文出版年: | 2006 |
畢業學年度: | 94 |
語文別: | 英文 |
論文頁數: | 70 |
中文關鍵詞: | 平行檔案系統 、平行輸入輸出 、高效能輸入輸出 、視窗作業系統 、隨機視訊系統 |
外文關鍵詞: | parallel file system, MPI-IO, VOD, .NET framework, C#, high performance I/O, Windows, DirectShow |
相關次數: | 點閱:2 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
近年來,因為處理器的迅速發展所以使其速度變得越來越快,然而,資料的存取系統卻無法跟上處理器的速度,這使得處理器會因為存取系統過於緩慢而常常處於閒置狀態,因此,平行檔案系統被廣泛的用來解決這個問題,除此之外,由於叢集電腦被使用於平行處理平台的普及率逐步上升,這造成使用於叢集電腦上的平行處理軟體需求量持續增加,但是,卻有可能缺乏平行I/O的系統,這讓平行檔案系統被要求為叢集電腦來提供高效能的平行I/O系統,雖然它非常適合用來解決因為貧乏的效能所導致的瓶頸,可是大多數現存的平行檔案系統都是基於像UNIX的作業系統,因為這個理由,我們利用微軟的.NET架構實做了一個基於視窗作業系統的平行檔案系統,這篇論文除了有描述這個系統的設計構思與實現方法還有做了些效能的測試,讀取跟寫入的效能都比只用單一磁碟還要好,結果顯示,寫入的效能最高可以達到110 MB/s,讀取的效能最好也有75 MB/s。
在我們的平行檔案系統上,我們發展了一個簡單的隨選視訊系統,這個系統還有使用到微軟的DirectShow,另外,我們還實現了幾個主要的MPI-IO函數,這些在這篇論文中都有被討論到,利用平行檔案系統的優點,隨選視訊系統有足夠的能力可以供給客戶更好的服務品質,而且MPI-IO的效能也因受惠於此而有更傑出的性能。
In recent year, the speed of CPU becomes more and more progressive. However, the development of the storage system is incompetent to catch up with that of CPU. For this reason, the parallel file system is widely used to overcome this problem. In addition, as the PC cluster has increased in popularity as a parallel computing platform, the demand for system software support is necessary for parallel computing to continue to grow on this famous platform. But it may be devoid of parallel I/O systems. The parallel file system is required for providing a high performance parallel I/O system for PC clusters. Although it is so applicable to solve the bottleneck due to poor I/O performance, most of the existing parallel file systems are based on UNIX-like operating systems. Therefore, we implement a Windows based parallel file system by using Microsoft .NET framework. Design and implementation of our system are described in this thesis and the performance evaluation is performed as well. Both write and read performance are better than those of only using a single disk. The results indicate that the write performance can achieve 110 MB/s at the utmost and the read performance can also reach a peak of 75 MB/s.
We develop a simple VOD system by using Microsoft DirectShow and some primary methods of MPI-IO on top of our parallel file system. These two implementations are discussed in the thesis, too. Taking advantage of the parallel file system, the VOD system could have sufficient ability to supply higher QoS for clients and the performance of MPI-IO also benefits from it and attains outstanding results without suffering serious overhead.
[1] S. Kleiman D., Walsh R. Sandberg, D. Goldberg, and B. Lyon, .Design and Im-
plementation of the Sun Network Filesystem., in Summer USENIX Conference
Proceedings, pages 119-130, June 1985.
[2] Gene M. Amdahl, Validity of the Single Processor Approach to Achieve Large
Scale Computing Capabilities, pages 79-81, Morgan Kaufmann Publishers Inc.,
San Francisco, CA, USA, 2000.
[3] N.R. Adiga, M. Blumrich, and T. Liebsch, .An Overview of the BlueGene/L
Supercomputer., in Proceedings of the 2002 ACM/IEEE Conference on Super-
computing, Baltimore, Maryland, 2002, pages 1-22.
[4] Philip H. Carns, Walter B. Ligon III, Robert B. Ross, and Rajeev Thakur,
.PVFS: A Parallel File System for Linux Clusters., in Fourth Annual Linux
Showcase and Conference, Atlanta, GA, October 2000, pages 317-327.
[5] The Parallel Virtual File System, version 2. http://www.pvfs.org/pvfs2.
[6] Walter B. Ligon III and Robert B. Ross, .An Overview of the Parallel Virtual
File System., in 1999 Extreme Linux Workgroup, June 1999.
[7] José María Pérez, Jesús Carretero, and José Daniel García, .A Parallel File Sys-
tem for Networks of Windows Workstations., in ACM International Conference
on Supercomputing, 2004.
[8] Christopher R. Hertel, Implementing CIFS: The Common Internet File System,
Prentice-Hall, 2003.
[9] Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals,
Fourth Edition: Microsoft Windows Server 2003, Windows XP, and Windows
2000, Microsoft Press, 2004.
[10] Rajeev Nagar, Windows NT File System Internals: A Developer.s Guide,
O.Relly & Associates, September 1997.
[11] Message Passing Interface (MPI) Forum. MPI-2: Extensions to Message Passing
Interface, July 1997. http://www.mpi-forum.org/docs/docs.html.
[12] R. D. Russel, .The Architecture of BPFS: A Basic Parallel File System., in Pro-
ceedings of Second IASTED Int. Conference on Parallel and Distributed Com-
puting and Networks, pages 214-220, Brisbanen, Australia, December 1998.
[13] R. Thakur, W. Gropp, and E. Lusk, .On Implementing MPI-IO Portably and
with High Performance., in Proceedings of the Sixth Workshop on Input/Output
in Parallel and Distributed Systems, May 1999.
[14] R. Thakur, W. Gropp, and E. Lusk, Users Guide for ROMIO: A High-
Performance, Portable MPI-IO Implementation, Tech. Report ANL/MCS-TM-
234, Mathematics and Computer Science Division, Argonne National Labora-
tory, October 1997.
[15] Samuel A. Fineberg, Parkson Wong, Bill Nitzberg, and Chris Kuszmaul, .PM-
PIO - A Portable Implementation of MPI-IO., in Proceedings of the Sixth Sym-
posium on the Frontiers of Massively Parallel Computation, pages 188-195, IEEE
Computer Society Press, October 1996.
[16] R. Thakur, W. Gropp, and E. Lusk, .An Abstract-Device Interface for Imple-
menting Portable Parallel-I/O Interfaces., in Proceedings of the Sixth Sympo-
sium on the Frontiers of Massively Parallel Computation, pages180-187, IEEE
Computer Society Press, October 1996.
[17] James V. Huber, Christopher L. Elford, Daniel A. Reed, Andrew A. Chien,
and David S. Blumenthal, .PPFS: A High Performance Portable Parallel File
System., in Proceedings of the Ninth ACM International Conference on Super-
computing, pages 385-394, ACM Press, July 1995.
[18] E. Schikuta, T. Fuerle, and H. Wanek, .ViPIOS: The Vienna Parallel In-
put/Output System., in Proceedings of the Fourth International Euro-Par Con-
ference on Parallel Processing, pages 953-958, September 1998.
[19] David A. Patterson, Garth Gibson, and Randy H. Katz, .A Case for Redundant
Arrays of Inexpensive Disks (RAID)., in Proceedings of the 1998 ACM SIGMOD
International Conference on Mangement of Data, pages 109-116, June 1998.
[20] S. Moyer and V. Sunderam, .PIOUS: A Scalable Parallel I/O System for
Distributed Computing Environments., in Proceedings of the Scalable High-
Performance Computing Conference, pages 71-78, 1994.
[21] Intel Scalable System Division. Paragon System User.s Guide. Order Number
312489-004, May 1995.
[22] IBM Corp. IBM AIX Parallel I/O File System: Installation, Administration,
and Use. Document Number SH34-6065-01, August 1995.
[23] O. Krieger and M. Stumm, .HFS: A Performance-Oriented Flexible File System
Based on Building-Block Compositions., in Proceedings of Fourth Workshop on
Input/Output in Parallel and Distributed Systems, pages 95-108, ACM Press,
May 1996.
[24] Adam Sweeney, Doug Doucette, Wei Hu, Curtis Anderson, Mike Nishimoto, and
Geo¤Peck, .Scalability in the XFS File System., in Proceedings of the USENIX
1996 Annual Technical Conference, January 1996.