簡易檢索 / 詳目顯示

研究生: 洪偉程
Hung, Wei-Cheng
論文名稱: 邊緣節點之容器資料備份
CSM-DBEN: Container Storage Manager for Data Backup on Edge Nodes
指導教授: 李哲榮
Lee, Che-Rung
口試委員: 周志遠
Chou, Jerry
賴冠州
Lai, Kuan-Chou
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 資訊工程學系
Computer Science
論文出版年: 2022
畢業學年度: 110
語文別: 英文
論文頁數: 36
中文關鍵詞: 容器虛擬化備份機器學習故障預測
外文關鍵詞: container, virtualization, backup, prediction, failure
相關次數: 點閱:3下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 邊緣計算將計算資源分散到距離資料來源更接近的地方,現在已經成為解決延遲與
    大量連接問題的最有效率的架構之一。在邊緣節點上,應用程式通常被包裝成容器
    來達到快速佈署與多租戶的需求。然而,邊緣節點有限的計算資源與特定的應用
    程式容器化使得資料永不丟失難以達成。在本論文中,我們提出了 CSM-DBEN,
    他用來幫助邊緣節點上容器的儲存資料做熱備份以及資料恢復。CSM-DBEN 利
    用虛擬機器來達到更好的安全性、資料隔離性以及儲存分層化。對於遠端備
    份,CSM-DBEN 會負責做資料的加密以及壓縮來加強安全性以及有效利用頻寬。
    此外,CSM-DBEN 還會利用硬碟故障預測模型來動態調整資料備份的頻率,以達
    到降低系統負擔的效果。實驗結果表明,CSM-DBEN 執行增量備份所需的時間比
    它的對手 (rdiff-backup) 少了約 75% ∼ 80%。


    Edge computing that distributes computing resources close to data sources has become one of the most efficient architectures to solve the latency and the massive connection problems. On top of edge nodes, applications are usually packed in containers to enable fast deployment and multi-tenancy. However, the limited computing resource of edge nodes and application specific containerization make systematic data resilience difficult on edge nodes. In this paper, we present a container storage manager for data backup on edge nodes (CSM-DBEN), which provides the live storage backup and recovery for container volumes in the system level of edge nodes. CSM-DBEN leverages virtual machines (VM) to support better security, data isolation, and storage layering. For remote backups, CSM-DBEN performs data encryption and compression for data transmission to enhance the security and bandwidth efficiency. It also dynamically adjusts the data backup frequency, based on the prediction of disk failures, to reduce the system overhead. Experiments show that the incremental data backup time of CSM-DBEN is reduced by 75% ~ 80% compared to the commonly used rdiff-backup system.

    Contents 中文摘要 1 Abstract 2 List of Figures 5 List of Tables 6 1 Introduction 7 2 Related Works 10 2.1 Container Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Data Backup for Containers . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Disk Failure Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3 Container Storage Manager 13 3.1 Design of CSM-DBEN . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3 Backup and Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4 Disk Failure Prediction 20 4.1 Dynamic Backup Frequency . . . . . . . . . . . . . . . . . . . . . . . . 20 4.2 SMART Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.3 Prediction Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5 Experiments 26 5.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.2 Results Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 5.3 Effectiveness of data compression . . . . . . . . . . . . . . . . . . . . . 30 5.4 Optimization of the sidecar VM . . . . . . . . . . . . . . . . . . . . . . 31 5.5 Disk Failure Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 6 Conclusion and Future Work 33 References 34

    [1] Andrew Tridgell. Efficient Algorithms for Sorting and Synchronization. 2000.
    [2] Lik-Hang Lee et al. All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda. 2021.
    [3] Mohammed Mohammed Sadeeq et al. “IoT and Cloud Computing Issues, Challenges and Opportunities: A Review”. Qubahan Academic Journal 1.2 (Mar.2021), pp. 1–7.
    [4] Keyan Cao et al. “An Overview on Edge Computing Research”. IEEE Access 8(2020), pp. 85714–85728.
    [5] Rajdeep Dua, A Reddy Raja, and Dharmesh Kakadia. “Virtualization vs Containerization to Support PaaS”. 2014 IEEE International Conference on Cloud Engineering. 2014, pp. 610–614.
    [6] Tuan Anh Nguyen, Dong Seong Kim, and Jong Sou Park. “Availability modeling and analysis of a data center for disaster tolerance”. Future Generation Computer Systems 56 (2016), pp. 27–50.
    [7] Mohammad Reza Mesbahi, Amir Masoud Rahmani, and Mehdi Hosseinzadeh.
    “Reliability and high availability in cloud computing environments: a reference roadmap”. Human-centric Computing and Information Sciences 8.1 (2018),pp. 1–31.
    [8] Chieh-Yu Yu et al. “Efficient Group Fault Tolerance for Multi-tier Services in Cloud Environments”. ICC 2020 - 2020 IEEE International Conference on Communications (ICC). 2020, pp. 1–7.
    [9] Lele Ma, Shanhe Yi, and Qun Li. “Efficient Service Handoff across Edge Servers via Docker Container Migration”. Proceedings of the Second ACM/IEEE Symposium on Edge Computing. SEC ’17. San Jose, California: Association for Computing Machinery, 2017. isbn: 9781450350877.
    [10] Carlo Puliafito et al. “Container migration in the fog: A performance evaluation”. Sensors 19.7 (2019), p. 1488.
    [11] Brendan Burns and David Oppenheimer. “Design patterns for container-based distributed systems”. 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 16). 2016.
    [12] Emré Celebi, Tal Garfinkel, and Min Cai. “The Design and Evolution of Live Storage Migration in VMware ESX”. 2011 USENIX Annual Technical Conference (USENIX ATC 11). Portland, OR: USENIX Association, June 2011.
    13] Yaodong Yang et al. “SnapMig: Accelerating VM Live Storage Migration by Leveraging the Existing VM Snapshots in the Cloud”. IEEE Transactions on Parallel and Distributed Systems 29.6 (2018), pp. 1416–1427.
    [14] Yaodong Yang et al. “WAIO: Improving Virtual Machine Live Storage Migration for the Cloud by Workload-Aware IO Outsourcing”. 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom). 2015, pp. 314–321.
    [15] Cheng-Hao Huang and Che-Rung Lee. “Enhancing the Availability of Docker Swarm Using Checkpoint-and-Restore”. 2017 14th International Symposium on Pervasive Systems, Algorithms and Networks 2017 11th International Conference on Frontier of Computer Science and Technology 2017 Third International Symposium of Creative Computing (ISPAN-FCST-ISCC). 2017, pp. 357–362.
    [16] Yong Xu et al. “Improving Service Availability of Cloud Systems by Predicting Disk Error”. 2018 USENIX Annual Technical Conference (USENIX ATC 18). Boston, MA: USENIX Association, July 2018, pp. 481–494. isbn: 978-1-939133-01-4.
    [17] Chuan Luo et al. “NTAM: Neighborhood-Temporal Attention Model for Disk Failure Prediction in Cloud Platforms”. Proceedings of the Web Conference 2021. WWW ’21. Ljubljana, Slovenia: Association for Computing Machinery, 2021, pp. 1181–1191. isbn: 9781450383127.
    [18] Maxime Amram et al. “Interpretable predictive maintenance for hard drives”. Machine Learning with Applications 5 (2021), p. 100042. issn: 2666-8270.
    [19] The speed of containers, the security of VMs. https://katacontainers.io/. Accessed: 2022-06-15.
    [20] Documentation of 9psetup. https://wiki.qemu.org/Documentation/ 9psetup. Accessed: 2022-06-15.
    [21] Gitlab of virtiofs. https://virtio-fs.gitlab.io/. Accessed: 2022-06-15.
    [22] Eduardo Pinheiro, Wolf-Dietrich Weber, and Luiz André Barroso. “Failure Trends in a Large Disk Drive Population”. 5th USENIX Conference on File and Storage Technologies (FAST 07). San Jose, CA: USENIX Association, Feb. 2007.
    [23] Sidi Lu et al. “Making Disk Failure Predictions SMARTer!” 18th USENIX Conference on File and Storage Technologies (FAST 20). Santa Clara, CA: USENIX Association, Feb. 2020, pp. 151–167. isbn: 978-1-939133-12-0.
    [24] BackBlaze quarterly hard drive reliability reports. https://www.backblaze.com/b2/hard-drive-test-data.html. Accessed: 2022-06-15.
    [25] Reverse differential backup tool, over a network or locally. https://rdiff-backup.net/index.html. Accessed: 2022-06-15.
    [26] sysbench. https://github.com/akopytov/sysbench.
    [27] Ubunutu minimal wiki. https://wiki.ubuntu.com/Minimal. Accessed: 2022-06-15.
    [28] Intel qemu-lite github. https://github.com/intel/qemu-lite. Accessed: 2022-06-15.
    [29] Chi-Cheng Chuang et al. “A Compression Algorithm for Fluctuant Data in Smart Grid Database Systems”. 2013 Data Compression Conference. 2013, pp. 485–485.
    [30] Chien-Hung Chen, Che-Rung Lee, and Walter Chen-Hua Lu. “Smart in-car camera system using mobile cloud computing framework for deep learning”. Veh. Commun. 10 (2017), pp. 84–90.

    QR CODE