簡易檢索 / 詳目顯示

研究生: 葉眉湘
Yeh, Mei-Xiang
論文名稱: 基於Q-Learning增強的Tsetlin Machine演算法的低功耗推論機制在間歇運算系統中的應用
Q-Learning Enhanced Energy-Efficient Inference Mechanisms for Tsetlin Machines in Intermittent Systems
指導教授: 石維寬
Shih, Wei-Kuan
口試委員: 梁郁珮
Liang, Yu-Pei
陳彥廷
Chen, Yen-Ting
張原豪
Chang, Yuan-Hao
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 資訊工程學系
Computer Science
論文出版年: 2024
畢業學年度: 112
語文別: 英文
論文頁數: 26
中文關鍵詞: 間歇系統機器學習低功耗分類
外文關鍵詞: Tsetlin-Machine, Battery-less
相關次數: 點閱:49下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 本研究主要探討新型機器學習演算法- Tsetlin Machine 於間歇供電系統上之
    運算效能增強的方法。隨著網路技術進步,物聯網技術已大量遍布於生活之
    中,無電池供電的計算模式替小型裝置帶來前景,主因為可以降低維護成本,
    並提高佈署程度。此種裝置的能量來源由環境中收集,因此會面臨間歇供電之
    時刻,故需將揮發性記憶體中的資料轉移至非揮發性記憶體中做備份,以利下
    個電源週期可接續前次的運算。而Tsetlin Machine 演算法因其架構簡單,以及
    其採邏輯運算為主的特性,相較於傳統結構複雜的DNN模型,在推論速度上佔
    有優勢,且具有可競爭性的準確率,並有較高的可讀性。因此本文認為更適合
    佈署至微型裝置上做機器學習應用,有探討之價值。
    Tsetlin Machine 演算法所使用的記憶體空間與其準確度會呈正相關,因此在
    空間使用上有其挑戰性,另外,當面臨能量稀缺的環境,可能造成運算停滯不
    前的困境,故本研究深入探討Tsetlin Machine 的模型架構與推論行為,訓練出
    相對高運算複雜度但精度高,和相對低運算複雜度但精度稍降的兩種模型,並
    採用Q-learning 機制作為決策者,能夠綜合考量運算複雜度與系統能量變化,
    盡可能達到當前最佳推論效率,並有效使推運過程能跨越多個電源週期。最終
    以TI MSP430FR5994 微處理機做為實驗平台,模擬非連續供電之環境,實驗數
    據顯示本方法可有效達成間歇性運算,並以輕微犧牲準確率換取能成功推論的
    保證,解決先前他人之研究的弊端,因此本研究可作為未來物聯網技術發展之
    參考。


    This study aims to enhance the performance of a novel machine learning algorithm, Tsetlin Machine (TM), on intermittent power systems. With the advancement of embedded systems, ”battery-less” presents a promising future for tiny
    devices by reducing maintenance costs and being more environmentally friendly.
    These devices harvest energy from the environment, which is unpredictable and
    may encounter intermittent power supply. Therefore, it has to backup data from
    volatile memory to non-volatile memory so that the computation can work seamlessly in the next power cycle. TM stands out for its simple architecture and
    logic-based operations, offering fast inference and low energy consumption. Compared to traditional DNN algorithms, it achieves competitive accuracy and higher
    readability. We believes that TM is more suitable to deploy on tiny devices for
    classification applications, meriting further exploration.
    The memory usage of TM is positively correlated with its accuracy, posing
    challenges for memory utilization. In addition, in energy-scarce environments,
    computational progress may not be made. Hence, this study further explores
    TM’s model architecture and inference behavior, and had two trained models: one
    with high computational complexity but high accuracy, and another one with low
    computational complexity but slightly reduced accuracy. Q-learning algorithm
    is adopted as a decision-maker to balance computational complexity with energy
    variations, aiming to achieve optimal inference efficiency under intermittent power.
    TI MSP430FR5994 micro-controller was used as the experimental platform, with
    commonly-used dataset serving as benchmark. Results show that our method
    effectively performs under intermittent power with a slight trade-off in accuracy,
    addressing issues in previous research by others. Thus, it can be served as a
    foundational study for future developments in IoT technology.

    Abstract (Chinese) I Acknowledgements (Chinese) II Abstract III Contents IV List of Tables VI List of Figures VII 1 Introduction 1 2 Background and Motivation 6 2.1 Intermittent and Checkpoint . . . . . . . . . . . . . . . . . 6 2.2 Tsetlin Machine . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.1 Model Architecture . . . . . . . . . . . . . . . . . . . . 7 2.2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Q-Learning . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3.1 Mechanism of Q-Learning . . . . . . . . . . . . . . . . . 11 2.3.2 On-Policy v.s. Off-Policy . . . . . . . . . . . . .. . . . 12 2.4 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 12 3 Methodology 14 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.1 Encoding Method . . . . . . . . . . . . . . . . . . . . . 15 3.2.2 Two-Version Model : Trained Model and Pruning Model . . . . 15 3.2.3 QLTM . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4 Experiment 19 4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . 19 4.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.2.1 Under Continuous Power . . . . . . . . . . . . . . . . . . 20 4.2.2 Under Intermittent Power . . . . . . . . . . . . . . . . . 21 5 Conclusion 23 Bibliography 24

    [1] Saad Ahmed, Naveed Anwar Bhatti, Muhammad Hamad Alizai, Junaid Haroon Siddiqui, and Luca Mottola. Eicient intermittent computing with diferential checkpointing. Proc. of ACM LCTES., 2019.
    [2] A. Bakar, T. Rahman, A. Montanari, J. Lei, R. Shafik, and F. Kawsar. Logicbased intelligence for batteryless sensors. Proceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications, 2022.
    [3] A. Bakar, T. Rahman, R. Shafik, F. Kawsar, and A. Montanari. Adaptive intelligence for batteryless sensors using software-accelerated tsetlin machines.
    Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, 2022.
    [4] D. Balsamo, A. S. Weddell, G. V. Merrett, B. M. Al-hashimi, D. Brunelli, and
    L. Benini. Hibernus: Sustaining computation during intermittent supply for
    energy-harvesting systems. IEEE Embedded Systems Letters, 7:15–18, 2015.
    [5] Domenico Balsamo, Alex S Weddell, Anup Das, Alberto Rodriguez Arreola,
    Davide Brunelli, Bashir M Al-Hashimi, Geof V Merrett, and Luca Benini.
    Hibernus++: A self-calibrating and adaptive system for transiently-powered
    embedded devices. IEEE TCAD 35, 2016.
    24
    [6] Wei-Ming Chen, Tei-Wei Kuo, and Pi-Cheng Hsiu. Enabling failure-resilient
    intermittent systems without runtime checkpointing. IEEE TCAD 39, 12,
    2020.
    [7] Alexei Colin and Brandon Lucia. Chain: Tasks and channels for reliable
    intermittent programs. In Proc. of ACM OOPSLA, 2016.
    [8] Ole-Christoffer. Granmo. The tsetlin machine–a game theoretic bandit
    driven approach to optimal pattern recognition with propositional logic.
    arXiv:1804.01508, 2018.
    [9] Texas Instruments. Msp430fr5994 mcu.
    http://www.ti.com/product/MSP430FR5994., 2018.
    [10] Hrishikesh Jayakumar, Arnab Raha, Jacob R. Stevens, and Vijay Raghunathan. Energy-aware memory mapping for hybrid fram-sram mcus in
    intermittently-powered iot devices. ACM TECS 16, 2017.
    [11] C. Jin, Z. Allen-Zhu, S. Bubeck, and M. I. Jordan. Is q-learning provably
    efficient? Advances in neural information processing systems 31, 2018.
    [12] C. K. Kang, H. R. Mendis, C. H. Lin, M. S. Chen, and P. C. Hsiu. More is
    less: Model augmentation for intermittent deep inference. acm transactions on
    embedded computing systems. ACM Transactions on Embedded Computing
    Systems (TECS), 2022.
    [13] Y. LeCun. The mnist database of handwritten digits.
    https://yann.lecun.com/exdb/mnist/, 1998.
    [14] J. Lei, A. Wheeldon, R. Shafik, A. Yakovlev, and O. C. Granmo. From
    arithmetic to logic based ai: A comparative analysis of neural networks and
    25
    tsetlin machine. 27th IEEE international conference on electronics, circuits
    and systems (ICECS), 2020.
    [15] Jie Lei, Tousif Rahman, Rishad Shafik, Adrian Wheeldon, Alex Yakovlev,
    Ole-Christoffer Granmo, Fahim Kawsar, and Akhil Mathur. Low-power audio
    keyword spotting using tsetlin machines. Journal of Low Power Electronics
    and Applications 11.2, 2021.
    [16] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra,
    and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv
    preprint arXiv:1312.5602, 2013.
    [17] Shree K. Nayar, Daniel C. Sims, and Mikhail Fridberg. Towards self-powered
    cameras. In Proc. of IEEE ICCP. 1´s10, 2015.
    [18] Benjamin Ransford, Jacob Sorber, and Kevin Fu. Mementos: System support
    for long-running computation on rfid-scale devices. Proc. of ACM ASPLOS.,
    2011.
    [19] Gavin A. Rummery and Mahesan Niranjan. On-line q-learning using connectionist systems. Journal of Foo, 37, 1994.
    [20] Saha, Rupsa, Ole-Christoffer Granmo, and Morten Goodwin. Using tsetlin
    machine to discover interpretable rules in natural language processing applications. Expert Systems 40.4, 2023.
    [21] Seraj, Raihan, Jivitesh Sharma, and Ole-Christoffer Granmo. Tsetlin machine
    for solving contextual bandit problems. Advances in Neural Information Processing Systems 35, 2022.
    [22] C.J.C.H. Watkins and P Dayan. Q-learning. Mach Learn, 8:279–292, 1992

    QR CODE