簡易檢索 / 詳目顯示

回結果列表
研究生： 李軒毅
Lee, Syuan-Yi
論文名稱： 自適應虛擬樣本生成機制於概念飄移小樣本資料集之應用
Self-Adaptive Virtual Sample Generation Mechanism in Concept Drift with Small Sample Size
指導教授： 陳建良
Chen, James C.
口試委員: 陳子立
Chen, Tzu-Li
陳盈彥
Chen, Yin-Yann
學位類別： 碩士
Master
系所名稱： 工學院 - 工業工程與工程管理學系
Department of Industrial Engineering and Engineering Management
論文出版年： 2024
畢業學年度： 112
語文別： 中文
論文頁數： 54
中文關鍵詞： 概念飄移 、小樣本資料集 、虛擬樣本生成 、即時學習 、工業 4.0
外文關鍵詞： Concept Drift, Small Sample Size, Virtual Sample Generation (VSG), Just-In-Time Learning (JITL), Industry 4.0
相關次數： 點閱：53  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 當前製造業領域中，為了有效應對漸趨複雜的市場需求，生產過程正向著工業4.0與永續製造的實踐方向轉變。這些變革包含了更小批次的產品客製化與更具堅韌性的供應鏈需求等，也顯著縮短了產品生命週期和製造週期。於此時代趨勢下，大數據和分析技術在提高生產力、靈活性和資源效率方面變得愈加重要，並在預測性維護、快速生產系統重組、減少浪費和能源消耗以及資源分配優化中發揮關鍵作用。然而，實施這些技術面臨著數據流中的概念漂移和小樣本學習的挑戰。
    概念漂移發生在資料集中的共變量偏移導致無法準確解釋目標集的變異性時；小樣本學習則會導致過擬合和高變異性等問題。在製造現場中，受限於抽樣成本與時間，「大數據，小樣本」已成常態，而當訓練集中的樣本數量有限時，演算法可能會輸出帶有偏差的預測結果。
    因此，本研究旨在開發一種自適應的預測模型框架，以同時應對概念漂移和小樣本學習的挑戰為目標，通過採用即時學習（Just-In-Time Learning, JITL）來確保模型性能的維持和恢復，並利用虛擬樣本生成（Virtual Sample Generation, VSG）來減輕與小樣本學習相關的高變異與過擬合問題，最後將以上框架結合超參數最佳化的自適應框架，達到穩定的預測結果。從實驗結果表明，採用本研究提出之SAJITL和SAVSG可以顯著提高常見的模型性能，如 Linear Regression、XGBoost 與 Extreme Learning Machine (ELM)。此外，本研究提出的框架可以彈性整合進其他時間序列或虛擬樣本生成技術，並與先進的預測模型協調運作，以進一步提升在概念漂移和小樣本規模限制下的模型性能。

    Modern manufacturing is moving towards Industry 4.0 and sustainable practices to meet complex market demands. This shift involves smaller batch sizes, more resilient supply chains, and shorter product and manufacturing cycles. In this context, big data and analytics are crucial for boosting productivity, flexibility, and resource efficiency, playing key roles in predictive maintenance, system reconfiguration, waste reduction, and resource optimization. However, implementing these technologies faces challenges like concept drift and small sample size learning in data streams.
    Concept drift occurs when dataset covariates shift, reducing explanatory power for target variability. Small sample size learning leads to overfitting and high variance. In manufacturing, due to high sampling costs and time constraints, "big data, small sample" is common, often resulting in biased predictions.
    This study develops a novel prediction framework to tackle concept drift and small sample size learning concurrently. By adopting Just-In-Time Learning (JITL), model performance is maintained and recovered, while Virtual Sample Generation (VSG) reduces high variance and overfitting issues. Lastly, the framework integrates these methods with hyperparameter optimization for adaptive and stable results. Experiments show that the proposed SAJITL and SAVSG methods significantly improve the performance of common models, such as Linear Regression, XGBoost and Extreme Learning Machine (ELM). In addition, the proposed framework can seamlessly combine with other time-series and virtual sample generation techniques as demonstrated in this research , and be coordinated with advanced prediction algorithms to further enhance model performance under the constraints of concept drift and small sample size.

    摘要 I Abstract II 致謝 III Contents IV List of Tables VI List of Figures VIII Chapter 1 Introduction 1 1.1 Background and Motivation 1 1.2 Research Objective 2 1.3 Organization of Thesis 3 Chapter 2 Literature Review 4 2.1 Reviews for Concept Drift in Data Stream 4 2.1.1 The Types of Concept Drift 5 2.1.2 Detection, Understanding and Adaption of Concept Drift 6 2.2 Reviews for Small Sample Size Learning 11 Chapter 3 Problem Statement and Assumptions 15 Chapter 4 Methodology 18 4.1 Overview for Self-Adaptive Data Science Architecture 18 4.2 Data Preprocessing 20 4.2.1 Data Standardization 20 4.2.2 Feature Selection 21 4.3 Self-Adaptive Algorithm Selection 22 4.3.1 Self-Adaptive Just-In-Time Learning (SAJITL) Algorithm 22 4.3.2 Self-Adaptive Virtual Sample Generation (SAVSG) Algorithm 25 4.4 Learning Model Selection 27 4.5 Hyperparameter Optimization 30 4.6 Prediction and Evaluation 31 Chapter 5 Experiment 33 5.1 Data Preparation 33 5.2 Experiment Design 34 5.3 Comparison of Experiment Performance in CalHousing 36 5.3.1 Large Initial Historical Instance Size (n=2,000) 36 5.3.2 Small Initial Historical Instance Size (n=200) 39 5.4 Comparison of Experiment Performance in House8L 42 5.4.1 Large Initial Historical Instance Size (n=2000) 42 5.4.2 Small Initial Historical Instance Size (n=200) 45 5.5 Summary 48 Chapter 6 Conclusion 50 Reference 51

    Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019). Optuna: A Next-generation Hyperparameter Optimization Framework (arXiv:1907.10902). arXiv. https://doi.org/10.48550/arXiv.1907.10902
    Bifet, A., & Gavaldà, R. (2007). Learning from Time-Changing Data with Adaptive Windowing. 7. https://doi.org/10.1137/1.9781611972771.42
    Brain, D., & Webb, G. (2000). On the Effect of Data Set Size on Bias and Variance in Classification Learning. Proceedings of the Fourth Australian Knowledge Acquisition Workshop.
    Cai, H., Feng, J., Zhu, F., Yang, Q., Li, X., & Lee, J. (2021). Adaptive virtual metrology method based on Just-in-time reference and particle filter for semiconductor manufacturing. Measurement, 168, 108338. https://doi.org/10.1016/j.measurement.2020.108338
    Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785–794. https://doi.org/10.1145/2939672.2939785
    Chen, Z.-S., Zhu, B., He, Y.-L., & Yu, L.-A. (2017). A PSO based virtual sample generation method for small sample sets: Applications to regression datasets. Engineering Applications of Artificial Intelligence, 59, 236–243. https://doi.org/10.1016/j.engappai.2016.12.024
    Cheng, C., & Chiu, M.-S. (2004). A new data-based methodology for nonlinear process modeling. Chemical Engineering Science, 59(13), 2801–2810. https://doi.org/10.1016/j.ces.2004.04.020
    Dasu, T., Krishnan, S., Venkatasubramanian, S., & Yi, K. (2006). An Information-Theoretic Approach to Detecting Changes in Multi-Dimensional Data Streams. https://www.semanticscholar.org/paper/An-Information-Theoretic-Approach-to-Detecting-in-Dasu-Krishnan/36faada44c7c1d738313eda607572902e91b0a29
    Fan, S.-K. S., Chang, X.-W., & Lin, Y.-Y. (2022). Product-to-Product Virtual Metrology of Color Filter Processes in Panel Industry. IEEE Transactions on Automation Science and Engineering, 19(4), 3496–3507. https://doi.org/10.1109/TASE.2021.3124157
    Farrés, M., Platikanov, S., Tsakovski, S., & Tauler, R. (2015). Comparison of the variable importance in projection (VIP) and of the selectivity ratio (SR) methods for variable selection and interpretation. Journal of Chemometrics, 29(10), 528–536. https://doi.org/10.1002/cem.2736
    Gama, J., Medas, P., Castillo, G., & Rodrigues, P. (2004). Learning with Drift Detection. 8, 286–295. https://doi.org/10.1007/978-3-540-28645-5_29
    Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, H. (2014). A Survey on Concept Drift Adaptation. ACM Computing Surveys (CSUR), 46. https://doi.org/10.1145/2523813
    He, Y.-L., Wang, P.-J., Zhang, M.-Q., Zhu, Q.-X., & Xu, Y. (2018). A novel and effective nonlinear interpolation virtual sample generation method for enhancing energy prediction and analysis on small data problem: A case study of Ethylene industry. Energy, 147, 418–427. https://doi.org/10.1016/j.energy.2018.01.059
    Javaid, M., Haleem, A., Singh, R., Suman, R., & Santibanez Gonzalez, E. (2022). Understanding the adoption of Industry 4.0 technologies in improving environmental sustainability. Sustainable Operations and Computers, 3. https://doi.org/10.1016/j.susoc.2022.01.008
    Jebri, M. A., El Adel, E. M., Graton, G., Ouladsine, M., & Pinaton, J. (2016). Virtual metrology on semiconductor manufacturing based on Just-in-time learning*. IFAC-PapersOnLine, 49(12), 89–94. https://doi.org/10.1016/j.ifacol.2016.07.555
    Jen, C.-H., Fan, S.-K. S., & Lin, Y.-Y. (2022). Data-Driven Virtual Metrology and Retraining Systems for Color Filter Processes of TFT-LCD Manufacturing. IEEE Transactions on Instrumentation and Measurement, 71, 1–12. https://doi.org/10.1109/TIM.2022.3207807
    Kiraci, E., Franciosa, P., Warnett, J. A., & Williams, M. A. (2017). Non-reproducible alignment and fitting algorithm effects on Laser Radar measurement. 2017 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 1–5. https://doi.org/10.1109/I2MTC.2017.7969967
    Li, D.-C., Chen, C.-C., Chang, C.-J., & Lin, W.-K. (2012). A tree-based-trend-diffusion prediction procedure for small sample sets in the early stages of manufacturing systems. Expert Systems with Applications, 39(1), 1575–1581. https://doi.org/10.1016/j.eswa.2011.08.071
    Li, D.-C., Chen, S.-C., Lin, Y.-S., & Huang, K.-C. (2021). A Generative Adversarial Network Structure for Learning with Small Numerical Data Sets. Applied Sciences, 11, 10823. https://doi.org/10.3390/app112210823
    Li, D.-C., & Wen, I.-H. (2014). A genetic algorithm-based virtual sample generation technique to improve small data set learning. Neurocomputing, 143, 222–230. https://doi.org/10.1016/j.neucom.2014.06.004
    Lima, M., Neto, M., Filho, T. S., & de A. Fagundes, R. A. (2022). Learning Under Concept Drift for Regression—A Systematic Literature Review. IEEE Access, 10, 45410–45429. https://doi.org/10.1109/ACCESS.2022.3169785
    Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018). Learning under Concept Drift: A Review. IEEE Transactions on Knowledge and Data Engineering, 1–1. https://doi.org/10.1109/TKDE.2018.2876857
    Maciel, B. I. F., Santos, S. G. T. C., & Barros, R. S. M. (2015). A Lightweight Concept Drift Detection Ensemble. 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), 1061–1068. https://doi.org/10.1109/ICTAI.2015.151
    OpenML. (Online). https://www.openml.org/
    Psarommatis, F., & Kiritsis, D. (2018). A Scheduling Tool for Achieving Zero Defect Manufacturing (ZDM): A Conceptual Framework: IFIP WG 5.7 International Conference, APMS 2018, Seoul, Korea, August 26-30, 2018, Proceedings, Part II (pp. 271–278). https://doi.org/10.1007/978-3-319-99707-0_34
    Schlimmer, J. C., & Granger, R. H. (1986). Incremental learning from noisy data. Machine Learning, 1(3), 317–354. https://doi.org/10.1007/BF00116895
    Song, Y., Zhang, G., Lu, J., & Lu, H. (2017). A fuzzy kernel c-means clustering model for handling concept drift in regression. 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–6. https://doi.org/10.1109/FUZZ-IEEE.2017.8015515
    Su, X., Yan, X., & Tsai, C.-L. (2012). Linear regression. WIREs Computational Statistics, 4(3), 275–294. https://doi.org/10.1002/wics.1198
    Wang, J., Lu, S., Wang, S.-H., & Zhang, Y.-D. (2022). A review on extreme learning machine. Multimedia Tools and Applications, 81(29), 41611–41660. https://doi.org/10.1007/s11042-021-11007-7
    Wold, S., Johansson, E., & Cocchi, M. (1993). PLS: Partial Least Squares Projections to Latent Structures. DEU. https://iris.unimore.it/handle/11380/607273
    Xie, X., Sun, B., Li, X., Olsson, T., Maleki, N., & Ahlgren, F. (2023). Fuel Consumption Prediction Models Based on Machine Learning and Mathematical Methods. Journal of Marine Science and Engineering, 11, 738. https://doi.org/10.3390/jmse11040738
    Ying, X. (2019). An Overview of Overfitting and its Solutions. Journal of Physics: Conference Series, 1168(2), 022022. https://doi.org/10.1088/1742-6596/1168/2/022022
    Yu, X., He, Y., Xu, Y., & Zhu, Q. (2019). A Mega-Trend-Diffusion and Monte Carlo based virtual sample generation method for small sample size problem. Journal of Physics: Conference Series, 1325(1), 012079. https://doi.org/10.1088/1742-6596/1325/1/012079
    Zhang, Y., Ren, G., Liu, X., Gao, G., & Zhu, M. (2022). Ensemble learning-based modeling and short-term forecasting algorithm for time series with small sample. Engineering Reports, 4(5), e12486. https://doi.org/10.1002/eng2.12486
    Zhu, Q.-X., Zhang, X.-H., & He, Y.-L. (2020). Novel Virtual Sample Generation Based on Locally Linear Embedding for Optimizing the Small Sample Problem: Case of Soft Sensor Applications. Industrial & Engineering Chemistry Research, 59(40), 17977–17986. https://doi.org/10.1021/acs.iecr.0c01942
    Zliobaite, I., & Kuncheva, L. I. (2009). Determining the Training Window for Small Sample Size Classification with Concept Drift. 2009 IEEE International Conference on Data Mining Workshops, 447–452. https://doi.org/10.1109/ICDMW.2009.20

    QR CODE