針對時間序列模型的峰點預測成效評估方法｜國立清華大學博碩士論文庫

簡易檢索 / 詳目顯示

回結果列表

研究生：	官大鈞 Kuan, Ta-Chun
論文名稱：	針對時間序列模型的峰點預測成效評估方法 Evaluating Peak-Capturing Performance of Time Series Forecasting Algorithms
指導教授：	徐茉莉 Shmueli, Galit
口試委員:	林福仁 Lin, Fu-Ren 李曉惠 Lee, Hsiao-Hui
學位類別：	碩士 Master
系所名稱：	科技管理學院 - 服務科學研究所 Institute of Service Science
論文出版年：	2020
畢業學年度：	108
語文別：	英文
論文頁數：	47
中文關鍵詞：	時間序列、預測、模型表現評估、迴歸樹、峰點
外文關鍵詞：	time series, forecasting, performance evaluation, regression trees, peak periods
相關次數：	點閱：2 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

在許多時間序列預測的實務應用中，特別於高點與低點的預測，其準確度容易產生顯著的後續影響。例如，低估用電量的高峰值可能會使供電量無法負荷用電需求進而造成停電。低估旅遊旺季時的旅客數或客房需求量容易導致嚴重的財務損失。以及，低估在空氣污然高峰期期間的PM2.5含量更會間接影響到人們的健康。因此，評估模型在高低點時的預測表現時常會比評估整體平均的預測表現更為重要。習慣上，選取最佳時間序列預測模型的方式是將各種演算法針對序列進行建模後，並以整體平均為基礎的衡量公式對各模型進行評估，如RMSE、MAE和MAPE等衡量標準。接著則是各模型的比較與選擇。然而，上述方式並不適用於模型峰點預測成效的評估。
本研究提出一種新穎的模型評估方法，稱作為EvalTree。此評估方法主要是設計用於衡量模型在峰點（或低點）時的預測表現。在使用者可預先得知峰點發生時機的條件下，EvalTree可協助使用者評估模型的峰點預測表現。此方法是基於迴歸樹演算法，藉由預測誤差值發展的延伸應用。EvalTree能自動偵測序列上某些特殊時間點，尤其是高點或低點的發生時機。其原因在於在此時間點的實際值對於各模型而言是較為難以準確掌握的。此研究除了說明EvalTree方法的使用外，也會展示如何使用EvalTree衍生的預測表現差異比較表進行模型選擇，而此表是由各種衡量標準以及EvalTree中屬於極值的終端節點所組成。此研究使用M4時間序列預測比賽提供的資料演示EvalTree的使用方式以及用處。
最後，這份研究會建構一套EvalTree與預測表現差異比較表的使用方針，使用者可依據此方針有效地使用EvalTree。此外，也會列出未來可研究的方向作為後續改善EvalTree的目標。

In many time series forecasting applications, the accuracy of forecasts on peak or dip periods has significant implications. Under-forecasting peak electricity usage can result in blackouts; under-forecasting tourism demand in peak seasons can lead to severe losses; under-forecasting air quality peaks in PM2.5 can lead to health risks for some populations. As a result, evaluating the performance of a forecasting method to accurately forecast peaks or dips is often more important than overall model performance. The conventional approach to forecasting time series is fitting and comparing different forecasting models to a time series of interest. Comparison and evaluation of forecasting performance is done using average-based evaluation metrics such as RMSE, MAE, and MAPE. These conventions are sometimes not suitable for evaluating and comparing peak performance of different methods.
In this research we develop an evaluation approach called EvalTree, designed for measuring forecasting performance during peak periods. EvalTree can be used to help analysts evaluate and compare forecasting models for series that contain peaks/dips on known periods. This approach is based on a novel use of regression trees, applied to forecast errors. The tree automatically detects periods, typically peaks and dips, that are the most difficult for each method to forecast. In addition to presenting the tree itself, we produce comparison tables of performance measures to compare forecasting models on the most extreme tree terminal nodes. We demonstrate the use and usefulness of EvalTree on data from the M4 forecasting competition.
Finally, we construct a guideline that can help analysts utilize EvalTree, and provide directions for future enhancements.

摘要.............................................................. ii
Abstract......................................................... iii
List of Tables................................................... v
List of Figures.................................................. vi
Chapter 1: Introduction.......................................... 1
1.1 The Importance of Accurately Predicting Peak Period Values... 1
1.2 What Is A Peak?.............................................. 2
1.3 The Challenge of Evaluating Peak Performance................. 3
Chapter 2: Background and Notation............................... 5
2.1 Notation & Terminology....................................... 5
2.2 Standard Performance Measures................................ 7
2.3 Example: The M4 Competition.................................. 9
Chapter 3: Using Trees for Peak Performance Evaluation........... 13
3.1 Overview of Decision Trees................................... 13
3.2 Proposed New Method: EvalTree................................ 16
Chapter 4: Applying EvalTree to M4 dataset....................... 19
4.1 Monthly data................................................. 19
4.1.1 Data description........................................... 19
4.1.2 Preprocessing.............................................. 20
4.1.3 EvalTree results for a single forecaster................... 22
4.1.4 Comparing three forecasters................................ 26
4.2 Hourly data.................................................. 28
4.2.1 Data description........................................... 28
4.2.2 Preprocessing.............................................. 30
4.2.3 EvalTree results for a single forecaster................... 32
4.2.4 Comparing three forecasters................................ 37
4.3 Comparing a large set of forecasters......................... 38
Chapter 5: Conclusions and Future Work........................... 40
References....................................................... 44
Appendix......................................................... 46
                                

Akouemo, H. N., & Povinelli, R. J. (2014). Time series outlier detection and imputation. In 2014 IEEE PES General Meeting| Conference & Exposition (pp. 1-5).
Baker, M. (2016). Reproducibility crisis?. Nature, 533(26), 353-66.
Ball, P. (2018). High-profile journals put to reproducibility test. Nature, 20198(10.1038).
Breiman, L. (2017). Classification and regression trees. Routledge.
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behavior, 2(9), 637–644.
Casella, G., & Berger, R. L. (2001). Statistical inference (2nd Edition). Cengage Learning.
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 1-58.
Haben, S., Ward, J., Greetham, D. V., Singleton, C., & Grindrod, P. (2014). A new error measure for forecasts of household-level, high resolution electrical energy consumption. International Journal of Forecasting, 30(2), 246-256.
Hyndman, R. J. (2006). Another look at forecast-accuracy metrics for intermittent demand. Foresight: The International Journal of Applied Forecasting, 4(4), 43-46.
Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International journal of forecasting, 22(4), 679-688.
Kuan, T.-C., Wu, S.-W., Liao, C.-C., Ashouri, M., Shmueli, G. and Lin, C. (2019), Forecasting Daily Accommodation Occupancy for Supply Preparation by a Sharing Economy Platform, Proceedings of IEEE 5th International Conference on Big Data Intelligence and Computing (DATACOM), Kaohsiung, Taiwan, Nov 2019. DOI 10.1109/DataCom.2019.00030
https://conferences.computer.org/datacom/2019/pdfs/DataCom2019-3MYdIPKpqxiurNWZaDmspf/7roMDUSl0xEvNEjYtW5odN/6RHduX0lf1C2e2J0PnC1Xt.pdf
Lavin, A., & Ahmad, S. (2015). Evaluating Real-Time Anomaly Detection Algorithms--The Numenta Anomaly Benchmark. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) (pp. 38-44).
Makridakis, S. (1993). Accuracy measures: theoretical and practical concerns. International Journal of Forecasting, 9(4), 527-529.
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2018). The M4 Competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34(4), 802-808.
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2020). Predicting/hypothesizing the findings of the M4 Competition. International Journal of Forecasting, 36(1), 29-36.
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2020a). The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1), 54-74.
Makridakis, S., Wheelwright, S., & Hyndman, R. J. (1998). Forecasting: methods and applications (3rd Edition). John Wiley & Sons.
Malhotra, P., Vig, L., Shroff, G., & Agarwal, P. (2015). Long short term memory networks for anomaly detection in time series. In Proceedings of European symposium on artificial neural networks, computational intelligence, and machine learning, Bruges, Belgium, 2015, 89, 89-94.
Montero-Manso, P., Netto, C., & Talagala, T. (2018). M4comp2018: Data from the M4-Competition. R package version: 0.1.0.
Palshikar, G. (2009). Simple algorithms for peak detection in time-series. In Proceedings of 1st IIMA International Conference on Advanced Data Analysis, Business Analytics and Intelligence, Ahmedabad, India, 2009. (Vol. 122).
Shmueli, G., Bruce, P. C., Yahav, I., Patel, N. R., & Lichtendahl Jr, K. C. (2017). Data mining for business analytics: concepts, techniques, and applications in R. John Wiley & Sons.
Shmueli, G., Bruce, P. C., Gedeck, P., & Patel, N. R. (2019). Data mining for business analytics: concepts, techniques, and applications in Python. John Wiley & Sons.
Shmueli, G., & Lichtendahl Jr, K. C. (2016). Practical time series forecasting with r: A hands-on guide (2nd Ed.). Axelrod Schnall Publishers.
Simpson, R. W., & Layton, A. P. (1983). Forecasting Peak Ozone Levels. Atmospheric Environment, Volume 17, Issue 9, 1649-1654.
Tan, P. N., Steinbach, M., & Kumar, V. (2016). Introduction to data mining. Pearson Education India.
Taylor, J. W. (2010). Triple seasonal methods for short-term electricity demand forecasting. European Journal of Operational Research, 204(1), 139-152.

簡易檢索 / 詳目顯示

相關論文