Recent trends in the Machine Learning (ML) and in particular Deep Learning (DL) domains have demonstrated that with the availability of massive amounts of time series, ML and DL techniques are competitive in time series forecasting. Nevertheless, the different forms of non-stationarities associated with time series challenge the capabilities of data-driven ML models. Furthermore, due to the domain of forecasting being fostered mainly by statisticians and econometricians over the years, the concepts related to forecast evaluation are not the mainstream knowledge among ML researchers. We demonstrate in our work that as a consequence, ML researchers oftentimes adopt flawed evaluation practices which results in spurious conclusions suggesting methods that are not competitive in reality to be seemingly competitive. Therefore, in this work we provide a tutorial-like compilation of the details associated with forecast evaluation. This way, we intend to impart the information associated with forecast evaluation to fit the context of ML, as means of bridging the knowledge gap between traditional methods of forecasting and adopting current state-of-the-art ML techniques.We elaborate the details of the different problematic characteristics of time series such as non-normality and non-stationarities and how they are associated with common pitfalls in forecast evaluation. Best practices in forecast evaluation are outlined with respect to the different steps such as data partitioning, error calculation, statistical testing, and others. Further guidelines are also provided along selecting valid and suitable error measures depending on the specific characteristics of the dataset at hand.
22 May 2013 in International Journal of Sports Medicine
The aims of this study were: to compare energy expenditure (EE) estimated from the existing GT3X accelerometer equations and EE measured with indirect calorimetry; to define new equations for EE estimation with the GT3X in youth, adults and older people; and to define GT3X vector magnitude (VM) cut points allowing to classify PA intensity in the aforementioned age-groups. The study comprised 31 youth, 31 adults and 35 older people. Participants wore the GT3X (setup: 1-s epoch) over their right hip during 6 conditions of 10-min duration each: resting, treadmill walking/running at 3, 5, 7, and 9 km · h−1, and repeated sit-stands (30 times · min−1). The GT3X proved to be a good tool to predict EE in youth and adults (able to discriminate between the aforementioned conditions), but not in the elderly. We defined the following equations: for all age-groups combined, EE (METs)=2.7406+0.00056 · VM activity counts (counts · min−1)−0.008542 · age (years)−0.01380 · body mass (kg); for youth, METs=1.546618+0.000658 · VM activity counts (counts · min−1); for adults, METs=2.8323+0.00054 · VM activity counts (counts · min−1)−0.059123 · body mass (kg)+1.4410 · gender (women=1, men=2); and for the elderly, METs=2.5878+0.00047 · VM activity counts (counts · min−1)−0.6453 · gender (women=1, men=2). Activity counts derived from the VM yielded a more accurate EE estimation than those derived from the Y-axis. The GT3X represents a step forward in triaxial technology estimating EE. However, age-specific equations must be used to ensure the correct use of this device.
29 June 2022 in Data Mining and Knowledge Discovery
We propose MultiRocket, a fast time series classification (TSC) algorithm that achieves state-of-the-art accuracy with a tiny fraction of the time and without the complex ensembling structure of many state-of-the-art methods. MultiRocket improves on MiniRocket, one of the fastest TSC algorithms to date, by adding multiple pooling operators and transformations to improve the diversity of the features generated. In addition to processing the raw input series, MultiRocket also applies first order differences to transform the original series. Convolutions are applied to both representations, and four pooling operators are applied to the convolution outputs. When benchmarked using the University of California Riverside TSC benchmark datasets, MultiRocket is significantly more accurate than MiniRocket, and competitive with the best ranked current method in terms of accuracy, HIVE-COTE 2.0, while being orders of magnitude faster.