Stock price prediction is an important issue in the financial world, as it contributes to the development of effective strategies for stock exchange transactions. In this paper, we propose a generic framework employing Long Short-Term Memory (LSTM) and convolutional neural network (CNN) for adversarial training to forecast high-frequency stock market. This model takes the publicly available index provided by trading software as input to avoid complex financial theory research and difficult technical analysis, which provides the convenience for the ordinary trader of nonfinancial specialty. Our study simulates the trading mode of the actual trader and uses the method of rolling partition training set and testing set to analyze the effect of the model update cycle on the prediction performance. Extensive experiments show that our proposed approach can effectively improve stock price direction prediction accuracy and reduce forecast error.

With the rapid progress of urbanization, predicting citywide crowd flows has become increasingly significant in many fields, such as traffic management and public security. However, influenced by the complex spatiotemporal relations in raw data and other factors, such as events and weather, obtaining a precise prediction is challenging. Some previous works attempted to address this problem using various ways, such as autoregressive integrated moving average, vector auto-regression and some deep learning models. However, seldom can these methods comprehensively capture the spatiotemporal correlations. In this paper, we propose a novel spatio-temporal prediction model that is based on densely connected convolutional networks and attention long short-term memory (ST-DCCNAL), to simultaneously predict the inflow and outflow of the crowds in regions divided within a specific city. The ST-DCCNAL model consists of three parts: spatial part, external factors part and temporal part. In the spatial part, we employ densely connected convolutional networks to extract spatial characteristics at different levels. The external factors part utilizes a fully connected network to extract features from auxiliary information. In the last part, an attention-based long short-term memory module is leveraged to capture the temporal pattern. To demonstrate the practicality and effectiveness of the proposed model, we evaluate it using two separate real-world datasets of taxis in Beijing and bikes in New York. The experimental results confirm that the performance of our model is better than that of other baseline methods.

Online time series prediction is the mainstream method in a wide range of fields, ranging from speech analysis and noise cancelation to stock market analysis. However, the data often contains many outliers with the increasing length of time series in real world. These outliers can mislead the learned model if treated as normal points in the process of prediction. To address this issue, in this paper, we propose a robust and adaptive online gradient learning method, RoAdam (Robust Adam), for long short-term memory (LSTM) to predict time series with outliers. This method tunes the learning rate of the stochastic gradient algorithm adaptively in the process of prediction, which reduces the adverse effect of outliers. It tracks the relative prediction error of the loss function with a weighted average through modifying Adam, a popular stochastic gradient method algorithm for training deep neural networks. In our algorithm, the large value of the relative prediction error corresponds to a small learning rate, and vice versa. The experiments on both synthetic data and real time series show that our method achieves better performance compared to the existing methods based on LSTM.

With the growing scale of pre-trained language models (PLMs), full parameter fine-tuning becomes prohibitively expensive and practically infeasible. Therefore, parameter-efficient adaptation techniques for PLMs have been proposed to learn through incremental updates of pre-trained weights, such as in low-rank adaptation (LoRA). However, LoRA relies on heuristics to select the modules and layers to which it is applied, and assigns them the same rank. As a consequence, any fine-tuning that ignores the structural information between modules and layers is suboptimal. In this work, we propose structure-aware low-rank adaptation (SaLoRA), which adaptively learns the intrinsic rank of each incremental matrix by removing rank-0 components during training. We conduct comprehensive experiments using pre-trained models of different scales in both task-oriented (GLUE) and task-agnostic (Yelp and GYAFC) settings. The experimental results show that SaLoRA effectively captures the structure-aware intrinsic rank. Moreover, our method consistently outperforms LoRA without significantly compromising training efficiency.