Recent work has explored the potential to adapt a pre-trained vision transformer (ViT) by updating only a few parameters so as to improve storage efficiency, called parameter-efficient transfer learning (PETL). Current PETL methods have shown that by tuning only 0.5% of the parameters, ViT can be adapted to downstream tasks with even better performance than full fine-tuning. In this paper, we aim to further promote the efficiency of PETL to meet the extreme storage constraint in real-world applications. To this end, we propose a tensorization-decomposition framework to store the weight increments, in which the weights of each ViT are tensorized into a single 3D tensor, and their increments are then decomposed into lightweight factors. In the fine-tuning process, only the factors need to be updated and stored, termed Factor-Tuning (FacT). On VTAB-1K benchmark, our method performs on par with NOAH, the state-of-the-art PETL method, while being 5x more parameter-efficient. We also present a tiny version that only uses 8K (0.01% of ViT's parameters) trainable parameters but outperforms full fine-tuning and many other PETL methods such as VPT and BitFit. In few-shot settings, FacT also beats all PETL baselines using the fewest parameters, demonstrating its strong capability in the low-data regime.
24 October 2017 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
A novel adaptive finite-time control method is investigated for the gyros systems in the presence of external disturbance. The fast nonsingular terminal sliding mode surface (NTSMS) is designed which has all advantages of NTSMS and general sliding mode surface together. Based on adaptive law and Lyapunov stability method, the novel proposed theorems are given and proven, and we can obtain that the synchronization errors converge to a small region including zero in finite time. Furthermore, it can be seen that the new adaptive control algorithm can guarantee that the chaotic gyros systems have some excellent properties, and all the signals are finite-time convergent. Finally, the new control scheme is verified by using a numerical example, and can achieve beneficial synchronization under the effectiveness of external disturbance of the gyros systems.
Motivated by tracking applications with sensor networks under non-Gaussian noise and intermittent observations, this paper considers a maximum correntropy unscented Kalman filter (MCUKF). MCUKF is based on maximum correntropy criterion (MCC) and unscented transformation (UT) which can deal with both non-Gaussian noise and intermittent observations. The intermittent observations are described by a binary sequence satisfying some properties. The MCC is used to deal with non-Gaussian noise and improves the robustness. Moreover, the arrival probabilities under non-Gaussian noise (shot noise and Gaussian mixture noise) and intermittent observations are given. The performance of the presented algorithm is verified by illustrating numerical examples.
16 November 2017 in IEEE/ASME Transactions on Mechatronics
Matching algorithm is the key technology in gravity-aided inertial navigation system. A matching algorithm combined with iterated closest contour point (ICCP) algorithm and Point Mass Filter (PMF) is proposed. The algorithm involves two-step matching processes. Firstly, the employment of point mass filter based on vehicle position variable can quickly and accurately obtain an instructional position given in a large initial position error. In particular, since the gravity anomaly database is tabulated in the form of a digital model, the parameter in filter model is estimated by Inertial Navigation System (INS) position short-term output and the base map resolution. Then ICCP algorithm with sliding window can be employed to further matching and real-time performance is guaranteed as well. Simulation and experimental tests show that compared to the conventional matching algorithms, the proposed algorithm is more reliable and achieves higher precision in the case of large initial position errors.
27 April 2018 in IEEE Transactions on Industrial Electronics
Gravity matching algorithm is a key of gravity-aided inertial navigation system (INS). The traditional matching algorithm only connects the measured gravity anomaly data to the position of the carrier according to a certain method to correct the inertial error. The gravitational field characteristic parameters should be also considered in the matching algorithm to improve the matching accuracy and reduce the number of mismatching. Therefore, based on the vector matching algorithm, a characteristic parame-ter matching algorithm is proposed. Meanwhile, this work pre-sents new methods to calculate the characteristic parameters and the range of particle filters to increase the accuracy of matching. The gravity anomaly value of each particle is calculated more accurately by the proposed method of continuous gravity anom-aly. Due to the high short-time accuracy of inertial navigation, the final trajectory is obtained by rigid transformation of a series of positions indicated by the INS corresponding to the trajectory. Simulation results prove that compared with the vector matching algorithm, the proposed method makes the matching results more accurate and more reliable.
21 April 2020 in IEEE Transactions on Industrial Electronics
Gravity-matching algorithm is a key to the gravity-aided inertial navigation system (INS). The traditional particle filter-based matching algorithm with a gravity sample vector is efficient. However, the range of particle filter and the probability model of the actual location are not specified in the algorithm. An improved particle filter-based matching algorithm with a gravity sample vector is proposed. Because of the high short-term accuracy of INS, the error range of INS in a short period is analyzed in a polar coordinate system in this algorithm. First, the attitude error angle model of INS is established. Relative angle error is proposed to calculate latitudes and longitudes of particles in the fan area at any position. Then a particle filter embedded in a particle filter is proposed to calculate the error range of the real position and establish the probability model of this position. Finally, in order to reduce the matching error, the relative displacements of the positions of the particles and the upper matching positions are added to the weights of the particles. Simulation results show that the proposed method has higher accuracy and better robustness.