Executing contact-rich manipulation tasks necessitates the fusion of tactile and visual feedback. However, the distinct nature of these modalities poses significant challenges. In this paper, we introduce a system that leverages visual and tactile sensory inputs to enable dexterous in-hand manipulation. Specifically, we propose Robot Synesthesia, a novel point cloudbased tactile representation inspired by human tactile-visual synesthesia. This approach allows for the simultaneous and seamless integration of both sensory inputs, offering richer spatial information and facilitating better reasoning about robot actions. Comprehensive ablations are performed on how the integration of vision and touch can improve reinforcement learning and Sim2Real performance. Our project page is available at https://yingyuan0414.github.io/visuotactile/.

This research investigates the detection of the detailed static air-gap eccentricity position (SAGE) at both ends of non-salient pole synchronous generators (SG). Differently from the previous studies, the detection method proposed does not necessitate the installation of invasive sensors inside the generator and instead it is based on two groups of symmetrically distributed search coils placed at both end-sides outside the generator. The root mean square (RMS) value comparison in electromotive forces (EMFs) of the search coils is then employed to assess the exact SAGE position of each end-side. Numerical 3D finite element calculation together with an experimental campaign on a two-pole 3000rpm CS-5 prototype synchronous generator validates the effectiveness of the proposed method. It is shown that the maximum assessment error by using four external search coils at each end is 15.1%, while by setting more coils the error can be significantly reduced. Since this proposed method has no need to install invasive search coils inside the generator, it is more convenient for field testing and helps improve the cost-performance of online SAGE condition monitoring.

This article presents a comprehensive analysis on the end winding vibration characteristics due to different stator inter-turn short circuit (SISC) positions in synchronous generators. Different from other studies, this article not only pays attention to the effect of SISC on the end winding electromagnetic forces, but also considers the impact of the short circuit position, which is related to the static air-gap eccentricity, on the end winding vibrations at the meantime. The whole work is based on the qualitatively theoretical analysis, the finite element analysis (FEA), and the experimental validation on a 6-pole synchronous generator. The study shows that SISC will considerably enlarge the electromagnetic force and intensify the radial vibration of the end winding, especially at 100 Hz, 200 Hz and 300 Hz. The closer the short circuit position is to the minimum air-gap point, the more severe the end winding vibration will be.

Periodic impacts are considered as the important defect signatures of wind turbine bearing. However, it is difficult to separate the weak periodic impacts from collected signal under serious interference environment. Facing with this issue, the recently developed signal processing technique named adaptive chirp mode decomposition (ACMD) is firstly introduced into the fault diagnosis field of wind turbine bearing. And the influences of instantaneous frequency and weighting coefficient parameters of ACMD are researched through simulated fault signal. In additional, the L-kurtosis fluctuation spectrum (LFS) guided instantaneous frequency selection strategy and the cuckoo search algorithm (CSA) guided weighting coefficient determination strategy are respectively put forward to automatically confirm the optimal influencing parameters without requiring priori knowledge. Combined with these two specific strategies, an innovation optimized adaptive chirp mode decomposition (OACMD) method is further proposed for defect identification of wind turbine bearing. The experimental signals and the engineering signal are in turn adopted to demonstrate the reliability of this proposed diagnostic method. The obtained analysis results show that the informative target mode with obvious periodic impact signatures can be effectively extracted, and the key defect characteristic frequencies are able to be successfully identified. Furthermore, the conventional spectral kurtosis method, the minimum entropy deconvolution method and the singular spectrum decomposition method are respectively employed for performance comparison. And it is validated that the proposed OACMD method has outstanding advantage on weak defect detection of wind turbine bearing.

In order to achieve status identification of rolling bearing at early fault stage under variable speed conditions, a method called IEWT-based enhanced envelope order spectrum is proposed by improving empirical wavelet transform (IEWT) using cuckoo search algorithm (CSA) and combing IEWT with the compute order tracking (COT) approach and singular value ratio spectrum (SVRS) denoising technology. The inherent deficiencies of traditional EWT limit its application in fault signal processing significantly. In order to address this issue, the support interval of an empirical wavelet is adaptively determined by CSA to ensure the accuracy of signal decomposition, and an IEWT method is proposed. For a given bearing fault signal under fluctuating speed conditions, firstly COT is utilized as a preprocessing technology to convert the original time domain signal into the resampled angle domain signal without special hardware. Then, IEWT is used to separate the informative component effectively from the resampled signal. Due to the weak features and the heavy noises in the early fault signal, IEWT is unable to remove the whole interference contained in the separated component directly. Thus, SVRS denoising technology is applied on the envelope signal of the separated signal to further enhance the potential impulsive features and suppress the residual noises. Finally, the fault type of rolling bearing can be determined by analyzing the obtained enhanced envelope order spectrum. Both the simulated and the experimental signals are used to verify the effectiveness of the proposed method, and the comparisons between the proposed method and the EEWT, the EMD, and the spectral kurtosis methods are carried out. The results indicate the proposed method possesses significant advantages in bearing weak fault diagnosis under variable speed condition.

We propose a new dataset and a novel approach to learning hand-object interaction priors for hand and articulated object pose estimation. We first collect a dataset using visual teleoperation, where the human operator can directly play within a physical simulator to manipulate the articulated objects. We record the data and obtain free and accurate annotations on object poses and contact information from the simulator. Our system only requires an iPhone to record human hand motion, which can be easily scaled up and largely lower the costs of data and annotation collection. With this data, we learn 3D interaction priors including a discriminator (in a GAN) capturing the distribution of how object parts are arranged, and a diffusion model which generates the contact regions on articulated objects, guiding the hand pose estimation. Such structural and contact priors can easily transfer to real-world data with barely any domain gap. By using our data and learned priors, our method significantly improves the performance on joint hand and articulated object poses estimation over the existing state-of-the-art methods. The project is available at https://zehaozhu.github.io/ContactArt/.