The separate design of vision algorithms and motion control algorithms in existing vision servo systems limits their application in high-speed and high-precision industrial scenarios. To address this problem, a vision-motion codesign method is proposed for integrating visual servo systems with motion controllers via real-time Ethernet connections. During the engineering design phase of our proposal, we developed a control kernel architecture to clarify the system design logic. Then, we proposed a runtime module for real-time trajectory generation to facilitate the unified deployment of different planning methods in the kernel. On this basis, we implemented a self-adaptive trajectory generation synthesis method capable of dynamically adjusting the motion resources for real-time interpolation. The experimental results demonstrate that in visual servo static tracking experiments with the SCARA robot, our proposed method has a faster response time and smaller impact than other mainstream trajectory generation methods in 1 ms real-time interpolation cycles.

Fog computing provides computation, storage and network services for smart manufacturing. However, in a smart factory, the task requests, terminal devices and fog nodes have very strong heterogeneity, such as the different task characteristics of terminal equipment: fault detection tasks have high real-time demands; production scheduling tasks require a large amount of calculation; inventory management tasks require a vast amount of storage space, and so on. In addition, the fog nodes have different processing abilities, such that strong fog nodes with considerable computing resources can help terminal equipment to complete the complex task processing, such as manufacturing inspection, fault detection, state analysis of devices, and so on. In this setting, a new problem has appeared, that is, determining how to perform task scheduling among the different fog nodes to minimize the delay and energy consumption as well as improve the smart manufacturing performance metrics, such as production efficiency, product quality and equipment utilization rate. Therefore, this paper studies the task scheduling strategy in the fog computing scenario. A task scheduling strategy based on a hybrid heuristic (HH) algorithm is proposed that mainly solves the problem of terminal devices with limited computing resources and high energy consumption and makes the scheme feasible for real-time and efficient processing tasks of terminal devices. Finally, the experimental results show that the proposed strategy achieves superior performance compared to other strategies.

Driving fatigue can be detected by measuring drivers' heart rate with a wearable device or extracting their facial features with an RGB camera. However, a wearable device causes inconvenience and discomfort to the driver, and an RGB camera's detection accuracy may be affected by light, glasses, and head orientation. Furthermore, most existing methods ignored the temporal information of fatigue features and the relationship between the features, lowering recognition accuracy. Additionally, some existing fatigue detection methods focused on dealing with fatigue features with a temporal slice, ignoring temporal variations in the features. To address these problems, a single RGB-D camera is first used to extract three fatigue features: heart rate, eye openness level, and mouth openness level. More importantly, this paper proposes a novel multimodal fusion recurrent neural network (MFRNN), integrating the three features to improve the accuracy of driver fatigue detection. Specifically, a recurrent neural network (RNN) layer is applied in the MFRNN to obtain the temporal information of the features. Since the heart rate feature is a physiological signal extracted indirectly, it contains more noise and is fuzzier than the other features. To deal with the fuzziness and noise, we combine fuzzy reasoning with RNN to extract the temporal information of the heart rate. To identify the relationship between the features, we develop a new relationship layer containing a two-level RNN, for which the input is the temporal information of the features. Both the simulation and field experiment results show that the proposed method provides better performance than similar methods.

The introduction of Industry 4.0 and rapid development of Manufacturing Cyber-Physical Systems (MCPS), as well as the increasing demand for multi-variety, small batch and personalized customization, pose a huge challenge to the traditional manufacturing systems. In order to meet the production requirements for fast iteration and realize agile and efficient manufacturing resource allocation, this paper proposes an ontology-based resource reconfiguration method from the perspective of resource utilization. First, an intelligent device ontology that describes the intelligent manufacturing resource is established using the Web Ontology Language (OWL). On this basis, the relational database is associated with the ontology of manufacturing system, which makes the manufacturing resources be mapped to the model instances. Finally, we analyze the equipment reconfiguration of intelligent manipulator as an application case, which explains the proposed method for resource reconfiguration based on ontology, and verifies its feasibility in manufacturing. Lastly, this study provides a new method for reconfigurable research of manufacturing resources.