A tactile image sensor employing a camera is capable of obtaining rich tactile information through image sequences with high spatial resolution. There have been many studies on the tactile image sensors from more than 30 years ago, and, recently, they have been applied in the field of robotics. Tactile image sensors can be classified into three typical categories according to the method of conversion from physical contact to light signals: Light conductive plate-based, marker displacement- based, and reflective membrane-based sensors. Other important elements of the sensor, such as the optical system, image sensor, and post-image analysis algorithm, have been developed. In this work, the literature is surveyed, and an overview of tactile image sensors employing a camera is provided with a focus on the sensing principle, typical design, and variation in the sensor configuration.

The frame rate of the digital high-speed video camera was 2000 frames per second (fps) in 1989, and has been exponentially increasing. A simulation study showed that a silicon image sensor made with a 130 nm process technology can achieve about 10¹⁰ fps. The frame rate seems to approach the upper bound. Rayleigh proposed an expression on the theoretical spatial resolution limit when the resolution of lenses approached the limit. In this paper, the temporal resolution limit of silicon image sensors was theoretically analyzed. It is revealed that the limit is mainly governed by mixing of charges with different travel times caused by the distribution of penetration depth of light. The derived expression of the limit is extremely simple, yet accurate. For example, the limit for green light of 550 nm incident to silicon image sensors at 300 K is 11.1 picoseconds. Therefore, the theoretical highest frame rate is 90.1 Gfps (about 10¹¹ fps)

Light in flight was captured by a single shot of a newly developed backside-illuminated multi-collection-gate image sensor at a frame interval of 10 ns without high-speed gating devices such as a streak camera or post data processes. This paper reports the achievement and further evolution of the image sensor toward the theoretical temporal resolution limit of 11.1 ps derived by the authors. The theoretical analysis revealed the conditions to minimize the temporal resolution. Simulations show that the image sensor designed following the specified conditions and fabricated by existing technology will achieve a frame interval of 50 ps. The sensor, 200 times faster than our latest sensor will innovate advanced analytical apparatuses using time-of-flight or lifetime measurements, such as imaging TOF-MS, FLIM, pulse neutron tomography, PET, LIDAR, and more, beyond these known applications.

This paper describes the concept, design and realization of a novel manipulator system for a multirotor aerial robot. The proposed manipulator system consists of three robotic arms attached to a multirotor frame. To overcome the limited payload capabilities in multirotor aerial platforms, the same set of on-board manipulators are used to realize various tasks, primarily the manipulation or grasping of objects of various sizes. Furthermore, they can provide aid in complex navigation tasks by doing physical, contact-based obstacle avoidance and are able to act as adaptive landing gear in uneven terrain. The design requirements and the analysis to realize these tasks is discussed. The proposal is supported with the successful demonstration of these tasks using the aerial multi manipulator system in an outdoor environment. GRAPHICAL ABSTRACT

We herein propose a long-reach aerial manipula-tion to allow the manipulation of objects several meters away from an unmanned aerial vehicle (UAV). The system consists of a multirotor platform with winch mechanism and wire-suspended gripper module equipped with a camera and swing-suppression device. The winch controls the distance of the wire-suspended device from the UAV. To stabilize the device suspended from the UAV by two wires, we developed a swing-suppression device using an IMU and ducted fans. Through experiments, we verified effectiveness of the swing-suppression system; that is, it successfully stabilized the wire-suspended device within a distance of 3.5 m from the UAV. Furthermore, we performed a pick-and-place task using a gripper that was lowered from the UAV by 3 m.

In the inspection work involving foodstuffs in food factories, there are cases where people not only visually inspect foodstuffs, but must also physically touch foodstuffs with their hands to find foreign or undesirable objects mixed in the product. To contribute to the automation of the inspection process, this paper proposes a method for detecting foreign objects in food based on differences in hardness using a camera-based tactile image sensor. Because the foreign objects to be detected are often small, the tactile sensor requires a high spatial resolution. In addition, inspection work in food factories requires a sufficient inspection speed. The proposed cylindrical tactile image sensor meets these requirements because it can efficiently acquire high-resolution tactile images with a camera mounted inside while rolling the cylindrical sensor surface over the target object. By analyzing the images obtained from the tactile image sensor, we detected the presence of foreign objects and their locations. By using a reflective membrane-type sensor surface with high sensitivity, small and hard foreign bodies of sub-millimeter size mixed in with soft food were successfully detected. The effectiveness of the proposed method was confirmed through experiments to detect shell fragments left on the surface of raw shrimp and bones left in fish fillets.