Object tracking is one of the most important problems in computer vision applications such as robotics, autonomous driving, and pedestrian movement. There has been a significant development in camera hardware where researchers are experimenting with the fusion of different sensors and developing image processing algorithms to track objects. Image processing and deep learning methods have significantly progressed in the last few decades. Different data association methods accompanied by image processing and deep learning are becoming crucial in object tracking tasks. The data requirement for deep learning methods has led to different public datasets that allow researchers to benchmark their methods. While there has been an improvement in object tracking methods, technology, and the availability of annotated object tracking datasets, there is still scope for improvement. This review contributes by systemically identifying different sensor equipment, datasets, methods, and applications, providing a taxonomy about the literature and the strengths and limitations of different approaches, thereby providing guidelines for selecting equipment, methods, and applications. Research questions and future scope to address the unresolved issues in the object tracking field are also presented with research direction guidelines.

Technologies that fall under the umbrella of Industry 4.0 can be classified into one of its four significant components: cyber-physical systems, the internet of things (IoT), on-demand availability of computer system resources, and cognitive computing. The success of this industrial revolution lies in how well these components can communicate with each other, and work together in finding the most optimised solution for an assigned task. It is achieved by sharing data collected from a network of sensors. This data is communicated via images, videos, and a variety of other signals, attracting unwanted attention of hackers. The protection of such data is therefore pivotal, as is maintaining its integrity. To this end, this paper proposes a novel image watermarking scheme with potential applications in Industry 4.0. The strategy presented is multipurpose; one such purpose is authenticating the transmitted image, another is curtailing the illegal distribution of the image by providing copyright protection. To this end, two new watermarking methods are introduced, one of which is for embedding the robust watermark, and the other is related to the fragile watermark. The robust watermark’s embedding is achieved in the frequency domain, wherein the frequency coefficients are selected using a novel mean-based coefficient selection procedure. Subsequently, the selected coefficients are manipulated in equal proportion to embed the robust watermark. The fragile watermark’s embedding is achieved in the spatial domain, wherein self-generated fragile watermark(s) is embedded by directly altering the pixel bits of the host image. The effective combination of two domains results in a hybrid scheme and attains the vital balance between the watermarking requirements of imperceptibility, security and capacity. Moreover, in the case of tampering, the proposed scheme not only authenticates and provides copyright protection to images but can also detect tampering and localise the tampered regions. An extensive evaluation of the proposed scheme on typical images has proven its superiority over existing state-of-the-art methods.

The presence of shadows in images significantly affects the performance of many computer vision tasks and visual processing applications such as object tracking, object classification and behaviour recognition. Most methods have been designed to detect shadows in specific situations, but they often fail to distinguish shadow points from the foreground object in many problematic situations, such as chromatic shadows, non-textured and dark surfaces, and foreground-background camouflage. In this paper, we propose a new feature-based image patch approximation and multi-independent sparse representation technique to tackle these environmental problems. In this method, two illumination-invariant features - binary patterns of local colour constancy (BPLCC) and light-based gradient matching (LGM) - are introduced, along with the intensityreduction histogram (IRH). These features are extracted from image patches and are used to construct two over-complete dictionaries for objects and shadows, respectively. Given a new image patch, its best approximation for a number of iterations is found from each dictionary. For each iteration, an independent class assignment is performed by finding its distances from the reference dictionaries. The patch is then assigned to a class based on its probability of occurrence. The proposed framework is evaluated on common shadow detection datasets, and it shows improved performance in terms of the shadow detection rate and discrimination rate compared with state-of-the-art methods.

Texture provides an important cue for many computer vision applications, and texture image classification has been an active research area over the past years. Recently, deep learning techniques using convolutional neural networks (CNN) have emerged as the state-of-the-art: CNN-based features provide a significant performance improvement over previous handcrafted features. In this study, we demonstrate that we can further improve the discriminative power of CNN-based features and achieve more accurate classification of texture images. In particular, we have designed a discriminative neural network-based feature transformation (NFT) method, with which the CNN-based features are transformed to lower dimensionality descriptors based on an ensemble of neural networks optimized for the classification objective. For evaluation, we used three standard benchmark datasets (KTH-TIPS2, FMD, and DTD) for texture image classification. Our experimental results show enhanced classification performance over the state-of-the-art.

Inexact graph matching algorithms have proved to be useful in many applications, such as character recognition, shape analysis, and image analysis. Inexact graph matching is, however, inherently an NP-hard problem with exponential computational complexity. Much of the previous research has focused on solving this problem using heuristics or estimations. Unfortunately, many of these techniques do not guarantee that an optimal solution will be found. It is the aim of the proposed algorithm to reduce the complexity of the inexact graph matching process, while still producing an optimal solution for a known application. This is achieved by greatly simplifying each individual matching process, and compensating for lost robustness by producing a hierarchy of matching processes. The creation of each matching process in the hierarchy is driven by an application-specific criterion that operates at the subgraph scale. To our knowledge, this problem has never before been approached in this manner. Results show that the proposed algorithm is faster than two existing methods based on graph edit operations. The proposed algorithm produces accurate results in terms of matching graphs, and shows promise for the application of shape matching. The proposed algorithm can easily be extended to produce a sub-optimal solution if required.

Robot hand-to-eye calibration is a necessary process for a robot arm to perceive and interact with its environment. Past approaches required collecting multiple images using a calibration board placed at different locations relative to the robot. When the robot or camera is displaced from its calibrated position, hand–eye calibration must be redone using the same tedious process. In this research, we developed a novel method that uses a semi-automatic process to perform hand-to-eye calibration with a stereo camera, generating a transformation matrix from the world to the camera coordinate frame from a single image. We use a robot-pointer tool attached to the robot’s end-effector to manually establish a relationship between the world and the robot coordinate frame. Then, we establish the relationship between the camera and the robot using a transformation matrix that maps points observed in the stereo image frame from two-dimensional space to the robot’s three-dimensional coordinate frame. Our analysis of the stereo calibration showed a reprojection error of 0.26 pixels. An evaluation metric was developed to test the camera-to-robot transformation matrix, and the experimental results showed median root mean square errors of less than 1 mm in the x and y directions and less than 2 mm in the z directions in the robot coordinate frame. The results show that, with this work, we contribute a hand-to-eye calibration method that uses three non-collinear points in a single stereo image to map camera-to-robot coordinate-frame transformations.