In this study, we develop a framework for an intelligent and self-supervised industrial pick-and-place operation for cluttered environments. Our target is to have the agent learn to perform prehensile and non-prehensile robotic manipulations to improve the efficiency and throughput of the pick-and-place task. To achieve this target, we specify the problem as a Markov decision process (MDP) and deploy a deep reinforcement learning (RL) temporal difference model-free algorithm known as the deep Q-network (DQN). We consider three actions in our MDP; one is ‘grasping’ from the prehensile manipulation category and the other two are ‘left-slide’ and ‘right-slide’ from the non-prehensile manipulation category. Our DQN is composed of three fully convolutional networks (FCN) based on the memory-efficient architecture of DenseNet-121 which are trained together without causing any bottleneck situations. Each FCN corresponds to each discrete action and outputs a pixel-wise map of affordances for the relevant action. Rewards are allocated after every forward pass and backpropagation is carried out for weight tuning in the corresponding FCN. In this manner, non-prehensile manipulations are learnt which can, in turn, lead to possible successful prehensile manipulations in the near future and vice versa, thus increasing the efficiency and throughput of the pick-and-place task. The Results section shows performance comparisons of our approach to a baseline deep learning approach and a ResNet architecture-based approach, along with very promising test results at varying clutter densities across a range of complex scenario test cases.

In video transmission, maintaining high visual quality under variable network conditions, including bandwidth and efficiency, is essential for optimal viewer experience. Channel errors or malicious attacks during transmission can cause degradation in video quality, affecting its secure transmission and putting its confidentiality and integrity at risk. This paper presents a novel approach to enhancing objective video quality by integrating an energy-efficient forward error correction (FEC) technique into video encoding and transmission processes. Moreover, it ensures that the video contents remain secure and unintelligible to unauthorized parties. This is achieved by combining H.264/AVC syntax-based encryption and decryption algorithms with error correction during the video coding process to provide end-to-end confidentiality. Unlike traditional error correction strategies, our approach dynamically adjusts redundancy levels based on real-time network conditions, optimizing bandwidth utilization without compromising quality. The proposed framework is evaluated across full reference objective video quality metrics, demonstrating significant improvements in the peak signal-to-noise ratio (PSNR) and PSNR611 of the recovered videos. Experiments are carried out on multiple test video sequences with different video resolutions having various characteristics, i.e., colors, motions, and structures, and confirm that the FEC-based solution effectively detects and corrects packet loss and transmission errors without the need for retransmission, reducing the impact of channel noise and accidental disruptions on visual quality in challenging network environments. This study contributes to the development of resilient video transmission systems with reduced computational complexity of the codec and provides insights into the role of FEC in addressing quality degradation in modern multimedia applications where low latency is crucial.

This research study represents a polydexterous deep reinforcement learning-based pick-and-place framework for industrial clutter scenarios. In the proposed framework, the agent tends to learn the pick-and-place of regularly and irregularly shaped objects in clutter by using the sequential combination of prehensile and non-prehensile robotic manipulations involving different robotic grippers in a completely self-supervised manner. The problem was tackled as a reinforcement learning problem; after the Markov decision process (MDP) was designed, the off-policy model-free Q-learning algorithm was deployed using deep Q-networks as a Q-function approximator. Four distinct robotic manipulations, i.e., grasp from the prehensile manipulation category and inward slide, outward slide, and suction grip from the non-prehensile manipulation category were considered as actions. The Q-function comprised four fully convolutional networks (FCN) corresponding to each action based on memory-efficient DenseNet-121 variants outputting pixel-wise maps of action-values jointly trained via the pixel-wise parametrization technique. Rewards were awarded according to the status of the action performed, and backpropagation was conducted accordingly for the FCN generating the maximum Q-value. The results showed that the agent learned the sequential combination of the polydexterous prehensile and non-prehensile manipulations, where the non-prehensile manipulations increased the possibility of prehensile manipulations. We achieved promising results in comparison to the baselines, differently designed variants, and density-based testing clutter.