Ensuring safe coexistence between human operators and industrial robot manipulators is a critical challenge in collaborative manufacturing environments. Existing approaches rely either on dedicated safety-rated hardware, which is expensive and difficult to retrofit, or on purely vision-based classifiers that discard the precise kinematic state available from the robot controller, leading to unresolved visual ambiguities when different joint configurations produce similar appearances from fixed camera viewpoints. Kinematics-only approaches, while precise, lack the spatial context needed to disambiguate configurations near workspace boundaries. We propose RGBJointsNet, a late-fusion multimodal deep learning classifier that combines RGB visual features extracted by a frozen EfficientNet-B2 convolutional backbone with a compact kinematic stream processing the 12-dimensional joint angle vector of a dual-UR5 robotic cell. The model maps each observation to one of five mutually exclusive workspace zones: rest (), nominal (), extended (), shared/collision-risk (), and joint-limit/singularity (). A dedicated simulation environment built on ROS 2 Humble Hawksbill and Gazebo Classic 11 was used to generate a labelled dataset of 54,309 frames and 162,927 RGB images from three calibrated overhead cameras, with analytic ground-truth labels derived from closed-form forward kinematics. Training on a CPU with a feature-caching strategy brings the per-epoch wall-clock time to seconds, making the approach tractable without GPU hardware. On the held-out test set, the model achieves 87.1% overall accuracy and a macro-averaged F1 score of 90.0%, with near-perfect recall of 99.3% for the safety-critical shared zone . The trained classifier is integrated as an ROS 2 inference node capable of running at 10 on a standard workstation. Our results demonstrate that joint angle information is a decisive complement to RGB imagery for fine-grained, safety-oriented workspace classification in simulation-derived settings.
This work presents a real-time, explainable Eye-to-Hand visual-feedback architecture for trajectory control of a planar 2RRR parallel manipulator. A fine-tuned YOLOv8n detector locates an end-effector marker, a calibrated homography converts image coordinates into Cartesian position, and the resulting visual error feeds a Cartesian PID correction before inverse kinematics generates the actuator commands. A previously validated ROS/Gazebo digital twin supplies reference trajectories, while the Dynamixel MX-64 encoders are retained only as an independent measurement source for training, calibration, and experimental validation. On 3,142 test images, the observer achieved a position RMSE of 3.455 mm (MAE = 2.804 mm). Wilcoxon tests detected no significant median offset on either Cartesian axis, although confidence intervals for the arithmetic mean differences excluded zero and indicate small mean offsets. The camera-in-the-loop controller was evaluated in 12 physical runs covering four trajectories; whole-trajectory errors ranged from 1.48 to 2.57 mm RMSE, with a pooled RMSE of 1.78 mm and a maximum error of 4.52 mm. Grad-CAM, LIME, and SHAP concentrated their attribution on the end-effector marker during successful detections and lost spatial agreement under synthetic occlusion. The framework combines real-time visual correction, independent encoder-based validation, digital-twin-assisted experimentation, and explainable perception. Physical validation was performed on the Dynamixel-equipped prototype; operation with actuators lacking joint-state telemetry remains to be investigated.