Monitoring the status of culture fish is an essential task for precision aquaculture using a smart underwater imaging device as a non-intrusive way of sensing to monitor freely swimming fish even in turbid or low-ambient-light waters. This paper developed a two-mode underwater surveillance camera system consisting of a sonar imaging device and a stereo camera. The sonar imaging device has two cloud-based Artificial Intelligence (AI) functions that estimate the quantity and the distribution of the length and weight of fish in a crowded fish school. Because sonar images can be noisy and fish instances of an overcrowded fish school are often overlapped, machine learning technologies, such as Mask R-CNN, Gaussian mixture models, convolutional neural networks, and semantic segmentation networks were employed to address the difficulty in the analysis of fish in sonar images. Furthermore, the sonar and stereo RGB images were aligned in the 3D space, offering an additional AI function for fish annotation based on RGB images. The proposed two-mode surveillance camera was tested to collect data from aquaculture tanks and off-shore net cages using a cloud-based AIoT system. The accuracy of the proposed AI functions based on human-annotated fish metric data sets were tested to verify the feasibility and suitability of the smart camera for the estimation of remote underwater fish metrics.

Precise fish metric estimation is essential in providing intelligent aquaculture farm decisions. Stereo vision has been widely used for size estimation. Still, many factors affect fish metrics accuracy using a low-cost underwater stereo camera, such as distance, ambient lighting, water velocity, and turbidity. Although such a system is affordable and energy-efficient, they are less accurate in estimating depths than its active counterparts. Since power source is always a problem in offshore aquaculture sites, energy-efficient devices are important. To deal with the accuracy problems of the camera, we propose an effective deep-learning-based object matching to optimize the fish metric estimation. In terms of the challenges of the underwater environment, an analysis of the accuracy of the fish 3D position calculation in the aquaculture cage based on the captured stereo camera images is performed. The analysis assumes a known geometrical configuration of the rectified camera system. The critical factor limiting the 3D fish metric estimation accuracy is the resolution of the computed depth maps of fish. An object-based matching is proposed for underwater fish tracking and depth computing to address this issue using reliable convolutional neural networks (CNNs). For each stereo video frame, an object classification and instance segmentation CNN separates the fish objects from their background. The fish objects are then cropped and matched using sub-pixel disparity computation of the video interpolation CNN. The calculated fish disparities and depth values are used for fish metric estimations. We also tracked each fish and computed the metrics across frames. The median metrics are calculated as the final result to reduce the noises introduced by the different gestures of the fish. Furthermore, underwater stereo video datasets with the actual metrics of sampled fish measured by humans are also constructed to verify the effectiveness of our approach. Our proposed method has less than a 5% error rate for fish length estimation.

This paper presents a low-cost and cloud-based autonomous drone system to survey and monitor aquaculture sites. We incorporated artificial intelligence (AI) services using computer vision and combined various deep learning recognition models to achieve scalability and added functionality, in order to perform aquaculture surveillance tasks. The recognition model is embedded in the aquaculture cloud, to analyze images and videos captured by the autonomous drone. The recognition models detect people, cages, and ship vessels at the aquaculture site. The inclusion of AI functions for face recognition, fish counting, fish length estimation and fish feeding intensity provides intelligent decision making. For the fish feeding intensity assessment, the large amount of data in the aquaculture cloud can be an input for analysis using the AI feeding system to optimize farmer production and income. The autonomous drone and aquaculture cloud services are cost-effective and an alternative to expensive surveillance systems and multiple fixed-camera installations. The aquaculture cloud enables the drone to execute its surveillance task more efficiently with an increased navigation time. The mobile drone navigation app is capable of sending surveillance alerts and reports to users. Our multifeatured surveillance system, with the integration of deep-learning models, yielded high-accuracy results.