The unique vegetation in riparian zones is fundamental for various ecological and socio-economic functions in these transitional areas. Sustainable management requires detailed spatial information about the occurring flora. Here, we present a Deep Learning (DL)-based approach for processing multimodal high-resolution remote sensing data (aerial RGB and near-infrared (NIR) images and elevation maps) to generate a classification map of the tidal Elbe and a section of the Rhine River (Germany). The ground truth was based on existing mappings of vegetation and biotope types. The results showed that (I) despite a large class imbalance, for the tidal Elbe, a high mean Intersection over Union (IoU) of about 78% was reached. (II) At the Rhine River, a lower mean IoU was reached due to the limited amount of training data and labelling errors. Applying transfer learning methods and labelling error correction increased the mean IoU to about 60%. (III) Early fusion of the modalities was beneficial. (IV) The performance benefits from using elevation maps and the NIR channel in addition to RGB images. (V) Model uncertainty was successfully calibrated by using temperature scaling. The generalization ability of the trained model can be improved by adding more data from future aerial surveys.