The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable tight frame proximal neural networks. Let $$\mathcal {H}$$H and $$\mathcal {K}$$K be real Hilbert spaces, $$b \in \mathcal {K}$$b∈K and $$T \in \mathcal {B} (\mathcal {H},\mathcal {K})$$T∈B(H,K) a linear operator with closed range and Moore–Penrose inverse $$T^\dagger $$T† . Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator $$\mathrm {Prox}:\mathcal {K}\rightarrow \mathcal {K}$$Prox:K→K the operator $$T^\dagger \, \mathrm {Prox}( T \cdot + b)$$T†Prox(T·+b) is a proximity operator on $$\mathcal {H}$$H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator $$\mathrm {Prox}= S_{\lambda }:\ell _2 \rightarrow \ell _2$$Prox=Sλ:ℓ2→ℓ2 and any frame analysis operator $$T:\mathcal {H}\rightarrow \ell _2$$T:H→ℓ2 that the frame shrinkage operator $$T^\dagger \, S_\lambda \, T$$T†SλT is a proximity operator on a suitable Hilbert space. The concatenation of proximity operators on $$\mathbb R^d$$Rd equipped with different norms establishes a PNN. If the network arises from tight frame analysis or synthesis operators, then it forms an averaged operator. In particular, it has Lipschitz constant 1 and belongs to the class of so-called Lipschitz networks, which were recently applied to defend against adversarial attacks. Moreover, due to its averaging property, PNNs can be used within so-called Plug-and-Play algorithms with convergence guarantee. In case of Parseval frames, we call the networks Parseval proximal neural networks (PPNNs). Then, the involved linear operators are in a Stiefel manifold and corresponding minimization methods can be applied for training of such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.

Learning neural networks using only few available information is an important ongoing research topic with tremendous potential for applications. In this paper, we introduce a powerful regularizer for the variational modeling of inverse problems in imaging. Our regularizer, called patch normalizing flow regularizer (patchNR), involves a normalizing flow learned on small patches of very few images. In particular, the training is independent of the considered inverse problem such that the same regularizer can be applied for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our model is indeed a maximum a posteriori approach. Numerical examples for low-dose and limited-angle computed tomography (CT) as well as superresolution of material images demonstrate that our method provides very high quality results. The training set consists of just six images for CT and one image for superresolution. Finally, we combine our patchNR with ideas from internal learning for performing superresolution of natural images directly from the low-resolution observation without knowledge of any high-resolution image.

Many recent superresolution methods are based on supervised learning. That means, that they require a large database of pairs of high- and low-resolution images as training data. However, for many applications, acquiring registered pairs of high and low resolution data or even imaging a large area with a high resolution is unrealistic. To overcome this problem, we introduce a Wasserstein patch prior for unsupervised superresolution of two- and three-dimensional images. In addition to the low-resolution observation, our method only requires one, possibly small, reference image which has a similar patch distribution as the high resolution ground truth. This assumption can e.g. be fulfilled when working with texture images or images of homogeneous material microstructures. The proposed regularizer penalizes the Wasserstein-2-distance of the patch distributions within the reconstruction and the reference image at different scales. We demonstrate the performance of the proposed method by applying it to two- and three-dimensional images of materials' microstructures.

Despite the rapid development of computational hardware, the treatment of large and high dimensional data sets is still a challenging problem. The contribution of this paper to the topic is twofold. First, we propose a Gaussian mixture model in conjunction with a reduction of the dimensionality of the data in each component of the model by principal component analysis, which we call PCA-GMM. To learn the (low dimensional) parameters of the mixture model we propose an EM algorithm whose M-step requires the solution of constrained optimization problems. Fortunately, these constrained problems do not depend on the usually large number of samples and can be solved efficiently by an (inertial) proximal alternating linearized minimization algorithm. Second, we apply our PCA-GMM for the superresolution of 2D and 3D material images based on the approach of Sandeep and Jacob. Numerical results confirm the moderate influence of the dimensionality reduction on the overall superresolution result.

In this paper, we consider maximum likelihood estimations of the degree of freedom parameterν, the location parameterμand the scatter matrixΣof the multivariate Studenttdistribution. In particular, we are interested in estimating the degree of freedom parameterνthat determines the tails of the corresponding probability density function and was rarely considered in detail in the literature so far. We prove that under certain assumptions a minimizer of the negative log-likelihood function exists, where we have to take special care of the case $\nu \rightarrow \infty $ν→∞ , for which the Studenttdistribution approaches the Gaussian distribution. As alternatives to the classical EM algorithm we propose three other algorithms which cannot be interpreted as EM algorithm. For fixedν, the first algorithm is an accelerated EM algorithm known from the literature. However, since we do not fixν, we cannot apply standard convergence results for the EM algorithm. The other two algorithms differ from this algorithm in the iteration step forν. We show how the objective function behaves for the different updates ofνand prove for all three algorithms that it decreases in each iteration step. We compare the algorithms as well as some accelerated versions by numerical simulation and apply one of them for estimating the degree of freedom parameter in images corrupted by Studenttnoise.

Exploiting image patches instead of whole images has proved to be a powerful approach to tackling various problems in image processing. Recently, Wasserstein patch priors (WPPs), which are based on the comparison of the patch distributions of the unknown image and a reference image, were successfully used as data-driven regularizers in the variational formulation of superresolution. However, for each input image, this approach requires the solution of a nonconvex minimization problem which is computationally costly. In this paper, we propose to learn two kinds of neural networks in an unsupervised way based on WPP loss functions. First, we show how convolutional neural networks (CNNs) can be incorporated. Once the network, called WPPNet, is learned, it can be very efficiently applied to any input image. Second, we incorporate conditional normalizing flows to provide a tool for uncertainty quantification. Numerical examples demonstrate the very good performance of WPPNets for superresolution in various image classes, even if the forward operator is known only approximately.