We introduce the sparse operator compression to compress a self-adjoint higher-order elliptic operator with rough coefficients and various boundary conditi
Randomized controlled experiment has long been accepted as the golden standard for establishing causal link and estimating causal effect in various scientific fields. Average treatment effect is often used to summarize the effect estimation, even though treatment effects are commonly believed to be varying among individuals. In the recent decade with the availability of "big data", more and more experiments have large sample size and increasingly rich side information that enable and require experimenters to discover and understand heterogeneous treatment effect (HTE). There are two aspects in HTE understanding, one is to predict the effect conditioned on a given set of side information or a given individual, the other is to interpret the HTE structure and summarize it in a memorable way. The former aspect can be treated as a regression problem, and the latter aspect focuses on concise summarization and interpretation. In this paper we propose a method that can achieve both at the same time. This method can be formulated as a convex optimization problem, for which we provide stable and scalable implementation.
Visual recognition is recently learned via either supervised learning on human-annotated image-label data or language-image contrastive learning with webly-crawled image-text pairs. While supervised learning may result in a more discriminative representation, language-image pretraining shows unprecedented zero-shot recognition capability, largely due to the different properties of data sources and learning objectives. In this work, we introduce a new formulation by combining the two data sources into a common image-text-label space. In this space, we propose a new learning paradigm, called Unified Contrastive Learning (UniCL) with a single learning objective to seamlessly prompt the synergy of two data types. Extensive experiments show that our UniCL is an effective way of learning semantically rich yet discriminative representations, universally for image recognition in zero-shot, linear-probe, fully finetuning and transfer learning scenarios. Particularly, it attains gains up to 9.2% and 14.5% in average on zero-shot recognition benchmarks over the language-image contrastive learning and supervised learning methods, respectively. In linear probe setting, it also boosts the performance over the two methods by 7.3% and 3.4%, respectively. Our study also indicates that UniCL stand-alone is a good learner on pure image-label data, rivaling the supervised learning methods across three image classification datasets and two types of vision backbones, ResNet and Swin Transformer. Code is available at https://github.com/microsoft/UniCL.
We propose a stochastic multiscale finite element method (StoMsFEM) to solve random elliptic partial differential equations with a high stochastic dimension. The key idea is to simultaneously upscale the stochastic solutions in the physical space for all random samples and explore the low stochastic dimensions of the stochastic solution within each local patch. We propose two effective methods to achieve this simultaneous local upscaling. The first method is a high order interpolation method in the stochastic space that explores the high regularity of the local upscaled quantities with respect to the random variables. The second method is a reduced-order method that explores the low rank property of the multiscale basis functions within each coarse grid patch. Our complexity analysis shows that compared with the standard FEM on a fine grid, the StoMsFEM can achieve computational saving in the order of $(H/h)^{d}/(\log(H/h))^k$, where $H/h$ is the ratio between the coarse and the fine gird sizes, $d$ is the physical dimension and $k$ is the local stochastic dimension. Several numerical examples are presented to demonstrate the accuracy and effectiveness of the proposed methods. In the high contrast example, we observe a factor of 2000 speed-up.
Suppose that $A \in \mathbb{R}^{N \times N}$ is symmetric positive semidefinite with rank $K \le N$. Our goal is to decompose $A$ into $K$ rank-one matrices $\sum_{k=1}^K g_k g_k^T$ where the modes $\{g_{k}\}_{k=1}^K$ are required to be as sparse as possible. In contrast to eigen decomposition, these sparse modes are not required to be orthogonal. Such a problem arises in random field parametrization where $A$ is the covariance function and is intractable to solve in general. In this paper, we partition the indices from 1 to $N$ into several patches and propose to quantify the sparseness of a vector by the number of patches on which it is nonzero, which is called patch-wise sparseness. Our aim is to find the decomposition which minimizes the total patch-wise sparseness of the decomposed modes. We propose a domain-decomposition type method, called intrinsic sparse mode decomposition (ISMD), which follows the "local-modes-construction + patching-up" procedure. The key step in the ISMD is to construct local pieces of the intrinsic sparse modes by a joint diagonalization problem. Thereafter a pivoted Cholesky decomposition is utilized to glue these local pieces together. Optimal sparse decomposition, consistency with different domain decomposition and robustness to small perturbation are proved under the so called regular-sparse assumption (see Definition 1.2). We provide simulation results to show the efficiency and robustness of the ISMD. We also compare the ISMD to other existing methods, e.g., eigen decomposition, pivoted Cholesky decomposition and convex relaxation of sparse principal component analysis [25] and [40].