Significance: Mean-field games (MFGs) is an emerging field that models large populations of agents. They play a central role in many disciplines, such as economics, data science, and engineering. Since many applications come in the form of high-dimensional stochastic MFGs, numerical methods that use spatial grids are prone to the curse of dimensionality. To this end, we exploit the variational structure of potential MFGs and reformulate it as a generative adversarial network (GAN) training problem. This reformulation allays a bit the curse of dimensionality when solving high-dimensional MFGs in the stochastic setting, by avoiding spatial grids or uniform sampling in high dimensions, and instead utilizes the structure of the MFG and its connection with GANs.
1 January 2018 in Communications in Mathematical Sciences
Belief and opinion evolution in social networks (SNs) can aid in understanding how people influence others’ decisions through social relationships. As a large number of users are involved in SNs, the complexity of traditional optimization techniques is high as they deal with the interactions between users separately. Moreover, the state variable (opinion) is high-dimensional because a person usually has opinions about many different social issues. Incorporating classical opinion dynamics, we formulate the opinion evolution in SNs as a high-dimensional stochastic mean field game (MFG). Numerical methods for high-dimensional MFGs are practically non-existent because of the need for grid-based spatial discretization. Thus, we propose a machine-learning method to tractably solve high-dimensional stochastic MFGs. With this approach, solving MFGs can be regarded as a special case of training a generative adversarial network. In the simulation, we analyze the effect of random social issues and stubbornness on the opinion evolution. Moreover, with the Social Evolution data set, we show that the proposed algorithm can efficiently predict the diffusion of opinions in SNs.
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained ResNet20 from $ \sim 46\% $ to $ \sim 69\% $ under the state-of-the-art Iterative Fast Gradient Sign Method (IFGSM) based adversarial attack. When we combine this data-dependent activation with total variation minimization on adversarial images and training data augmentation, we achieve an improvement in robust accuracy by 38.9$ \% $ for ResNet56 under the strongest IFGSM attack. Furthermore, We provide an intuitive explanation of our defense by analyzing the geometry of the feature space.
2 April 2024 in IEEE Transactions on Artificial Intelligence
Data augmentation is a critical component in building modern deep-learning systems. In this paper, we propose MFG Augment , a novel data augmentation method based on the Mean-Field-Game (MFG) theory, that can synthesize a sequence of data between every two images or features. The central idea is to consider every image as a distribution over its pixel or feature space. Using Mean-field Game theory, we can generate a time-continuous “path” from one distribution to another so that the points along the “path” are augmented images or features. Empirically, the experiment results on MNIST, CIFAR-10, and ImageNet demonstrate that the proposed technology has better generalization ability and higher classification accuracy as compared to several benchmark methods. More importantly, our MFG Augment improves the test accuracy significantly when the dataset size is small. MFG Augment consistently shows better affinity and diversity scores, two important empirical metrics for evaluating the generalization of data augmentation techniques.
Recent observations have been made that bridge splitting methods arising from optimization, to the Hopf and Lax formulas for Hamilton-Jacobi Equations with Hamiltonians $H(p)$. This has produced extremely fast algorithms in computing solutions of these PDEs. More recent observations were made in generalizing the Hopf and Lax formulas to state-and-time-dependent cases $H(x,p,t)$. In this article, we apply a new splitting method based on the Primal Dual Hybrid Gradient algorithm (a.k.a. Chambolle-Pock) to nonlinear optimal control and differential games problems, based on techniques from the derivation of the new Hopf and Lax formulas, which allow us to compute solutions at points $(x,t)$ directly, i.e. without the use of grids in space. This algorithm also allows us to create trajectories directly. Thus we are able to lift the curse of dimensionality a bit, and therefore compute solutions in much higher dimensions than before. And in our numerical experiments, we actually observe that our computations scale polynomially in time. Furthermore, this new algorithm is embarrassingly parallelizable.