Nvidia RTX is a proprietary hardware-accelerated ray tracing technology. Since implementation details are unknown, there were many questions in the development community about the hardware implementation: which stages of the ray tracing pipeline are hardware-accelerated and which of them can be efficiently implemented in software. In this paper we present the results of our experiments with RTX aimed at understanding the inner workings of this technology. We tried to address the questions intriguing developers around the world: What kind of acceleration can be obtained in practical applications compared to software implementations and what is the technological basis of this acceleration? How arduous is it to develop a rendering system with support for hardware acceleration, which at the same time can work on a GPU without RTX (using software implementation of ray tracing), or even perform calculations on a CPU? How effective is the software emulation of RTX (available on some of the previous generation Nvidia GPUs) and to what extent is it possible to bring its effectiveness to that of the hardware accelerated one? How hard will it be to create an analog of RTX if one needs to run application on GPUs from other manufacturers?
The rapid development of machine learning technologies in recent years has led to the emergence of CNN-based sensors or ML-enabled smart sensor systems, which are intensively used in medical analytics, unmanned driving of cars, Earth sensing, etc. In practice, the accuracy of CNN-based sensors is highly dependent on the quality of the training datasets. The preparation of such datasets faces two fundamental challenges: data quantity and data quality. In this paper, we propose an approach aimed to solve both of these problems and investigate its efficiency. Our solution improves training datasets and validates it in several different applications: object classification and detection, depth buffer reconstruction, panoptic segmentation. We present a pipeline for image dataset augmentation by synthesis with computer graphics and generative neural networks approaches. Our solution is well-controlled and allows us to generate datasets in a reproducible manner with the desired distribution of features which is essential to conduct specific experiments in computer vision. We developed a content creation pipeline targeted to create realistic image sequences with highly variable content. Our technique allows rendering of a single 3D object or 3D scene in a variety of ways, including changing of geometry, materials and lighting. By using synthetic data in training, we have improved the accuracy of CNN-based sensors compared to using only real-life data.
1 January 2021 in Proceedings of the Institute for System Programming of the RAS
The modern realistic computer graphics is based on physically correct lighting simulation. One of the main and computationally difficult problems is the calculation of light transport or global illumination, i.e. the distribution of light in a virtual scene taking into account multiple reflections, light scattering and various interactions of light with the scene objects. This problem is studied in hundreds of books and papers. They describe dozens of computational methods and their modifications. Our survey not only lists and briefly describes them but also gives some kind of a “map” of existing works that helps the reader to find one’s bearings, understand the advantages and drawbacks of these methods and thus select an appropriate basic approach. Special attention is paid to such characteristics of the methods as their robustness and universality with respect to models, the clarity of their verification, the possibility of efficient implementation on GPUs and the constraints imposed on the scene or illumination phenomena. In contrast to existing surveys, we try to analyze not only the efficiency of the methods but their limitations and the complexity of software implementation as well. In addition, the results of the authors' own numerical experiments that illustrate some of our conclusions are presented.
A software implementation of the graphics library OpenGL SC (Safety Critical) for the promising real-time onboard operating system JetOS for Russian aircraft is considered. The library...
We propose a novel high-level approach for software development on GPU using Vulkan API. Our goal is to speed-up development and performance studies for complex algorithms on GPU, which is quite difficult and laborious for Vulkan due to large number of HW features low level details. The proposed approach uses auto programming to translate ordinary C++ to optimized Vulkan implementation with automatic shaders generation, resource binding and fine-grained barriers placement. Our model is not general-purpose programming, but is extendible and customer-focused. For a single C++ input our tool can generate multiple different implementations of algorithm in Vulkan for different cases or types of hardware. For example, we automatically detect reduction in C++ source code and then generate several variants of parallel reduction on GPU: with optimization for different warp size, with or without atomics, using or not subgroup operations. Another example is GPU ray tracing applications for which we can generate different variants: pure software implementation in compute shader, using hardware accelerated ray queries, using full RTX pipeline. The goal of our work is to increase productivity of developers who are forced to use Vulkan due to various required hardware features in their software but still do care about cross-platform ability of the developed software and want to debug their algorithm logic on the CPU. Therefore, we assume that the user will take generated code and integrate it with hand-written Vulkan code.