Lev Sorokin

Lev Sorokin

Auto-generated profile, based on publicly available information

Lev Sorokin

Technical University of Munich, Munich, Germany and BMW Group, Munich, Germany

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 April 2024
Verified
Conference Paper
Open access
LSLev Sorokin
Lev Sorokin
TMTiziano Munaro
Tiziano Munaro
DSDamir Safin
Damir Safin
BLBrian Hsuan-Cheng Liao
Brian Hsuan-Cheng Liao
AMAdam Molin
Adam Molin
Search-based software testing (SBST) is an effective and efficient approach for testing automated driving systems (ADS). However, testing pipelines for ADS testing are particularly challenging as they involve integrating complex driving simulation platforms and establishing communication protocols and APIs with the desired search algorithm. This complexity prevents a wide adoption of SBST and thorough empirical comparative experiments with different simulators and search approaches. We present OpenSBT, an open-source, modular and extensible framework to facilitate the SBT of ADS. With OpenSBT, it is possible to integrate simulators with an embedded system under test, search algorithms and fitness functions for testing. We describe the architecture and show the usage of our framework by applying different search algorithms for testing Automated Emergency Braking Systems in CARLA as well as in the industrial and high-fidelity simulator Prescan in collaboration with our industrial partner DENSO. OpenSBT is available at https://git.fortiss.org/opensbt. A demo video is provided here: https://www.youtube.com/watch?v=qi_CTTzrk5s.

16 November 2024 in Empirical Software Engineering
Verified
Journal Article
Open access
PDF file
LSLev Sorokin
Lev Sorokin
DSDamir Safin
Damir Safin
SNShiva Nejati
Shiva Nejati
Search-based software testing (SBST) is a widely-adopted technique for testing complex systems with large input spaces, such as Deep Learning-enabled (DL-enabled) systems. Many SBST techniques focus on Pareto-based optimization where multiple objectives are optimized in parallel to reveal failures. However, it is important to ensure that identified failures are spread throughout the entire failure-inducing area of a search domain, and not clustered in a sub-region. This ensures that identified failures are semantically diverse and reveal a wide range of underlying causes. In this paper, we present a theoretical argument explaining why testing based on Pareto optimization is inadequate for covering failure-inducing areas within a search domain. We support our argument with empirical results obtained by applying two widely used types of Pareto-based optimization techniques, namely NSGA-II (an evolutionary algorithm) and OMOPSO (a swarm-based algorithm), to two DL-enabled systems: an industrial Automated Valet Parking (AVP) system and a system for classifying handwritten digits. We measure the coverage of failure-revealing test inputs in the input space using a metric, that we refer to as the Coverage Inverted Distance (CID) quality indicator. Our results show that NSGA-II and OMOPSO are not more effective than a naïve random search baseline in covering test inputs that reveal failures. We show that this comparison remains valid for failure-inducing regions of various sizes of these two case studies. Further, we show that incorporating a diversity-focused fitness function as well as a repopulation operator in NSGA-II improves, on average, the coverage difference between NSGA-II and random search by 52.1%. However, even after diversification, NSGA-II still does not outperform random testing in covering test inputs that reveal failures. The replication package for this study is available in a GitHub repository (Replication package. https://github.com/ast-fortiss-tum/coverage-emse-24 2024.

20 February 2026 in Empirical Software Engineering
Verified
Journal Article
Open access
PDF file
LSLev Sorokin
Lev Sorokin
MBMatteo Biagiola
Matteo Biagiola
ASAndrea Stocco
Andrea Stocco
Scenario-based testing with driving simulators is extensively used to identify failing conditions of automated driving assistance systems (ADAS) and reduce the amount of in-field road testing. However, existing studies have shown that repeated test execution in the same as well as in distinct simulators can yield different outcomes, which can be attributed to sources of flakiness or different implementations of the physics, among other factors. In this paper, we present , a novel approach to multi-simulation ADAS testing based on a search-based testing approach that leverages an ensemble of simulators to identify failure-inducing, simulator-agnostic test scenarios. During the search, each scenario is evaluated jointly on multiple simulators. Scenarios that produce consistent results across simulators are prioritized for further exploration, while those that fail on only a subset of simulators are given less priority, as they may reflect simulator-specific issues rather than generalizable failures. Our empirical study, which involves testing three lane-keeping ADAS with increasing complexity on different pairs of three widely used simulators, demonstrates that outperforms single-simulator testing by achieving, on average, a higher rate of simulator-agnostic failures by 66%. Compared to a state-of-the-art multi-simulator approach that combines the outcome of independent test generation campaigns obtained in different simulators, identifies on average up to $$3.4\times$$ more simulator-agnostic failing tests and higher failure rates. To avoid the costly execution of test inputs on which simulators disagree, we propose an enhancement of that leverages surrogate models to predict simulator disagreements and bypass test executions. Our results show that utilizing a surrogate model during the search does not only retain the average number of valid failures but also improves its efficiency in finding the first valid failure. These findings indicate that combining an ensemble of simulators during the search is a promising approach for the automated cross-replication in ADAS testing.

20 April 2024
Verified
Conference Paper
Open access
LSLev Sorokin
Lev Sorokin
NKNiklas Kerscher
Niklas Kerscher
In this paper, we present NSGA-II-SVM (Non-dominated Sorting Genetic Algorithm with Support Vector Machine Guidance), a novel learnable evolutionary and search-based testing algorithm that leverages Support Vector Machine (SVM) classification models to direct the search towards failure-revealing test inputs. Supported by genetic search, NSGA-II-SVM creates iteratively SVM-based models of the test input space, learning which regions in the search space are promising to be explored. A subsequent sampling and repetition of evolutionary search iterations allow to refine and make the model more accurate in the prediction. Our preliminary evaluation of NSGA-II-SVM by testing an Automated Valet Parking system shows that NSGA-II-SVM is more effective in identifying more critical test cases than a state of the art learnable evolutionary testing technique as well as naïve random search.

24 March 2024
Verified
Book Chapter
Closed access
PDF file
LSLev Sorokin
Lev Sorokin
RBRadouane Bouchekir
Radouane Bouchekir
TBTewodros A. Beyene
Tewodros A. Beyene
BLBrian Hsuan-Cheng Liao
Brian Hsuan-Cheng Liao
AMAdam Molin
Adam Molin

5 May 2026 in Journal of Systems and Software
Verified
Journal Article
Open access
PHPhilipp Habicht
Philipp Habicht
LSLev Sorokin
Lev Sorokin
ASAbdullah Saydemir
Abdullah Saydemir
KFKen Friedl
Ken Friedl
ASAndrea Stocco
Andrea Stocco