Mithilesh Kumar Singh

Mr. Mithilesh Kumar Singh

orcid

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 March 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
YKYoonsang Kim
Yoonsang Kim
ZAZainab Aamir
Zainab Aamir
MSMithilesh Kumar Singh
Mithilesh Kumar Singh
SBSaeed Boorboor
Saeed Boorboor
KMKlaus Mueller
Klaus Mueller
AKArie Kaufman
Arie Kaufman
We present Explainable XR, an end-to-end framework for analyzing user behavior in diverse eXtended Reality (XR) environments by leveraging Large Language Models (LLMs) for data interpretation assistance. Existing XR user analytics frameworks face challenges in handling cross-virtuality – AR, VR, MR – transitions, multi-user collaborative application scenarios, and the complexity of multimodal data. Explainable XR addresses these challenges by providing a virtuality-agnostic solution for the collection, analysis, and visualization of immersive sessions. We propose three main components in our framework: (1) A novel user data recording schema, called User Action Descriptor (UAD), that can capture the users' multimodal actions, along with their intents and the contexts; (2) a platform-agnostic XR session recorder, and (3) a visual analytics interface that offers LLM-assisted insights tailored to the analysts' perspectives, facilitating the exploration and analysis of the recorded XR session data. We demonstrate the versatility of Explainable XR by demonstrating five use-case scenarios, in both individual and collaborative XR applications across virtualities. Our technical evaluation and user studies show that Explainable XR provides a highly usable analytics solution for understanding user actions and delivering multifaceted, actionable insights into user behaviors in immersive environments.

2 May 2025 in Proceedings of the ACM on Human-Computer Interaction
Verified
Journal Article
Open access
TBTina Behzad
Tina Behzad
MSMithilesh Kumar Singh
Mithilesh Kumar Singh
ARAnthony J. Ripa
Anthony J. Ripa
KMKlaus Mueller
Klaus Mueller
The issue of fairness in decision-making is a critical one, especially given the variety of stakeholder demands for differing and mutually incompatible versions of fairness. Adopting a strategic interaction of perspectives provides an alternative to enforcing a singular standard of fairness. We present a web-based software application, FairPlay, that enables multiple stakeholders to debias datasets collaboratively. With FairPlay, users can negotiate and arrive at a mutually acceptable outcome without a universally agreed-upon theory of fairness. In the absence of such a tool, reaching a consensus would be highly challenging due to the lack of a systematic negotiation process and the inability to modify and observe changes. We have conducted user studies that demonstrate the success of FairPlay, as users could reach a consensus within about five rounds of gameplay, illustrating the application's potential for enhancing fairness in AI systems.

18 October 2020
Verified
Book Chapter
Closed access
PDF file
MSMithilesh Kumar Singh
Mithilesh Kumar Singh
WFWarren Mark Fernandes
Warren Mark Fernandes
MRMohammad Saad Rashid
Mohammad Saad Rashid

2 April 2021
Verified
Conference Paper
Closed access
MSMithilesh Kumar Singh
Mithilesh Kumar Singh
RDRaghavendra D
Raghavendra D
DPDeepak Pandian
Deepak Pandian
ASAnubhav Sadana
Anubhav Sadana
One of the biggest challenges of any Robotic Process Automation (RPA) tools is automating scenarios in tricky environments such as virtual machines (VMs) and remote desktops or when dealing with legacy applications where the technical know-how of the interfaces are limited. Typical approach to interacting with applications under regular scenarios is to use application technology specific APIs to build knowledge metadata. However, these approaches fail when it comes to automating in the black box environments because we cannot have technical understanding or APIs of the applications. Now, the only way to automate here is by taking the screenshot of the applications and understanding the controls available inside the screenshots using image processing algorithms. In this paper we present a novel approach for an end-to-end automation of such scenarios. We employ various AI routines to understand structural and semantic aspects of enterprise applications which in turn are used for building automations and we group all these techniques under a common umbrella of Surface Automation. Our approach involves using robust Optical Character Recognition (OCR) which internally uses LSTMs for reading texts like labels, Natural Language Processing (NLP) for semantic understanding of the page, Image Region Similarity for matching icons like home button and Computer Vision based Object Detection for recognizing predefined controls [text-fields, buttons, dropdown etc.].

6 November 2020
Verified
Conference Paper
Closed access
MSMithilesh Kumar Singh
Mithilesh Kumar Singh
_ One of the biggest challenges faced in deep learning community is the image data annotation, we know how data is essential for any machine learning problems but when it comes to object detection and segmentation (ODS) problems the data preparation process becomes extremely difficult and time-consuming. The data preparation becomes two steps process, first the data collection itself and second the data annotation. In this paper I present AADGen, a novel way of automatic data collection and annotation (DCA) for text detection and segmentation (TDS) problems. This method reduces the time taken for DCA to almost zero and provides error-free data annotation process. The DCA process will help us in building online learning model training by providing on demand continuous stream of training data. I have done extensive experiments on the data annotation process and achieved more than 99% annotation accuracy. I present a fast, scalable, customizable and error free data annotation approach.

21 August 2025 in Data
Verified
Journal Article
Open access
PDF file
MSMithilesh Kumar Singh
Mithilesh Kumar Singh
KMKlaus Mueller
Klaus Mueller
Tracking group membership dynamics over time is a persistent challenge in visual analytics, particularly when dealing with complex, multidimensional datasets. Existing tools often struggle to visualize dynamic group transitions while preserving attribute relationships and maintaining consistent group definitions. We present GroupView, a visual framework designed to explore temporal data and group dynamics to address this. GroupView enables users to slice data into time-based segments and create dynamic groupings, facilitating the identification of trends and patterns that may otherwise remain hidden. Its features include automated grouping based on data similarities, combinatorial grouping for richer insights, and custom grouping for tailored analysis. A heuristic user study involving visualization experts provided feedback on usability and analytical value, highlighting the strengths of GroupView in intuitive exploration and insight discovery. These features position GroupView as a valuable tool for analysts and researchers working with evolving datasets, offering new avenues for uncovering trends and tracking group-level changes over time.