Kaushik Roy

Mr. Kaushik Roy

orcid

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 August 2023 in Neural Networks
Verified
Journal Article
Open access
KRKaushik Roy
Kaushik Roy
CSChristian Simon
Christian Simon
PMPeyman Moghadam
Peyman Moghadam
Mehrtash HarandiMehrtash Harandi
Mehrtash Harandi
An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL.


23 November 2023 in Journal of Imaging
Verified
Journal Article
Open access
PDF file
KRKaushik Roy
Kaushik Roy
CSChristian Simon
Christian Simon
PMPeyman Moghadam
Peyman Moghadam
Mehrtash HarandiMehrtash Harandi
Mehrtash Harandi
Lifelong learning portrays learning gradually in nonstationary environments and emulates the process of human learning, which is efficient, robust, and able to learn new concepts incrementally from sequential experience. To equip neural networks with such a capability, one needs to overcome the problem of catastrophic forgetting, the phenomenon of forgetting past knowledge while learning new concepts. In this work, we propose a novel knowledge distillation algorithm that makes use of contrastive learning to help a neural network to preserve its past knowledge while learning from a series of tasks. Our proposed generalized form of contrastive distillation strategy tackles catastrophic forgetting of old knowledge, and minimizes semantic drift by maintaining a similar embedding space, as well as ensures compactness in feature distribution to accommodate novel tasks in a current model. Our comprehensive study shows that our method achieves improved performances in the challenging class-incremental, task-incremental, and domain-incremental learning for supervised scenarios.

13 June 2018 in IEEE Access
Verified
Journal Article
Open access
KRKaushik Roy
Kaushik Roy
RARifat Arefin
Rifat Arefin
FMFarkhod Makhmudkhujaev
Farkhod Makhmudkhujaev
OCOksam Chae
Oksam Chae
JKJaemyun Kim
Jaemyun Kim
Background subtraction technique is commonly used in foreground segmentation problem. Most of the existing color or intensity feature-based background subtraction methods suffer in non-stationary environments, such as presence of illumination variations. To address the issue, we introduce a new spatial feature descriptor that extracts the prominent directional information in a local neighborhood of a pixel. We use this local feature along with color information as the core component of a sample consensus-based model evolved from visual background extractor method. We also introduce an adaptive way to fuse color and local feature at each pixel to determine if the current observed features match to its corresponding background model while classification. The discriminative nature of our proposed local feature descriptor makes our model robust against the change of illumination. Extensive experiments on CDnet 2012 (changedetection.net) data set demonstrate that our background subtraction method outperforms many state-of-the-art methods.

27 November 2025 in IEEE Access
Verified
Journal Article
Open access
KRKaushik Roy
Kaushik Roy
Mehrtash HarandiMehrtash Harandi
Mehrtash Harandi
PMPeyman Moghadam
Peyman Moghadam
Continual Federated Learning (CFL) for 3D object classification requires learning generalizable representations across distributed clients while maintaining feature-space consistency as new classes emerge. However, existing CFL methods, which are primarily designed for 2D vision tasks, fail to address the geometric complexity of 3D point clouds and lack mechanisms to simultaneously mitigate intra-task and inter-task forgetting. To address these limitations, we propose DUAL3D-Fed, a dual-distillation framework that supports scalable, privacy-preserving, and semantically coherent 3D continual federated learning. First, we propose an embedding-space distillation, where we align local and global representation distributions using Sinkhorn divergence, a regularized optimal transport metric that enforces consistency across client models and over time. Next, we introduce semantic grounding withVision-Language Models (VLMs), which uses fixed language prompts to anchor the 3D feature space to a rich semantic prior. This promotes clustering of point cloud samples with similar textual descriptions and separation of semantically dissimilar ones. Finally, a Logit-space distillation is added, where we stabilize class predictions using a hybrid distillation scheme that incorporates buffered exemplars to retain historical knowledge while adapting to novel classes. Experiments on four synthetic and real-world 3D benchmarks under CFL settings show that DUAL3D-Fed improves classification accuracy, enhanced feature representations, and exhibits greater robustness to class drift and forgetting than baseline methods.

1 August 2023 in arXiv
Verified
Preprint
Open access
PDF file
KRKaushik Roy
Kaushik Roy
CSChristian Simon
Christian Simon
PMPeyman Moghadam
Peyman Moghadam
Mehrtash HarandiMehrtash Harandi
Mehrtash Harandi
An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL.