Along with recent achievements in deep learning empowered by enormous amounts of training data, preserving privacy of an individual related to the gathered data is becoming an essential part of public data collection and publication. Advancements in deep learning threaten traditional image anonymization techniques with model inversion attacks that try to reconstruct the original image from the anonymized image. In this paper, we propose a privacy-preserving adversarial protector network (PPAPNet) as an image anonymization tool to convert an image into another synthetic image that is both realistic and immune to model inversion attacks. Our experiments on various datasets show that PPAPNet can effectively convert a sensitive image into a high-quality and attack-immune synthetic image.
There is a strong positive correlation between the development of deep learning and the amount of public data available. Not all data can be released in their raw form because of the risk to the privacy of the related individuals. The main objective of privacy-preserving data publication is to anonymize the data while maintaining their utility. In this paper, we propose a privacy-preserving semi-generative adversarial network (PPSGAN) that selectively adds noise to class-independent features of each image to enable the processed image to maintain its original class label. Our experiments on training classifiers with synthetic datasets anonymized with various methods confirm that PPSGAN shows better utility than other conventional methods, including blurring, noise-adding, filtering, and generation using GANs.
Illicit online communication, such as drug-dealing dialogues, is increasingly conducted through covert, context dependent language patterns that evade traditional detection techniques in South Korea. However, developing reliable AI based detection systems remains challenging due to the scarcity of real world training data in such sensitive domains. This paper proposes a novel persona-driven data augmentation framework using Large Language Model(LLM) to generate realistic synthetic drug-dealing dialogues. By encoding domain specific buyer and seller personas along with linguistic behaviour rules, the method produces contextually coherent and semantically diverse dialogues that reflect authentic communication styles. Evaluation results demonstrate that the augmented data preserves key stylistic features (high cosine similarity), maintains lexical diversity (TTR), improves fluency (perplexity), and enhances coherence and lexical richness (ROUGE-L), outperforming traditional augmentation method. Furthermore, statistical validation confirms the semantic consistency and stability of the generated data. These findings highlight the viability of LLM-based augmentation in low-resource, high-risk domains and suggest its potential transferability to other specialized NLP applications requiring context-preserving generation.
Video captioning is a task of describing the visual scene of a given video in natural language. There have been several lines of research focused on developing large-scale models in a transfer learning paradigm, with major challenge being the tradeoff between scalability and performance in limited environments. To address this problem, we propose a simple yet effective encoder–decoder-based video captioning model integrating transformers and CLIP, both of which are widely adopted in the vision and language domains, together with appropriate temporal feature embedding modules. Taking this proposal a step further, we also address the challenge of human-interactive video captioning, where the captions are tailored to specific information desired by humans. To design a human-interactive environment, we assume that a human offers an object or action in the video as a short prompt; in turn, the system then provides a detailed explanation regarding the prompt. We embed human prompts within an LSTM-based prompt encoder and leverage soft prompting to tune the model effectively. We extensively evaluated our model on benchmark datasets, demonstrating comparable results, particularly on the MSR-VTT dataset, where we achieve state-of-the-art performance with 4% improvement. In addition, we also show potential for human-interactive video captioning through quantitative and qualitative analysis.
Understanding complex multimodal documents remains challenging due to their structural inconsistencies and limited training data availability. We introduce \textit{DocsRay}, a training-free document understanding system that integrates pseudo Table of Contents (TOC) generation with hierarchical Retrieval-Augmented Generation (RAG). Our approach leverages multimodal Large Language Models' (LLMs) native capabilities to seamlessly process documents containing diverse elements such as text, images, charts, and tables without requiring specialized models or additional training. DocsRay's framework synergistically combines three key techniques: (1) a semantic structuring module using prompt-based LLM interactions to generate a hierarchical pseudo-TOC, (2) zero-shot multimodal analysis that converts diverse document elements into unified, text-centric representations using the inherent capabilities of multimodal LLMs, and (3) an efficient two-stage hierarchical retrieval system that reduces retrieval complexity from $O(N)$ to $O(S + k_1 \cdot N_s)$. Evaluated on documents averaging 49.4 pages and 20,971 textual tokens, DocsRay reduced query latency from 3.89 to 2.12 seconds, achieving a 45% efficiency improvement. On the MMLongBench-Doc benchmark, DocsRay-Pro attains an accuracy of 64.7%, substantially surpassing previous state-of-the-art results.
About Me
Machine Learning Researcher. Specialized in Computer Vision & Multimodal tasks. Currently focusing on Interactive Multimodal AI.
Machine Learning Researcher. Specialized in Computer Vision & Multimodal tasks. Currently focusing on Interactive Multimodal AI.