Large language models are beginning to enter peer review as tools for summarizing manuscripts, drafting evaluations, and reducing reviewer workload. Yet this use creates a security problem specific to evaluative settings: the manuscript being judged can also contain hidden instructions that shape the model’s judgment. We investigate this risk through indirect prompt injection, where hidden text embedded in a manuscript is processed like a benign prompt by a public chatbot during review generation. We introduce an Author-Reviewer-Organizer framework that distinguishes author-side manipulation from organizer-side integrity checks, and use it to evaluate both offensive payloads, which steer reviews toward positive or negative judgments, and integrity-probing payloads, which trigger refusal, watermarking, or external-site redirection. The redirection payload is introduced here as a novel integrity-check mechanism. In experiments on 100 OpenReview papers, two public chatbot systems, five payload families, two custom prompt approaches, three injection positions, and five repeated runs per condition, we generate 42000 model outputs. Hidden instructions are highly effective: positive steering, refusal, and external-site redirection exceed 98% success on both systems; watermarking remains high but less reliable, reaching 94.27% on ChatGPT and 88.17% on Gemini; and negative steering is near-ceiling on ChatGPT but lower on Gemini. Vulnerability varies with payload design, model, and document position. These findings show that current models exhibit “contextual blindness”: they do not reliably separate manuscript content from embedded control text. This weakness threatens the validity of AI-assisted review, while also creating a dual-use channel that organizers may repurpose for detection. Peer-review systems that rely on public LLMs therefore require stronger safeguards between document content and instructions before they can be trusted in evaluative workflows.
The GENERAL (GENerative, Explainable and Reasonable Artificial Learning) workshop, held at CHITALY 2023, delves into the advancements in General and Generative Artificial Intelligence (GGAI), with a focus on breakthroughs in natural language processing (NLP) and computer vision (CV). The workshop highlights the capabilities of Large Language Models (LLMs) and Latent Diffusion Models (LDMs) in generating human-like content across text and images. It emphasizes the importance of AI explainability, aiming to understand, explain, and control the complexities of these AI systems in terms of fairness, accountability, and transparency. The workshop encourages interdisciplinary collaboration across fields like HCI, psychology, social studies, and the arts to better understand AI’s societal and cultural impacts. Topics of interest include user perceptions of generative AIs, machine psychology, AI assistants, ethical issues in Generative AI, and safety and control mechanisms for large language models.