Computational analysis of therapeutic communication presents challenges in multi-label classification, severe class imbalance, and heterogeneous multimodal data integration. We introduce a bidirectional analytical framework addressing patient emotion recognition and provider behavior analysis. For patient-side analysis, we employ ClinicalBERT on human-annotated CounselChat (1482 interactions, 25 categories, imbalance 60:1), achieving a macro-F1 of 0.74 through class weighting and threshold optimization, representing a six-fold improvement over naive baselines and 6–13 point improvement over modern imbalance methods. For provider-side analysis, we process 330 YouTube therapy sessions through automated pipelines (speaker diarization, automatic speech recognition, temporal segmentation), yielding 14,086 annotated segments. Our architecture combines DeBERTa-v3-base with WavLM-base-plus through cross-modal attention mechanisms adapted from multimodal Transformer frameworks. On controlled human-annotated HOPE data (178 sessions, 12,500 utterances), the model achieves a macro-F1 of 0.91 with Cohen’s kappa of 0.87, comparable to inter-rater reliability reported in psychotherapy process research. On YouTube data, a macro-F1 of 0.71 demonstrates feasibility while highlighting annotation quality impacts. Cross-dataset transfer and systematic attention analyses validate domain-specific effectiveness and interpretability.

The computational analysis of therapeutic communication presents fundamental challenges in multi-label classification, severe class imbalance, and heterogeneous multimodal data integration. We introduce a comprehensive bidirectional framework that addresses patient emotion recognition and provider behavior analysis through advanced data mining techniques. For patient-side emotion recognition, we employ ClinicalBERT fine-tuned on human-annotated CounselChat comprising 1,482 counseling interactions across 25 emotion categories exhibiting class imbalance ratios reaching 60:1. Through frequency-stratified class weighting combined with dynamic per-class threshold optimization, we achieve macro-F1 of 0.74, representing a six-fold improvement over baseline multi-label approaches. Recognizing that patient emotion detection alone provides insufficient analytic utility, we extend our framework to provider-side behavior recognition using real-world psychotherapy sessions. We process 330 YouTube therapy sessions through an automated pipeline incorporating speaker diarization, automatic speech recognition, and temporal segmentation, yielding 14,086 annotated 10-second communication segments. Our provider-side architecture combines DeBERTa-v3-base for contextual text encoding with WavLM-base-plus for self-supervised audio representation learning, integrated through cross-modal attention mechanisms that learn content-dependent prosodic associations. On controlled human-annotated HOPE data comprising 178 sessions with approximately 12,500 utterances, the provider model achieves macro-F1 of 0.91 with Cohen's kappa of 0.87, comparable to inter-rater reliability reported among trained human annotators in psychotherapy process research, outperforming simple concatenation-based fusion by 12 percentage points. On automatically annotated YouTube data, the model achieves macro-F1 of 0.71, demonstrating feasibility of analyzing naturalistic clinical communication at scale while highlighting the performance gap between controlled and real-world scenarios.

Speech-language pathology (SLP) training programs face persistent challenges in providing sufficient opportunities for clinical placements or simulations that allow students to practice clinical communication with patients with communication disorders. We designed and implemented V.O.I.C.E., an AI-powered virtual patient system that enables realistic, open-ended clinical simulation for SLP students. V.O.I.C.E. integrates (1) a large language model (LLM) role-play agent designed to produce responses consistent with symptoms of post-stroke expressive aphasia (Broca’s aphasia), (2) a multimodal emotional expression pipeline that renders context-appropriate emotions of the virtual patient through coordinated speech prosody, facial expressions, and gestures, and (3) an LLM-based, rubric-guided debriefing module that generates structured formative feedback across core clinical communication competencies. We conducted a pilot mixed-methods, quasi-experimental evaluation with 11 graduate-level SLP students, where they completed three simulation sessions using the V.O.I.C.E system with progressive challenges. Results show significant gains in students' overall self-efficacy for interacting with Broca's aphasia patients in clinical settings. The self-efficacy gains were moderated by students’ perceptions of multimodal realism of the simulations. Qualitative findings highlight the system’s value as a low-stakes “bridge” for transition to clinical practice, which supports learning through iterative practice, reflection, and strategy refinement. This work provides a framework for AI-based simulation training applications that emphasizes interpersonal communication.

Accurate estimation of pasture biomass components from imagery is essential for sustainable grazing management and precision agriculture. Conventional methods such as destructive harvesting, rising plate meters, and remote sensing are limited by scalability, reliability, or the ability to disaggregate biomass by species. We propose a parameter-efficient multi-output regression framework predicting five biomass components (dry green, dry dead, dry clover, green dry matter, and total dry biomass) from high-resolution top-view pasture images. It employs a pretrained DINOv3 Vision Transformer backbone adapted via a dynamic, depth-aware Low-Rank Adaptation (LoRA) strategy, in which the adaptation rank and scaling factor increase exponentially with layer depth: early layers encoding generic visual primitives are minimally perturbed, while deeper layers receive stronger task-specific adaptation. This schedule is effective in low-data regimes, where uniform adaptation or full fine-tuning overfits. To handle rectangular image geometry, each image is split into two square halves processed as a dual-view stream with a contrastive alignment loss. The system ensembles ViT-Large and ViT-Huge backbones with test-time augmentation across five-fold cross-validation. On the CSIRO Image2Biomass benchmark, the full pipeline attains a cross-validated weighted R-squared of 0.81, indicating that depth-aware, parameter-efficient adaptation of large vision models is effective for non-invasive biomass estimation under data scarcity.

BACKGROUND Speech-language pathology (SLP) students need repeated opportunities to practice clinical communication with patients who have communication disorders, including aphasia. Existing simulations support case-based learning, but many provide limited open-ended dialogue and limited affective feedback during communication breakdowns. OBJECTIVE This study aimed to describe and pilot-evaluate Virtual Observation and Interactive Clinical Education (V.O.I.C.E.), an emotionally responsive artificial intelligence (AI) virtual patient system for aphasia-focused communication training in graduate SLP education. METHODS We conducted a single-group, mixed methods pilot study with 11 graduate SLP students. V.O.I.C.E. combines a large language model patient persona with poststroke Broca aphasia, a multimodal affect pipeline that renders patient emotion through speech prosody, facial expression, and gesture, and an automated rubric-guided debriefing report. Participants completed three approximately 10-minute simulations with progressively more severe aphasia presentations. Quantitative measures included pre-post self-efficacy for clinical communication and poststudy ratings of dialogue realism, voice realism, facial/gesture realism, usability, and educational value, which were analyzed descriptively. Qualitative measures including open-ended questions examining participants’ experience with and attitudes towards the AI patient, which were analyzed using hybrid deductive-inductive content analysis. RESULTS Overall self-efficacy improved significantly after the simulations, with a mean within-person gain of 1.09 points on a 6-point scale (95% CI 0.33-1.84; p=.03). Significant gains were observed for confidence in interviewing clients with aphasia (mean 2.33 to 3.89; change +1.56; p=.04) and responding to unexpected patient behavior (mean 3.73 to 5.09; change +1.36; p=.03). Confidence gains were positively associated with perceived dialogue realism (Spearman rho=0.79; p=.01), facial/gesture realism (rho=0.81; p=.01), and voice realism (rho=0.75; p=.02). Qualitatively, students described V.O.I.C.E. as a safe first step toward clinical practice that supported iterative practice, feedback-driven strategy refinement, metacognitive awareness, and rehearsal of emotionally challenging interactions; they also identified limitations related to speech recognition, emotional subtlety, feedback specificity, and the need for direct patient contact. CONCLUSIONS This pilot study suggests that emotionally responsive AI virtual patients may provide a useful opportunity for SLP students in their early stages of learning and practicing aphasia-focused clinical communication. The findings highlight the importance of multimodal realism, emotionally contingent patient behavior, and actionable debriefing. Larger controlled studies are needed to further evaluate the efficacy of the V.O.I.C.E. system.

Many of the questions computer vision asked in 2017 are still being asked today. Can a machine understand a scene the way a person does? Can it find a tumor in a tissue sample? Can it tell who someone is from the way they move or read? The core questions are familiar, but the way we answer them has changed almost completely, and in the process some of the questions have grown. This paper tells that story through one line of research that started in the era of hand-built features and early deep networks and continues into the era of foundation models and AI agents. We walk through what we built then, what we build now, and what changed in between. We describe six shifts in plain terms: where a model's knowledge comes from, how much labeled data it needs, how it handles context, how it combines different kinds of signals, how it is adapted to a new task, and what it finally produces. We close with the questions that did not go away, and that matter more now than before.