Faaz Khan Mohammad

Faaz Khan Mohammad


Research Focus & Metrics

Academic Career Length

1Year (2026-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 November 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
FMFaaz Khan Mohammad
Faaz Khan Mohammad
A simulated psychiatric patient must identify what a learner is asking about before it can decide whether to disclose a case detail. We evaluated this matching step on 512 AI-generated questions across four fictional psychiatric cases. The benchmark writer saw descriptions of the case memories but had no access to the matching rules. We reserved 248 questions for development and 264 for testing. On the test set, keyword matching recalled 35.1% of expected risk memories. Adding a small sentence encoder raised recall to 72.7%, while the share of questions about non-risk memories incorrectly matched to a risk memory rose from 6.1% to 15.2%. The hybrid matcher took a median of 2.1 ms per question on a laptop CPU. Adding hand-set heads raised recall to 77.9% and the false-hit rate to 21.2%. All methods fell short of the 95% recall target. The hybrid matcher scored 90.7% on the development set, where we also used missed questions to write probes. A separate attempt to maximize recall reached 100% on development data at a 36.9% false-hit rate. Tested in scripted conversations, that configuration sometimes prompted unasked disclosures. The results show why evaluation should report both recall and false hits on held-out questions.