Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: The U.S. National Cancer Institute (NCI) developed software to collect symptomatic adverse events directly from patients participating in cancer clinical trials. This software administers surveys to patients using items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) via web-based or automated telephone interfaces, and enables management of survey administration and the resultant data by professionals (clinicians and research associates). Objective: The purpose of this study was to iteratively evaluate and improve the usability of the PRO-CTCAE software in both patients and professional users. Methods: Heuristic evaluation of software functionality was followed by semi-scripted, think-aloud protocols in sequential rounds of usability testing among patients, clinicians, and research associates. Patient testing was conducted both in clinic and remotely (i.e., from home) for both the web and telephone interfaces. Software was refined between rounds and retested. Results: Heuristic evaluation identified functionality problems and deviations from best practices across 10 standardized categories, which informed initial software improvement. Subsequently, user-based testing was conducted among 169 patients with cancer (54% female, 67% white, 26% high school education or less, 19% with limited computer experience, 18% older than 65), and 47 professionals (26% physicians, 26% nurses, 49% research associates) at three cancer centers in two rounds of usability testing. Software modifications between rounds addressed identified issues, including difficulty using radio buttons, absence of survey progress indicators, and login problems (for patients); and scheduling of patient surveys (for professionals). Following modifications, there were improvements in task performance for both patients and professional users. Conclusions: Software modifications, informed by rigorous assessment, rendered a usable system which is now in use in multiple NCI-sponsored multicenter cancer clinical trials. Clinical Trial: ClinicalTrials.gov Identifier: NCT01031641