Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Introduction: Computable phenotypes are algorithms that translate clinical features into code that can be run against electronic health record (EHR) data to define patient cohorts. However, computable phenotypes that only make use of structured EHR data do not capture the full richness of a patient’s medical record. While natural language processing (NLP) methods have shown success in extracting clinical features from text, the use of such tools has generally been limited to research groups with substantial NLP expertise. Our goal was to develop open-source phenotyping software, Clinical Annotation Research Kit (CLARK), that enables clinical and translational researchers to use machine-learning based NLP for computable phenotyping without requiring deep informatics expertise. Methods: CLARK enables non-expert users to mine text using machine learning classifiers by specifying features for the software to match in clinical notes. Once the features are defined, the user-friendly CLARK interface allows the user to choose from a variety of standard machine learning algorithms (linear support vector machine, Gaussian Naïve Bayes, decision tree, and random forest), cross-validation methods, and the number of folds (cross-validation splits) to be used in evaluation of the classifier. Results: Example phenotypes where CLARK has been applied include pediatric diabetes (0.91/0.98 sensitivity/specificity), symptomatic uterine fibroids (0.81/0.54 PPV/NPV), nonalcoholic fatty liver disease (0.90/0.94 sensitivity/specificity), and primary ciliary dyskinesia (0.88 / 1.0 sensitivity/specificity). Discussion: In each of these use cases, CLARK allowed investigators to incorporate variables into their phenotype algorithm that would not be available as structured data. Moreover, the fact that non-expert users can get started with machine learning-based NLP with limited informatics involvement is a significant improvement over the status quo. Our hope is to disseminate CLARK to other organizations that may not have NLP or machine learning specialists available, enabling wider use of these methods.