Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Electronic health records (EHR) is an important data resource for clinical studies and applications. Physicians or clinicians describe patients’ disorders or treatment procedures using free texts in EHR. The narrative information play an important role in patient treatment and clinical research. However, it is challenging to made machine understand the clinical narratives. Objective: This study aimed to automatically identify Chinese clinical entities from free texts in EHR, and make machine semantically understand diagnosis, test, body part, symptom, treatment and, etc. Methods: The data set we used for this study is the benchmark data set with human annotated Chinese electronic health records, released by CCKS 2017 CNER challenge task. Two machine learning (ML) models, conditional random fields (CRF) method and bidirectional LSTM-CRF, were applied to recognize clinical entities from Chinese EHR data. For training the CRF-based model, we selected features as bag of Chinese characters, part-of-speech (POS) tags, character types and the position of characters. For the bidirectional LSTM-CRF-based model, character embeddings and segmentation information were used as features. In addition, we also employed a dictionary-based approach as the baseline for performance evaluation purpose. Precision, Recall and the harmonic average of the precision and recall (F1 score) were used to evaluate the performance of the methods. Results: Experiments on the test set showed that our methods were able to automatically identify types of Chinese clinical entities such as diagnosis, test, symptom body part and treatment in one-round running. The identification overall performance of CRF and bidirectional LSTM-CRF achieved Precision of 0.9203 and 0.9112, Recall of 0.8709 and 0.8974, F1 score of 0.8949 and 0.9043 respectively. The result also indicated that our methods performed well on recognizing each type of clinical entities, in which the “symptom” type achieved the best with F1 score over 0.96. Moreover, as the number of features increased, F1 score of the CRF model increases from of 0.8547 to 0.8949. Conclusions: In this study, we employed two computational methods to simultaneously identify types of Chinese clinical entities from free texts in EHRs. Via training, it can effectively identify various types of clinical entities (e.g., symptom and treatment) with high accuracy. The deep learning model, bidirectional LSTM-CRF, can achieve better performance than the CRF model with little feature engineering. This study contributed to translating human-readable health information into machine-readable one.