Background: Automated ICD coding on medical texts via machine learning has been a hot topic. Related studies from medical field heavily relies on conventional bag-of-words (BoW) as the feature extraction method, and do not commonly use more complicated methods, such as word2vec (W2V) and large pretrained models likeBERT. This study aimed at uncovering the most effective feature extraction methods for coding models by comparingBoW,W2VandBERTvariants.Methods: We experimented with a Chinese dataset from Fuwai Hospital, which contains 6947 records and 1532 unique ICD codes, and a public Spanish dataset, which contains 1000 records and 2557 unique ICD codes. We designed coding tasks with different code frequency thresholds (denoted as $$f_s$$ ), with a lower threshold indicating a more complex task. Using traditional classifiers, we comparedBoW,W2VandBERTvariants on accomplishing these coding tasks.Results: When $$f_s$$ was equal to or greater than 140 for Fuwai dataset, and 60 for the Spanish dataset, theBERTvariants with the whole network fine-tuned was the best method, leading to aMicro-F1 of 93.9% for Fuwai data when $$f_s=200$$ , and aMicro-F1 of 85.41% for the Spanish dataset when $$f_s=180$$ . When $$f_s$$ fell below 140 for Fuwai dataset, and 60 for the Spanish dataset,BoWturned out to be the best, leading to aMicro-F1 of 83% for Fuwai dataset when $$f_s=20$$ , and aMicro-F1 of 39.1% for the Spanish dataset when $$f_s=20$$ . Our experiments also showed that both theBERTvariants andBoWpossessed good interpretability, which is important for medical applications of coding models.Conclusions: This study shed light on building promising machine learning models for automated ICD coding by revealing the most effective feature extraction methods. Concretely, our results indicated that fine-tuning the whole network of theBERTvariants was the optimal method for tasks covering only frequent codes, especially codes that represented unspecified diseases, whileBoWwas the best for tasks involving both frequent and infrequent codes. The frequency threshold where the best-performing method varied differed between different datasets due to factors like language and codeset.