This paper presents a new two-step approach for record linkage, focusing on the creation of high-quality training data in the first step. The approach employs the unsupervised random forest model as a similarity measure to produce a similarity score vector for record matching. Three constructions were proposed to select non-match pairs for the training data, with both balanced (symmetry) and imbalanced (asymmetry) distributions tested. The top and imbalanced construction was found to be the most effective in producing training data with 100% correct labels. Random forest and support vector machine classification algorithms were compared, and random forest with the top and imbalanced construction produced an F₁-score comparable to probabilistic record linkage using the expectation maximisation algorithm and EpiLink. On average, the proposed approach using random forests and the top and imbalanced construction improved the F₁-score by 1% and recall by 6.45% compared to existing record linkage methods. By emphasising the creation of high-quality training data, this new approach has the potential to improve the accuracy and efficiency of record linkage for a wide range of applications.

Analysing data can be quite a challenge sometimes due to the nature of the data and the vast options of methods and techniques that can be used on the data. In this study, for example, a six years Cleft Lip and Palate dataset were gathered on these patients’ conditions in the quest to identify the contributing factors for a successful pre-graft orthodontic treatment. The challenges faced was in the small number of datasets and imbalance sample class. Therefore, this study had taken a step back and tried to approach the dataset with a combination of unsupervised and supervised learning methods to tackle the challenges by incorporating clustering - for testing records creation and; resampling - for balancing sample class. We also observed if the auto-created testing records are replaceable with the manually selected testing records by looking at the performances of the classification models. Based on the feature that was selected, k-Means and PAM were implemented as the clustering algorithm using the Euclidean formula as the distance measure. Resampling was done using SMOTE and Random Forest as the classification model. When the comparison was done on the models, the ones that were fed by resampled training records showed an increase in the AUC values and decrease in the OOB error. Comparable results were also achieved between the training records produced by PAM and by manual selection as both models, based on the AUC values, was classified as excellent classification models.