Palpation is the most popular method of measuring keel bone damage on live birds, although it has been criticized for being subjective and inaccurate. The goals of this study were to examine intra- and inter-rater reliability when trained with feedback of accuracy, as well as determine the accuracy of portable radiography and sonography. Four evaluators palpated 50 103-week old Lohmann LSL-lite hens immediately following euthanasia. Of those birds, 34 were then radiographed, sonographed, and all 50 were re-palpated. Lastly, the keels were dissected and scored. The presence of deviations (DEV), fractures (FR), and tip fractures (TFR) was scored for each method. Reliability of palpation was analyzed using Cronbach’s Alpha (intra) and Fleiss’ Kappa (inter) tests. Radiography and Sonography scores were further compared with dissection scores to determine sensitivity and specificity. Initial inter-observer reliability was 0.39 DEV, 0.53 FR, and 0.12 TFR, with similar scores for the second round of palpation. Scores for intra-observer reliability ranged from 0.58–0.79 DEV, 0.66–0.90 FR, and 0.37–0.87 TFR. A high prevalence of TFR, but low assessor agreement, warrants the development of specialized training for the palpation of this area. Both radiography and sonography showed relatively high sensitivity for FR and TFR, but low for DEV. On the other hand, specificity was generally high across all damage types. Even with feedback, palpation reliability was poor. However, portable radiography and sonography show promise for detecting keel fractures.