Rapidly progressive hepatocellular carcinoma (RPHCC) is a subset of hepatocellular carcinoma that demonstrates accelerated growth, and the radiographic features of RPHCC versus non-RPHCC have not been determined. The purpose of this retrospective study was to use baseline radiologic features and texture analysis for the accurate detection of RPHCC and subsequent improvement of clinical outcomes. We conducted a qualitative visual analysis and texture analysis, which selectively extracted and enhanced imaging features of different sizes and intensity variation including mean gray-level intensity (mean), standard deviation (SD), entropy, mean of the positive pixels (MPP), skewness, and kurtosis at each spatial scaling factor (SSF) value of RPHCC and non-RPHCC tumors in a computed tomography (CT) cohort of n = 11 RPHCC and n = 11 non-RPHCC and a magnetic resonance imaging (MRI) cohort of n = 13 RPHCC and n = 10 non-RPHCC. There was a statistically significant difference across visual CT irregular margins p = 0.030 and CT texture features in SSF between RPHCC and non-RPHCC for SSF-6, coarse-texture scale, mean p = 0.023, SD p = 0.053, MPP p = 0.023. A composite score of mean SSF-6 binarized + SD SSF-6 binarized + MPP SSF-6 binarized + irregular margins was significantly different between RPHCC and non-RPHCC (p = 0.001). A composite score ≥3 identified RPHCC with a sensitivity of 81.8% and specificity of 81.8% (AUC = 0.884, p = 0.002). CT coarse-texture-scale features in combination with visually detected irregular margins were able to statistically differentiate between RPHCC and non-RPHCC. By developing an image-based, non-invasive diagnostic criterion, we created a composite score that can identify RPHCC patients at their early stages when they are still eligible for transplantation, improving the clinical course of patient care.