This paper investigates the progressive damage and failure behavior of unidirectional graphite fiber-reinforced aluminum composites (CF/Al composites) under transverse and longitudinal tensile loadings. Micromechanical finite element analyses are carried out using different assumptions regarding fiber, matrix alloy, and interface properties. The validity of these numerical analyses is examined by comparing the predicted stress-strain curves with the experimental data measured under transverse and longitudinal tensile loadings. Assuming a perfect interface, the transverse tensile strength is overestimated by more than 180% and the transverse fracture induced by fiber failure is unrealistic based on the experimental observations. In fact, the simulation and experiment results indicate that the interface debonding arising from the matrix alloy failure dominates the transverse fracture, and the influence of matrix alloy properties on the mechanical behavior is inconspicuous. In the case of longitudinal tensile testing, however, the characteristic of interface bonding has no significant effect on the macroscopic mechanical response due to the low in-situ strength of the fibers. It is demonstrated that ultimate longitudinal fracture is mainly controlled by fiber failure mechanisms, which is confirmed by the fracture morphology of the tensile samples.