Composite materials with carbon nanotube and graphene additives have long been considered as exciting prospects among nanotechnology applications. However, after nearly two decades of work in the area, questions remain about the practical impact of nanotube and graphene composites. This uncertainty stems from factors that include poor load transfer, interfacial engineering, dispersion, and viscosity-related issues that lead to processing challenges in such nanocomposites. Moreover, there has been little effort to identify selection rules for the use of nanotubes or graphene in composite matrices for specific applications. This review is a critical look at the status of composites for developing high-strength, low-density, high-conductivity materials with nanotubes or graphene. An outlook of the different approaches that can lead to practically useful nanotube and graphene composites is presented, pointing out the challenges and opportunities that exist in the field.