Olives’ consumption produces copious agricultural byproducts that have accompanied humanity for millennia, but the increasing worldwide production complicates its management. Most wastes are generated during olive oil production in form of olive stones and other lignocellulosic derivatives. Industrial processes of chemical or physical nature to recover economically compounds from biomass residues are costly, difficult, and non-environmentally friendly. Cellulose, hemicellulose, and lignin biopolymers are the principal components of olive stones, which present interesting qualities as lignocellulosic fillers in polymeric composites. This review will summarize examples of composites based on thermoplastic polymers, such as polystyrene (PS), polylactide (PLA), polyvinyl chloride (PVC), polypropylene (PP), and polycaprolactone (PCL); thermosetting resins (phenol-formaldehyde, unsaturated polyesters, and epoxy) and acrylonitrile butadiene rubber/devulcanized waste rubber (NBR/DWR) blends focusing on the fabrication procedures, characterization, and possible applications. Finally, thanks to the wide disparity in polymer matrix types, the variability in applications is important, from adsorption to mechanical enhancement, showing the easiness and benefit of olive stone integration in many materials.