Flavonoids are a group of polyphenols that provide health-promoting benefits upon consumption. However, poor bioavailability has been a major hurdle in their use as drugs or nutraceuticals. Low bioavailability has been associated with flavonoid interactions at various stages of the digestion, absorption and distribution process, which is strongly affected by their molecular structure. In this review, we use structure–activity/property relationship to discuss various flavonoid interactions with food matrices, digestive enzymes, intestinal transporters and blood proteins. This approach reveals specific bioactive properties of flavonoids in the gastrointestinal tract as well as various barriers for their bioavailability. In the last part of this review, we use these insights to determine the effect of different structural characteristics on the overall bioavailability of flavonoids. Such information is crucial when flavonoid or flavonoid derivatives are used as active ingredients in foods or drugs.