The self-aggregation of tau, a microtubule-binding protein, has been linked to the onset of Alzheimer’s Disease. Recent studies indicate that the disordered tau aggregates, or oligomers, are more toxic than the ordered fibrils found in the intracellular neurofibrillary tangles of tau. At present, details of tau oligomer interactions with lipid rafts, a model of neuronal membranes, are not known. Using molecular dynamics simulations, the lipid-binding events, membrane-damage, and protein folding of tau oligomers on various lipid raft surfaces were investigated. Tau oligomers preferred to bind to the boundary domains (Lod) created by the coexisting liquid-ordered (Lo) and liquid-disordered (Ld) domains in the lipid rafts. Additionally, stronger binding of tau oligomers to the ganglioside (GM1) and phosphatidylserine (PS) domains, and subsequent protein-induced lipid chain order disruption and beta-sheet formation were detected. Our results suggest that GM1 and PS domains, located exclusively in the outer and inner leaflets, respectively, of the neuronal membranes, are specific membrane domain targets, whereas the Lod domains are non-specific targets, of tau oligomers binding to neurons. The molecular details of these specific and non-specific tau bindings to lipid rafts may provide new insights into understanding membrane-associated tauopathies leading to Alzheimer’s Disease.