Flaviviruses circulate worldwide and cause a number of medically relevant human diseases, such as dengue, Zika, yellow fever, and tick-borne encephalitis (TBE). Serology plays an important role in the diagnosis of flavivirus infections, but can be impeded by antigenic cross-reactivities among flaviviruses. Therefore, serological diagnosis of a recent infection can be insufficiently specific, especially in areas where flaviviruses co-circulate and/or vaccination coverage against certain flaviviruses is high. In this study, we developed a new IgM assay format, which is well suited for the specific diagnosis of TBE, Zika and dengue virus infections. In the case of TBE and Zika, the IgM response proved to be highly specific for the infecting virus. In contrast, primary dengue virus infections induced substantial amounts of cross-reactive IgM antibodies, which is most likely explained by structural peculiarities of dengue virus particles. Despite the presence of cross-reactive IgM, the standardized nature and the quantitative read-out of the assay even allowed the serotype-specific diagnosis of recent dengue virus infections in most instances.