Free movement of production factors in the enlarged EU has led to immigration flows from East to West and from South to North with a significant impact on EU labor markets. Fiscal federalism also determined large immigration flows into EU area and affected unemployment rate in the EU countries. The aim of this paper is to investigate the impact of factors such as number of immigrants, tax on profits, social contributions, economic growth and population growth on unemployment rates in EU area using a panel quantile regression and an PMG-ARDL approach as robustness test during 1991–2020. The results show a positive association between population growth and unemployment rate, whereas the remaining exogenous factors are negatively associated with unemployment rate. Still, social contributions are statistically significant only for upper quantiles. The overall impact of social contribution on unemployment rate is positive as per PMG-ARDL estimations. We have also demonstrated that the immigrant flows impact on unemployment rate is very weak. The factors that are exerting the most significance influence on unemployment rate, are economic and population growth, followed by tax on profits. Findings support policy recommendation in EU area in terms of fiscal policy.