Previous research has shown that readers experience stronger immersion while reading first-person narratives than third-person narratives, but whether this difference in processing is time-sensitive remains unclear. We report on a self-paced reading experiment in which French participants read short stories in first and in third person. Additionally, participants were directly addressed with either formal or informal second-person pronouns in the final sentence of the narrative, as well as in a subsequent (off-line) questionnaire soliciting post-story immersion reports. The suitability of a particular pronoun of address depends on the social context, and misplaced use can impact the way in which people perceive and assess a particular situation. We did not find significant differences between reading times with first- or third-person pronouns, but participants reported higher immersion and emotional engagement after reading first-person stories than third-person stories. Moreover, the effect of story perspective on reported immersion only occurred when participants were addressed with formal second-person pronouns. We take these findings as evidence for an effect of first- versus third-person pronouns on immersion via perspective shifts. Moreover, we argue that the effect of being addressed with an unexpected (in this case, informal) pronoun can overrule such an effect, while being addressed with an expected (in this case, formal) pronoun can be conducive of it. This finding highlights the importance of research into the impact of pronouns of address.