Where we look lets others know what we are interested in and allows them to join our focus of attention. In several studies our group investigated the neuronal basis of gaze following behavior in primates and described a gaze following patch (GFP) as the underlying functional unit. This makes the GFP a promising neurobiological correlate of Simon Baron-Cohen's eye direction detector (EDD), an integral part of his mindreading system. With the latter, Baron-Cohen proposed a set of domain-specific neurocognitive modules implementing a Theory of Mind - the attribution of mental states to others. The tenet of domain-specificity requires that the EDD processes exclusively eye-like stimuli. In this preregistered fMRI study, we aimed to critically test if the GFP fulfills this criterion. Contrary to previous studies, we found that this was not the case and that it functionally and anatomically overlapped with the MT+ complex, putting the plausibility of Baron-Cohen's mindreading model as a set of domain-specific modules into question.