Host-parasite interactions are predicted to exhibit geographic heterogeneity, creating the opportunity for local adaptation. This is difficult to detect because it requires knowledge of selection pressures and fitness consequences of phenotypic variation. We explored the adaptive landscape of Hawaiian cricket populations which are parasitised by larvae of a fly that targets singing male crickets, in which multiple protective male-silencing morphs have recently spread. We resequenced cricket genomes from several locations across three islands, finding genetic variation is dominated by massive inversions, and observing sharp declines in effective population size following the fly\'s introduction. We find flies continue to impose extreme but heterogeneous selection on cricket populations, influencing the distribution of a recently discovered male-silencing phenotype and allelic variation at genes with functions related to evading or resisting larval infestation. Our study illuminates the extreme conditions contributing to and resulting from recurrent and remarkably rapid adaptive evolution in these wild populations.