Ancient DNA has revolutionized our understanding of human evolutionary history, but studies focusing solely on genetic variation tell an incomplete story by neglecting phenotypic outcomes. The relationships between genotype and phenotype can change over time, making it desirable to study them directly in ancient populations rather than present-day data. Here, we present a large-scale integration of ancient genomic and phenotypic data, analyzing femur length as a proxy for stature in 568 individuals with published whole-genome ancient DNA data across western Eurasia. Polygenic scores derived from modern European and East Asian genome-wide association studies retain predictive power in ancient populations, explaining up to 10% of phenotypic variance. Contrary to longstanding archaeological hypotheses, we find that Neolithic populations were only modestly shorter than preceding Mesolithic groups, with differences at least partly attributable to genetic rather than environmental factors, challenging narratives of systematic stature decline following the transition to agriculture. Finally, we find that the lactase persistence allele had a large positive effect on stature in ancient individuals (0.24 standard deviations), even though it shows no association with height in modern populations. This gene-environment interaction highlights the limitation of using present-day genetic data to infer past phenotypic relationships. Our results underscore the value of integrating genetic and morphological data from ancient populations to reconstruct the dynamics of human adaptation.