Ancient DNA-based selection scans have revealed multiple diet-associated single nucleotide polymorphisms (SNPs) that exhibit robust signatures of natural selection. To test whether the selection was indeed driven by diet, we conducted a paleo-empirical investigation. We compiled an isotopic dataset comprising 6,064 ancient human samples and 5,635 food resource samples from Britain. We developed a Bayesian mixing model to estimate individual dietary proportions based on isotopic data and constructed a temporal dietary model spanning the last 10,000 years. Using 1,038 ancient DNA samples, we reconstructed derived allele frequency trajectories for 20 strongly selected SNPs via bootstrap resampling. We then applied a generalized additive model (GAM) to estimate both mean and time-varying selection coefficients, while accounting for various evolutionary forces beyond natural selection. Finally, we applied the convergent cross mapping (CCM) algorithm for causal discovery between the time-varying selection coefficients and their corresponding dietary variable. The findings demonstrate that specific dietary elements, including marine resources, C3 plants, and dairy consumption, have acted as selective pressures on specific SNPs whose associated phenotypes are primarily shaped by diet. However, diet-driven signals become less detectable when the phenotypes are influenced by multiple environmental factors, including but not limited to diet.