Accurate and comprehensive peptide spectrum annotation is a crucial step to interpret mass spectrometry-based proteomics data. While peak assignment in peptide fragmentation spectra is central to a broad range of proteomics applications, current tools tend to be specialized to a specific task. Here we present a more comprehensive tool, the Annotator, which unifies spectrum annotation for bottom-up, middle-down, top-down, cross-linked, and glycopeptide fragmentation mass spectra, from all fragmentation methods, including all ion types: a/b/c, x/y/z, d/v/w, and immonium ions. The Annotator integrates all known post-translational modifications from common databases and additionally allows for the definition of custom modifications and diagnostic fragment ions. The underlying library used for the theoretical fragmentation and matching is based on the unified peptidoform notation ProForma 2.0 and made available as a Rust library (rustyms) with Python bindings. This enables spectrum annotation of diverse and complex peptidoforms across the broad range of mass spectrometry-based proteomics applications.