Reprogramming of DNA methylation plays a vital role in the establishment of cell identity during early mammalian development. To gain deeper mechanistic insights into this process requires capturing the entire dynamics of DNA methylation - both 5-methylcytosine (5mC) and its downstream oxidation product 5-hydroxymethylcytosine (5hmC) - in individual cells. Therefore, in this work, we report a new single-cell genome-wide strand-specific sequencing method, scMHT-seq, to jointly profile 5mC, 5hmC, and the transcriptome from individual cells. Using human embryonic stem cells (hESCs), we first show that scMHT-seq can accurately detect both 5mC and 5hmC from the same cell with minimal crosstalk in quantifying these two DNA modifications, and that the multi-modal measurements are in close agreement with individual measurements of 5mC and 5hmC in single cells. After establishing the method, we next applied scMHT-seq to gain insights into human primordial germ cell (hPGC) development. After specification, hPGCs undergo rapid global demethylation as they mature, and this reprogramming is critical for normal development of gametes. However, it has not been possible to fully overcome this key epigenetic barrier in culture, thereby limiting our ability to generate mature hPGC-like cells (hPGCLCs) and accomplish in vitro gametogenesis. To gain deeper understanding of the molecular factors involved in germ cell maturation, we applied scMHT-seq to an extended in vitro culture system for generating hPGCLCs and observed partial and heterogeneous erasure of the methylome across single cells that is mechanistically predominantly driven by passive demethylation due to reduced DNMT1-mediated maintenance methylation activity. Notably, we discover that hPGCLCs in extended culture can be transcriptionally classified into two distinct states, with one population enriched with more mature hPGCLCs exhibiting genome-wide loss of DNA methylation. Moreover, analysis of these two cell states identifies DND1 and SOX15 as two factors that are potentially key drivers of hPGCLC demethylation and maturation. Overall, we demonstrate that scMHT-seq is a robust and high-throughput technology that can provide insights into the mechanisms driving DNA methylation dynamics and their effect on cell states.