The cell fate participants characterization based on single-cell RNA sequencing (scRNA-seq) data greatly facilitates the mechanism understandings of cellular differentiation. However, inferring these fate factors dynamics along the pseudotime is challenging. Based on cell-state density and pseudotime regression weights, we present an algorithm TimeFactorKernel (FT-Kernel), to predict the key cell fate factors, not only the minimum lineage transition genes, but also the related genesets/pathways, and cellular interaction dynamics along the pseudotime. By extrapolating the pseudotime-related key genes from spectral data as a pseudotime-kernel, FT-Kernel outperformed previous methods. Beyond time-related genes, FT-Kernel offered a comprehensive analysis of the inferred cellular interactions dynamics and lineage pathways dynamics along the pseudotime, which were limited in other methods. Additionally, it facilitated time-related fate factors prediction across various data modalities. Our work developed an important pseudotime-kernel in predicting the fate factors, and provided insights into the cellular hierarchies during development.