Cerebrospinal fluid (CSF) flow in the brain is tightly regulated and essential for brain health, and imaging techniques are needed to quantitatively establish the properties of this flow system. Flow-sensitive fMRI has recently emerged as a tool to measure large scale CSF flow dynamics with high sensitivity and temporal resolution; however, the measured signal is not quantitative. Here, we developed a dynamic model to simulate fMRI inflow signals based on time-varying flow velocities. We validated the model in both human and phantom data, and used it to identify important properties of the fMRI inflow signal that inform how the signal should be interpreted. Additionally, we developed a physics-based deep learning framework to invert the model, which enables direct estimation of velocity using fMRI inflow data. This work allows new quantitative information to be obtained from fMRI, which will enable neuroimaging researchers to take advantage of the high sensitivity, high temporal resolution, and wide availability of fMRI to obtain flow signals that are physically interpretable.