Granger causality analysis has been suggested as a method of estimating causal modulation without specifying the direction of information flow a priori. Using BOLD-contrast functional MRI (fMRI) data, such analysis has been typically implemented in the time domain. In this study, we used magnetic resonance inverse imaging, a method of fast fMRI enabled by massively parallel detection allowing up to 10 Hz sampling rate, to investigate the causal modulation at different frequencies up to 5 Hz. Using a visuomotor two-choice reaction-time task, both the spectral decomposition of Granger causality and isolated effective coherence revealed that the BOLD signal at frequency up to 3 Hz can still be used to estimate significant dominant directions of information flow consistent with results from the time-domain Granger causality analysis. We showed the specificity of estimated dominant directions of information flow at high frequencies by contrasting causality estimates using data collected during the visuomotor task and resting state. Our data suggest that hemodynamic responses carry physiological information related to inter-regional modulation at frequency higher than what has been commonly considered.