The purpose of this study is to investigate whether and to what extent fiscal decentralization affects tax effort of local governments in China after the Tax Sharing System (TSS). This research provides different indexes of tax effort and fiscal decentralization in analysis. By using the panel data of 31 regions in China during the period of 1996–2006 and the two-way fixed-effect model, the empirical results show that fiscal decentralization has a significantly positive impact on tax effort of local governments. In addition, this positive influence of fiscal decentralization on tax effort increases over time. Finally, trade openness and industrialization level also will enhance the local government's tax effort.