Quantum anomalous Hall effect has been predicted in HgMnTe quantum wells with an out-of-plane magnetization of Mn atoms. However, since HgMnTe quantum wells are paramagnetic, an out-of-plane magnetic field is required to polarize magnetic moments of Mn atoms, which inevitably induces Landau levels and makes it difficult to identify the origin of the quantized Hall conductance experimentally. In this work, we study the quantum anomalous Hall effect in the presence of an in-plane magnetic field in Mn doped HgTe quantum wells. For a small out-of-plane magnetic field, the in-plane magnetic field can drive the system from a normal insulating state to a quantum anomalous Hall state. When the out-of-plane magnetic field is slightly above the transition point, the system shows a reentrant behavior of Hall conductance, varying from [Math Processing Error] to 0 and back to [Math Processing Error], with increasing in-plane magnetic fields. The reentrant quantum anomalous Hall effect originates from the interplay between the exchange coupling of magnetic moments and the direct Zeeman coupling of magnetic fields. The calculation incorporating Landau levels shows that there is no qualitative change of the reentrant behavior.