Recently, liquid flow over monolayer graphene has been experimentally demonstrated to generate an induced voltage in the flow direction, and various physical mechanisms have been proposed to explain the electricity-generating process between liquid and graphene. However, there are significant discrepancies in the reported results with non-ionic liquid: the observed voltage responses with deionized (DI) water vary from lab to lab under presumably similar flowing conditions. Here, a graphene-piezoelectric material heterostructure is proposed for harvesting energy from water flow; it is shown that the introduction of a piezoelectric template beneath graphene results in an obvious voltage output up to 0.1 V even with DI water. This potential arises from a continuous charging–discharging process in graphene, which is suggested to be a result of a relatively retarded screening effect of the water for the generated piezoelectric charges than that of the graphene layer, as revealed by first-principles calculations. This work considers a dynamic charge interaction among water, graphene, and the substrate, highlighting the crucial role of the underlying substrate in the electricity-generating process, which will greatly enhance understanding of the flow-induced voltage and push the graphene-water nanogenerator close to practical applications.