Increasing agricultural water productivity will be a key factor in China's ability to maintain food security in the face of rapidly growing water demand from other sectors. This paper highlights how such achievements can be considered and made by examining and contrasting water institutions, policies and management strategies across scales in two irrigation systems in China situated within strikingly different environments, the Zhanghe Irrigation System (ZIS) in the relatively water-abundant Yangtze River basin and the Liuyuankou Irrigation System (LIS) in the highly water-stressed Yellow River basin. The results show, perhaps surprisingly, that institutional arrangements which have evolved in the water-abundant system are more conducive to water savings. However, the particular conclusions on water productivity and savings also depend in part on definitions of scale and other factors. These findings form part of a changing trend in thinking about irrigation, water productivity and water savings that considers the analysis of scales, multiple uses, and practices of irrigation in the context of water scarcity and has direct implications for China's efforts to better use its scarce water resources.