This paper describes the difficulties inherent in gathering reliable data and estimating water savings of installing water saving devices for residential water conservation. The Saint Paul Water Utility conducted a study to evaluate the water savings of its 1992 and 1993 residential toilet retrofit programs. The authors found that the technique used to estimate the amount of water used per flush was not as accurate as they had thought. The established technique was to measure the volume of water in the tanks used before and after the retrofit. The tank measurements could not take into account the flow rates of the water entering and leaving the tank during the flush cycle. The flow rate of water entering the tank was found to be essential in the determination of actual water use. In addition, there was not a correlation between the size of the tank and the water used during the flush cycle, for a number of reasons. The researchers found that the most accurate measurement was to determine the flow rate and multiply it by the cycle time. Two gallons of water are poured into an empty tank. The level is marked and a stop watch is used to measure the number of seconds it takes to reach the two gallon mark. This determines the number of gallons per second. The authors caution against using manufacturers' claims for water savings because each toilet is different.