Whole building energy model (BEM) is difficult to beused in the classical model-based optimal control (MOC)because of its high-dimension nature and intensive computationalspeed. This study proposes a novel deep reinforcementlearning framework to use BEM for MOCof HVAC systems. A case study based on a real officebuilding in Pennsylvania is presented in this paper todemonstrate the workflow, including building modeling,model calibration and deep reinforcement learning training.The learned optimal control policy can potentiallyachieve 15% of heating energy saving by simply controllingthe heating system supply water temperature.