Building energy modeling is critical for retrofit design, but it is labor-intensive. We present Data2BEM (Data to Building Energy Model), a large language model-based multi-agent framework that parses architectural drawings, specifications, and sensor data to automatically generate and calibrate building energy simulations. Applied to an existing University of Cambridge office building, Data2BEM produced a calibrated model meeting industry accuracy benchmarks and enabled the assessment of heat-electrification retrofits. Relative to professional practice, the system reduced total modeling time by over 90% (48 min versus 8-32 h) with minimal human input. The workflow integrates information extraction, model generation, and data-driven calibration, delivering end-to-end automation while accurately reflecting measured performance. These results indicate that large language model-driven multi-agent methods can accelerate retrofit analysis, lowering expertise and time barriers for practitioners and supporting scalable pathways to building-sector decarbonization.