Climate change is accelerating the introduction of new fruit varieties worldwide, yet machine learning models for agricultural quality prediction require years of historical data to be reliable. This bottleneck severely limits AI deployment in smart farming: a new variety planted today may lack sufficient quality records for a decade. Transfer learning from established crops offers a promising shortcut, but our investigation reveals a critical pitfall: naively combining data from multiple established crops is highly unreliable—improving results in some target domains (+23.7%) while causing catastrophic performance degradation in others (R2 drops from 0.553 to −0.494). We term this the source overwhelming problem, where abundant source data drowns the signal from scarce target observations. We present Balanced Multi-Source Transfer Learning (BMTL), a principled framework that controls source-target balance to enable effective cross-crop knowledge transfer, operating in a few-shot regime where a small target validation set (nvalT ≥ 10) is available. We establish a counterintuitive finding: source-target balance matters more than domain similarity for successful transfer. On fruit quality prediction spanning 4 crops (apple, citrus, grape, pear; 3,721 quality measurements with 20,896 integrated weather records from the Korea Meteorological Administration), BMTL improves R2 by +0.143 points (from 0.393 to 0.536, a +36.4% relative gain) over single-crop training in 7 of 8 tested scenarios, and up to +117.8% when target data is extremely scarce (nT =50). On UCI benchmarks, BMTL is the only transfer-based method avoiding catastrophic negative transfer on high-shift domains. The proposed method reduces the data collection burden for new crop varieties, enabling agricultural AI systems to become operational years earlier—with direct benefits for food security and supply chain management.