This paper proposes a methodological architecture to integrate Q-learning into economic modelling systematically. It addresses a common gap: the lack of a shared framework linking economic foundations to Reinforcement Learning components. Rather than introducing a new algorithm, it specifies and reports how preferences, frictions, information structures, and time horizons map to the reward function, discount factor, and learning environment design. Equilibrium outcomes serve as benchmarks for comparing learned policies, not as imposed axioms. This approach interprets learning dynamics through standard economic categories and enables comparability across studies. The architecture organizes models along explicit dimensions: behavioural preferences, institutional frictions, economic environment class, information structure, learning and exploration mechanisms, and evaluation metrics. A simulation illustrates how variations in frictions, risk attitudes, and intertemporal preferences affect learned policies, their stability, and their relationship to static benchmarks. The paper aims to promote the cumulative use of Reinforcement Learning in applied economics by providing a general specification that improves interpretability, comparability, and reproducibility, turning deviations from theoretical equilibria into measurable diagnostics that refine economic fundamentals.