An ex vivo model of human colon cancer enables study of the drivers of tumor development and invasion. Refined cancer models are needed to bridge the gaps between cell line, animal and clinical research. Here we describe the engineering of an organotypic colon cancer model by recellularization of a native human matrix that contains cell-populated mucosa and an intact muscularis mucosa layer. This ex vivo system recapitulates the pathophysiological progression from APC-mutant neoplasia to submucosal invasive tumor. We used it to perform a Sleeping Beauty transposon mutagenesis screen to identify genes that cooperate with mutant APC in driving invasive neoplasia. We identified 38 candidate invasion-driver genes, 17 of which, including TCF7L2, TWIST2, MSH2, DCC, EPHB1 and EPHB2 have been previously implicated in colorectal cancer progression. Six invasion-driver genes that have not, to our knowledge, been previously described were validated in vitro using cell proliferation, migration and invasion assays and ex vivo using recellularized human colon. These results demonstrate the utility of our organoid model for studying cancer biology.