This study evaluates a 12-item instrument for subjective measurement of mental load (ML) and mental effort (ME) by analysing different sources of validity evidence. The findings of an expert judgement (N = 8) provide evidence based on test content that the formulation of the items corresponds to the meaning of ML and ME. An empirical study was conducted in which secondary school students (N = 602) worked on multiple choice (mc)-tasks and thereafter using the developed instrument to self-report ML and ME. The findings show that the instrument reliably measures the two positively correlated constructs ML and ME (evidence based on internal structure). Students working on mc-tasks with high complexity self-reported higher amounts of ML and ME than students working on mc-tasks with low complexity, and there is a negative relation between test performance and ML (evidence based in relation to other variables). Implications for educational assessment and limitations of the study are discussed.