We introduce GOLEM --- an open-source optimization framework for automated design of graph-based structures in various scientific domains. It solves the problem of finding optimal topology and parameters of graphs using evolutionary algorithms, and does it in a modular domain-agnostic way. The paper describes the framework and its flexible approach to domain adaptation. Experimental studies provide several examples of GOLEM application in different fields: Bayesian networks, drug design, and robotics.