Existing frameworks for transfer learning are incomplete from a systems theoretic perspective. They place emphasis on notions of domain and task, and neglect notions of structure and behavior. In doing so, they limit the extent to which formalism can be carried through into the elaboration of their frameworks. Herein, we use the Mesarovician systems theory to define transfer learning as a relation on sets, and subsequently, characterize the general nature of transfer learning as a mathematical construct. We interpret existing frameworks in terms of ours and go beyond existing frameworks to define notions of transferability, transfer roughness, and transfer distance. Importantly, despite its formalism, our framework avoids the detailed mathematics of the learning theory or machine learning solution methods without excluding their consideration. As such, we provide a formal, general systems framework for modeling transfer learning that offers a rigorous foundation for system design and analysis.

Classical machine learning approachesare sensitive to nonstationarity. Transfer learning can address nonstationarity by sharing knowledge from one system to another, however, in areas like machine prognostics and defense, data are fundamentally limited. Therefore, transfer learning algorithms have little, if any, examples from which to learn. Herein, the authors suggest that these constraints on algorithmic learning can be addressed by systems engineering. We formally define transfer distance in general terms and demonstrate its use in empirically quantifying the transferability of models. We consider the use of transfer distance in the design of machine rebuild procedures to allow for transferable prognostic models. We also consider the use of transfer distance in predicting operational performance in computer vision. Practitioners can use the presented methodology to design and operate systems with consideration for the learning theoretic challenges faced by component learning systems.

The machine learning formulation of transfer learning is incomplete from a systems theoretic perspective. It focuses on algorithm parameters, features, and samples, and neglects the perspective offered by considering system structure and system dynamics. Furthermore, while the machine learning formulation serves to classify methods and literature, the systems theoretic formulation presented herein serves to provide a framework for the top-down design of transfer learning systems, including a novel definition of transfer learning and identification of key design parameters. We dichotomize the transfer learning problem into a question of transferring system structure and dynamics. We formulate our framework in the context of input-output systems. We use an actuator system as a case study throughout the paper to ground the discussion in a real-world transfer problem.

There is a lack of formalism for some key foundational concepts in systems engineering. One of the most recently acknowledged deficits is the inadequacy of systems engineering practices for engineering intelligent systems. In our previous works, we proposed that closed systems precepts could be used to accomplish a required paradigm shift for the systems engineering of intelligent systems. However, to enable such a shift, formal foundations for closed systems precepts that expand the theory of systems engineering are needed. The concept of closure is a critical concept in the formalism underlying closed systems precepts. In this paper, we provide formal, systems- and information-theoretic definitions of closure to identify and distinguish different types of closed systems. Then, we assert a mathematical framework to evaluate the subjective formation of the boundaries and constraints of such systems. Finally, we present a high-level overview of the engineering implications stemming from this formalism, with a specific focus on its application in engineering intelligent systems. In the main, this framework lays the groundwork for understanding and applying closed systems precepts in the engineering of complex systems, especially those characterized by high levels of intelligence.