This paper summarizes the technical activities of the IEEE Task Force on Solving Large Scale Optimization Problems in Electricity Market and Power System Applications. This Task Force was established by the IEEE Technology and Innovation Subcommittee to first review the state-of-the-art of the security-constrained unit commitment (SCUC) business model, its mathematical formulation, and solution techniques in solving electricity market clearing problems. The Task Force then investigated the emerging challenges of future market clearing problems and presented efforts in building benchmark mathematical and business models.

To reduce energy costs and emissions of microgrids, daily operation is critical. The problem is to commit and dispatch distributed devices with renewable generation to minimize the total energy and emission cost while meeting the forecasted energy demand. The problem is challenging because of the intermittent nature of renewables. In this paper, photovoltaic (PV) uncertainties are modeled by a Markovian process. For effective coordination, other devices are modeled as Markov processes with states depending on PV states. The entire problem is Markovian. This combinatorial problem is solved using branch-and-cut. Beyond energy and emission costs, to consider capital and maintenance costs in the long run, microgrid design is also essential. The problem is to decide device sizes with given types to minimize the lifetime cost while meeting energy demand. Its complexity increases exponentially with the problem size. To evaluate the lifetime cost including the reliability cost and the classic components such as capital and fuel costs, a linear model is established. By selecting a limited number of possible combinations of device sizes, exhaustive search is used to find the optimized design. The results show that the operation method is efficient in saving cost and scalable, and microgrids have lower lifetime costs than conventional energy systems. Implications for regulators and distribution utilities are also discussed.

Many operation optimization problems such as scheduling and assignment of interest to the automation community are mixed-integer linear programming (MILP) problems. Because of their combinatorial nature, the effort required to obtain optimal solutions increases drastically as the problem size increases. Such operation optimization problems typically need to be solved several times a day and require short solving times (e.g., 5, 10, or 20 min). The goal is, therefore, to obtain near-optimal solutions with quantifiable quality in a computationally efficient manner. Existing MILP methods, however, suffer from slow convergence and may not efficiently achieve this goal. In this paper, motivated by fast convergence of augmented Lagrangian relaxation (LR), a novel advanced price-based decomposition and coordination “surrogate absolute-value LR” (SAVLR) approach is developed. Within the method, convergence of our recent surrogate LR (SLR), which has overcome all major difficulties of traditional LR, is significantly improved by penalizing constraint violations by adding “absolute-value” penalties. Moreover, such penalties are efficiently linearized in a standard way, thereby enabling the use of MILP solvers. By exploiting the beautiful property of exponential reduction of complexity of subproblems upon decomposition, subproblems are efficiently solved and their solutions are efficiently coordinated by updating Lagrangian multipliers. Convergence is then proved under novel adjustment of penalty coefficients. A series of generalized assignment problems is considered, and for these problems, superior performance of SAVLR over other state-of-the-art and state-of-the-practice methods is demonstrated. Accompanying CPLEX codes, whereby SAVLR is implemented, are also included. Note to Practitioners-Examples of important problems that arise in automation community include scheduling and assignment problems. Because of their combinatorial nature, the effort required to obtain optimal solutions increases drastically as the problem size increases. Existing mixed-integer linear programming (MILP) methods, however, may suffer from slow convergence and may not efficiently achieve this goal. The new method revolutionizes the way such problems can be solved with major improvements on the overall performance. It is based on our recent breakthrough “surrogate Lagrangian relaxation” (LR), which has overcome all major difficulties of traditional LR while exploiting the beautiful property of exponential reduction of complexity upon decomposition. To significantly improve convergence while maintaining linearity so as to use MILP solvers, our idea is to penalize violations of relaxed constraints by the infrequently used “absolute-value” penalty functions. Although not differentiable, absolute-value penalties have the advantage of being exactly linearizable through extra variables and constraints. The difficulties caused by those extra constraints, which couple subproblems, are resolved by adaptive adjustment of penalty coefficients. A series of generalized assignment problems is considered and superior performance of the new method against state-of-the-art and state-of-the-practice methods is demonstrated. Accompanying CPLEX codes whereby the new method is implemented are also included.

Unit Commitment is usually formulated as a Mixed Binary Linear Programming (MBLP) problem. When considering a large number of units, state-of-the-art methods such as branch-and-cut may experience difficulties. To address this, an important but much overlooked direction is formulation transformation since if the problem constraints can be transformed to directly delineate the convex hull in the data pre-processing stage, then a solution can be obtained by using linear programming methods without combinatorial difficulties. In the literature, a few tightened formulations for single units with constant ramp rates were reported without presenting how they were derived. In this paper, a systematic approach is developed to tighten formulations in the data pre-processing stage. The idea is to derive vertices of the convex hull without binary requirements. From them, vertices of the original convex hull can be innovatively obtained. These vertices are converted to tightened constraints, which are then parameterized based on unit parameters for general use, tremendously reducing online computational requirements. By analyzing short-time horizons, e.g., two or three hours, tightened formulations for single units with constant and generation-dependent ramp rates are obtained, beyond what is in the literature. Results demonstrate computational efficiency and solution quality benefits of formulation tightening.

Unit Commitment (UC) is important for power system operations. With increasing challenges, e.g., growing intermittent renewables and intra-hour net load variability, traditional mathematical optimization could be time-consuming. Machine learning (ML) is a promising alternative. However, directly learning good solutions is difficult in view of the combinatorial nature of UC. This paper synergistically integrates ML within our recent decomposition and coordination method of Surrogate Lagrangian Relaxation to learn “good enough” subproblem solutions of deterministic UC. Compared to original UC, a subproblem is much easier to learn. Nevertheless, predicting good-enough subproblem solutions is still challenging because of the “jumps” of binary decisions and many types of constraints. To overcome these issues, subproblem dimensionality is reduced via aggregating multipliers. Multiplier distributions are novelly specified based on “jumps” for effective learning. Loss functions are innovatively designed to improve prediction qualities. Ordinal Optimization and branch-and-cut are used as backups for unfamiliar cases. Furthermore, online self-learning is seamlessly integrated with offline learning to exploit solutions from daily operations. Results on the IEEE 118-bus system and the Polish 2383-bus system demonstrate that continual learning keeps on improving the subproblem-solving process with near-optimality of the overall solutions maintained. Our method opens a new direction to solve complicated UC.