We propose a general Markovian model for the optimal control of admissions and subsequent routing of customers for service provided by a collection of heterogeneous stations. Queue-length information is available to inform all decisions. Admitted customers will abandon the system if required to wait too long for service. The optimisation goal is the maximisation of reward rate earned from service completions, net of the penalties paid whenever admission is denied, and the costs incurred upon every customer loss through impatience. We show that the system is indexable under mild conditions on model parameters and give an explicit construction of an index policy for admission control and routing founded on a proposal of Whittle for restless bandits. We are able to gain insights regarding the strength of performance of the index policy from the nature of solutions to the Lagrangian relaxation used to develop the indices. These insights are strengthened by the development of performance bounds. Although we are able to assert the optimality of the index heuristic in a range of asymptotic regimes, the performance bounds are also able to identify instances where its performance is relatively weak. Numerical studies are used to illustrate and support the theoretical analyses.

We develop appropriately generalized notions of indexability for problems of dynamic resource allocation where the resource concerned may be assigned more flexibility than is allowed, for example, in classical multi-armed bandits. Most especially we have in mind the allocation of a divisible resource (manpower, money, equipment) to a collection of objects (projects) requiring it in cases where its over-concentration would usually be far from optimal. The resulting project indices are functions of both a resource level and a state. They have a simple interpretation as a fair charge for increasing the resource available to the project from the specified resource level when in the specified state. We illustrate ideas by reference to two model classes which are of independent interest. In the first, a pool of servers is assigned dynamically to a collection of service teams, each of which mans a service station. We demonstrate indexability under a natural assumption that the service rate delivered is increasing and concave in the team size. The second model class is a generalization of the spinning plates model for the optimal deployment of a divisible investment resource to a collection of reward generating assets. Asset indexability is established under appropriately drawn laws of diminishing returns for resource deployment. For both model classes numerical studies provide evidence that the proposed greedy index heuristic performs strongly.

In this study, we consider the dynamic and stochastic resource-constrained multi-project scheduling problem where projects generate rewards at their completion, completions later than a due date cause tardiness costs, task duration is uncertain, and new projects arrive randomly during the ongoing project execution both of which disturb the existing project scheduling plan. We model this problem as a discrete-time Markov decision process and explore the performance and computational limitations of solving the problem by dynamic programming. We run and compare five different solution approaches, which are: a dynamic programming algorithm to determine a policy that maximises the time-average profit, a genetic algorithm and an optimal reactive baseline algorithm, both generate a schedule to maximise the total profit of ongoing projects, a rule-based algorithm which prioritises processing of tasks with the highest processing durations, and a worst decision algorithm to seek a non-idling policy that minimises the time-average profit. The performance of the optimal reactive baseline algorithm is the closest to the optimal policies of the dynamic programming algorithm, but its results are suboptimal, up to 37.6%. Alternative scheduling algorithms are close to optimal with low project arrival probability but quickly deteriorate their performance as the probability increases.

An important issue in the manufacturing and supply chain literature concerns the optimisation of inventory decisions. Single-product inventory problems are widely studied and have been optimally solved under a variety of assumptions and settings. However, as systems become more complex, inventory decisions become more complicated for which the methods/approaches for optimising single inventory systems are incapable of deriving optimal policies. Manufacturing process flexibility provides an example of such a complex application area. Decisions involving the interrelated product inventories and production facilities form a highly multidimensional, non-decomposable system for which optimal policies cannot be readily obtained. We propose the methodology of approximate dynamic programming (ADP) to overcome the computational challenge imposed by this multidimensionality. Incorporating a sample backup simulation approach, ADP develops policies by utilising only a fraction of the computations required by classical dynamic programming. However, there are few studies in the literature that optimise production decisions in a stochastic, multi-factory, multi-product inventory system of this complexity. This paper aims to explore the feasibility and relevancy of ADP algorithms for this application. We present the results from numerical experiments that establish the strong performance of policies developed via temporal difference ADP algorithms in comparison to optimal policies and to policies derived from a deterministic approximation of the problem.

Journal of the Operational Research Society (JORS) is the world's longest established OR journal and an official journal of The OR Society. It is the aim of the Journal to present papers which are relevant to practitioners, researchers, teachers, students and consumers of operational research, and which cover the theory, practice, history or methodology of OR.