In the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate. This paper identifies several challenges responsible for the non-coordination of independent agents: Pareto-selection, non-stationarity, stochasticity, alter-exploration and shadowed equilibria. A selection of multi-agent domains is classified according to those challenges: matrix games, Boutilier's coordination game, predators pursuit domains and a special multi-state game. Moreover, the performance of a range of algorithms for independent reinforcement learners is evaluated empirically. Those algorithms are Q-learning variants: decentralized Q-learning, distributed Q-learning, hysteretic Q-learning, recursive frequency maximum Q-value and win-or-learn fast policy hill climbing. An overview of the learning algorithms’ strengths and weaknesses against each challenge concludes the paper and can serve as a basis for choosing the appropriate algorithm for a new domain. Furthermore, the distilled challenges may assist in the design of new learning algorithms that overcome these problems and achieve higher performance in multi-agent applications.

In multi-agent systems, the presence of learning agents can cause the environment to be non-Markovian from an agent's perspective thus violating the property that traditional single-agent learning methods rely upon. This paper formalizes some known i

At the nanoscale, accurate control and position sensing are recurrent issues conditioning advances of technologies and instruments in diverse fields of application. Recent advances were reported that combine optical microscopy and computer vision for non-contact and multi-axis position sensing of end-effectors with a high accuracy. This paper contributes to this domain by introducing a micro-encoded target aimed to high accurate (x,y,θ) measurements over an extended range. The micro-encoded target is designed to allow linear phase measurements, leading to high accuracy in position sensing, and unambiguous binary encoding leading to absolute position determination. In this way, ultimate performances and allowed measurement ranges are independent of each other and a current zone of observation can be registered with respect to the whole encoded area with an extra-large range-to-resolution ratio. Experiments reported in this paper demonstrate a precision below one nanometer in lateral position and of ${4\cdot10^{-6}}$ rad. in rotation with an encoded target of more than ${11\times11\ \rm{cm^{2}}}$ . The device demonstrated can be either inserted into experimental setups to allow accurate position control or used to calibrate in-plane displacements of one to three degrees of freedom actuators.

The demand in the biomedical field for fast and precise devices for in vitro applications has increased in recent years. Mobile microrobots are significantly suitable for such applications and are developing rapidly. These microrobots offer untethered actuation toward a contamination-free environment while allowing for fast and precise handling of biological entities for applications such as positioning, sensing, delivery, and cell surgery that are highly effective for new drug discoveries and improving our understanding of cells’ behavior on the single-cell level. Here, we present a review of the recent state of the art in the actuation and implementation of mobile microrobots for in vitro applications. We first explore the widely used methods of wireless actuation. Next, we address the challenge of implementing an on-board interaction technique to handle the target biological entity without affecting the actuation of the microrobot. Finally, we discuss the future directions that would draw the basic outline for the next generation of mobile microrobots for in vitro applications.

Concentric tube robots (CTRs) have drawn significant research attention over the years, particularly due to their applications in minimally invasive surgery (MIS). Indeed, their small size, flexibility, and high dexterity enable several potential benefits for MIS. The research has led to an increasing number of discoveries and scientific breakthroughs in CTR design, fabrication, control, and applications. Numerous prototypes have emerged from different research groups, each with its own design and specifications. This survey paper provides an overview of the state of the art of the mechatronics aspects of CTRs, including approaches for the design and fabrication of the tubes, actuation unit, and end effector. In addition to the various hardware and associated fabrication methods, we propose to the research community, a unifying way of classifying CTRs based on their actuation unit architecture, as well as a set of specification details for the evaluation of future CTR prototypes. Finally, we also aim to highlight the current advancements, challenges, and perspectives of CTR design and fabrication.

In many cases, soft and continuum robots represent an interesting alternative to articulated robots because they have the advantages of miniaturization capability, safer interactions with humans and often simpler fabricating and integration. However, these benefits are usually considered to arise at the expense of accuracy and precision because of the soft or flexible limbs. This paper demonstrates that, with a proper design, a planar parallel continuum robot is capable of great precision. Indeed, the proposed 3-Degrees-of-Freedom planar parallel continuum robot exhibits a precision of 9.13 nm in position and 1.2 μrad in orientation. In addition, the novel robotic design leverages the effect of the actuators' defects, making the robot more precise than its own actuators. Finally, the workspace of the proposed robot (62.3 mm2, 36.97 deg) is significantly larger than most compliant mechanisms, which is particularly interesting when both very high precision and relatively large displacements are required.