Purpose – This paper seeks to develop universal software (a programming framework) enabling the implementation of service robot controllers. The software should distinguish the hardware-oriented part of the system from the task-oriented one. Moreover, force, vision as well as other sensors should be taken into account. Multi-effector systems have to be considered. Design/methodology/approach – The robot programming framework MRROC++ has been implemented as a hierarchical structure composed of processes, potentially consisting of threads. All of the software is written in an object-oriented manner using C++ and is supervised by a QNX real-time operating system. The framework has been verified on several systems executing diverse tasks. Here, a Rubik's cube puzzle-solving system, consisting of two arms and utilizing force control and visual servos, is presented. Findings – The presented framework is well suited to tasks requiring two-handed manipulation with force sensing, visual servoing and online construction of plans of actions. The Rubik's cube puzzle is a reasonable initial benchmark for validation of fundamental service robot capabilities. It requires force sensing and sight coupled with two-handed manipulation and logical reasoning, as do the majority of service tasks. Owing to the use of force sensing during manipulation, jamming of the faces has always been avoided; however, visual servoing could only cope with slow handing over of the cube due to the volume of computations associated with vision processing. Research limitations/implications – The proposed software structure does not limit the implementation of service robot controllers. However, some of the specific algorithms used for the solution of the benchmark task (i.e. Rubik's cube puzzle) need to be less time-consuming. Practical implications – The MRROC++ robot programming framework can be applied to the implementation of diverse robot controllers executing complex service tasks. Originality/value – A demanding benchmark task for service robots has been formulated. This task, as well as many others, has been used to validate the MRROC++ robot programming framework which significantly facilitates the implementation of diverse robot systems.

The paper presents a systematic approach to robot control system design. The robot is treated as an embodied agent decomposed into effectors, receptors, both real and virtual, and a control subsystem. Those entities communicate through communication buffers. The activities of those entities are governed by FSMs that invoke behaviours formulated in terms of transition functions taking as arguments the contents of input buffers and producing the values inserted into output buffers. The method is exemplified by applying it to a design of a control system of a robot capable of locating an open box and covering it with a lid. Other systems that have been designed in a similar way are presented as well, to demonstrate the versatility of the approach.

Visual Question Answering (VQA) is a novel problem domain where multi-modal inputs must be processed in order to solve the task given in the form of a natural language. As the solutions inherently require to combine visual and natural language processing with abstract reasoning, the problem is considered as AI-complete. Recent advances indicate that using high-level, abstract facts extracted from the inputs might facilitate reasoning. Following that direction we decided to develop a solution combining state-of-the-art object detection and reasoning modules. The results, achieved on the well-balanced CLEVR dataset, confirm the promises and show significant, few percent improvements of accuracy on the complex "counting" task.