Memory is the biggest limiting factor to the widespread use of FPGAs for high-level image processing, which require complete frame(s) to be stored in situ. Since FPGAs have limited on-chip memory capabilities, efficient use of such resources is essential to meet performance, size and power constraints. In this paper, we investigate allocation of on-chip memory resources in order to minimize resource usage and power consumption, contributing to the realization of power-efficient high-level image processing fully contained on FPGAs. We propose methods for generating memory architectures, from both Hardware Description Languages and High Level Synthesis designs, which minimize memory usage and power consumption. Based on a formalization of on-chip memory configuration options and a power model, we demonstrate how our partitioning algorithms can outperform traditional strategies. Compared to commercial FPGA synthesis and High Level Synthesis tools, our results show that the proposed algorithms can result in up to 60% higher utilization efficiency, increasing the sizes and/or number of frames that can be accommodated, and reduce frame buffers’ dynamic power consumption by up to approximately 70%. In our experiments using Optical Flow and MeanShift Tracking, representative high-level algorithms, data show that partitioning algorithms can reduce total power by up to 25% and 30%, respectively, without impacting performance.

Specialized FPGA implementations can deliver higher performance and greater power efficiency than embedded CPU or GPU implementations for real-time image processing. Programming challenges limit their wider use, because the implementation of FPGA architectures at the register transfer level is time consuming and error prone. Existing software languages supported by high-level synthesis (HLS), although providing a productivity improvement, are too general purpose to generate efficient hardware without the use of hardware-specific code optimizations. Such optimizations leak hardware details into the abstractions that software languages are there to provide, and they require knowledge of FPGAs to generate efficient hardware, such as by using language pragmas to partition data structures across memory blocks. This article presents a thorough account of the Rathlin image processing language (RIPL), a high-level image processing domain-specific language for FPGAs. We motivate its design, based on higher-order algorithmic skeletons, with requirements from the image processing domain. RIPL’s skeletons suffice to elegantly describe image processing stencils, as well as recursive algorithms with nonlocal random access patterns. At its core, RIPL employs a dataflow intermediate representation. We give a formal account of the compilation scheme from RIPL skeletons to static and cyclostatic dataflow models to describe their data rates and static scheduling on FPGAs. RIPL compares favorably to the Vivado HLS OpenCV library and C++ compiled with Vivado HLS. RIPL achieves between 54 and 191 frames per second (FPS) at 100MHz for four synthetic benchmarks, faster than HLS OpenCV in three cases. Two real-world algorithms are implemented in RIPL: visual saliency and mean shift segmentation. For the visual saliency algorithm, RIPL achieves 71 FPS compared to optimized C++ at 28 FPS. RIPL is also concise, being 5x shorter than C++ and 111x shorter than an equivalent direct dataflow implementation. For mean shift segmentation, RIPL achieves 7 FPS compared to optimized C++ on 64 CPU cores at 1.1, and RIPL is 10x shorter than the direct dataflow FPGA implementation.

In the development of real-time systems, predictability is often hindered by technological factors which break the timing abstractions offered by real time operating systems (RTOSs); namely, the priority space separation between threads and interrupts induces the rate-monotonic problem. Software approaches have tackled this issue, attempting to unify the priority space with varying degrees of success. We present a hardware approach to the problem: unifying the priority space at the interrupt handling subsystem, predictability is greatly enhanced with minimum software modifications. Our solution provides the interrupt controller with awareness of the currently running task's priority making the solution independent of the used operating system. We show how our approach is minimally intrusive at hardware architecture level and provides benefits beyond the capabilities of previous approaches. Our technique shows a 0.05% run-time overhead if no interrupts occur, and run-time reduction proportional to interrupt rate for rates higher than 5 per s, for a interrupt workload around 0.07 ms.

The emergence of hardware multithread (HW-MT) architectures increased the performance of MT applications. However, traditional HW-MT architectures are not suitable to real-time operating systems as their performance-oriented scheduling algorithm may conflict with RTOS software scheduling. This letter presents RT-SHADOWS, a portable architecture which provides a unified hardware-software scheduling, bringing the benefits of HW-MT to the RTOS domain. We show that tightly-coupled real-time compliant hardware integration achieves throughput benefits, maintaining the RTOS scheduling policy intact while increasing the predictability of RTOSes. Our solution shows on average, speed-ups between 3 and 4 times over the native versions with very low area usage/performance overhead ratio, due to its minimal cost (2% of extra slices per hardware-supported thread). This work surpasses related work by providing a complete and agnostic hardware solution which is independent of any specific RTOS.

The use of technological solutions to address the production of goods and offering of services is ubiquitous. Health and social issues, however, have only slowly been permeated by technological solutions. Whilst several advances have been made in health in recent years, the adoption of technology to combat social problems has lagged behind. In this paper, we explore Big Data-driven Artificial Intelligence (AI) applied to social systems; i.e., social computing, the concept of artificial intelligence as an enabler of novel social solutions. Through a critical analysis of the literature, we elaborate on the social and human interaction aspects of technology that must be in place to achieve such enabling and address the limitations of the current state of the art in this regard. We review cultural, political, and other societal impacts of social computing, impact on vulnerable groups, and ethically-aligned design of social computing systems. We show that this is not merely an engineering problem, but rather the intersection of engineering with health sciences, social sciences, psychology, policy, and law. We then illustrate the concept of ethically-designed social computing with a use case of our ongoing research, where social computing is used to support safety and security in home-sharing settings, in an attempt to simultaneously combat youth homelessness and address loneliness in seniors, identifying the risks and potential rewards of such a social computing application.

This paper presents a novel hypervisor, developed for aerospace applications using an object oriented approach that embodies time and space partitioning (TSP) on a PowerPC (PPC) core embedded in a FPGA, for the NetworkCentric core avionics [1] - an architecture of cooperating components and managed by a real-time operating system, to implement dependable computing and targeting simplicity. To support Integrated Modular Architecture (IMA) [2] partitioned software architectures, the proposed hypervisor adapted to the aerospace application domain the Popek and Goldberg's [3] fidelity, efficiency and resource control virtualization requirements, and extends them with additional ones like timing determinism, reactivity and improved dependability. A distinctive feature of this hypervisor is its I/O device virtualization approach that guarantees real-time performance and small trusted computing base. The object oriented approach will be particularly useful to customize key components of the hypervisor (with different granularity levels) such as partition scheduling and the communications manager using generative programming techniques (Aspect Oriented Programming (AOP) and template meta-programming).