Deshanand Singh

Deshanand Singh

Auto-generated profile, based on publicly available information

Deshanand Singh

Intel, Toronto, ON, Canada

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 June 2024
Verified
Conference Paper
Open access
AAAhmed Abouelhamayed
Ahmed Abouelhamayed
SBSusanne Balle
Susanne Balle
DSDeshanand Singh
Deshanand Singh
MAMohamed Abdelfattah
Mohamed Abdelfattah
Deep neural network (DNN) inference has become an important part of many data-center workloads. This has prompted focused efforts to design ever-faster deep learning accelerators such as GPUs and TPUs. However, an end-to-end DNN-based vision application contains more than just DNN inference, including input decompression, resizing, sampling, normalization, and data transfer. In this paper, we perform a thorough evaluation of computer vision inference requests performed on a throughput-optimized serving system. We quantify the performance impact of server overheads such as data movement, preprocessing, and message brokers between two DNNs producing outputs at different rates. Our empirical analysis encompasses many computer vision tasks including image classification, segmentation, detection, depth-estimation, and more complex processing pipelines with multiple DNNs. Our results consistently demonstrate that end-to-end application performance can easily be dominated by data processing and data movement functions (up to 56% of end-to-end latency in a medium-sized image, and ~ 80% impact on system throughput in a large image), even though these functions have been conventionally overlooked in deep learning system design. Our work identifies important performance bottlenecks in different application scenarios, achieves 2.25× better throughput compared to prior work, and paves the way for more holistic deep learning system design.

7 May 2024
Verified
Conference Paper
Open access
YMYuan Meng
Yuan Meng
MKMichael Kinsner
Michael Kinsner
DSDeshanand Singh
Deshanand Singh
MIMahesh Iyer
Mahesh Iyer
VPViktor Prasanna
Viktor Prasanna
Deep Reinforcement Learning (DRL) is vital in various AI applications. DRL algorithms comprise diverse compute kernels, which may not be simultaneously optimized using a homogeneous architecture. However, even with available heterogeneous architectures, optimizing DRL performance remains a challenge due to the complexity of hardware and programming models employed in modern data centers. To address this, we introduce PEARL, a toolkit for composing parallel DRL systems on heterogeneous platforms consisting of general-purpose processors (CPUs) and accelerators (GPUs, FPGAs). Our innovations include: 1. A general training protocol agnostic of the underlying hardware, enabling portable implementations across various platforms. 2. Incorporation of DRL-specific optimizations on runtime scheduling and resource allocation, facilitating parallelized training and enhancing the overall system performance. 3. Automatic optimization of DRL task-to-device assignments through throughput estimation. 4. High-level API for productive development using the toolkit. We showcase our toolkit through experimentation with two widely used DRL algorithms, DQN and DDPG, on two diverse heterogeneous platforms. The generated implementations outperform state-of-the-art libraries for CPU-GPU platforms by up to 2.2× throughput improvements, and 2.4× higher performance portability across platforms.

26 January 2024 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
RMRui Ma
Rui Ma
JHJia-Ching Hsu
Jia-Ching Hsu
AMAli Mansoorshahi
Ali Mansoorshahi
JGJoseph Garvey
Joseph Garvey
MKMichael Kinsner
Michael Kinsner
DCDerek Chiou
Derek Chiou
Overlay processors on FPGAs enable (i) software programmability through sequential code calling library functions, (ii) high performance by converting the library calls to invocations of corresponding accelerators, and (iii) faster deployment than reprogramming the FPGA. Traditionally, overlays have been hand-written in RTL and programmed through handwritten assembly. We present the Primate framework, which automatically generates overlays from applications written in annotated C++. We evaluated Primate on Whippersnapper [1] P4 benchmarks. Primate Overlay latencies are 0.06x - 0.15x compared to PISCES [2], a high-performance CPU solution, and 0.25× - 2.3× compared to solutions generated by P4FPGA [3], a P4 HLS compiler on FPGA.