Venmugil Elango

Venmugil Elango

Auto-generated profile, based on publicly available information

Venmugil Elango

Microsoft, Redmond, USA

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 June 2023
Verified
Conference Paper
Open access
BRBita Darvish Rouhani
Bita Darvish Rouhani
RZRitchie Zhao
Ritchie Zhao
VEVenmugil Elango
Venmugil Elango
RSRasoul Shafipour
Rasoul Shafipour
MHMathew Hall
Mathew Hall
MNMaxim Naumov
Maxim Naumov
This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables comparison of popular quantization standards, and through BDR, new formats based on shared microexponents (MX) are identified, which outperform other state-of-the-art quantization approaches, including narrow-precision floating-point and block floating-point. MX utilizes multiple levels of quantization scaling with ultra-fine scaling factors based on shared microexponents in the hardware. The effectiveness of MX is demonstrated on real-world models including large-scale generative pretraining and inferencing, and production-scale recommendation systems.

1 July 2014 in Computer Physics Communications
Verified
Journal Article
Closed access
Bertrand Rouet-LeducBertrand Rouet-Leduc
Bertrand Rouet-Leduc
KBKipton Barros
Kipton Barros
ECEmmanuel Cieren
Emmanuel Cieren
VEVenmugil Elango
Venmugil Elango
Christoph JunghansChristoph Junghans
Christoph Junghans
TGTimothy C. Germann
Timothy C. Germann

9 January 2015 in ACM Transactions on Architecture and Code Optimization
Verified
Journal Article
Open access
VEVenmugil Elango
Venmugil Elango
NSNaser Sedaghati
Naser Sedaghati
FRFabrice Rastello
Fabrice Rastello
LPLouis-Noël Pouchet
Louis-Noël Pouchet
JRJagannathan Ramanujam
Jagannathan Ramanujam
PSP. Sadayappan
P. Sadayappan
The roofline model is a popular approach for “bound and bottleneck” performance analysis. It focuses on the limits to the performance of processors because of limited bandwidth to off-chip memory. It models upper bounds on performance as a function of operational intensity, the ratio of computational operations per byte of data moved from/to memory. While operational intensity can be directly measured for a specific implementation of an algorithm on a particular target platform, it is of interest to obtain broader insights on bottlenecks, where various semantically equivalent implementations of an algorithm are considered, along with analysis for variations in architectural parameters. This is currently very cumbersome and requires performance modeling and analysis of many variants. In this article, we address this problem by using the roofline model in conjunction with upper bounds on the operational intensity of computations as a function of cache capacity, derived from lower bounds on data movement. This enables bottleneck analysis that holds across all dependence-preserving semantically equivalent implementations of an algorithm. We demonstrate the utility of the approach in assessing fundamental limits to performance and energy efficiency for several benchmark algorithms across a design space of architectural variations.