Niranda Perera

Niranda Perera

Auto-generated profile, based on publicly available information

Niranda Perera

Computing, and Engineering Luddy School of Informatics Indiana University 47408 Bloomington IN USA

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 July 2023 in Future Generation Computer Systems
Verified
Journal Article
Closed access
NPNiranda Perera
Niranda Perera
ASArup Kumar Sarker
Arup Kumar Sarker
MSMills Staylor
Mills Staylor
GLGregor von Laszewski
Gregor von Laszewski
KSKaiying Shan
Kaiying Shan
GFGeoffrey Fox
Geoffrey Fox

28 April 2023
Verified
Book Chapter
Closed access
PDF file
NPNiranda Perera
Niranda Perera
SKSupun Kamburugamuve
Supun Kamburugamuve
CWChathura Widanage
Chathura Widanage
VAVibhatha Abeykoon
Vibhatha Abeykoon
AUAhmet Uyar
Ahmet Uyar
GFGeoffrey Fox
Geoffrey Fox

15 December 2020 in arXiv
Verified
Preprint
Open access
PDF file
NPNiranda Perera
Niranda Perera
VAVibhatha Abeykoon
Vibhatha Abeykoon
CWChathura Widanage
Chathura Widanage
SKSupun Kamburugamuve
Supun Kamburugamuve
TKThejaka Amila Kanewala
Thejaka Amila Kanewala
GFGeoffrey Fox
Geoffrey Fox
In the current era of Big Data, data engineering has transformed into an essential field of study across many branches of science. Advancements in Artificial Intelligence (AI) have broadened the scope of data engineering and opened up new applications in both enterprise and research communities. Aggregations (also termed reduce in functional programming) are an integral functionality in these applications. They are traditionally aimed at generating meaningful information on large data-sets, and today, they are being used for engineering more effective features for complex AI models. Aggregations are usually carried out on top of data abstractions such as tables/ arrays and are combined with other operations such as grouping of values. There are frameworks that excel in the said domains individually. But, we believe that there is an essential requirement for a data analytics tool that can universally integrate with existing frameworks, and thereby increase the productivity and efficiency of the entire data analytics pipeline. Cylon endeavors to fulfill this void. In this paper, we present Cylon's fast and scalable aggregation operations implemented on top of a distributed in-memory table structure that universally integrates with existing frameworks.

29 December 2022 in arXiv
Verified
Preprint
Open access
PDF file
KSKaiying Shan
Kaiying Shan
NPNiranda Perera
Niranda Perera
DLDamitha Lenadora
Damitha Lenadora
TZTianle Zhong
Tianle Zhong
ASArup Sarker
Arup Sarker
Geoffrey Charles FoxGeoffrey Charles Fox
Geoffrey Charles Fox
Data pre-processing is a fundamental component in any data-driven application. With the increasing complexity of data processing operations and volume of data, Cylon, a distributed dataframe system, is developed to facilitate data processing both as a standalone application and as a library, especially for Python applications. While Cylon shows promising performance results, we experienced difficulties trying to integrate with frameworks incompatible with the traditional Message Passing Interface (MPI). While MPI implementations encompass scalable and efficient communication routines, their process launching mechanisms work well with mainstream HPC systems but are incompatible with some environments that adopt their own resource management systems. In this work, we alleviated this issue by directly integrating the Unified Communication X (UCX) framework, which supports a variety of classic HPC and non-HPC process-bootstrapping mechanisms as our communication framework. While we experimented with our methodology on Cylon, the same technique can be used to bring MPI communication to other applications that do not employ MPI's built-in process management approach.

30 July 2021 in arXiv
Verified
Preprint
Open access
PDF file
SKSupun Kamburugamuve
Supun Kamburugamuve
CWChathura Widanage
Chathura Widanage
NPNiranda Perera
Niranda Perera
VAVibhatha Abeykoon
Vibhatha Abeykoon
AUAhmet Uyar
Ahmet Uyar
Geoffrey Charles FoxGeoffrey Charles Fox
Geoffrey Charles Fox
Data-intensive applications impact many domains, and their steadily increasing size and complexity demands high-performance, highly usable environments. We integrate a set of ideas developed in various data science and data engineering frameworks. They employ a set of operators on specific data abstractions that include vectors, matrices, tensors, graphs, and tables. Our key concepts are inspired from systems like MPI, HPF (High-Performance Fortran), NumPy, Pandas, Spark, Modin, PyTorch, TensorFlow, RAPIDS(NVIDIA), and OneAPI (Intel). Further, it is crucial to support different languages in everyday use in the Big Data arena, including Python, R, C++, and Java. We note the importance of Apache Arrow and Parquet for enabling language agnostic high performance and interoperability. In this paper, we propose High-Performance Tensors, Matrices and Tables (HPTMT), an operator-based architecture for data-intensive applications, and identify the fundamental principles needed for performance and usability success. We illustrate these principles by a discussion of examples using our software environments, Cylon and Twister2 that embody HPTMT.

13 October 2020 in arXiv
Verified
Preprint
Open access
PDF file
VAVibhatha Abeykoon
Vibhatha Abeykoon
NPNiranda Perera
Niranda Perera
CWChathura Widanage
Chathura Widanage
SKSupun Kamburugamuve
Supun Kamburugamuve
TKThejaka Amila Kanewala
Thejaka Amila Kanewala
GFGeoffrey Fox
Geoffrey Fox
Data engineering is becoming an increasingly important part of scientific discoveries with the adoption of deep learning and machine learning. Data engineering deals with a variety of data formats, storage, data extraction, transformation, and data movements. One goal of data engineering is to transform data from original data to vector/matrix/tensor formats accepted by deep learning and machine learning applications. There are many structures such as tables, graphs, and trees to represent data in these data engineering phases. Among them, tables are a versatile and commonly used format to load and process data. In this paper, we present a distributed Python API based on table abstraction for representing and processing data. Unlike existing state-of-the-art data engineering tools written purely in Python, our solution adopts high performance compute kernels in C++, with an in-memory table representation with Cython-based Python bindings. In the core system, we use MPI for distributed memory computations with a data-parallel approach for processing large datasets in HPC clusters.