Chulaka Gunasekara

Chulaka Gunasekara

Auto-generated profile, based on publicly available information

Chulaka Gunasekara

IBM Research

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2019
Verified
Conference Paper
Open access
JKJonathan Kay Kummerfeld
Jonathan Kay Kummerfeld
SGSai R. Gouravajhala
Sai R. Gouravajhala
JPJoseph J. Peper
Joseph J. Peper
VAVignesh Athreya
Vignesh Athreya
CGChulaka Gunasekara
Chulaka Gunasekara
WLWalter Lasecki
Walter Lasecki

7 May 2021 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
SKSeokhwan Kim
Seokhwan Kim
MGMichel Galley
Michel Galley
CGChulaka Gunasekara
Chulaka Gunasekara
SLSungjin Lee
Sungjin Lee
AAAdam Atkinson
Adam Atkinson
RGRaghav Gupta
Raghav Gupta
This paper introduces the Eighth Dialog System Technology Challenge. In line with recent challenges, the eighth edition focuses on applying end-to-end dialog technologies in a pragmatic way for multi-domain task-completion, noetic response selection, audio visual scene-aware dialog, and schema-guided dialog state tracking tasks. This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track. We also summarize the results of the submitted systems to highlight the overall trends of the state-of-the-art technologies for the tasks.

11 July 2024 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
CGChulaka Gunasekara
Chulaka Gunasekara
SKSeokhwan Kim
Seokhwan Kim
LDLuis Fernando D'Haro
Luis Fernando D'Haro
ARAbhinav Rastogi
Abhinav Rastogi
Yun-Nung (Vivian) ChenYun-Nung (Vivian) Chen
Yun-Nung (Vivian) Chen
RSRajen Subba
Rajen Subba
This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling with Unstructured Knowledge Access, 2. Multi-domain task-oriented dialog, 3. Interactive evaluation of dialog and 4. Situated interactive multimodal dialog. This paper describes the task definition, provided datasets, baselines, and evaluation setup for each track. We also summarize the results of the submitted systems to highlight the general trends of the state-of-the-art technologies for the tasks.

29 May 2015 in Social Network Analysis and Mining
Verified
Journal Article
Closed access
PDF file
RGR. Chulaka Gunasekara
R. Chulaka Gunasekara
KMKishan Mehrotra
Kishan Mehrotra
Chilukuri K. MohanChilukuri K. Mohan
Chilukuri K. Mohan

19 August 2018
Verified
Book Chapter
Closed access
PDF file
RGR. Chulaka Gunasekara
R. Chulaka Gunasekara
CMChilukuri K. Mohan
Chilukuri K. Mohan
KMKishan Mehrotra
Kishan Mehrotra

7 January 2025 in arXiv
Verified
Preprint
Open access
PDF file
YKYannis Katsis
Yannis Katsis
SRSara Rosenthal
Sara Rosenthal
KFKshitij Fadnis
Kshitij Fadnis
CGChulaka Gunasekara
Chulaka Gunasekara
YLYoung-Suk Lee
Young-Suk Lee
MDMarina Danilevsky
Marina Danilevsky
Retrieval-augmented generation (RAG) has recently become a very popular task for Large Language Models (LLMs). Evaluating them on multi-turn RAG conversations, where the system is asked to generate a response to a question in the context of a preceding conversation is an important and often overlooked task with several additional challenges. We present MTRAG: an end-to-end human-generated multi-turn RAG benchmark that reflects several real-world properties across diverse dimensions for evaluating the full RAG pipeline. MTRAG contains 110 conversations averaging 7.7 turns each across four domains for a total of 842 tasks. We also explore automation paths via synthetic data and LLM-as-a-Judge evaluation. Our human and automatic evaluations show that even state-of-the-art LLM RAG systems struggle on MTRAG. We demonstrate the need for strong retrieval and generation systems that can handle later turns, unanswerable questions, non-standalone questions, and multiple domains. MTRAG is available at https://github.com/ibm/mt-rag-benchmark.