Hao Song

Hao Song

Auto-generated profile, based on publicly available information

Hao Song

Intelligent Systems Laboratory, University of Bristol, Bristol, United Kingdom

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 May 2023 in Machine Learning
Verified
Journal Article
Open access
PDF file
TFTelmo Silva Filho
Telmo Silva Filho
HSHao Song
Hao Song
MPMiquel Perello-Nieto
Miquel Perello-Nieto
RSRaul Santos-Rodriguez
Raul Santos-Rodriguez
MKMeelis Kull
Meelis Kull
PFPeter Flach
Peter Flach
This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the level of uncertainty or confidence associated with its instance-wise predictions. This is essential for critical applications, optimal decision making, cost-sensitive classification, and for some types of context change. Calibration research has a rich history which predates the birth of machine learning as an academic field by decades. However, a recent increase in the interest on calibration has led to new methods and the extension from binary to the multiclass setting. The space of options and issues to consider is large, and navigating it requires the right set of concepts and tools. We provide both introductory material and up-to-date technical details of the main concepts and methods, including proper scoring rules and other evaluation metrics, visualisation approaches, a comprehensive account of post-hoc calibration methods for binary and multiclass classification, and several advanced topics.

1 September 2022 in International Journal of Transportation Science and Technology
Verified
Journal Article
Open access
JWJunhua Wang
Junhua Wang
HSHao Song
Hao Song
TFTing Fu
Ting Fu
MBMolly Behan
Molly Behan
JLJie LEI
Jie LEI
QSQiangqiang Shangguan
Qiangqiang Shangguan

23 March 2023 in Scientific Data
Verified
Journal Article
Open access
PDF file
ETEmma Tonkin
Emma Tonkin
MHMichael Holmes
Michael Holmes
HSHao Song
Hao Song
NTNiall Twomey
Niall Twomey
TDTom Diethe
Tom Diethe
Ian James CraddockIan James Craddock
Ian James Craddock
SPHERE is a large multidisciplinary project to research and develop a sensor network to facilitate home healthcare by activity monitoring, specifically towards activities of daily living. It aims to use the latest technologies in low powered sensors, internet of things, machine learning and automated decision making to provide benefits to patients and clinicians. This dataset comprises data collected from a SPHERE sensor network deployment during a set of experiments conducted in the ‘SPHERE House’ in Bristol, UK, during 2016, including video tracking, accelerometer and environmental sensor data obtained by volunteers undertaking both scripted and non-scripted activities of daily living in a domestic residence. Trained annotators provided ground-truth labels annotating posture, ambulation, activity and location. This dataset is a valuable resource both within and outside the machine learning community, particularly in developing and evaluating algorithms for identifying activities of daily living from multi-modal sensor data in real-world environments. A subset of this dataset was released as a machine learning competition in association with the European Conference on Machine Learning (ECML-PKDD 2016).

28 October 2019 in arXiv
Verified
Preprint
Open access
PDF file
MKMeelis Kull
Meelis Kull
MPMiquel Perello-Nieto
Miquel Perello-Nieto
MKMarkus Kängsepp
Markus Kängsepp
TFTelmo Silva Filho
Telmo Silva Filho
HSHao Song
Hao Song
PFPeter Flach
Peter Flach
Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single corrective multiplicative factor for inputs to the last softmax layer. On non-neural models the existing methods apply binary calibration in a pairwise or one-vs-rest fashion. We propose a natively multiclass calibration method applicable to classifiers from any model class, derived from Dirichlet distributions and generalising the beta calibration method from binary classification. It is easily implemented with neural nets since it is equivalent to log-transforming the uncalibrated probabilities, followed by one linear layer and softmax. Experiments demonstrate improved probabilistic predictions according to multiple measures (confidence-ECE, classwise-ECE, log-loss, Brier score) across a wide range of datasets and classifiers. Parameters of the learned Dirichlet calibration map provide insights to the biases in the uncalibrated model.

4 September 2016
Verified
Book Chapter
Closed access
PDF file
HSHao Song
Hao Song
MKMeelis Kull
Meelis Kull
PFPeter Flach
Peter Flach
GKGeorgios Kalogridis
Georgios Kalogridis

16 February 2023 in arXiv
Verified
Preprint
Open access
PDF file
TFTelmo Silva Filho
Telmo Silva Filho
HSHao Song
Hao Song
MPMiquel Perello-Nieto
Miquel Perello-Nieto
RSRaul Santos-Rodriguez
Raul Santos-Rodriguez
MKMeelis Kull
Meelis Kull
PFPeter Flach
Peter Flach
This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the level of uncertainty or confidence associated with its instance-wise predictions. This is essential for critical applications, optimal decision making, cost-sensitive classification, and for some types of context change. Calibration research has a rich history which predates the birth of machine learning as an academic field by decades. However, a recent increase in the interest on calibration has led to new methods and the extension from binary to the multiclass setting. The space of options and issues to consider is large, and navigating it requires the right set of concepts and tools. We provide both introductory material and up-to-date technical details of the main concepts and methods, including proper scoring rules and other evaluation metrics, visualisation approaches, a comprehensive account of post-hoc calibration methods for binary and multiclass classification, and several advanced topics.