neural networks research group
areas
people
projects
demos
publications
software/data
Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel (2020)
Xin Qiu
,
Elliot Meyerson
,
Risto Miikkulainen
Neural Networks (NNs) have been extensively used for a wide spectrum of real-world regression tasks, where the goal is to predict a numerical outcome such as revenue, effectiveness, or a quantitative result. In many such tasks, the point prediction is not enough: the uncertainty (i.e. risk or confidence) of that prediction must also be estimated. Standard NNs, which are most often used in such tasks, do not provide uncertainty information. Existing approaches address this issue by combining Bayesian models with NNs, but these models are hard to implement, more expensive to train, and usually do not predict as accurately as standard NNs. In this paper, a new framework (RIO) is developed that makes it possible to estimate uncertainty in any pretrained standard NN. The behavior of the NN is captured by modeling its prediction residuals with a Gaussian Process, whose kernel includes both the NN's input and its output. The framework is justified theoretically and evaluated in twelve real-world datasets, where it is found to (1) provide reliable estimates of uncertainty, (2) reduce the error of the point predictions, and (3) scale well to large datasets. Given that RIO can be applied to any standard NN without modifications to model architecture or training pipeline, it provides an important ingredient for building real-world NN applications.
View:
PDF
Citation:
In
International Conference on Learning Representations
, 2020.
Bibtex:
@inproceedings{qiu:iclr20, title={Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel}, author={Xin Qiu and Elliot Meyerson and Risto Miikkulainen}, booktitle={International Conference on Learning Representations}, month={ }, url="http://nn.cs.utexas.edu/?qiu:iclr20", year={2020} }
Presentation:
Video
People
Elliot Meyerson
Ph.D. Alumni
ekm [at] cs utexas edu
Risto Miikkulainen
Faculty
risto [at] cs utexas edu
Xin Qiu
Collaborator
xin qiu [at] cognizant com
Areas of Interest
Supervised Learning
Other Areas