neural networks research group
areas
people
projects
demos
publications
software/data
Evolutionary Neural AutoML for Deep Learning (2019)
Jason Liang
,
Elliot Meyerson
, Babak Hodjat, Dan Fink, Karl Mutch, and
Risto Miikkulainen
Deep neural networks (DNNs) have produced state-of-the-art results in many benchmarks and problem domains. However, the success of DNNs depends on the proper configuration of its architecture and hyperparameters. Such a configuration is difficult and as a result, DNNs are often not used to their full potential. In addition, DNNs in commercial applications often need to satisfy real-world design constraints such as size or number of parameters. To make configuration easier, automatic machine learning (AutoML) systems for deep learning have been developed, focusing mostly on optimization of hyperparameters. This paper takes AutoML a step further. It introduces an evolutionary AutoML framework called LEAF that not only optimizes hyperparameters but also network architectures and the size of the network. LEAF makes use of both state-of-the-art evolutionary algorithms (EAs) and distributed computing frameworks. Experimental results on medical image classification and natural language analysis show that the framework can be used to achieve state-of-the-art performance. In particular, LEAF demonstrates that architecture optimization provides a significant boost over hyperparameter optimization, and that networks can be minimized at the same time with little drop in performance. LEAF therefore forms a foundation for democratizing and improving AI, as well as making AI practical in future applications.
View:
PDF
Citation:
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2019)
, 401–409, 2019.
Bibtex:
@inproceedings{liang:gecco19, title={Evolutionary Neural AutoML for Deep Learning}, author={Jason Liang and Elliot Meyerson and Babak Hodjat and Dan Fink and Karl Mutch and Risto Miikkulainen}, booktitle={Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2019)}, pages={401–409}, url="http://nn.cs.utexas.edu/?liang:gecco19", year={2019} }
People
Jason Zhi Liang
Ph.D. Alumni
jasonzliang [at] utexas edu
Elliot Meyerson
Ph.D. Alumni
ekm [at] cs utexas edu
Risto Miikkulainen
Faculty
risto [at] cs utexas edu
Areas of Interest
Deep Learning
Evolutionary Computation
Neuroevolution