Automated Machine Learning : : Methods, Systems, Challenges.

• Superior document: The Springer Series on Challenges in Machine Learning Series
• Author: Hutter, Frank.
• TeilnehmendeR: Kotthoff, Lars.; Vanschoren, Joaquin.
• Place / Publishing House: Cham : : Springer International Publishing AG,, 2019.; Ã2019.
• Year of Publication: 2019
• Edition: 1st ed.
• Language: English
• Series: The Springer Series on Challenges in Machine Learning Series
• Physical Description: 1 online resource (223 pages)

Table of Contents

• Intro
• Foreword
• Preface
• Acknowledgments
• Contents
• Part I AutoML Methods
• 1 Hyperparameter Optimization
• 1.1 Introduction
• 1.2 Problem Statement
• 1.2.1 Alternatives to Optimization: Ensembling and Marginalization
• 1.2.2 Optimizing for Multiple Objectives
• 1.3 Blackbox Hyperparameter Optimization
• 1.3.1 Model-Free Blackbox Optimization Methods
• 1.3.2 Bayesian Optimization
• 1.3.2.1 Bayesian Optimization in a Nutshell
• 1.3.2.2 Surrogate Models
• 1.3.2.3 Configuration Space Description
• 1.3.2.4 Constrained Bayesian Optimization
• 1.4 Multi-fidelity Optimization
• 1.4.1 Learning Curve-Based Prediction for Early Stopping
• 1.4.2 Bandit-Based Algorithm Selection Methods
• 1.4.3 Adaptive Choices of Fidelities
• 1.5 Applications to AutoML
• 1.6 Open Problems and Future Research Directions
• 1.6.1 Benchmarks and Comparability
• 1.6.2 Gradient-Based Optimization
• 1.6.3 Scalability
• 1.6.4 Overfitting and Generalization
• 1.6.5 Arbitrary-Size Pipeline Construction
• Bibliography
• 2 Meta-Learning
• 2.1 Introduction
• 2.2 Learning from Model Evaluations
• 2.2.1 Task-Independent Recommendations
• 2.2.2 Configuration Space Design
• 2.2.3 Configuration Transfer
• 2.2.3.1 Relative Landmarks
• 2.2.3.2 Surrogate Models
• 2.2.3.3 Warm-Started Multi-task Learning
• 2.2.3.4 Other Techniques
• 2.2.4 Learning Curves
• 2.3 Learning from Task Properties
• 2.3.1 Meta-Features
• 2.3.2 Learning Meta-Features
• 2.3.3 Warm-Starting Optimization from Similar Tasks
• 2.3.4 Meta-Models
• 2.3.4.1 Ranking
• 2.3.4.2 Performance Prediction
• 2.3.5 Pipeline Synthesis
• 2.3.6 To Tune or Not to Tune?
• 2.4 Learning from Prior Models
• 2.4.1 Transfer Learning
• 2.4.2 Meta-Learning in Neural Networks
• 2.4.3 Few-Shot Learning
• 2.4.4 Beyond Supervised Learning
• 2.5 Conclusion
• Bibliography.
• 3 Neural Architecture Search
• 3.1 Introduction
• 3.2 Search Space
• 3.3 Search Strategy
• 3.4 Performance Estimation Strategy
• 3.5 Future Directions
• Bibliography
• Part II AutoML Systems
• 4 Auto-WEKA: Automatic Model Selection and Hyperparameter Optimization in WEKA
• 4.1 Introduction
• 4.2 Preliminaries
• 4.2.1 Model Selection
• 4.2.2 Hyperparameter Optimization
• 4.3 CASH
• 4.3.1 Sequential Model-Based Algorithm Configuration (SMAC)
• 4.4 Auto-WEKA
• 4.5 Experimental Evaluation
• 4.5.1 Baseline Methods
• 4.5.2 Results for Cross-Validation Performance
• 4.5.3 Results for Test Performance
• 4.6 Conclusion
• 4.6.1 Community Adoption
• Bibliography
• 5 Hyperopt-Sklearn
• 5.1 Introduction
• 5.2 Background: Hyperopt for Optimization
• 5.3 Scikit-Learn Model Selection as a Search Problem
• 5.4 Example Usage
• 5.5 Experiments
• 5.6 Discussion and Future Work
• 5.7 Conclusions
• Bibliography
• 6 Auto-sklearn: Efficient and Robust Automated MachineLearning
• 6.1 Introduction
• 6.2 AutoML as a CASH Problem
• 6.3 New Methods for Increasing Efficiency and Robustness of AutoML
• 6.3.1 Meta-learning for Finding Good Instantiations of Machine Learning Frameworks
• 6.3.2 Automated Ensemble Construction of Models Evaluated During Optimization
• 6.4 A Practical Automated Machine Learning System
• 6.5 Comparing Auto-sklearn to Auto-WEKA and Hyperopt-Sklearn
• 6.6 Evaluation of the Proposed AutoML Improvements
• 6.7 Detailed Analysis of Auto-sklearn Components
• 6.8 Discussion and Conclusion
• 6.8.1 Discussion
• 6.8.2 Usage
• 6.8.3 Extensions in PoSH Auto-sklearn
• 6.8.4 Conclusion and Future Work
• Bibliography
• 7 Towards Automatically-Tuned Deep Neural Networks
• 7.1 Introduction
• 7.2 Auto-Net 1.0
• 7.3 Auto-Net 2.0
• 7.4 Experiments
• 7.4.1 Baseline Evaluation of Auto-Net 1.0 and Auto-sklearn.
• 7.4.2 Results for AutoML Competition Datasets
• 7.4.3 Comparing AutoNet 1.0 and 2.0
• 7.5 Conclusion
• Bibliography
• 8 TPOT: A Tree-Based Pipeline Optimization Toolfor Automating Machine Learning
• 8.1 Introduction
• 8.2 Methods
• 8.2.1 Machine Learning Pipeline Operators
• 8.2.2 Constructing Tree-Based Pipelines
• 8.2.3 Optimizing Tree-Based Pipelines
• 8.2.4 Benchmark Data
• 8.3 Results
• 8.4 Conclusions and Future Work
• Bibliography
• 9 The Automatic Statistician
• 9.1 Introduction
• 9.2 Basic Anatomy of an Automatic Statistician
• 9.2.1 Related Work
• 9.3 An Automatic Statistician for Time Series Data
• 9.3.1 The Grammar over Kernels
• 9.3.2 The Search and Evaluation Procedure
• 9.3.3 Generating Descriptions in Natural Language
• 9.3.4 Comparison with Humans
• 9.4 Other Automatic Statistician Systems
• 9.4.1 Core Components
• 9.4.2 Design Challenges
• 9.4.2.1 User Interaction
• 9.4.2.2 Missing and Messy Data
• 9.4.2.3 Resource Allocation
• 9.5 Conclusion
• Bibliography
• Part III AutoML Challenges
• 10 Analysis of the AutoML Challenge Series 2015-2018
• 10.1 Introduction
• 10.2 Problem Formalization and Overview
• 10.2.1 Scope of the Problem
• 10.2.2 Full Model Selection
• 10.2.3 Optimization of Hyper-parameters
• 10.2.4 Strategies of Model Search
• 10.3 Data
• 10.4 Challenge Protocol
• 10.4.1 Time Budget and Computational Resources
• 10.4.2 Scoring Metrics
• 10.4.3 Rounds and Phases in the 2015/2016 Challenge
• 10.4.4 Phases in the 2018 Challenge
• 10.5 Results
• 10.5.1 Scores Obtained in the 2015/2016 Challenge
• 10.5.2 Scores Obtained in the 2018 Challenge
• 10.5.3 Difficulty of Datasets/Tasks
• 10.5.4 Hyper-parameter Optimization
• 10.5.5 Meta-learning
• 10.5.6 Methods Used in the Challenges
• 10.6 Discussion
• 10.7 Conclusion
• Bibliography
• Correction to: Neural Architecture Search.