The field of automatic machine learning (abbreviated as AutoML) concerns all endeavours to automate the process of machine learning. To provide a sense of what could constitute AutoML, let me post a list from the "Call for Papers" of the International Workshop on Automatic Machine Learning (ICML 2018) [1]:
* Model selection, hyper-parameter optimization, and model search
* Neural architecture search
* Meta learning and transfer learning
* Automatic feature extraction / construction
* Demonstrations (demos) of working AutoML systems
* Automatic generation of workflows / workflow reuse
* Automatic problem "ingestion" (from raw data and miscellaneous formats)
* Automatic feature transformation to match algorithm requirements
* Automatic detection and handling of skewed data and/or missing values
* Automatic acquisition of new data (active learning, experimental design)
* Automatic report writing (providing insight on automatic data analysis)
* Automatic selection of evaluation metrics / validation procedures
* Automatic selection of algorithms under time/space/power constraints
* Automatic prediction post-processing and calibration
* Automatic leakage detection
* Automatic inference and differentiation
* User interfaces and human-in-the-loop approaches for AutoML
> I don't see "Automatic design of novel algorithms" in this list. Can AutoML produce something as novel as a GAN, CapsNet, WaveNet, Transformer, Neural ODE, etc? Is that even considered to be one of its goals. In my opinion, there's a clear separation between a group of people trying to improve AutoML so that it's more useful in doing all those tasks on the list, and a group of people trying to invent next gen ML algorithms or DL architectures.
I agree with you from the view of the current state of the art methods and the current state of the AutoML / fundamental ML research communities. Current methods are very limited, but I can not think of a reason why a sufficiently general searchspace of architectures/pipelines could not produce something like a GAN or a WaveNet.
I do not think that designing algorithms as novel as the ones you listed is currently a goal of AutoML, as that is not something we have an attack for. However, I do think that with increasing capabilities, the field of AutoML will seek to automate every step of the machine learning pipeline - including the design of algorithms. E.g., once/if there are attacks to apply NAS for yielding truly novel architectures, I think NAS researchers will be happy to do just that -- wouldn't you call that AutoML then?
I don't see "Automatic design of novel algorithms" in this list.
Can AutoML produce something as novel as a GAN, CapsNet, WaveNet, Transformer, Neural ODE, etc? Is that even considered to be one of its goals?
In my opinion, there's a clear separation between a group of people trying to improve AutoML so that it's more useful in doing all those tasks on the list, and a group of people trying to invent next gen ML algorithms or DL architectures.
