metalearn: ensembling controller

[cosmicBboy]

The ensembler can be implemented as an additional component to the metalearn controller architecture.

High-level approach

The RNN controller produces k machine learning frameworks (MLFs), where each hyperparameter is represented as an embedding in the shared RNN embedding space.

Take the last hidden layer of each ML framework and feed them as input to an ensembler RNN that produces a probability [0, 1] that serves as a weight to the ensemble of k MLFs.

These weights are then applied to the validation predictions of the k MLFs as a weighted average to produce a validation performance and reward signal for the ensembler.