Quick start¶
If you are already using
tf.estimator.Estimator
, the
fastest way to get up and running with AdaNet is to use the
adanet.AutoEnsembleEstimator
.
This estimator will automatically convert a list of estimators into subnetworks,
and learn to ensemble them for you.
AutoEnsembleEstimator¶
Next you will want to define which estimators you want to ensemble. For example,
if you don’t know if the best model a linear model, or a neural network, or some
combination, then you can try using tf.estimator.LinearEstimator
and
tf.estimator.DNNEstimator
as subnetworks:
import adanet
import tensorflow as tf
# Define the model head for computing loss and evaluation metrics.
head = MultiClassHead(n_classes=10)
# Feature columns define how to process examples.
feature_columns = ...
# Learn to ensemble linear and neural network models.
estimator = adanet.AutoEnsembleEstimator(
head=head,
candidate_pool=lambda config: {
"linear":
tf.estimator.LinearEstimator(
head=head,
feature_columns=feature_columns,
config=config,
optimizer=...),
"dnn":
tf.estimator.DNNEstimator(
head=head,
feature_columns=feature_columns,
config=config,
optimizer=...,
hidden_units=[1000, 500, 100])},
max_iteration_steps=50)
estimator.train(input_fn=train_input_fn, steps=100)
metrics = estimator.evaluate(input_fn=eval_input_fn)
predictions = estimator.predict(input_fn=predict_input_fn)
The above code will train both the linear
and dnn
subnetworks in parallel,
and will average their predictions. After max_iteration_steps=100
steps, the
best subnetwork will compose the ensemble according to its performance on the
training set.
Ensemble strategies¶
The way AdaNet chooses which subnetworks to include in a candidate ensemble is via ensemble strategies.
Grow strategy¶
The default ensemble strategy is adanet.ensemble.GrowStrategy
which will only
select the subnetwork that most improved the ensemble’s performance. The
remaining subnetworks will be pruned from the graph.
All strategy¶
Suppose instead of only selecting the single best subnetwork, you want to
ensemble all of the subnetworks, regardless of their individual performance.
You can pass an instance of the adanet.ensemble.AllStrategy
to the
adanet.AutoEnsembleEstimator
constructor:
estimator = adanet.AutoEnsembleEstimator(
[...]
ensemble_strategies=[adanet.ensemble.AllStrategy()]
candidate_pool={
"linear": ...,
"dnn": ...,
},
[...])
Tutorials¶
To play with AdaNet in Colab notebooks, and learn about more advanced features like customizing AdaNet and training on TPU, see our tutorials section.