FTRL¶

This section describes the FTRL (Follow the Regularized Leader) model as implemented in datatable.

FTRL Model Information¶

The Follow the Regularized Leader (FTRL) model is a datatable implementation of the FTRL-Proximal online learning algorithm for binomial logistic regression. It uses a hashing trick for feature vectorization and the Hogwild approach for parallelization. FTRL for multinomial classification and continuous targets are implemented experimentally.

Create an FTRL Model¶

The FTRL model is implemented as the Ftrl Python class, which is a part of datatable.models, so to use the model you should first do

from datatable.models import Ftrl

and then create a model as

ftrl_model = Ftrl()

FTRL Model Parameters¶

The FTRL model requires a list of parameters for training and making predictions, namely:

alpha – learning rate, defaults to 0.005.
beta – beta parameter, defaults to 1.0.
lambda1 – L1 regularization parameter, defaults to 0.0.
lambda2 – L2 regularization parameter, defaults to 1.0.
nbins – the number of bins for the hashing trick, defaults to 1000000.
nepochs – the number of epochs to train the model for, defaults to 1.
interactions – whether to enable second order feature interactions, defaults to False.

If some parameters need to be changed, this can be done either when creating the model, as

ftrl_model = Ftrl(alpha = 0.1, nbins = 100, interactions = False)

or, if the model already exists, as

ftrl_model.alpha = 0.1
ftrl_model.nbins = 100
ftrl_model.interactions = False

If some parameters were not set explicitely, they will be assigned the default values.

Training a Model¶

Use the fit() method to train a model for a binomial logistic regression problem:

ftrl_model.fit(X, y)

where X is a frame of shape (nrows, ncols) to be trained on, and y is a frame of shape (nrows, 1) having a bool type of the target column. The following datatable column types are supported for the X frame: bool, int, real and str.

Resetting a Model¶

Use the reset() method to reset a model:

ftrl_model.reset()

This will reset model weights, but it will not affect learning parameters. To reset parameters to default values, you can do

ftrl_model.params = Ftrl().params

Making Predictions¶

Use the predict() method to make predictions:

targets = ftrl_model.predict(X)

where X is a frame of shape (nrows, ncols) to make predictions for. X should have the same number of columns as the training frame. The predict() method returns a new frame of shape (nrows, 1) with the predicted probability for each row of frame X.

Feature Importances¶

To estimate feature importances, the overall weight contributions are calculated feature-wise during training and predicting. Feature importances can be accessed as

fi = ftrl_model.feature_importances

where fi will be a frame of shape (nfeatures, 2) containing feature names and their importances, that are normalized to [0; 1] range.