datatable.models.aggregate()

datatable.models.

aggregate

(

frame

,

min_rows=500

,

n_bins=500

,

nx_bins=50

,

ny_bins=50

,

nd_max_bins=500

,

max_dimensions=50

,

seed=0

,

double_precision=False

,

fixed_radius=None

)

sourcetests

Aggregate a frame into clusters. Each cluster consists of a set of members, i.e. a subset of the input frame, and is represented by an exemplar, i.e. one of the members.

For one- and two-column frames the aggregation is based on the standard equal-interval binning for numeric columns, and grouping for string columns.

When the input frame has more columns than two, a parallel one-pass Ad-Hoc algorithm is employed, see description of Aggregator<T>::group_nd() method for more details. This algorithm takes into account the numeric columns only, and all the string columns are ignored.

Parameters

frame

Frame

The input frame containing numeric or string columns.

min_rows

int

Minimum number of rows the input frame should have to be aggregated. If frame has less rows than min_rows, aggregation is bypassed, in the sence that all the input rows become exemplars.

n_bins

int

Number of bins for 1D aggregation.

nx_bins

int

Number of bins for the first column for 2D aggregation.

ny_bins

int

Number of bins for the second column for 2D aggregation.

nd_max_bins

int

Maximum number of exemplars for ND aggregation. It is guaranteed that the ND algorithm will return less than nd_max_bins exemplars, but the exact number may vary from run to run due to parallelization.

max_dimensions

int

Number of columns at which the projection method is used for ND aggregation.

seed

int

Seed to be used for the projection method.

double_precision

bool

An option to indicate whether double precision, i.e. float64, or single precision, i.e. float32, arithmetic should be used for computations.

fixed_radius

float

Fixed radius for ND aggregation, use it with caution. If set, nd_max_bins will have no effect and in the worst case number of exemplars may be equal to the number of rows in the data. For big data this may result in extremly large execution times. Since all the columns are normalized to [0, 1), the fixed_radius value should be choosen accordingly.

return

Tuple[Frame, Frame]

The first element in the tuple is the aggregated frame, i.e. the frame containing exemplars, with the shape of (nexemplars, frame.ncols + 1), where nexemplars is the number of gathered exemplars. The first frame.ncols columns are the columns from the input frame, and the last column is the members_count that has stype int32 containing number of members per exemplar.

The second element in the tuple is the members frame with the shape of (frame.nrows, 1), each row in this frame corresponds to the row with the same id in the input frame. The only column exemplar_id has an stype of int32 and contains the exemplar ids a particular member belongs to. These ids are effectively the ids of the exemplar’s rows from the input frame.

except

ValueError

The exception is raised if the input frame is missing.

except

TypeError

The exception is raised when one of the frame’s columns has an unsupported stype, i.e. the column is both non-numeric and non-string.