|Next release:||Version 0.10.0|
|Previous release:||Version 0.8.0|
.__str__()now returns a string containing the preview of the frame’s data. This allows datatable frames to be used with
dt.median()can be used to compute median of a certain column or expression, either per group or for the entire Frame. #1530
Frameclass is now defined fully in C++, improving code robustness and performance. The property
Frame.internalwas removed, as it no longer represents anything. Certain internal properties of
Framecan be accessed via functions declared in the
.rbind()can now also accept a list or tuple of frames (previously only a vararg sequence was allowed).
Frame can now be treated as an iterable over the columns. Thus, a Frame object can now be used in a for-loop, producing its individual columns.
A Frame can now be treated as a mapping; in particular both
**frameare now valid.
Single-column frames can be be used as sources for Frame construction.
.to_csv(). The accepted values are the 4 constants from the standard
.to_csv()now quotes fields containing single-quote mark (
.to_csv(), with possibility to request gzip compression for the output file. By default the compression method will be inferred from the file name.
.to_numpy()now returns a numpy
masked_arrayif the frame has any NA values. #1619
A Frame will no longer be shown in “interactive” mode in console by default. The previous behavior can be restored with
dt.options.display.interactive = True. Alternatively, you can explore a Frame interactively using
Improved performance of type-casting a view column: now the code avoids materializing the column before performing the cast.
Fixed crash in certain circumstances when a key was applied after a groupby. #1639
A keyed frame will now be rendered correctly when viewing it in python console via
str32column can no longer overflow during the
.replace()operation, or when converting from python, numpy or pandas, etc. In all these cases we will now transparently create a
str64column instead. #1694
The reported frame size (
sys.getsizeof(DT)) is now more accurate; in particular the content of string columns is no longer ignored. #1697
Type casting into
str32no longer produces an error if the resulting column is larger than 2GB. Now a
str64column will be returned instead. #1695
Fixed memory leak during computation of a generic
DT[i, j]expression. Another memory leak was during generation of string columns, now also fixed. #1705
Pandas “boolean column with NAs” (of dtype
object) now converts into datatable
bool8column when pandas DataFrame is converted into a datatable Frame. #1730
Fixed conversion to numpy of a view Frame which contains NAs. #1738
Fixed issue with mis-aligned frame headers in IPython, caused by IPython inserting
Out[X]:in front of the rendered Frame display. #1793
Improved rendering of Frames in terminals with white background: we no longer use
'bright_white'color for emphasis, only
Fixed crash when a new column was created via partial assignment, i.e.
DT[i, "new_col"] = expr. #1800
Fixed memory leaks/crashes when materializing an object column. #1805
Fixed creating a Frame from a pandas DataFrame that has duplicate column names. #1816
Fixed a UnicodeDecodeError that could be thrown when viewing a Frame with unicode characters in Jupyter notebook. The error only manifested for strings that were longer than 50 bytes in length. #1825
Fixed crash when
Frame.colindex()was used without any arguments, now this properly raises an exception. #1834
Fixed possible crash when writing to disk that doesn’t have enough free space on it. #1837
Fixed invalid Frame being created when reading a large string column (str64) with fread, and the column contains NA values.
Fixed crash that occurred when sorting by multiple columns, and the first column is of low cardinality. #1857
Fixed display of NA values produced during a join, when a Frame was displayed in Jupyter Lab. #1872
Fixed a crash when replacing values in a str64 column. #1890
.cbind()no longer throws an error when passed a generator producing temporary frames. #1905
Fixed comparison of string columns vs. value
Fixed a crash when trying to select individual cells from a joined Frame, for the cells that were un-matched during the join. #1917
Fixed a crash when writing a joined frame into CSV. #1919
Fixed a crash when writing into CSV string view columns, especially of str64 type. #1921
Removed deprecated Frame methods
DT[col]has been restored (was previously deprecated in 0.7.0), however it works only for col an integer or a string. Support for slices may be added in the future, or not: there is a potential to confuse
DT[a:b]for a row selection. A column slice may still be selected via the i-j selector
.to_csv()was removed. If needed, please set the global option
.scalar()is now deprecated and will be removed in release 0.10.0. Please use
.append()is now deprecated and will be removed in release 0.10.0. Please use
.save()was renamed into
.to_jay()(for consistency with other
.to_*()methods). The old name is still usable, but marked as deprecated and will be removed in 0.10.0.
.len()can be applied to a string column to obtain the lengths of strings in each row.
.re_match()applies to a string column, and produces boolean indicator whether each value matches the regular expression
reor not. The method matches the entire string, not just the beginning. Thus, it most closely resembles Python function
dt.options.describe(), which will print the available options together with their values and descriptions.
dt.options.context(option=value), which can be used in a with- statement to temporarily change the value of one or more options, and then go back to their original values at the end of the with-block.
fread.log.escape_unicode(controls treatment of unicode characters in fread’s verbose log); and
display.use_colors(allows to turn on/off colored output in the console).
Some long-running operations in datatable will now show a progress bar. Its behavior can be controlled via
dt.options.progressset of options.
Added an internal function
dt.internal.compiler_version()which reports the compiler version used for compiling the core
datatable.mathmodule is a library of various mathematical functions that can be applied to datatable Frames. The set of functions is close to what is available in the standard python
mathmodule. See documentation for more details.
datatable.sphinxext.dtframe_directive, which can be used as a plugin for Sphinx. This module adds directive
.. dtframethat allows to easily include a Frame display in an .rst document.
datatable no longer uses OpenMP for parallelism. Instead, we use our own thread pool to perform multi-threaded computations #1736.
dt.optionsnow helps the user when they make a typo: if an option with a certain name does not exist, the error message will suggest the correct spelling.
Fixed crash upon exiting from a python terminal, if the user ever called function
datatablecan now be safely used with
multiprocessing, or other modules that perform fork-without-exec.#1758 The child process will spawn its own thread pool that will have the same number of threads as the parent. Adjust
dt.options.nthreadsin the child process(es) if different number of threads is required.
The interactive mode is no longer improperly turned on in IPython. #1789
dt.models.kfold()to prepare indices for k-fold splitting. This function will return
nsplitspairs of row selectors such that when these selectors are applied to an
nrows-rows frame, that frame will be split into train and test part according to the K-fold splitting scheme.
dt.models.kfold_random(), which is similar to
dt.models.kfold(), except that the assignment of rows into folds is randomized instead of being deterministic.
dt.models.aggregate()is removed. In its place you can set the global option
Added early stopping support to FTRL algo, that can now do binomial and multinomial classification for categorical targets, as well as regression for continuous targets.
Fixed FTRL model not resuming properly after unpickling. #1846
This release was created with the help of 5 people who contributed code and documentation, and 16 more people who submitted bug reports and feature requests.
Code & documentation contributors: