|Next release:||Version 0.3.1|
|Previous release:||Version 0.2.2|
Frame.tonumpy()now has argument
stypewhich will force conversion into a numpy array of the specific stype.
ltypethat encapsulate the type-system of the
It is now possible to fread from a
Allow columns to be renamed by setting the
namesproperty on the frame.
Internal “MemoryMapManager” will make datatable more robust when opening a frame with many columns on Linux systems. In particular, error 12 “not enough memory” should become much more rare now.
Number of threads used by fread can now be controlled via parameter
It is now possible to supply string argument to
dt.DataTableconstructor, which in turn will try to interpret that argument via
fread()can now read compressed
fread()now automatically skips Ctrl+Z / NUL characters at the end of the file.
It is now possible to create a datatable from string numpy array.
Single-column files with blank lines can now be read successfully.
Fread now recognizes
\r\r\nas a valid line ending.
fread(), as well as ability to detect URLs automatically. The
urlparameter downloads file from HTTP/HTTPS/FTP server into a temporary location and reads it from there. The
cmdparameter executesthe provided shell command and then reads the data from the stdout.
It is now possible to pass
fread()(or any objects exposing method
File path given to
fread()can now transparently select files within
.ziparchives. This doesn’t work with archives-within-archives.
GenericReader now supports auto-detecting and reading UTF-16 files.
GenericReader now attempts to detect whether the input file is an HTML, and if so raises an exception with the appropriate error message.
Datatable can now use either
llvm-5.0depending on what the user has.
sep="", causing the file to be read line-by-line.
rangearguments can now be passed to a DataTable constructor.
Datatable will now fall back to eager execution if it cannot detect LLVM runtime.
Added simple Excel file reader.
It is now possible to select columns from DataTable by type:
df[int]selects all integer columns from
Allow creating DataTable from list, while forcing a specific stype(s).
Added ability to delete rows from a DataTable:
del df[rows, :]
DataTable can now accept pandas/numpy frames with columns of float16 dtype (which will be automatically converted to
isna()function now works on strings too.
save()is now a method of
Warnings now have custom display hook.
Added global option
nthreadswhich control the number of Omp threads used by
datatablefor parallel execution. Example:
dt.options.nthreads = 1.
.scalar()to quickly convert a 1x1 Frame into a python scalar.
.countna1()that are similar to
.max(), etc. but return a scalar instead of a Frame (however they only work with a 1-column Frames).
.nunique()to compute the number of unique values in each column.
Added stats functions
When writing “round” doubles/floats to CSV, they’ll now always have trailing zero. For example,
[0.0, 1.0, 1e23]now produces
df.stypesnow returns a tuple of
stypeelements (previously it was returning a list of strings). Likewise,
df.typeswas renamed into
df.ltypesand now it returns a tuple of
ltypeelements instead of strings.
colnames=in DataTable constructor was renamed to
names=. The old parameter may still be used, but it will result in a warning.
DataTable can no longer have duplicate column names. If such names are given, they will be mangled to make them unique, and a warning will be issued.
Special characters (in the ASCII range
\x00 - \x1F) are no longer permitted in the column names. If encountered, they will be replaced with a dot
Fread now ignores trailing whitespace on each line, even if
' 'separator is used.
Fread on an empty file now produces an empty DataTable, instead of an exception.
skip_lineswas replaced with
skip_to_line, so that it’s more in sync with the similar argument
When saving datatable containing
obj64columns, they will no longer be saved, and user warning will be shown (previously saving this column would eventually lead to a segfault).
DataTableclass was renamed into
“eager” evaluation engine is now the default.
dt.Frame.rbind()was removed: instead you can now rbind frames to an empty frame:
datatablewill no longer cause the C locale settings to change upon importing.
reading a csv file with invalid UTF-8 characters in column names will no longer throw an exception.
colnameswill no longer ignore those column names.
fread(fill=True)will correctly fill missing fields with NAs.
fread(columns=set(...))will correctly handle the case when the input contains multiple columns with the same names.
fread will no longer crash if the input dataset contains invalid utf8/win1252 data in the column headers. #594 #628
fixed bug in exception handling, which occasionally caused empty exception messages.
fixed bug in fread where string fields starting with “NaN” caused an assertion error.
Fixed bug when saving a
DataTablewith unicode column names into
.nffformat on systems where default encoding is not unicode-aware.
Quoted fields are now correctly unquoted in
Fixed a bug in fread which occurred if the number of rows in the CSV file was estimated too low. #664
Fixed fread bug where an invalid
DataTablewas constructed if parameter
max_nrowswas used and there were any string columns. #671
Fixed a rare bug in fread which produced error message “Jump X did not finish reading where jump X+1 started”. #682
Prevented memory leak when using
PyObjectcolumns in conjunction with
View frames can now be properly saved.
Fixed crash when sorting view frame by a string column.
Deleting 0 columns is no longer an error.
Rows filter now works properly when applied to a view table and using “eager” evaluation engine.
Computed columns expression can now be combined with rows expression, or applied to a view Frame.
This release was created with the help of 5 people who contributed code and documentation, and 6 more people who submitted bug reports and feature requests.
Code & documentation contributors: