IOPro Release Notes
- Remove warnings and documentation for unsupported Numba use
- Rewrite most documentation for clarity and accuracy
- Improve unit tests
- Add PostgresAdapter for reading data from PostgreSQL databases
- Add AccumuloAdapter for reading data from Accumulo databases
- Fix an issue with pyodbc where result NumPy arrays could return
uninitialized data after the actual data null character. Now it pads
the results with nulls.
- Properly cache output string objects for better performance
- Add Python 3 support
- Add support for parsing utf8 text files
- Add ability to set/get field types in MongoAdapter
- Fix issue with escape char not being parsed correctly inside quoted strings
- Fix issue with using field filters with json parser
- Fix issue with json field names getting mixed up
- Fix issue with return nulls returning wrong “null” for large queries
(more than 10000 rows) in some circumstances.
- Fix issue with reading slices of json data
- Change json parser so that strings fields of numbers do not get converted
to number type by default
- Allow json field names to be specified with field_names constructor
- If user does not specify json field names, use json attribute names as
field names in array result
- Fix issue when reading more than 10000 rows containing unicode strings in platform where ODBC uses UTF-16/UCS2 encoding (notably Windows and unixODBC). The resulting data could be corrupt.
- Fix possible segfault when dealing with unicode strings in platforms where ODBC uses UTF-16/UCS2 encoding (notably Windows and unixODBC)
- Add iopro_set_text_limit function to iopro. It globally limits the size of text fields read by fetchdictarray and fetchsarray. By default it is set to 1024 characters.
- Fix possible segfault in fetchdictarray and fetchsarray when failing to allocate some NumPy array. This could notably happen in the presence of “TEXT” fields. Now it will raise an OutOfMemory error.
- Add lazy loading of submodules in IOPro. This reduces upfront import time of IOPro. Features are imported as they are used for the first time.
- Fix crash when building textadapter index
- Fix default value for null strings in IOPro/pyodbc changed to be an empty string instead of ‘NA’. NA was not appropriate as it can collide with valid data (Namibia country code is ‘NA’, for example), and it failed with single character columns.
- Ignore SQlRowCount when performing queries with fetchsarray and fetchdictarray, since SQLRowCount sometimes returns incorrect number of rows.
- Fix SQL TINYINT is now returned as an unsigned 8 bit integer in fetchdictarray/fetchsarray. This is to match the range specified in SQL (0…255). It was being returned as a signed 8 bit integer before (range -128…127)
- Add Preliminary unicode string support in fetchdictarray/fetchsarray.
- Disable Numba support for version 0.12 due to lack of string support.
- Fix a regression that made possible some garbage in string fields when using fetchdictarray/fetchsarray.
- Fix a problem where heap corruption could happen in IOPro.pyodbc fetchdictarray/fetchsarray related to nullable string fields.
- Fix the allocation guard debugging code: iopro.pyodbc.enable_mem_guards(True|False) should no longer crash.
- Merge Vertica fix for cancelling queries
- Add JSON support
- Misc bug fixes
- Fix crash in IOPro.pyodbc when dealing with nullable datetimes in fetch_dictarray and fetch_sarray.
- Fix issue parsing negative ints with leading whitespace in csv data.
- Allow delimiter to be set to None for csv files with single field.
- Fill in missing csv fields with fill values.
- Fill in blank csv lines with fill values for pandas dataframe output.
- Allow list of field names for TextAdapter field_names parameter.
- Change default missing fill value to empty string for string fields.
- Temporary fix for IndexError exception in TextAdapter.__read_slice method.
- Add ability to specify escape character in csv data
- fixed coredump when using datetime with numpy < 1.7
- Added a cancel method to the Cursor object in iopro.pyodbc.
This method wraps ODBC SQLCancel.
- DECIMAL and NUMERIC types are now working on iopro.pyodbc on regular fetch
functions. They are still unsupported in fetchsarray and fetchdict and
- Add ftp support
- Performance improvements to S3 support
- Misc bug fixes
- Update loadtxt and genfromtxt to reflect numpy versions’ behavior
for dealing with whitespace (default to any whitespace as delimiter,
and treat multiple whitespace as one delimiter)
- Add read/write field_names property
- Add support for pandas dataframes as output
- Misc bug fixes