0.5.0 (2020-12-08)

Highlights:

• Some functionalities have been moved to plugin packages to lighten the core atoti package.

• The web application and the JupyterLab extension have been rewritten from scratch to provide better performances and a simpler experience. atoti’s JupyterLab extension leverages JupyterLab 3’s federated extension system meaning that Node.js and the rebuilding of JupyterLab are not required anymore for its installation. It is also distributed as an atoti plugin instead of a separate npm or Conda package.

• Data can be loaded from more sources: Amazon S3, Azure Blob Storage, Google Cloud Storage, and SQL databases. On the fly decompression of CSV files stored in .gz, .tar.gz, or .zip archives has also been added.

• The name of the default dimension of a hierarchy has changed from Hierarchies to the name of the store on which the hierarchy is based.

Added

• Plugins bringing additional features:

  • atoti-azure to load CSV and parquet file from Azure Blob Storage.

  • atoti-gcp to load CSV and parquet file from Google Cloud Storage.

  • atoti-jupyterlab to make interactive visualizations on top of atoti cubes in JupyterLab. It enables the session.Session.visualize() and query.session.QuerySession.visualize() methods.

  • atoti-sql to load results of SQL queries into atoti stores.

• Reports about the data loaded into stores including number of lines, errors and duration. A warning is now issued in the notebook if an error occured during the loading (#58 and #64).

• Support for path parameters in endpoint()’s route parameter.

• Hierarchy visibility can be toggled through the visible attribute.

• Support for reading .gz, .tar.gz and .zip files containing compressed CSV(s) (#123).

• array.n_lowest_indices() and array.n_greatest_indices() to retrieve the indices of the lowest or greatest values of an array measure (#153).

• array.prod() to do the product of all the elements of an array (#113).

• value() to create a measure based on the value of a store column.

• hierarchized_columns parameter to select which columns of a store are converted into hierarchies. It is available in these methods:

  • session.Session.create_store()

  • session.Session.read_csv()

  • session.Session.read_numpy()

  • session.Session.read_pandas()

  • session.Session.read_parquet()

  • session.Session.read_spark()

  • session.Session.read_sql()

• config.create_ldap_authentication() to setup LDAP authentication in Atoti+.

• Support for multiple hierarchies in total().

• Support for negative value in array indexing (#149).

• Support for named_measure.NamedMeasure representing booleans in where()’s condition parameter (#94).

• cube.Cube.create_store_column_parameter_hierarchy() to create parameter hierarchies from existing store columns.

• array.quantile_index() returning the index of the desired quantile.

• Measure description can be changed (#167).

• Runtime type checking on all the public API functions.

• branding, extra_jars, https, and same_site parameters to create_config().

Experimental

The atoti.experimental module regroups new features that can go through breaking changes in minor and/or patch releases. Its initial content is:

• atoti.experimental.distributed to create distributed clusters of atoti cubes.

• atoti.experimental.finance.irr() to compute an internal rate of return.

• atoti.experimental.stats providing the probability distribution functions pdf, cdf and ppf for Normal, Chi-square, Student’s t, Beta and F distributions.

Changed

• BREAKING: The web application requires a new initial file structure in the metadata DB. Metadata DBs created in previous versions are not compatible with this version and will have to be recreated.

• BREAKING: Cube.visualize() has been replaced with session.Session.visualize() that requires the atoti-jupyterlab plugin. Widgets made with Cube.visualize() will have to be rebuilt with the new JupyterLab extension.

• BREAKING: AWS S3 and Kafka loading are no longer supported in the base package, they require the plugins atoti-aws and atoti-kafka respectively.

• BREAKING: Hierarchy are put in a dimension with the same name as the store which feeds their levels.

• BREAKING: math functions have been moved to the atoti.math module: math.abs(), math.ceil(), math.cos(), math.exp(), math.floor(), math.log(), math.log10(), math.max(), math.min(), math.round(), math.sin(), math.sqrt() and math.tan().

• BREAKING: parent_value()’s degree parameter has been replaced by a degrees mapping to support multiple hierarchies.

• BREAKING: comparator.first_members()’s members parameter has been made variadic instead of accepting a collection.

• BREAKING: atoti.types module and AtotiType class have been respectively renamed atoti.type and DataType. Array and nullable types have also been renamed for improved grammar and consistency.

• BREAKING: session.Session.endpoint()’s method parameter has been made keyword-only.

• BREAKING: level.Level.data_type’s type changed from str to DataType.

• BREAKING: The constructors of the following classes are no longer part of the API and have been replaced by factory functions:

  • config.SessionConfiguration → config.create_config()

  • config.BasicAuthentication → config.create_basic_authentication()

  • config.BasicUser → config.create_basic_user()

  • config.OidcAuthentication → config.create_oidc_authentication()

  • query.basic_auth.BasicAuthentication → query.create_basic_authentication()

• BREAKING: cube.Cube.create_parameter_hierarchy() has been renamed cube.Cube.create_static_parameter_hierarchy().

• BREAKING: Store names inferred from file paths are capitalized.

• BREAKING: Key columns cannot be nullable anymore and are automatically made non nullable. String and date columns are also inferred as non nullable.

• BREAKING: config.create_config()’s inherit parameter has been renamed inherit_global_config.

• BREAKING: JSON responses generated from endpoint() are no longer encapsulated into an object with data and status keys.

• BREAKING: .VALUE measures are no longer automatically created from numeric columns of joined stores.

• The first MDX query run by an atoti widget in JupyterLab is no longer executed in Python. Instead, the query is executed client-side like before 0.4.3 and the call to visualize() will block until this first query is done.

• session.Session.query_mdx() and query.session.QuerySession.query_mdx() support any MDX SELECT query (more than 2 axes, measures on rows, or totals). Empty measure values will also be kept as None in the resulting DataFrame instead of being converted to NaN.

• ROLE_USER is no longer automatically added to the role mapping of config.create_oidc_authentication() and must be given explicitly.

• atoti’s Conda package depends on jdk4py so the installation of the openjdk Conda package is no longer required.

• The data loaded while a sampling mode is active is now consistent between store and cube manipulations.

Deprecated

• BREAKING: agg.stop() moved to agg._stop() as its behavior can be replicated with where():

  - m["Stopped price"] = tt.agg.stop(m["Price"], lvl["Product"], lvl["Shop"])
  + m["Stopped price"] = tt.where(
  +     (lvl["Product"] != None) & (lvl["Shop"] != None), m["Price"]
  + )
  

• BREAKING: agg.single_value() moved to agg._single_value() as value() can be used for its main use-case: creating a measure based on the value of a store column.

Fixed

• HTML entities are correctly encoded in widget snpashots (#148).

• Issue with boolean type in Parquet files (#157).

• Issue when passing a measure to the n parameter of array.n_lowest(), array.nth_lowest(), array.n_greatest() and array.nth_greatest() (#159).

• agg.count_distinct() support for measure and scope parameters.

• Issue with date_shift() and date_diff() when applied on a date level with the default "N/A" member (#180).