0.5.0 (December 08, 2020)#

Highlights:

  • Some functionalities have been moved to plugin packages to lighten the core atoti package.

  • The web app and the JupyterLab extension have been rewritten from scratch to provide better performances and a simpler experience. Atoti’s JupyterLab extension leverages JupyterLab 3’s federated extension system meaning that Node.js and the rebuilding of JupyterLab are not required anymore for its installation. It is also distributed as an Atoti plugin instead of a separate npm or Conda package.

  • Data can be loaded from more sources: Amazon S3, Azure Blob Storage, Google Cloud Storage, and SQL databases. On the fly decompression of CSV files stored in .gz, .tar.gz, or .zip archives has also been added.

  • The name of the default dimension of a hierarchy has changed from Hierarchies to the name of the store on which the hierarchy is based.

Added#

  • Plugins bringing additional features:

    • atoti-azure to load CSV and parquet file from Azure Blob Storage.

    • atoti-gcp to load CSV and parquet file from Google Cloud Storage.

    • atoti-jupyterlab to make interactive visualizations on top of Atoti cubes in JupyterLab. It enables the atoti.Session.visualize() and atoti_query.QuerySession.visualize() methods.

    • atoti-sql to load results of SQL queries into Atoti stores.

  • Reports about the data loaded into stores including number of lines, errors and duration. A warning is now issued in the notebook if an error occurred during the loading (issue #58, issue #64).

  • Support for path parameters in atoti.Session.endpoint()’s route parameter.

  • Hierarchy visibility can be toggled through the atoti.Hierarchy.visible attribute.

  • Support for reading .gz, .tar.gz and .zip files containing compressed CSV(s) (issue #123).

  • atoti.array.n_lowest_indices() and atoti.array.n_greatest_indices() to retrieve the indices of the lowest or greatest values of an array measure (issue #153).

  • atoti.array.prod() to do the product of all the elements of an array (issue #113).

  • atoti.value() to create a measure based on the value of a store column.

  • hierarchized_columns parameter to select which columns of a store are converted into hierarchies. It is available in these methods:

  • atoti.config.create_ldap_authentication() to setup LDAP authentication in Atoti+.

  • Support for multiple hierarchies in atoti.total().

  • Support for negative value in array indexing (issue #149).

  • Support for atoti.named_measure.NamedMeasure representing booleans in atoti.where()’s condition parameter (issue #94).

  • atoti.Cube.create_store_column_parameter_hierarchy() to create parameter hierarchies from existing store columns.

  • atoti.array.quantile_index() returning the index of the desired quantile.

  • Measure ~atoti.named_measure.NamedMeasure.description can be changed (issue #167).

  • Runtime type checking on all the public API functions.

  • branding, extra_jars, https, and same_site parameters to atoti.config.create_config().

Experimental#

The atoti.experimental module regroups new features that can go through breaking changes in minor and/or patch releases. Its initial content is:

  • atoti.experimental.distributed to create distributed clusters of Atoti cubes.

  • atoti.experimental.finance.irr() to compute an internal rate of return.

  • atoti.experimental.stats providing the probability distribution functions pdf, cdf and ppf for Normal, Chi-square, Student’s t, Beta and F distributions.

Changed#

  • BREAKING: The web app requires a new initial file structure in the metadata DB. Metadata DBs created in previous versions are not compatible with this version and will have to be recreated.

  • BREAKING: Cube.visualize() has been replaced with atoti.Session.visualize() that requires the atoti-jupyterlab plugin. Widgets made with Cube.visualize() will have to be rebuilt with the new JupyterLab extension.

  • BREAKING: AWS S3 and Kafka loading are no longer supported in the base package, they require the plugins atoti-aws and atoti-kafka respectively.

  • BREAKING: atoti.Hierarchy are put in a dimension with the same name as the store which feeds their levels.

  • BREAKING: math functions have been moved to the atoti.math module:

  • BREAKING: atoti.parent_value()’s degree parameter has been replaced by a degrees mapping to support multiple hierarchies.

  • BREAKING: atoti.comparator.first_members()’s members parameter has been made variadic instead of accepting a collection.

  • BREAKING: atoti.types module and AtotiType class have been respectively renamed atoti.type and atoti.DataType. Array and nullable types have also been renamed for improved grammar and consistency.

  • BREAKING: atoti.Session.endpoint()’s method parameter has been made keyword-only.

  • BREAKING: atoti.Level.data_type’s type changed from str to atoti.DataType.

  • BREAKING: The constructors of the following classes are no longer part of the API and have been replaced by factory functions:

    • atoti.config.SessionConfiguration → atoti.config.create_config()

    • atoti.config.BasicAuthentication → atoti.config.create_basic_authentication()

    • atoti.config.BasicUser → atoti.config.create_basic_user()

    • atoti.config.OidcAuthentication → atoti.config.create_oidc_authentication()

    • atoti_query.basic_auth.BasicAuthentication → atoti.query.create_basic_authentication

  • BREAKING: atoti.Cube.create_parameter_hierarchy() has been renamed atoti.Cube.create_static_parameter_hierarchy.

  • BREAKING: Store names inferred from file paths are capitalized.

  • BREAKING: Key columns cannot be nullable anymore and are automatically made non nullable. String and date columns are also inferred as non nullable.

  • BREAKING: atoti.config.create_config()’s inherit parameter has been renamed inherit_global_config.

  • BREAKING: JSON responses generated from atoti.Session.endpoint() are no longer encapsulated into an object with data and status keys.

  • BREAKING: .VALUE measures are no longer automatically created from numeric columns of joined stores.

  • The first MDX query run by an Atoti widget in JupyterLab is no longer executed in Python. Instead, the query is executed client-side like before 0.4.3 and the call to atoti.Session.visualize() will block until this first query is done.

  • atoti.Session.query_mdx() and atoti_query.QuerySession.query_mdx support any MDX SELECT query (more than 2 axes, measures on rows, or totals). Empty measure values will also be kept as None in the resulting DataFrame instead of being converted to NaN.

  • ROLE_USER is no longer automatically added to the role mapping of atoti.config.create_oidc_authentication() and must be given explicitly.

  • Atoti’s Conda package depends on jdk4py so the installation of the openjdk Conda package is no longer required.

  • The data loaded while a sampling mode is active is now consistent between store and cube manipulations.

Deprecated#

  • BREAKING: atoti.agg.stop() moved to atoti.agg._stop() as its behavior can be replicated with atoti.where():

    - m["Stopped price"] = tt.agg.stop(m["Price"], l["Product"], l["Shop"])
+ m["Stopped price"] = tt.where(
+     (l["Product"] != None) & (l["Shop"] != None), m["Price"]
+ )

  • BREAKING: atoti.agg.single_value() moved to atoti.agg._single_value() as atoti.value() can be used for its main use-case: creating a measure based on the value of a store column.

Fixed#