Changelog

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

0.4.3 (2020-09-01)

Added

• agg.sum_product() return a measure equal to the product and sum of two measures or fields.

• The modulo % operation between measures (#48).

• Support reading Parquet files from AWS S3 (#27).

• store.Store.drop() remove rows from a store by specifying column names and values.

• atoti widgets are snapshotted as SVG images in the notebook on save. These images will appear in HTML exports or in GitHub previews of the notebook.

• Context menu actions for widgets of the atoti JupyterLab extension:

  • Undo and Redo (#71).

  • Convert to atoti Widget Below available in DataFrames returned by session.Session.query_mdx() and cube.Cube.query() to start an interactive exploration from the same MDX query (#49).

  • Open state editor to navigate to the notebook cell metadata editor (#104).

Changed

• Java is no longer required in the pip installation since the atoti Python wheel now relies on jdk4py.

• The Measure simulations and Source simulation editors have been redesigned. They now both have their dedicated page instead of being widgets embeddable in dashboards. The Measure simulations editor also issue more minimal updates to the underlying store (#70).

• Upgraded from ActiveUI SDK 4.3.8 to 4.3.11 (#30, #59, #74, #79, #84, #87 and #110).

• Upgraded from ActivePivot 5.9.1 to 5.9.2 (#98 and #99).

• session.Session.query_mdx(), cube.Cube.query(), query.session.QuerySession.query_mdx(), and query.cube.QueryCube.query() return DataFrames displaying formatted values and providing a style attribute reflecting the potential styling properties of the corresponding cell set (#91).

• The first MDX query ran by an atoti widget in JupyterLab is now executed in Python and its resulting cell set is outputted to the corresponding notebook cell. It allows to ensure that the data displayed by the widget reflects the expected state of the cube without having to block the IPython kernel in a fragile way.

• Added ability to append rows in all scenarios of a store with store.Store.append().

Removed

• The creation of calculated measures and the action to add a measure computing the difference between to table columns have been disabled in the JupyterLab widget extension to incentivize creating these measures in Python instead. These features are still available in the app (#21).

Deprecated

• The constructors of config.SessionConfiguration, config.BasicAuthentication, config.BasicUser and config.OidcAuthentication have been made private and using them is deprecated. They have been replaced by factory functions: config.create_config(), config.create_basic_authentication(), config.create_basic_user() and config.create_oidc_authentication().

• query.basic_auth.BasicAuthentication, replaced by query.basic_auth.create_basic_authentication().

Fixed

• Issue with dates not correctly converted to Python datetime in store.Store.head() (#97).

• Issue getting vector element with a measure containing long (#115).

• Issue with aggregation function not preserved when creating a simulated measure (#121).

0.4.2 (2020-07-15)

Added

• Kafka streaming data source through store.Store.load_kafka().

• session.Session.endpoint() decorator adds HTTP endpoints to the session from a Python callback.

• array.sort() has a new ascending parameter. True by default, it allows to choose the sorting order.

• scope.cumulative() has a new dense parameter. False by default, it allows to choose whether to include all of a level’s members in the cumulative aggregation, even those for which the underlying measure has no values.

• Atoti+ now supports i18n. en-US is the only locale supported by default but additional locales can be made available by providing custom translation files. These can be configured with config.create_config() A good starting point for adding new locales is to use the template containing all the translatable items, which can be obtained by using session.Session.export_translations_template().

• The Gauss error function math.erf() and its complementary math.erfc() (#92).

Changed

• The name_attribute parameter used to select the displayed username when using an OpenID Connect can be configured.

• The scope parameter used to select the requested scopes when using an OpenID Connect provider can be configured. The openid scope is always passed by default.

Fixed

• Missing images in the tutorial (#80).

• Wrong results when using where() (#17).

• Issue when reading pandas DataFrame with NaN (#77).

• hierarchy.Hierarchy.isin() and level.Level.isin() can be used with more than 2 values (#93).

• Issue with array types not displayed correctly in the stores schema.

• Issue when joining a column of type int to a column of type long if the store is based on a parquet file (#76).

0.4.1 (2020-06-17)

Added

• New tutorial exploring the main basic features of atoti.

• rank() returns a measure ranking the members of a given hierarchy based on the value of another measure.

• array.prefix_sum() performs the prefix sum of array measures.

• Hierarchies can have the same name if they are in different dimensions. To avoid conflicts, a hierarchy can be accessed via a tuple containing the dimension and the hierarchy: cube.hierarchies["Product", "Size"].

Changed

• Bumped the minimal required version of JupyterLab to 2.1.

• Upgraded from ActiveUI SDK 4.3.7 to 4.3.8.

• Better messages for Java known errors (#43).

• The Auth0 support in Atoti+ has been replaced by the more general OpenID Connect authentication protocol. The structure of the configuration can be seen in the configuration tutorial.

Fixed

• filter()’s measure parameter accepts any value that can be converted to a measure (#22).

• filter() and where() support inequalities on dates as conditions.

• Issue when loading data into a scenario with truncate set to True (#53).

• Issue with agg.quantile() combined with scope.origin().

• Issue when aggregating .VALUE measures using any of the agg.xxx functions (#52).

• Type issue that sometimes happened when chaining operators such as array.quantile() and date_shift().

• Blinking cell updates not appearing in pivot tables with real time queries.

0.4.0 (2020-05-25)

Added

• agg.max_member() and agg.min_member() return a measure equal to the member reaching the correspinding extremum of the passed measure on the given level.

• hierarchy.Hierarchy.isin(), query.hierarchy.QueryHierarchy.isin(), level.Level.isin(), and query.level.QueryLevel.isin() create conditions expressing that a hierarchy or a level should be on one of the given members.

• stores.Stores.schema and cube.Cube.schema: SVG graphs of, respectively, all the session’s stores and the stores used by a cube.

• Pip installation guide.

• store.StoreScenarios.load_csv() loads a directory of CSV files into a store, automatically generating scenarios based on the directory’s structure.

• total() returns the total value on each hierarchy member.

• session.Session.create_store() creates an empty store from a schema.

• Exponentiation operation between measures: measure_a ** measure_b.

Changed

• BREAKING: Hierarchies, levels, and measures can no longer be passed by name, instances of the corresponding class are expected instead.

• BREAKING: atoti.create_session()’s port, max_memory, java_args and sampling_mode parameters and the ATOTI_URL_PATTERN environment variable have been moved to the config.SessionConfiguration changing these signatures:

  • create_session(): (name='Unnamed', sampling_mode=SamplingMode(name='limit_lines', parameters=[10000]), port=None, max_memory=None, java_args=None, config=None, **kwargs) → (name='Unnamed', *, config=None)

  • config.create_config(): (inherit=True, metadata_db=None, roles=None, authentication=None, properties=None) → (*, inherit=True, port=None, url_pattern=None, metadata_db=None, roles=None, authentication=None, sampling_mode=None, max_memory=None, java_args=None)

• BREAKING: New structure for the authentication configuration in YAML as shown in the configuration tutorial <tutorial/02-configuration:Auth0>.

• BREAKING: config.BasicUser.roles and config.Auth0Authentication.role_mapping do not accept role instances anymore, only role names.

• BREAKING: The wildcard value in measure simulations has been changed from * to None.

• BREAKING: session.Session.read_pandas(), session.Session.read_spark() and session.Session.read_numpy() require a name for the created store:

  • session.Session.read_numpy(): (data, columns, store_name, keys, in_all_scenarios=True, partitioning=None, sep='|') → (array, columns, store_name, *, keys=None, in_all_scenarios=True, partitioning=None, **kwargs)

  • session.Session.read_pandas(): (dataframe, keys=None, store_name=None, partitioning=None, types=None, **kwargs) → (dataframe, store_name, *, keys=None, in_all_scenarios=True, partitioning=None, types=None, **kwargs)

  • session.Session.read_spark(): (dataframe, keys=None, store_name=None, partitioning=None) → (dataframe, store_name, *, keys=None, in_all_scenarios=True, partitioning=None)

• BREAKING: simulation.Scenario.insert(row) and store.Store.insert_rows(rows) have been renamed simulation.Scenario.append() and store.Store.append(). They take a variadic *rows parameter and in place addition of a single row is still supported with +=.

• BREAKING: percentile and variance functions have been renamed quantile and var:

  • agg.percentile(measure, percentile_value, mode='inc', interpolation='linear', scope=None) → agg.quantile() and (measure, q, *, mode='inc', interpolation='linear', scope=None)

  • array.percentile(measure, percentile_value, mode='inc', interpolation='linear') → array.quantile() and (measure, q, *, mode='inc', interpolation='linear')

  • agg.variance(measure, mode='sample', scope=None) → agg.var() and (measure, *, mode='sample', scope=None)

  • array.variance(measure, mode='sample') → array.var() and (measure, *, mode='sample')

• BREAKING: avg has been renamed mean with .MEAN suffix for automatically created measures instead of .AVG:

  • agg.avg(measure, scope=None) → agg.mean() and (measure, *, scope=None)

  • array.avg() → array.mean()

• BREAKING: Some function signatures have changed:

  • cube.Cube.create_parameter_hierarchy(): (level_and_hierarchy_name, members, indices=None, slicing=True, index_measure='', level_type=None) → (name, members, *, data_type=None, index_measure=None, indices=None, store_name=None) where slicing has been removed since it can be set afterwards through: hierarchy.Hierarchy.slicing.

  • parent_value(): (measure, on_hierarchies=None, top_value=None) → (measure, on, *, apply_filters=False, degree=1, total_value=None). The two new parameters default to values equivalent to the previous behavior; see the function documentation for more details.

  • scope.cumulative(): (level, partitioning=None, window=range(-2147483648, 0), exclude_self=False) → (level, *, partitioning=None, window=range(-2147483648, 0), exclude_self=False) where window can also accept a tuple of two time offsets to perform a rolling time period aggregation.

  • simulation.Scenario.load_csv(): (file, delimiter=',') → (path, *, sep=',')

• BREAKING: Some other function signatures have changed only to adopt keyword-only parameters (denoted by a * in the parameter list):

  • agg.count_distinct()

  • agg.long()

  • agg.max()

  • agg.median()

  • agg.min()

  • agg.prod()

  • agg.short()

  • agg.single_value()

  • agg.square_sum()

  • agg.std()

  • agg.sum()

  • array.std()

  • cube.Cube.setup_simulation()

  • date_diff()

  • open_query_session()

  • query.session.QuerySession.query_mdx()

  • scope.siblings()

  • session.Session.create_cube()

  • session.Session.explain_mdx_query()

  • session.Session.query_mdx()

  • session.Session.read_csv()

  • session.Session.read_parquet()

  • shift()

  • simulation.Simulation.load_csv()

  • store.Store.join()

  • store.Store.load_csv()

  • store.Store.load_pandas()

  • store.Store.load_parquet()

  • store.Store.load_spark()

• Upgraded from ActiveUI SDK 4.3.5 to 4.3.7. Pivot tables support new Tree, Pivot, and Table layouts, the latter making the Tabular View widget redundant so it has been removed from the available widgets.

• session.Session.read_pandas(), store.Store.load_pandas(), simulation.Simulation.load_pandas(), and simulation.Scenario.load_pandas() automatically load columns made of numerical Python lists or Numpy one-dimensional ndarrays as arrays.

• Stores without key columns are partitioned on their non-numerical columns by default.

• Changed the behavior of agg.single_value() aggregation function to be more consistent with other aggregation functions (#40).

• Cube names are not restricted to alphanumeric strings without spaces anymore.

• The path parameter of all CSV loading functions accepts glob patterns (e.g. /path/**/*.csv).

Removed

• BREAKING: simulation.Priority. Directly pass numbers to rank simulation rules instead.

• BREAKING: Cube.create_bucketing() has moved to Cube._setup_bucketing() and is not part of the public API anymore. It might change in future releases without notice.

• BREAKING: config.create_config()’s properties parameter. max_memory can be passed directly as a named-parameter instead. The other properties have been removed.

• BREAKING: pow(measure_a, measure_b) replaced by measure_a ** measure_b.

Fixed

• Inability to install atoti alongside Python > 3.7 when using Conda.

• Issue with filter() not being aggregated correctly (#17, #28).

• Metadata DBs created in atoti can be used in Atoti+ and reciprocally (#15).

• Inability to create some measures or hierarchies after some partial joins (#4, #10).

• Inability to load CSV folders from AWS S3 storage.

• Slow read of files on AWS S3 when anonymous due to multiple timeouts in the credentials provider (#26).

• Inability to use wildcards on fields types other than strings.

• Inability to use numeric levels for measure simulations.

0.3.1 (2020-04-14)

First public release of atoti.