Changelog

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

0.4.2 (2020-07-16)

Added

• Kafka streaming data source through store.Store.load_kafka().

• session.Session.endpoint() decorator adds HTTP endpoints to the session from a Python callback.

• array.sort() has a new ascending parameter. True by default, it allows to choose the sorting order.

• scope.cumulative() has a new dense parameter. False by default, it allows to choose whether to include all of a level’s members in the cumulative aggregation, even those for which the underlying measure has no values.

• Atoti+ now supports i18n. en-US is the only locale supported by default but additional locales can be made available by providing custom translation files. These can be configured with config.create_config() A good starting point for adding new locales is to use the template containing all the translatable items, which can be obtained by using session.Session.export_translations_template().

• The Gauss error function math.erf() and its complementary math.erfc() (#92).

Changed

• The name_attribute parameter used to select the displayed username when using an OpenID Connect can be configured.

• The scope parameter used to select the requested scopes when using an OpenID Connect provider can be configured. The openid scope is always passed by default.

Fixed

• Missing images in the tutorial (#80).

• Wrong results when using where() (#17).

• Issue when reading pandas DataFrame with NaN (#77).

• hierarchy.Hierarchy.isin() and level.Level.isin() can be used with more than 2 values (#93).

• Issue with array types not displayed correctly in the stores schema.

• Issue when joining a column of type int to a column of type long if the store is based on a parquet file (#76).

0.4.1.0 (2020-06-17)

Added

• New tutorial exploring the main basic features of atoti.

• rank() returns a measure ranking the members of a given hierarchy based on the value of another measure.

• array.prefix_sum() performs the prefix sum of array measures.

• Hierarchies can have the same name if they are in different dimensions. To avoid conflicts, a hierarchy can be accessed via a tuple containing the dimension and the hierarchy: cube.hierarchies["Product", "Size"].

Changed

• Bumped the minimal required version of JupyterLab to 2.1.

• Upgraded from ActiveUI SDK 4.3.7 to 4.3.8.

• Better messages for Java known errors (#43).

• The Auth0 support in Atoti+ has been replaced by the more general OpenID Connect authentication protocol. The structure of the configuration can be seen in the configuration tutorial.

Fixed

• filter()’s measure parameter accepts any value that can be converted to a measure (#22).

• filter() and where() support inequalities on dates as conditions.

• Issue when loading data into a scenario with truncate set to True (#53).

• Issue with agg.quantile() combined with scope.origin().

• Issue when aggregating .VALUE measures using any of the agg.xxx functions (#52).

• Type issue that sometimes happened when chaining operators such as array.quantile() and date_shift().

• Blinking cell updates not appearing in pivot tables with real time queries.

0.4.0 (2020-05-25)

Added

• agg.max_member() and agg.min_member() return a measure equal to the member reaching the correspinding extremum of the passed measure on the given level.

• hierarchy.Hierarchy.isin(), query.hierarchy.QueryHierarchy.isin(), level.Level.isin(), and query.level.QueryLevel.isin() create conditions expressing that a hierarchy or a level should be on one of the given members.

• stores.Stores.schema and cube.Cube.schema: SVG graphs of, respectively, all the session’s stores and the stores used by a cube.

• Pip installation guide.

• store.StoreScenarios.load_csv() loads a directory of CSV files into a store, automatically generating scenarios based on the directory’s structure.

• total() returns the total value on each hierarchy member.

• session.Session.create_store() creates an empty store from a schema.

• Exponentiation operation between measures: measure_a ** measure_b.

Changed

• BREAKING: Hierarchies, levels, and measures can no longer be passed by name, instances of the corresponding class are expected instead.

• BREAKING: atoti.create_session()’s port, max_memory, java_args and sampling_mode parameters and the ATOTI_URL_PATTERN environment variable have been moved to the config.SessionConfiguration changing these signatures:

  • create_session(): (name='Unnamed', sampling_mode=SamplingMode(name='limit_lines', parameters=[10000]), port=None, max_memory=None, java_args=None, config=None, **kwargs) → (name='Unnamed', *, config=None)

  • config.create_config(): (inherit=True, metadata_db=None, roles=None, authentication=None, properties=None) → (*, inherit=True, port=None, url_pattern=None, metadata_db=None, roles=None, authentication=None, sampling_mode=None, max_memory=None, java_args=None)

• BREAKING: New structure for the authentication configuration in YAML as shown in the configuration tutorial <tutorial/02-configuration:Auth0>.

• BREAKING: config.BasicUser.roles and config.Auth0Authentication.role_mapping do not accept role instances anymore, only role names.

• BREAKING: The wildcard value in measure simulations has been changed from * to None.

• BREAKING: session.Session.read_pandas(), session.Session.read_spark() and session.Session.read_numpy() require a name for the created store:

  • session.Session.read_numpy(): (data, columns, store_name, keys, in_all_scenarios=True, partitioning=None, sep='|') → (array, columns, store_name, *, keys=None, in_all_scenarios=True, partitioning=None, **kwargs)

  • session.Session.read_pandas(): (dataframe, keys=None, store_name=None, partitioning=None, types=None, **kwargs) → (dataframe, store_name, *, keys=None, in_all_scenarios=True, partitioning=None, types=None, **kwargs)

  • session.Session.read_spark(): (dataframe, keys=None, store_name=None, partitioning=None) → (dataframe, store_name, *, keys=None, in_all_scenarios=True, partitioning=None)

• BREAKING: simulation.Scenario.insert(row) and store.Store.insert_rows(rows) have been renamed simulation.Scenario.append() and store.Store.append(). They take a variadic *rows parameter and in place addition of a single row is still supported with +=.

• BREAKING: percentile and variance functions have been renamed quantile and var:

  • agg.percentile(measure, percentile_value, mode='inc', interpolation='linear', scope=None) → agg.quantile() and (measure, q, *, mode='inc', interpolation='linear', scope=None)

  • array.percentile(measure, percentile_value, mode='inc', interpolation='linear') → array.quantile() and (measure, q, *, mode='inc', interpolation='linear')

  • agg.variance(measure, mode='sample', scope=None) → agg.var() and (measure, *, mode='sample', scope=None)

  • array.variance(measure, mode='sample') → array.var() and (measure, *, mode='sample')

• BREAKING: avg has been renamed mean with .MEAN suffix for automatically created measures instead of .AVG:

  • agg.avg(measure, scope=None) → agg.mean() and (measure, *, scope=None)

  • array.avg() → array.mean()

• BREAKING: Some function signatures have changed:

  • cube.Cube.create_parameter_hierarchy(): (level_and_hierarchy_name, members, indices=None, slicing=True, index_measure='', level_type=None) → (name, members, *, data_type=None, index_measure=None, indices=None, store_name=None) where slicing has been removed since it can be set afterwards through: hierarchy.Hierarchy.slicing.

  • parent_value(): (measure, on_hierarchies=None, top_value=None) → (measure, on, *, apply_filters=False, degree=1, total_value=None). The two new parameters default to values equivalent to the previous behavior; see the function documentation for more details.

  • scope.cumulative(): (level, partitioning=None, window=range(-2147483648, 0), exclude_self=False) → (level, *, partitioning=None, window=range(-2147483648, 0), exclude_self=False) where window can also accept a tuple of two time offsets to perform a rolling time period aggregation.

  • simulation.Scenario.load_csv(): (file, delimiter=',') → (path, *, sep=',')

• BREAKING: Some other function signatures have changed only to adopt keyword-only parameters (denoted by a * in the parameter list):

  • agg.count_distinct()

  • agg.long()

  • agg.max()

  • agg.median()

  • agg.min()

  • agg.prod()

  • agg.short()

  • agg.single_value()

  • agg.square_sum()

  • agg.std()

  • agg.sum()

  • array.std()

  • cube.Cube.setup_simulation()

  • date_diff()

  • open_query_session()

  • query.session.QuerySession.query_mdx()

  • scope.siblings()

  • session.Session.create_cube()

  • session.Session.explain_mdx_query()

  • session.Session.query_mdx()

  • session.Session.read_csv()

  • session.Session.read_parquet()

  • shift()

  • simulation.Simulation.load_csv()

  • store.Store.join()

  • store.Store.load_csv()

  • store.Store.load_pandas()

  • store.Store.load_parquet()

  • store.Store.load_spark()

• Upgraded from ActiveUI SDK 4.3.5 to 4.3.7. Pivot tables support new Tree, Pivot, and Table layouts, the latter making the Tabular View widget redundant so it has been removed from the available widgets.

• session.Session.read_pandas(), store.Store.load_pandas(), simulation.Simulation.load_pandas(), and simulation.Scenario.load_pandas() automatically load columns made of numerical Python lists or Numpy one-dimensional ndarrays as arrays.

• Stores without key columns are partitioned on their non-numerical columns by default.

• Changed the behavior of agg.single_value() aggregation function to be more consistent with other aggregation functions (#40).

• Cube names are not restricted to alphanumeric strings without spaces anymore.

• The path parameter of all CSV loading functions accepts glob patterns (e.g. /path/**/*.csv).

Removed

• BREAKING: simulation.Priority. Directly pass numbers to rank simulation rules instead.

• BREAKING: Cube.create_bucketing() has moved to Cube._setup_bucketing() and is not part of the public API anymore. It might change in future releases without notice.

• BREAKING: config.create_config()’s properties parameter. max_memory can be passed directly as a named-parameter instead. The other properties have been removed.

• BREAKING: pow(measure_a, measure_b) replaced by measure_a ** measure_b.

Fixed

• Inability to install atoti alongside Python > 3.7 when using Conda.

• Issue with filter() not being aggregated correctly (#17, #28).

• Metadata DBs created in atoti can be used in Atoti+ and reciprocally (#15).

• Inability to create some measures or hierarchies after some partial joins (#4, #10).

• Inability to load CSV folders from AWS S3 storage.

• Slow read of files on AWS S3 when anonymous due to multiple timeouts in the credentials provider (#26).

• Inability to use wildcards on fields types other than strings.

• Inability to use numeric levels for measure simulations.

0.3.1 (2020-04-14)

First public release of atoti.