LogicBlox 4.25


LogicBlox 4.25.1

Release Date: July 16, 2020

Corrected Issues

The issues listed below have been corrected since the 4.25.0 release:

  • Modeler IDE
    • Entities are no longer accumulated for services used for importing/exporting measures in the Modeler IDE.

Known Issues

The issues listed below are known to exist in the 4.25.1 release.

  • Tools, Database, and Services
    • This release contains changes to some lb-workflow tasks related to workbook creation. These fixes have not been applied yet to lb.wb.CreateWorkbook, which means errors related to recent changes in the LB runtime that imposes restrictions on string lengths can still occur. Please note that this known issue does apply to lb.wb.util.CreateWorkbook.

LogicBlox 4.25.0

Release Date: July 7, 2020

What's New

LogicBlox 4.25.0 introduces the following enhancements:

  • Modeler
    • In this LogicBlox release, we've dropped support for static Vega charts in Modeler. Static Vega charts in your application can be migrated manually to sheet views that use Chart Mode (based on Vega-Lite). For more information, see the Upgrade Information section.

  • Modeler IDE
    • We've redesigned the measure selection functionality in the "Displayable Measures", "Workbook Templates", and "TDX" views of the Modeler IDE. These views use this functionality to allow the configuration of the measures which should either be included or excluded from the configuration:

      • Displayable Measures - select the measures that should be included/excluded in the list of measures shown in the configuration mode of sheet views
      • Workbook Templates - select the measures that should be included/excluded in commit and refresh groups
      • TDX - select the measures that should be included/excluded in the TDX service generation

      When selecting measures in these views, the Modeler IDE shows a list of measures based on the filters specified by the user. The following filters are available for measure selection:

      • Filter by measure component item(s)
      • Filter by measure name(s)
      • Filter by presence of rules and/or inverses

      Example 1. 

      The following screenshot shows the configuration of a commit group in the "Workbook Templates" view. In this example, the user added two filters:

      • "Version: WP" - this filter specifies we only want to select derived measures of measure variants that contain the item WP (Working Plan) of the measure component Version.
      • "Rule/Inverse: Rule - No, Inverse - No" - this filter specifies we only want to select measures that have no CubiQL rules and/or inverse. This is particularly useful when configuring commit and refresh groups for workbook templates because the measure selection of commit and refresh groups is always considered to be a subset of the default commit and refresh groups that only contain measures without measures rules (i.e. EDB and NOT_DERIVED measures).

      These filters are joined, resulting in a selection of only WP measures that have no CubiQL rules and/or inverse.

    • The Modeler IDE now supports the configuration of shifting for actions of type "Copy".

      Shifting is typically used to populate historical versions of measures while adjusting data from previous years to account for the fact that some holidays, like Easter, do not fall in the same week every year. This is significant for planning and forecasting applications because holidays tend to have very different consumer behavior (e.g. higher sales).

      This functionality adds two new configuration options in the "Actions" view for copy actions:

      • Slide - a slide is a tool used by CubiQL to define an aggregation step outside of the predefined level-to-level mappings (e.g. Sku to Subclass) in the hierarchy of a dimension. To populate historical versions of measures, this slide needs to move positions of the Week or Month level of the Calendar dimension to different positions in the same Week or Month level. When mapping a Month to the next year, this is always going to be a 12-month offset, while Week mappings are typically implemented using the 4-5-4 calendar.
      • Scaling Measure - a scaling measure holds a modifier for every Week of the Calendar dimension which will adjust values up or down based on whether or not that week was a holiday week.

      Example 2. 

      The following example shows the configuration of an example copy action in the Modeler IDE.

      This copy action uses the slide TimeShiftTYtoLY to map data to the next year. Roaming holidays like Easter are handled using the scaling measure ScaleWeek.

      The slide TimeShiftTYtoLY is configured in the new "Slides" view of the Modeler IDE.

      This example slide is configured with the following settings:
      • TimeShiftTYtoLY - the name of the slide which will be used in our CubiQL expressions (e.g. collect Sales_U_TY_PLN by slide TimeShiftTYtoLY) executed by the copy action.
      • Calendar_Week_next_year - the name of the predicate (configured using LogiQL) which maps positions from one intersection to positions in another. In this case, the predicate maps each Week to another Week in the following year.
      • Calendar:Week - the source level of the slide.
      • Calendar:Week - the target level of the slide. In our case, the target level is the same as the source level since our predicate defines a mapping between Week levels.

  • Tools, Database, and Services
    • Updated jackson Java libraries used by lb-web, lb-jobs, lb-workflow, and cloud-store to version 2.10.4 to get some security updates and bug fixes to those libraries.

    • Updated the netty library used by lb-web and lb-workflow to version 4.1.46 to get some security fixes.

    • The cover function in LogiQL now allows different timelines per group. See the LogicBlox Core Reference Manual for more detail.

    • The inventory LogiQL functions (cover and uncover) have been updated with many performance enhancements and code improvements. The main highlights of changes include:

      • Added support for specifying order over int/int128 with first/last predicates.

      • Only allow the input variables (e.g. sales and inventory) to be the same type as the output variable (e.g. cover).

      • Make user-specified order mandatory.

      • Avoid sorting each group, using the known position of a week in the order as the index into the buffer, thus avoiding an O(n log n) operation on each group.

      • Updated documentation in code, and general code refactoring and cleanup, including additional assertion checks.

      • New cleaner and more correct implementation of the P2P argument parser.

      • Fixed a bug when int128 is used for sales/inventory values.

      • Fixed several bugs in handling a non-functional atom in the head: incorrect errors were reported about variable bindings, the position of the output variable was often incorrect, and another bug in the parallel maintenance code.

    • Improved the performance of an internal predicate iterator, which in turn improved the join performance for queries that have negated predicates, lookup predicates (e.g. predicates in the script body of TDX rules), and for queries where predicates in the body match on most of their tuples.

    • This release contains performance improvements for SIP rules. In particular, if P is a large predicate and there is a transaction in which a relatively small set of changes (insertions or retractions) is applied to it through an IDB rule, significant improvements can be expected in the running time of the SIP rule for P in that transaction. The improvements affect only the samples maintenance part of SIP rules, thus the impact may be less pronounced for SIP rules whose running time is dominated by index maintenance (i.e., if we have several indices on P). Moreover, the improvements apply only on incremental SIP rule maintenance, thus no impact is expected in the initial evaluation of SIP rules (i.e., the first time an empty predicate is populated through an IDB rule).

      Moreover, this PR introduces parallel SIP full evaluation, as a more efficient alternative to parallel incremental SIP maintenance for cases in which the size of a predicate decreases significantly during a transaction. Apart from improving performance, this also helps with the completeness of samples in some extreme cases where a very large part of a predicate's contents is retracted during a transaction. In some such cases, incremental SIP rule maintenance could produce fewer than ideal samples for the resulting predicate size, while with the changes in this PR this is less likely to happen, and the number of the samples that is produced is generally closer to the ideal.

  • Measure service
    • Introduced an optional limit argument for the sort and keysort aggregations, written as sort<E1> E2 @ I or sort<E1> E2 by G1 … Gn and keysort<E1> E2 @ I or keysort<E1> E2 by G1 … Gn. The expression E1 must have a signature of {} => int. The result of evaluating the expression will indicate how many of the first n position to include in the result.

    • Introduced syntax for named aggregation methods, written as agg<Name> E @ I or agg<Name> E by G1 … Gn. Previously it was possible to refer to named aggregations when writing protobuf or JSON queries, but not in the textual syntax.

    • Improved the measure service one-to-one analysis for expression and path predicates so that more will now be generated with a one-to-one property, allowing for additional optimizations in the runtime.

    • Improved the detection of identity aggregations of min and max on more types so that they can be optimized away.

Corrected Issues

The issues listed below have been corrected since the 4.24.0 release:

  • Modeler
    • Resolved a functional dependency violation (FDV) error that could occur when loading multiple app nodes with the same hostname.

  • Modeler IDE
    • The Modeler IDE generated incorrect CubiQL update expressions for copy actions in cases where the intersection of the filter is partially higher and lower than the intersection of the source measure. This issue has been resolved.

    • Resolved an issue in the "Workbook Templates" view related to the selection of the level maps that should either be included or excluded in a commit or refresh group.

    • The Modeler IDE now deletes an existing build completely before starting a new build of the project. This resolves issues during build that could occur when the user changed the name of an action of type "Service".

    • In the "Displayable Measures" view, the user can now choose to "Exclude" and then configure no measures which results in no measures excluded in the view. This can be useful when a displayable measures configuration has been set at the workbook template level, and the application developer wants to create a view for that workbook that should include all measures configured in the model (e.g. an ad hoc sheet).

    • The "Actions" view now only shows the "Allow HTML?" setting for actions of type "Service". This setting is not relevant for other action types such as "Commit", "Refresh", and "Copy".

  • Tools, Database, and Services
    • Made a performance improvement to lb-web that can significantly reduce the time taken to transmit and process large protobuf messages in lb-web handlers.

    • Fixed some lb-web memory management bugs that could cause lb-web memory consumption to grow continually over time. In some extreme cases with previous releases, hundreds of thousands of lb-web queries could cause the process to run out of memory and crash.

  • Measure service
    • Fixed an issue that could cause a stack overflow when pretty-printing expressions for some updates in the logs.

    • Fixed an issue where querying results with an integer dimension in the REPL could cause an effectively infinite loop.

Known Issues

The issues listed below are known to exist in the 4.25.0 release.

  • Tools, Database, and Services
    • This release contains changes to some lb-workflow tasks related to workbook creation. These fixes have not been applied yet to lb.wb.CreateWorkbook, which means errors related to recent changes in the LB runtime that imposes restrictions on string lengths can still occur. Please note that this known issue does apply to lb.wb.util.CreateWorkbook.

Installation Information

Install LogicBlox 4.25.0 by following the steps outlined below:

  1. Download the LogicBlox installation package for Linux or macOS from the LogicBlox developer website.
  2. Extract the tarball into a directory, which we refer to as <LOGICBLOX_HOME>.
  3. Run the following commands:
    source <LOGICBLOX_HOME>/etc/profile.d/logicblox.sh
    source <LOGICBLOX_HOME>/etc/bash_completion.d/logicblox.sh
    NOTE: these scripts automatically set all the necessary environment variables (please be aware that this script only works if you use bash as your shell). You might want to add these commands to your .bashrc.

Upgrade Information

  • Modeler
    • In this LogicBlox release, we've dropped support for static Vega charts in Modeler. This support has been dropped due to the complexity of the static Vega configuration in the JSON canvas configuration, the low usage rate, and the significant time efforts involved in supporting this functionality. Static Vega charts in your application can be migrated manually to sheet views that use Chart Mode (based on Vega-Lite).
    • The library containing the Modeler migration tool is included in the distribution under <LOGICBLOX_HOME>/lib/npm/modeler-migrations-<VERSION>.tgz and contains a command line tool for doing various transformations/migrations of modeler configurations.

      To use the npm module, you first need to install it by running npm install --no-save <LOGICBLOX_HOME>/lib/npm/modeler-migrations-<VERSION>.tgz. You will then be able to use the command line tool. The module contains a README.md file that lists all available transformations as well as detailed instructions for use.

      To automatically upgrade a Modeler application from one version to another, run the upgrade script to upgrade between versions:

      ./node_modules/.bin/migrate-modeler --fromVersion <prevVer> --toVersion <toVer> /path/to/my/modelerapp

      where the version numbers refer to LogicBlox releases (e.g. 4.25.0).

      NOTE: If you installed the npm module globally, you won't need the ./node_modules/.bin prefix on the command.

Release Information

  • Server requirements
    • Operating System: 64-bit Linux; Apple macOS 10.10+ is supported for local development
    • Java Runtime Environment 8, update 101 or higher
    • Python 3.5 or higher
  • Client requirements
    • Applications using LogicBlox Modeler: Modeler supports major browsers not older than 1 year. Google Chrome provides the best performance.
    • Requirements for applications using non-LogicBlox Modeler components may vary per application.