LogicBlox 4.3.14

Release Date: September 2nd 2016

Executive Summary

LogicBlox 4.3.14 moves the database and services further toward supporting self service in two ways:

  • The database includes improved automatic optimization strategies that reduce the need for manual performance tuning of logic.

  • The Measure Service includes its first alpha release of a purely "managed mode", where the declaration of the schema is entirely managed by the service. Users manage the Measure Service model only, and no longer need to manage both the Measure Service model, as well as the database schema.

As this is the first release of these features, we expect to continually improve their effectiveness over the next releases.

LogicBlox 4.3.14 additionally includes various improvements to the LogiQL language, web services, workflow framework, and the cloud-store developer tool.

What's New

Database
  • InsideOut optimizer: Inside-out is a rule rewriting mechanism that transforms a rule into a set of rules that can be more efficiently evaluated. The technique applies to a rule whose joins can be independently, and more efficiently, evaluated as separate rules, with the results of these intermediate rules then aggregated into the final result.

    The theory of Inside-Out is published in and awarded the Best Paper Award by PODS 2016.

    Example 11. 

    An example of where this optimization applies is a common pattern that appears in analytical applications:

    _[class,region,month] = sls_total <- 
      agg<< sls_total = total(sls) >>
        sales[sku, store, week] = sls,
        product_sku_subclass[sku] = subclass, 
        product_subclass_class[subclass] = class,
        location_store_city[store] = city, 
        location_city_region[city] = region,
        time_week_month[week] = month.

    A common optimization rewrite applied manually today to rules such as the above, is to pull out the path computation from sku to class, and store to region, to be separately evaluated. Inside-out optimization obviates the need for such manual tuning.

  • Prefix-join mechanism is now applied more comprehensively to all queries, including those with default-value predicates, and those rules that write into file predicates. We expect some positive performance impacts on application data loads; a full evaluation of the mechanism is in progress and will be released with further prefix-join enhancements in a subsequent release.

LogiQL Language
  • New arithmetic functions int:negate and float:negate predicates. Additionally, {int,float,decimal}:negate can be used as long as at least one variable is bound, regardless of whether the variable is in the key or the value position.

Services Framework
  • lb-web's built-in authentication mechanism now supports case-insensitive usernames, which is now also the default configuration. In previous releases, usernames were treated in a case-sensitive manner (e.g. user Joe was different from user joe). To revert back to the case-sensitive mode, the username_case_sentitive() flag can be pulsed, for example by running the following command from the command line:

    $ lb exec test '+lb:web:credentials:database:username_case_sensitive().' 

    Note

    Note: For any workspace that has two usernames that only differ in case, migrating to this release can raise the following error: Error. Duplicated users found, and case sensitive flag is true.

Measure Service
  • The lb measure-service query command now accepts JSON input via the --in-format json option, and can output JSON via the --out-format json option.

  • It is now possible to configure the maximum number of expressions for which optimized versions and generated logic should be cached. These can be controlled by setting the max_compile_cache and max_optimize_cache fields in lb-measure-service.config.

  • In the editability analysis, visible metrics may now be at intersections incomparable to or less than the metric's base intersection (this feature will be used in a later release to support dimension splitting).

  • When the complete_model field of the MeasureModel protobuf message is set to true, the Measure Service will attempt to manage all predicates referenced in the model. If a predicate backs a level in the measure model that does not exist in the workspace, the Measure Service will generate its predicate declaration; if the predicate is removed from the measure model, it will also be removed from the workspace.

    Note

    Note that this feature is experimental at this moment and will be used in future releases to support some of the self-service related features in applications using modeler-js User Interface Components.

Developer Tools
  • The cloud-store list command now supports listing information on versions of an object.

  • The new command cloud-store du can now be used to provide the size of a bucket or folder.

Workflow
  • The lb workflow documentation generation tool now supports the --input-dir and --input-file options, to specify the location of the .wf files that should be used for generating the documentation.

Corrected Issues

The issues listed below have been corrected since the 4.3.13 release.

  • When using separate compilation, a number of decimal and float built-ins that were not previously recognized are now accepted.

  • Fixed a memory leak due to caching in the Measure Service.

  • The CubiQL REPL now supports the use of the character !.

  • A relabel expression now gets the same default value as the underlying expression, which allows performance optimizations (e.g. in aggregations).

  • modeler-js User Interface Components:
    • Resolved an issue where slice synchronization did not work if the Slice panel was not visible on the grid.

    • Resolved an issue where the Measures shown in Configuration Mode were not limited by the displayableMeasures configuration option.

    • Resolved an issue with the sorting of inline aggregations.

Installation and Upgrade information

Installation Instructions

Installing LogicBlox 4.3.14 is as simple as following the steps outlined below:

  1. Download the installation package.
  2. Extract the tarball in <YourPreferredInstallDirectory>
  3. Run the following command:
    source <YourPreferredInstallDirectory>/logicblox-4.3.14/etc/profile.d/logicblox.sh
    
    NOTE: this script will set all the necessary environment variables. You might want to add this command to your .bashrc.

Upgrade Information

  • The warning FUNC_SEMICOLON_DEPRECATED has been eliminated. The compiler will still recognize the error code just in case there is logic attempting to alter the warning/error status with a pragma.

  • The multi_dim_level_map field of MeasureModel has been deprecated (scheduled to be removed in 4.3.16). Slide messages and the slide field of MeasureModel should now be used instead.

Release Information

Server requirements
Operating System: 64 bit Linux; OSX 10.10+ is supported for local development.
Java Runtime Environment 7, update 11 or higher
Python 2.7 or higher
CPU compatible with the corei7 architecture
Client requirements
Applications using modeler-js User Interface Components: Google Chrome
Requirements for applications using non-modeler-js components may vary per application.