LogicBlox 4.4.7

Release Date: September 7th 2017

Executive Summary

LogicBlox 4.4.7 introduces the following enhancements:

  • The database includes a number of fixes around memory usage as well as an important correctness fix for linear recursion.

  • New commands for cloud-store to delete and rename objects.

  • The lb measure-service now uses JSON format by default, and has been extended with the new model subcommand for querying and installing models.

  • Measure model refresh requests are handled much faster now.

  • The specification of the branch containing the master workbook can now be postponed until an installed workbook template is instantiated to create a workbook.

  • The release also comes with some exciting features for Modeler-js based applications, such as:

    • A complete revamp of the UI to improve the usability. See the LogicBlox 4.4.7 release video for an overview of the changes.

    • Ability to configure data validations to help the user with the data entry.

    • Ability to add a confirmation dialog to action buttons.

    • Improvements to the address editor, such as the ability to sort the list of regions and countries as well as support for adding validations to postal codes.

What's New

Measure Service

  • All the lb measure-service subcommands now use JSON as their default input and output format. Previously, some subcommands used the protobuf text format by default.

  • A new lb measure-service model subcommand has been added for querying and installing models.

    Example 19. 

    Querying the model can be done with for example:

    lb measure-service model query --uri <your uri> --out-format <json, proto, etc.>

    Example of installing a model:

    lb measure-service model install --workspace <wsname> --in-format <json, proto, etc.> <file>

  • Model refresh has been optimized heavily. The sending of an admin refresh request has now much lower latency.

Workbook Framework

  • It is now possible to delay the specification of the branch containing the master workbook when installing a workbook template. If the master is not specified at installation time, it must be specified when the template is instantiated to create a workbook.

Modeler-js

  • New Look and Feel: The Modeler has received a complete revamp. To improve the usability, all panels, such as "slices", "rows" and columns" were moved to the top of the sheet. The layout of sheets is not configurable anymore, but it is possible to hide individual panels. Users can also hide / reveal individual panels via the settings cogwheel. Take a look at the LogicBlox 4.4.7 release video for an overview of all the changes.

  • Measure Validations: The Modeler now supports asynchronous as well as synchronous validations. In the UI, the validations are applied for each cell, async validations are applied at the base intersection. The table below gives an overview of all the validation types that are currently supported.

    Validation Async Sync (UI) Metric Attribute
    Min Value x x x x
    Max Value x x x x
    Min Length x x x x
    Max Length x x x x
    Value Regex x x x x
    Allowable Characters x x x x
    Precision   x x x
    Required x x x x
    By Metrics x   x x
    Custom Expression x   x x

    Current Limitations

    • UI validations are applied at the cell being edited regardless of the intersection.

    • Async validations always apply to the base intersection.

    • Synchronous validations have not been implemented yet for the level creation forms.

    • The validation messages of asynchronous validation failures can't be customized yet.

    • Min/Max Value: Numeric type cells cannot be edited to contain a value less than or greater than the specified min/max values. Aggregate values that are spread to the base intersection will be validated asynchronously.

      Example 20. 

      In the example below we have specified that the value of Sales must be within the range of 0 and 1,000,000.

       
      +pivot:config:constraints:metric_min_value[m] = 0.0f,
      +pivot:config:constraints:metric_max_value[m] = 1000000.0f
          <-
          lb:web:measure:Metric_name[m]="Sales".

    • Min/Max Length: String type cells cannot be edited to contain a string whose length is less than or greater than the specified min/max length values. Aggregate values that are spread to the base intersection will be validated asynchronously.

      Example 21. 

      In the example below we have specified that the metric SkuDescription needs to have at least 10 characters and at most 100 characters. Note that it is note mandatory to have both constraints specified.

      +pivot:config:constraints:metric_min_length[m] = 10,
      +pivot:config:constraints:metric_max_length[m] = 100
          <-
          lb:web:measure:Metric_name[m]="SkuDescription".
      

    • Value Regex: Confirms that the entered value adheres to a specified regex pattern. Values will also be validated asynchronously to validate that data at the base intersection also matches the supplied pattern.

      Example 22. 

      The example below shows a regex validation for the metric Str. In this example, the values for Str can only be lowercase letters, 3 to 5 characters long and can have a number in the end:

      +pivot:config:constraints:metric_value_regex[m] = "^[a-z]{3,5}[0-9]?$"
          <-
          lb:web:measure:Metric_name[m]="Str". 

    • Allowable Characters: This constraint is applied to String type cells and is composed of four properties. This constraint is not applied unless one of the four following properties are specified:

      • allowUppercase: value can contain uppercase characters.

      • allowLowercase: value can contain lowercase characters.

      • allowNumerals: value can contain numerals.

      • specialCharacters: user-defined list of allowed special characters.

      If a property is not set, it is assumed to be not allowed. A regex pattern will be created from these flags and will be applied in the UI and asynchronously.

      Example 23. 

      The validation below states that for metric AllowableUpperNumbers only uppercase and numerals are allowed:

      +pivot:config:constraints:metric_allow_numerals[m] = true,
      +pivot:config:constraints:metric_allow_uppercase[m] = true
          <-
          lb:web:measure:Metric_name[m]="AllowableUpperNumbers".

      The validation below states that for the metric AllowableLowerNumSpecial lowercase, numerals as well as the specified special characters []{} are allowed:

      +pivot:config:constraints:metric_allow_numerals[m] = true,
      +pivot:config:constraints:metric_allow_lowercase[m] = true,
      +pivot:config:constraints:metric_special_characters[m] = "{}[]"
          <-
          lb:web:measure:Metric_name[m]="AllowableLowerNumSpecial".

      The validation below states that for the level attribute Location:store:label, only lower characters are allowed:

      +pivot:config:constraints:level_attribute[l, a] = 0,
      +pivot:config:constraints:level_attribute_allow_lowercase[l, a] = true
          <-
          lb:web:measure:Dimension:Level_name[l] = "store",
          lb:web:measure:Dimension:Level_attribute(l, a),
          lb:web:measure:Attribute_name[a] = "label". 

    • Precision: This is a UI-only constraint that applies to numeric cell types. Users will be prevented from entering decimal values at a precision greater than the specified amount.

      Example 24. 

      Below we specify the prevision for the metric SalesEuros to be 3:

      +pivot:config:constraints:metric_value_precision[m] = 3
          <-
          lb:web:measure:Metric_name[m]="SalesEuros".

    • Required: This is also a UI-only constraint that can be applied to Number, String, Boolean, or Entity typed measures. A measure with this constraint will not allow users to delete values from it without specifying a new value. This validation also ignore empty values. However, when users edit an empty value position (cell) they will be required to set a valid value for it.

      Example 25. 

      In the example below, we have marked the metric SkuColor as required:

      +pivot:config:constraints:metric_is_required(m)
          <-
          lb:web:measure:Metric_name[m]="SkuColor".

    • By Metrics: This is an asynchronously-only constraint that allows clients to specify a set of metrics to use for a validation constraints. If the provided metrics are populated with any values, validation errors will be returned. One constraint is generated for each provided metric.

      Example 26. 

      The edits to Location:store:label are restrained in the example below by the metrics ConstraintRestrictSalesBySize and ConstraintDuplicateLabels. Both of these metrics have installed rules that are populated when invalid edits are made:

      +pivot:config:constraints:level_attribute_by_metrics(l, a, m),
          <-
          lb:web:measure:Dimension:Level_name[l] = "store",
          lb:web:measure:Dimension:Level_attribute(l, a),
          lb:web:measure:Attribute_name[a] = "label",
          (
            lb:web:measure:Metric_name[m]="ConstraintRestrictSalesBySize"
          ;
            lb:web:measure:Metric_name[m]="ConstraintDuplicateLabels"
          ).

    • Custom Expression: This is an asynchronously-only constraint that functions similar to the By Metrics constraint but only takes a single metric value.

      Example 27. 

      In the example below we added a constraint on the metric StoreSize based on the metric ConstrintRestrictSalesBySize.

      +pivot:config:constraints:metric_value_custom_expr[m] = "ConstraintRestrictSalesBySize"
          <-
          lb:web:measure:Metric_name[m]="StoreSize".

  • Confirmation dialog for action buttons: It is now possible to add a confirmation dialog to action buttons. When an action gets triggered that is configured to have a confirmation dialog, the user first has to confirm in a separate pop-up window the actual execution of the action.

    Example 28. 

    The example below shows the json configuration for such an action button. Both the title as well as the text that gets displayed on the window are configurable:

    {
        "id": "action_id",
        "is_enabled": true,
        "links": {
            "execute": "/path-to-action"
        },
        "confirmation": true,
        "confirmationTitle": "Please Confirm",
        "confirmationText": "Do you want to continue with the action?"
    }

  • Improvements to address editor: We have added the following improvements to the address editor:

    • We added the ability to sort the list of regions and countries within their respective dropdowns in the address editor. This allows users to place the more commonly used countries at the top of the dropdown list.

      To use this functionality, the region_order and country_order predicates need to be populated. Countries and regions with an order value always take precedent over non-ordered countries and regions. If no order is provided or the orders match, the sorting is based on label. We have also updated the search functionality to search by ID as well as label and we now search using the token prefix instead of matching any set of characters within the token.

    • We now provide two predicates, postal_validation_regex and postal_validation_message that can be configured on a per country basis to allow for validation of address postal codes. If populated, postal codes will be marked as invalid if they do not match the configured regex. The message predicate is used to display the proper format as help text when an invalid code is entered.

  • External dependencies to modules generated out of the CSV specs: Many projects reuse LogiQL libraries and do not want to auto-generate their schema entirely out of CSV files. Instead they want to use predicates defined in external libraries for part of their schema. The Modeler now allows you to define your own level predicates instead of generating them.

    Note

    Note that for now, you can only do this for levels that are at the bottom of their corresponding hierarchies.

    Example 29. 

    This can be done as follows. In Levels.csv:

    Level,Label,Dimension,ElementType,IsOrdered,OrderAttribute,OrderTransform,TransformedType,UseExisting,UseExistingId,UseExistingLabel
    store,store,Location,string,,,,,,,
    sku,sku,Product,string,true,model:external_dependency:addon:sku_index,model:external_dependency:addon:sku_transform,int,model:external_dependency:addon:sku,
    model:external_dependency:addon:sku_id,model:external_dependency:addon:sku_label

    We have extended the file with 3 optional columns. The example above shows two levels:

    • store: Which will be treated as usual, meaning that we will auto-generated a corresponding schema and logic for it.

    • sku: For which we don't want to auto-generate a corresponding schema and instead use the schema defined in the model:external_dependency:addon module.

    In config.py:

    # Compiling the external_dependency library here. But typically this library would be external to the project.
    lb_library(
        name = 'external_dependency',
        srcdir = 'src')
    
    # All we need to do is to declare the external_dependency library/module as a dependency to the generated_schema module.
    modeler.modeler_library(name='generated_schema',
        libraries=['external_dependency'],
        config_dir='src/config')

    Make sure to declare the external library that contains the model that you want to include instead of generating it from CSV files as a dependency to your "generated schema" module.

Developer Tools

  • The new cloud-store delete command supports removing objects and folders from a store. A --force option can be used to allow the delete command to succeed even if the specified object does not exist. If the specified object URL ends in a slash, the command will find and delete all top-level files with the URL as their prefix. Sub-folders will remain unless the --recursive flag is specified. If the specified object URL does not end in a slash, the command will delete at most one object whose key exactly matches the URL.

    Example 30. 

    The command below will report an error if abc.txt does not exist:

    cloud-store delete gs://my-bucket/abc.txt

    The command below will succeed if abc.txt does not exist:

    cloud-store delete gs://my-bucket/abc.txt --force

    The command below will report an error if no files are found with test_folder/ prefix. It will also delete all "top-level" files that match test_folder/ prefix, not recursing:

    cloud-store delete s3://my-bucket/test_folder/

    The command below will succeed if no files are found with the test_folder/ prefix. It will also delete all files that match the test_folder/ prefix, recursing through all sub-folders:

    cloud-store delete s3://my-bucket/test_folder/ --force --recursive

  • A new rename command has been added to cloud-store. The command supports renaming a single object as well as folders (optionally recursively). When renaming a single existing object, the command will act like a move into a folder if the destination looks like a folder (ends in a slash), keeping the original file name. Recursive folder rename operations have the same semantics as the copy command. Without the --recursive flag, all top-level objects in the directory are moved into another directory. With the --recursive flag, all objects that match the source directory prefix will be moved into the destination. The renamed objects should retain the same access control as the original objects. Object renames can cross S3 buckets, but cannot cross different store services (cannot copy from AWS to GCS, for example).

    Example 31. 

    In the example below the file abc.txt is renamed to def.txt in the same bucket:

    cloud-store rename gs://my-bucket/abc.txt gs://my-bucket/def.txt

    In the next example we rename abc.txt to def.txt in subdir/:

    cloud-store rename gs://my-bucket/abc.txt gs://my-bucket/subdir/def.txt

    Here we "move" abc.txt into subdir/ in a different bucket:

    cloud-store rename gs://my-bucket/abc.txt gs://my-other-bucket/subdir/

    In the example below we rename all top-level objects found in test_folder/ to be in some_other_folder/. No sub-folders of test_folder/ will be renamed

    cloud-store rename s3://my-bucket/test_folder/ s3://my-bucket/some_other_folder/

    In the final example we recursively find all objects that match the /test_folder/ prefix in my-bucket and rename them to have the /some_other_folder/ prefix:

    cloud-store rename --recursive s3://my-bucket/test_folder/ s3://my-bucket/some_other_folder/

  • The --overwrite option to the cloud-store download command has been improved to consistently check for local files regardless of whether you are downloading a single file or recursively downloading into directory structures. Previous releases would overwrite without warning in some situations. The download command will also do a better job of cleaning up if a failure happens during the download, deleting any newly created directories or files.

  • A --dry-run option has been added to all potentially destructive cloud-store commands. These include upload, download, copy, rename, and delete.

Corrected Issues

The issues listed below have been corrected since the 4.4.6 release:

  • Resolved a number of issues that were causing a high memory consumption.

  • The linear_recursion P2P now maintains its output correctly when the time dimensions changes.

    Note

    Please note that the LB_LINEAR_REC_MAINTENANCE environment variable that was previously introduced as a workaround has been removed and will no longer have any effect.

  • Resolved an issue that was causing an Attempt to modify the successor-set of immutable page exception.

  • Resolved an issue where IDB predicate contents were not cleared after the last defining rule was removed.

  • Resolved an issue where derivation types were not updated properly when resolved via script<<>> rules.

  • Resolved an issue that was causing an Internal error: Assertion failure: begin+len <= _len exception.

  • Resolved an issue with the string formatting of UUIDs.

  • Resolved an issue that was causing an Illegal index in BoxTupleVectorLookup exception.

  • The cloud-store ls command will no longer fail if the store contains objects with space characters in their key.

  • Model messages with caption fields (Metric, Level, etc.) will now use the name field for the caption if the caption field is not set or is the empty string.

  • Modeler-js:
    • Resolved an issue where under certain circumstances the grid became unresponsive when adding additional levels.

    • Resolved an issue where the scroll position in the viewstate was no longer respected.

    • Resolved an issue where hitting Enter directly after selecting a measure in the Measure Browser was placing the measure onto the Visible Measures panel, even if the Measures pill was not on the grid.

    • Recalc and percent parent measures are now editable when the view configuration is using dimension splitting.

    • Resolved an issue that would cause an exception when double clicking on a cell that is formatted as link or mouse-over.

    • The search field in the Model Browser does not get cleared anymore after selection.

Installation and Upgrade information

Installation Instructions

Install LogicBlox 4.4.7 by following the steps outlined below:

  1. Download the installation package.
  2. Extract the tarball in <YourPreferredInstallDirectory>
  3. Run the following command:
    source <YourPreferredInstallDirectory>/logicblox-4.4.7/etc/profile.d/logicblox.sh
    NOTE: this script will set all the necessary environment variables. You might want to add this command to your .bashrc.

Upgrade Information

  • The library containing the modeler migration tool is included in the distribution under lib/npm/modeler-migrations-<version>.tgz and contains a command line tool for doing various transformations/migrations of modeler configurations.

    To use the npm module, you first need to install it by running npm install path/to/modeler-migrations-<version>.tgz. You will then be able to use the command line tool. The module contains a README.md file that lists all available transformations as well as detailed instructions for use.

    To automatically upgrade a Modeler-js based application from one version to another, run the upgrade script to upgrade between versions:
    ./node_modules/.bin/migrate-modeler --fromVersion <prevVer> --toVersion <toVer> /path/to/my/modelerapp
    where the version numbers refer to LogicBlox releases.

    Note

    If you installed the npm module globally, you won't need the ./node_modules/.bin prefix on the command.

    Example 32. 

    To upgrade a Modeler-js based application from LogicBlox 4.4.3 to 4.4.7, you can run the following command:

    ./node_modules/.bin/migrate-modeler --fromVersion 4.4.3 --toVersion 4.4.7 /path/to/my/modelerapp

Release Information

Server requirements
Operating System: 64 bit Linux; OSX 10.10+ is supported for local development.
Java Runtime Environment 8
Python 2.7 or higher
Client requirements
Applications using modeler-js User Interface Components: Google Chrome or Internet Explorer 11+
Requirements for applications using non-modeler-js components may vary per application.