LogicBlox 4.4.4

Release Date: May 9th 2017

Executive Summary

LogicBlox 4.4.4 introduces the following enhancements:

  • The database includes stability improvements to replication.

  • The Measure Service includes improvements to help developers with debugging, as well as some performance optimizations.

  • Modeler-js is extending its support for self-service by allowing end-users to resize as well as rearrange sheets on the canvas. Support has also been added for an address cell type together with a new entry form for addresses, that is configurable per country. Finally, improvements are included to reduce development effort, such as native support for a workbook list page and a UI for Single-Sign-On.

Note that this release contains a number of backwards incompatible changes for Modeler-js based applications. The upgrade steps are described in detail in the the Upgrade Information section.

What's New

Database

  • Replication now uses a send timeout: if the remote side stops processing data, eventually the primary will give up and the replicator will go into a failed state. In synchronous replication mode, this causes any waiting transactions to fail. Note that this does not mean that replication of a transaction has to take less than 5 minutes, just that it has to keep making progress by reading some data every 5 minutes. The default timeout can be overridden via the replication_send_timeout workspace option.

  • Replication now uses TCP keepalive: This allows detecting the case where a primary or mirror has become unreachable. It also prevents NAT routers and stateful firewalls from killing the connection when there are no transactions for a while. The keepalive idle time is 300 seconds. That is, the kernel will start to send keepalive messages after 300 seconds of idleness.

Language

  • Module name associated with protobuf now must conform to valid LogiQL identifier rules.

    Note

    Note that this is a backward incompatible change.

Services Framework

  • SAML realms can now be configured with a credentials service to create new users in the credentials database when they log for the first time. Additionally, a list of authorization roles can be declared to be given to such new users.

Measure Service

  • Improved debug reporting on inverse selection for updates. This should make it easier to understand why an edit was not allowed or why a different update was performed than expected.

  • Added support for binding intersection and expressions variables in the model (via the var_binding field of the MeasureModel message) and in requests (via the var_binding field of the Request message). This makes it easier to generate smaller queries via avoiding repetition, as well as allows for controlling whether the bound expressions are optimized once or at every use site.

  • Other protocol changes:

    • The multi_dim_level_map field has been removed from the MeasureModel message.

    • The binary_decimal_column and default_values fields of the QueryRequest message now default to true rather than false.

    • The measure_str field has been removed from the QueryRequest message.

    • The expr_str and pred fields have been removed from the ExprSubst message.

    • The MultiDimensionalMap message has been removed.

    • The meausre_str field has been removed from the ExprMetric message.

    • The signature and metric fields have been removed from the UpdateExpr message.

    • The expr_str field has been removed from the UpdateExpr.Query message.

    • The Inverse message is now a child of the InverseGroup message, and its expr_str field has been removed.

    • The expr_str field has been removed from the MetricLock message.

    • The expr_str field has been removed from the DimensionLock message.

    • The multi_dim_map has been removed from the AggExpr.Grouping message.

    • The MULT_MAP kind of AggExpr.Grouping.Kind has been removed.

    • The missing MIN operator kind was added to Op.Kind.

Modeler-js

  • Resizable and Draggable Sheets on a Canvas: The self-service capabilities of Modeler-js have been further extended by allowing users to resize as well as move sheets on a canvas via drag and drop. To resize a sheet, simply click on the lower right hand corner of the sheet and drag to increase or decrease its size. To move a sheet, click on the sheet's title area (next to the link to the sheet-only page) and drag the sheet into its new position.

    Layout changes are saved per user. Therefore, any alterations will still be present upon leaving and returning to the canvas. To reset the canvas to its original layout, users can use the right-click menu of any sheet on the canvas. Select the Reset Canvas Layout option from the menu after right-clicking on the pivot grid's cells or headers. This will return the canvas to it's original layout, regardless of which sheet was right-clicked.

  • Address measure abstraction and cell type: The new cell type address allows users to view addresses in a single cell in the grid and enter the data in the new address editor that is configurable per country. The predicates that support the address abstraction are a part of the pivot library.

    The following components are provided:

    • pivot:config:addressmodel:address_def:

      • This block contains the definition of the address predicate and the component predicates (address lines, city, region).

      • It also contains some validations that are enforced on the address data type, for instance: each address must have a valid region/country; text fields cannot contain special characters.

      • You can also find here predicates that are used to parse the address refmode which is a JSON string representation or the address.

    • pivot:config:addressmodel:country_config: Contains address requirements that are configurable on a per-country basis, such as the number of lines available/required and the labels to use for the address editor (for instance to display the header Province instead of State). The validations on address values are based on the provided country configurations.

    • pivot:config:addressmodel:address_service: Service definitions for importing regions, countries, and the country configurations.

    Below you can find an extensive example on how to configure this feature.

    Example 47. 

    To be able to use the new address measure abstraction, you will first need to extend your build configuration with the execution of the block pivot:config:addressmodel:address_mapping, which will generate the additional data model that you need, such as the Address dimension, and the address level and the required measures, such as the address lines, city and region.

    lb execblock my_ws pivot:config:addressmodel:address_mapping

    To populate the country as well as the region dropdowns, you will need a comma delimited file similar to the example below:

    region,country,country_label
    CA-AB,CAN,Canada
    ..
    US-GA,USA,United States of America
    US-NY,USA,United States of America
    ...
    GB-ABC,GBR,United Kingdom
    GB-LND,GBR,United Kingdom
    ... 

    You can import (and export) this data using the service /addressmodel/address_def. This service populates the predicates defined in the block pivot:config:addressmodel:address_def.

    Next you can configure per country which fields to display on the address editor as well as which of those should be required. The configuration needs to be added to a comma delimited file similar to the example below, that can be imported using the service /addressmodel/country_config.

    country_code,numlines,numlines_required,subregion_required,postal_required,country_label,region_label,subregion_label,postal_label,address_label
    GBR,3,2,true,false,true,Province,Municipality,,
    USA,3,1,true,true,true,,,,

    The images below illustrate how the various address forms per country look like based on the country configuration from above.

    For Canada, a country that has been imported using the /addressmodel/address_def service, but for which no country configuration has been provided, the default address form is being presented to the user, including the default settings with regard to required fields:

    The configuration for the United Kingdom states that 3 address lines should be presented, where the first two lines are required. Additionally, the subregion_label is set to Municipality and the region_label to Province. Municipality and the Postal are required, due to the default setting.

    For the United States of America, we configured 3 address lines to be displayed, from which only 1 is required.

    Once the basic address configuration is in place, it is possible to define measures of type pivot:config:addressmodel:address_def:address. The example below shows the measure configuration in the Measures.csv file for the measure StoreAddress. Notice that we use address as the Format of this measure:

    Measure,Label,Intersection,DataType,DefaultValue,DefaultAgg,DefaultSpread,RecalcRuleName,PercentBase,PercentParentDimension,Format,
    HAlignment,Readonly,SpreadByMetric,DerivationType
    StoreAddress,Store Address,Store,pivot:config:addressmodel:address_def:address,,none,none,,,,address,,,,

    Once the measure has been defined, it can be used on any view, just as any other measure. You can write your own services to import/export data to and from this predicate.

  • Changed URLs and native workbook list page support: The URL suffixes used for accessing the various pages of the modeler application have changed. As a result of this change the workbooks list page is now natively supported for Modeler-js based applications and application teams no longer have to create a separate page for it. The new displayWorkbookList property on the list of props controls whether to show the workbook list as the default index page.

    If your application does not use workbooks, you can continue to use the navigationTreeId property to configure which navigation tree to show by default.

    The new URLs are as follows:

    Suffix Description
    /modeler/signin Sign-in page
    /modeler/sso Single-sign-on page
    /modeler/workbooks Workbooks list page
    /modeler/workbook/'workbook-id'/app/'wb-nav-tree-id' Main page of the workbook
    /modeler/workbook/'workbook-id'/app/'wb-nav-tree-id'/canvas/'canvas-id' Page for the specified canvas
    /modeler/workbook/'workbook-id'/app/'wb-nav-tree-id'/sheet/'sheet-id' Sheet's own page
    /modeler/app/'nav-tree-id' Master workspace's main page
    /modeler/app/'nav-tree-id'/canvas/'canvas-id' Page for the specified canvas in the master
    /modeler/app/'nav-tree-id'/sheet/'sheet-id' Sheet's own page in the master

    Note

    Please follow the upgrade steps listed in the the Upgrade Information section.

  • Built-in UI for Single-Sign-On: Modeler-js now has a built-in UI for Single-Sign-On. The various aspects of the page and the UI can be configured via a property in the Modeler config object.

    The following new properties are available:

    • requestUrl: the full URL that will handle the SSO request and response from the application.

    • appLogo: - relative path to the location of the application logo provided by the client. It will be displayed at the top of the SSO login UI.

    • appName: name of the application to be displayed in the SSO login UI.

    • serviceProviderLogo: relative path to the location of the logo of the service provider. This will be displayed below the text 'Powered by' in the SSO login UI.

    Example 48. 

    Here is an example,

      ...
      {
          ...
          appPrefix: ...,
          workspace: ...,
          singleSignOn: {
              requestUrl: "http://localhost:8080/sso/request",
              appLogo: "/images/client_logo.png",
              appName: "My LogicBlox Application",
              serviceProviderLogo: "/images/my_sp_logo.png"
          },
          ...
      },
      ...

  • The Modeler authorization has been simplified: Previously, applications were responsible for loading a list of Permissions, Operations and Roles specific to the Modeler and then associating users with said Roles. This resulted in applications copying and pasting these CSV files from project to project despite having no control over what's in them. Now, we have built-in the Permissions, Operations and Roles that the Modeler uses. The only thing an application needs to do is load the User/Role mapping which is actually application specific. Currently, there is only one user Role, Modeler User, though we plan on adding more in the future.

    Note

    Please follow the upgrade steps listed in the the Upgrade Information section.

Developer Tools

  • The lb predinfo command now supports an option --transitive. This explicit option replaces the previous confusing behavior, where ‘lb predinfo with one parameter would only return that predicate, and with more parameters would return those predicates plus additionally the entities used in the predicates.

Corrected Issues

The issues listed below have been corrected since the 4.4.3 release:

  • Fixed an issue with type inference of constructors in the head without a corresponding head entity reference.

  • Incorrectly terminated clauses will not report strange errors when adding/executing logic through the runtime.

  • Derived-only nullary predicates are now inlined correctly.

  • Fixed an issue with using named branches in separately compiled inactive logic.

  • Resolved a rare issue in the meta-engine that could correct reference counts.

  • GlobalProtobuf services now use the same method used on the request (GET, POST, etc) to call the target services.

  • Fixed an issue with lb-web's Proxy handler, that would not abort the connection with the target service once the client request is aborted.

  • Resolved an issue where the Measure Service would fail to report a 500 HTTP status code.

  • Resolved an issue where non-convertible locks were not being correctly reported by the Measure Service in model requests.

  • Modeler-js:
    • Resolved an issue where it was not possible to configure measures to be read-only per user was not possible when using workbooks.

    • Resolved an issue that caused the UI to hang when typing into a dropdown cell (Internet Explorer 11 only).

    • It is now possible to set a default column width to a grid when there are no levels or measures on an axis. The following configuration can be used for this in the sheet configuration:

      ..
      "gridOptions": {
        "display": {
          "_PLACEHOLDER_": {
            "width": 150
          }
        }
      },
      ..

    • Resolved an issue where the formatting was not respected for measures using # or #0 as formatting option.

    • Visual hints are now displayed on updates to visible cells both for the active user as well as other users viewing the cell that was updated.

    • Resolved an issue where under certain circumstances a spreading error could occur for users in deferred calc mode.

    • Search terms separated by spaces are now interpreted as disjunction for Exactly Matches, Starts with, Ends with, as well as Contains.

Installation and Upgrade information

Installation Instructions

Install LogicBlox 4.4.4 by following the steps outlined below:

  1. Download the installation package.
  2. Extract the tarball in <YourPreferredInstallDirectory>
  3. Run the following command:
    source <YourPreferredInstallDirectory>/logicblox-4.4.4/etc/profile.d/logicblox.sh
    NOTE: this script will set all the necessary environment variables. You might want to add this command to your .bashrc.

Upgrade Information

  • Modeler-js based applications can use the migration tool that is shipped together with the distribution to upgrade to 4.4.4. Additionally, a number of manual steps will need to be performed, mostly in the custom html and JavaScript files. Due to the nature of these custom files, we were not able to include these in the migration scripts. However, the goal of this upgrade is to reduce project-custom code so that we can automate more upgrades in the future.

    1. Run the migration tool: To run the migration tool, you first need to install the npm module by running npm install path/to/modeler-migrations-<version>.tgz. You will then be able to use the command line tool. The module contains a README.md file that lists all available transformations as well as detailed instructions for use.

      Run the following command to upgrade between versions:

      ./node_modules/.bin/migrate-modeler --fromVersion <prevVer> --toVersion <toVer> /path/to/my/modelerapp

      where the version numbers refer to LogicBlox releases (e.g. 4.4.4 or 4.4.3).

      Note

      If you installed the npm module globally, you won't need the ./node_modules/.bin prefix on the command.

      Example 49. 

      To upgrade a Modeler-js based application from LogicBlox 4.4.3 to 4.4.4, you can run the following command:

      ./node_modules/.bin/migrate-modeler --fromVersion 4.4.3 --toVersion 4.4.4 /path/to/my/modelerapp

    2. Perform changes with regard to Modeler Authorization: Remove the Modeler specific data from the operations.csv, permissions.csv, permission-operations.csv, role-permissions.csv and roles.csv, and any references to these files in workflows / batch scripts.

      Change the user-roles.csv to user the new Modeler User role.

      Example 50. 

      The csv files can typically be found under src/config/authorization. The user-roles.csv would need to be updated from:

      USER|ROLE
      user1|general access role
      user2|general access role

      to:

      USER|ROLE
      user1|Modeler User
      user2|Modeler User

    3. Update navigation trees and workbook template configurations: Starting this release, each workbook template is now expected to have its own navigation tree configuration file. The id used in the navtree configuration file should match the workbook template name.

    4. Perform changes to html files:

      By having defined routes for workbooks and for different navigation trees, applications should only need to have a single HTML file.

      We changed the name of the topmost React component we export from AuthenticatedModelerController to Modeler. The configuration options remain largely unchanged unless specifically noted.

      The onStart prop was removed along with the AuthenticatedModelerController. We are not aware of anyone that was using it and it should therefore not impact you, but if you were, please reach out to us for assistance.

    5. Update sheets and canvases with regard to deprecated features: ModelBrowser is replaced by SheetConfigurationPanel and has been moved from the canvas to the sheet. The ModelBrowser view module in canvases (usually named as browser-tabs-view) is not supported anymore. From now on, the model browser (which is now called SheetConfigurationPanel) should be a part of the sheet view. Any existing canvas with a ModelBrowser view will for now continue to be rendered correctly along with a warning message in the console.

      The sheet configuration no longer needs the Measures view module (usually named as visibleMeasures). It should be deleted from the sheet configuration files. Any existing sheet with a Measures view will for now continue to be rendered correctly along with a warning message in the console.

      Example 51. 

      Below you can find an example use case for the new SheetConfigurationPanel view in the sheet config:
      {
          ...
          "views": {
              "SheetConfigurationPanel": {
                  "module": "SheetConfigurationPanel"
              },
              "rows": {
                  "module": "Rows"
              },
              ...
          },
          "layout": {
              "shape": {
                  "columns": "200px,1,110px",
                  "rows": "39px,1,39px"
              },
              "viewPositions": {
                  "SheetConfigurationPanel": {
                      "col": 0,
                      "row": 0,
                      "colspan": 1,
                      "rowspan": "fill"
                  },
                  "rows": {
                      "col": 1,
                      "row": 2,
                      "colspan": 1,
                      "rowspan": 1
                  },
                  ...
              }
          }
      }
    6. Update the nginx configuration file: The config should direct any non-matching URL request to the main html page of the application. That is the page which instantiates the Modeler component.

      Example 52. 

      Here is an example of how it can be done:

      location = / {
          index "index.html";
          root /static/;
      }
      
      location = /modeler {
          try_files $uri /index.html;
      }
      
      location /modeler/ {
          try_files $uri /index.html;
      }
      
      location / {
          try_files $uri @bloxweb;
          root /static/;
      }

      In the example above, all requests to the root will be served by index.html. Similarly, all requests to /modeler/* will also serve the index.html. All other requests will fall back to bloxweb.

      Note

      No changes should be necessary to the deployed nginx.conf unless you change the rootPrefix to be something other than /modeler.

    7. Step only needed for applications upgrading from 4.3.12 or earlier:

      The /canvas and /sheet services are now only available on the internal ports. These services were deprecated as of release 4.3.13 in favor of the /view and /viewstate services. We have left them available for backwards compatibility so that applications can do a one time migration between the two services. As another step in the deprecation of these services, they are now only available on the internal port. This will not affect anyone that has already upgraded to the new services.

Release Information

Server requirements
Operating System: 64 bit Linux; OSX 10.10+ is supported for local development.
Java Runtime Environment 8
Python 2.7 or higher
Client requirements
Applications using modeler-js User Interface Components: Google Chrome or Internet Explorer 11+
Requirements for applications using non-modeler-js components may vary per application.