LogicBlox 4.11.0

Release Date: April 16, 2019

What's New

LogicBlox 4.11.0 introduces the following enhancements:

  • Modeler
    • Users can now create copies of both preconfigured and user-defined views in Modeler. This new functionality can be accessed by clicking on the ••• icon available on all items in the navigation panel except those configured as an internal or external link. Deleting user views as well as resetting a canvas layout are both accessible by clicking the same icon.

      Example 2. 

      In the following example, the user modifies the view configuration of a preconfigured canvas that contains two sheet views. After copying this view, a new user-defined view appears in the navigation panel which has a default view configuration equal to the source view at the time of copying. This user-defined view can also be copied over to another user-defined view, allowing the user to create and save variations of the same type of view.

    • Modeler's performance related to user permissions has been improved significantly.

    • Significant changes have been made to the underlying model used for data access permissions in the Modeler. These changes likely require updates to the project scripts that are used to build/compile your Modeler project. For more information, see the Upgrade Information section.

    • The measure format definition in Measures.csv has been changed from a string definition to a JSON object. The JSON format is based on numbro.js format with some additional properties specifically added for LogicBlox. For more information, see the Upgrade Information section.

    • Modeler now respects the user's browser locale when formatting numbers. This means you no longer have to specify thousands and decimal separators in Modeler's new measure format definition.

      Example 3. 

      10000.12 will look like 10,000.12 for locale en-US versus 10.000,12 for locale es-ES.

    • It's now possible to configure a scale factor as a part of the new measure format definition. Scale factor changes the displayed value of a scaled metric. The scale factor formatting is maintained when exporting data to CSV/Excel.

      Example 4. 

      For example, you have defined a measure SalesScaled with a scale factor set to 100 in Measures.csv.

      Measure,Label,Intersection,DataType,DefaultValue,DefaultAgg,DefaultSpread,RecalcRuleName,PercentBase,PercentParentDimension,Format,HAlignment,Readonly,SpreadByMetric,DerivationType,EmptyValue,BaseAmbiguous
      ..
      SalesScaled,Sales (Scaled),sku_store_day,decimal,,total,ratioEven,,,,"{""currencyPosition"":""prefix"",""currencySymbol"":""$"",""thousandSeparated"":true,""mantissa"":2, ""scaleFactor"":100}",right,,,,,                  
      ..
      

      Let's say this measure has the value 34050 at sku-1 in the database. Because of the scale factor, you'll see 340.50 as the value at sku-1 in Modeler's pivot grid. Typing 500 would result in the value 50000 being sent to the database.

    • The usage of prefixes and postfixes when editing data in Modeler has been changed. You now enter the equals sign = first to be able to use prefixes and postfixes. Make sure to use unformatted input, using formatted input is not supported.

      Example 5. 

      Some examples:

      • Use "=+10" instead of "+10". This will add the value 10 to the current value.
      • Use "=10R" instead of "10R". This will replicate the value 10 at the base intersection. Moreover, lowercased spread suffixes are allowed now. "=10r" is equal to "=10R".

    • Modeler's count queries are now included in the generated view warmup queries.

  • Tools, Database, and Services
    • Added tcp_timeout parameter for lb web-client. This can be set in lb-web-client.config and defaults to 300 seconds. Details on these timeouts are discussed in the LogicBlox Administration Manual.

    • Added support for 15 minute resolution of timezones.

    • A new P2P has been implemented that computes the inventory cover (or uncover) for a time series of sales data and inventory data (or weeks of stock data). This implementation is significantly faster than previous LogiQL implementations of cover and uncover in LB 4.

      Example 6. 

      An example of a rule computing cover using the P2P:

      weeksOfStock[week] = wos <-
          inventory<< wos = cover<week_first,week_next>[week](inventory, sales) >>
            sales = salesData[week], inventory = inventoryData[week].
      

      Example 7. 

      An example of a rule computing uncover using the P2P:

      inventoryData[week] = inv <-
          inventory<< inv = uncover<week_first,week_next>[week](wos, sales) >>
            sales = salesData[week], wos = wosData[week].
      

      The parameter list <week_first,week_next> is optional and specifies a linear order over the time series. The week type can either be a primitive or entity type, but if it is an entity type, then an order must be specified.

      To extend the time series with additional sales data, specify the extension predicate in the argument of the P2P as follows:

      weeksOfStock[week] = wos <-
          inventory<< wos = cover<week_first,week_next>[week](inventory, sales) extend with extendData[week] = sales >>
            sales = salesData[week], inventory = inventoryData[week].
      

      This is supported for both cover and uncover.

      For more detail, see the LogicBlox Reference Manual.

    • Enhancements to tracing infrastructure:

      • Added long-running-count and rule-count data to transaction spans.

      • Added names of changed predicates and head predicates to rule spans. Also separated rule ID from rule name and added rule source for these spans.

      • Changed location of the span popups in the web UI so they would not be truncated along the left edge with the popups are large.

      • Set the span border color to grey so adjacent but separate spans can be visually separated in the web UI.

      • Double-clicking on a span will copy all the span data to the clipboard in JSON format.

  • Measure service
    • Added support for all-paths aggregations. This is done by specifying the hierarchy as '*'. For example, in textual syntax, total Sales by to Product.*.department. This will use all possible paths in the dimension from the starting level to the target level. There is no "double counting" as long as the paths commute. The same level member in the starting level maps to the same level member in the target along all possible paths.

    • If you aren't using the measure language (indicated by not explicitly enabling it), the CubiQL desugaring phase is now implemented via call-by-need normalization. Call-by-need is often called "lazy evaluation" and is most famously used in the language Haskell. This ensures that we do not spend time normalizing variable bindings, metrics and abstraction arguments that aren't used, and when we do, we'll only do it once.

      Because we are now using a normalization based approach instead of a term rewriting one, desugaring will not make more than a single pass over the AST. Overall the performance will be significantly better, particularly for large expressions.

      This also fixes some annoying cases found in the prior term rewriting approach, where there would frequently not be enough context to determine whether it was possible to delay reducing an abstraction or metric.

    • Problem messages in the protocol now include a unique per request 64-bit ID and a dependencies field. The dependencies field can reference the IDs of other problems in the same request that are related. This can make it easier to determine the root cause of some errors.

    • Relaxed the requirements for what must be included as a transform in UpdateExprs. It is now possible to omit a transform for both spreads if there are no intermediate transform steps and default spreading methods. The same applies for removals. Alternatively, it is now possible to provide a single empty transform message in both cases. Finally, for multi-step spreads, it is possible to omit the final distribution in the chain of transforms. This is mostly useful in the case that the client cannot predict what the final intersection of the spread will need to be due to not being able to guess the choice of inverses ahead of time.

    • It is now possible to use any one-to-one predicate as the conversion predicate for named types, rather than just refmodes and constructors.

    • Removed the protocol fields and messages for expression substitutions, which have not been used for several releases now.

    • Another major revision to how we represent LogiQL was made. This is relevant to other projects that build against our libraries.

    • Made various optimizations to the measure service.

    • Improved the clarity of a number of error messages.

Corrected Issues

The issues listed below have been corrected since the 4.10.0 release:

  • Modeler
    • In some situation, the layout of Modeler's slice panel looked a bit off. This has been fixed.

    • Resolved an issue that could occur when opening the context menu of a cell with no Measures field configured on the view.

    • Importing previously exported view states and view metadata resulted in the navigation having no labels for user-defined views. This issue has been resolved. Please note that in order to backup your saved user-defined views data, it's now required to backup user-defined views as well. For more information, see the Upgrade Information section.

    • In certain situations, it was still possible to drag measures onto the Measures axis while there was no Measures field configured on the view. This has been fixed.

    • The menu options belonging to menus invoked from the toolbar are now rendered correctly in Microsoft Edge.

    • Performance of the Filter panel has been improved. This panel now opens quickly, even when there is a large number of measures on the grid.

    • Cell and position locks are now always correctly displayed when the intersection or position at which the lock was applied is visible on the grid. The presence or absence of parent levels doesn't influence the display of these locks.

    • Modeler's header cells have been restyled so the icons are better aligned.

    • Resolved an issue where the user couldn't open the Measure Properties panel when the Measures field was placed left-most on the Rows axis.

    • Resolved a number of issues with Export to Excel functionality. Exporting reference data to Excel no longer fails when there the view contains one or more position-only measures. In some situations, the data in the export for ALL level didn't match the data on the view, this has been fixed. Also, exports with reference data no longer fail when there the view contains one or more position-only measures.

    • The level columns in the workbook list now correctly contain levels by label instead of ID.

    • Percent parent measures with outline mode enabled now behave correctly when being sorted.

    • Resolved an issue that could occur when attempting to filter measures which are using the first/last aggregation method and are updated via a CubiQL expression.

    • In certain cases, Modeler issued a high number of Measure service requests within a small timeframe causing Modeler's UI to become less responsive. This has been fixed.

    • To make sure there are not multiple dimensions on the floating axis with a sort in place (which is not allowed) when switching to Form Mode, Modeler now removes the sort before a field is moved to Slices as a part of the switch to Form Fode. Additionally, the sort options in the context menu of the cells are now grayed out when they're not allowed in the current view configuration.

    • Resolved an issue that could occur when trying to sort measures with a single key. Sorting is now disabled for this type of measure when a level from the measure's key dimension is not on the floating axis.

    • Fixed an exception that could occur when using composite spread with first/last on CubiQL measures.

    • Inputting a mask into a Slice dropdown now works as expected.

    • Checkboxes are no longer out of alignment when a cell is locked.

    • In certain cases, the Modeler migration tool had trouble handling the version numbers specified in the --fromVersion and --toVersion arguments. This has been fixed.

    • To prevent error situations in Modeler when sending down update requests to the Measure Service, we no longer include distributions in these request.

    • Fixed an exception that could occur while processing editability requests when editing data or committing changes in Deferred Calc Mode.

    • Fixed an offscreen rendering issue with String-type filters in the Filters panel.

    • Pasting either blank cells or cells with values from Modeler or Excel continues to work as expected.

  • Tools, Database, and Services
    • Fixed an issue with rule evaluation when there is a disjunction in the body and the rule is qualified for the prefix join.

    • Fixed an internal error that occasionally occurred when incremental maintenance was performed for some uses of negated or scalar predicates.

    • Fixed an internal error that was generated when using the lb predinfo command with a scalar int128-valued predicate.

    • Corrected an internal error that reported a missing index during some rule evaluations.

    • Corrected an internal error produced for rule evaluation where there is disjunction in the body and the rule qualifies for prefix join.

  • Measure service
    • Potentially fixed a long-standing bug in the REPL where we would fail to format a response due to a level member not having a defined label. We now fall back on the ID in that case.

    • Fixed an error in the measure service that would produce "Composite or named aggregation should have been desugared prior to typeOf" error messages.

Known Issues

The issues listed below are known to exist in the 4.11.0 release.

  • Tools, Database, and Services
    • This release contains changes to some lb-workflow tasks related to workbook creation. These fixes have not been applied yet to lb.wb.CreateWorkbook, which means errors related to recent changes in the LB runtime that imposes restrictions on string lengths can still occur. Please note that this known issue does apply to lb.wb.util.CreateWorkbook.

Installation Information

Install LogicBlox 4.11 by following the steps outlined below:

  1. Download the LogicBlox installation package for Linux or macOS from the LogicBlox developer website.
  2. Extract the tarball into a directory, which we refer to as <LOGICBLOX_HOME>.
  3. Run the following commands:
    source <LOGICBLOX_HOME>/etc/profile.d/logicblox.sh
    source <LOGICBLOX_HOME>/etc/bash_completion.d/logicblox.sh
    NOTE: these scripts automatically set all the necessary environment variables (please be aware that this script only works if you use bash as your shell). You might want to add these commands to your .bashrc.

Upgrade Information

  • Modeler
    • In order to backup your saved user-defined views data, it's now required to backup user-defined views as well. Thus, to backup all user-defined views data you must do the following:
      • Export current user-defined views. The service is accesible at /user_views.
      • Export current user-defined views metadata. The service is accesible at /viewmetadata_tdx.
      • Export current user viewstates. The service is accesible at /user_preferences_viewstates.
      • Import them into a corresponding service in the same order as in the export (user-defined views, user-defined views metadata, user viewstates).
    • Significant changes have been made to the underlying model used for data access permissions in the Modeler. These changes likely require updates to the project scripts that are used to build/compile your Modeler project.

      • The function modeler_library in modeler.py now includes a flag include_permissions that must be set to True if you want to enable data permissions in your project. This is required in addition to adding a dependency to the permissions library in your .project file.
      • Like modeler.py, generate_schema_from_config.py also includes the include_permissions flag. This script is called by modeler.py but some projects call this script directly. If your project uses this script, you will be required to pass this additional parameter to use permissions in the modeler along with adding the dependency in your project file.
      • Previously, we exposed the UserPermissions and UpOnSome measures in the measure model to allow application developers to include these measures in their refresh rules. We no longer expose these measures due to the security concern where these measures could be edited. A future release of LogicBlox will support refreshing the user permissions table in your workbooks by allowing you to create a refresh group that pulls the permissions predicates directly. You can then call the refresh during your workflow.

    • The measure format definition in Measures.csv has been changed from a string definition to a JSON object. The JSON format is based on numbro.js format with some additional properties specifically added for LogicBlox.

      Currently, there are two additional properties:

      • currencySymbol - allows you to define a currency symbol. For example for Euros.
      • scaleFactor - this property is used for the new scale factor feature listed under the What's New section.

      Example 8. 

      Let's say you have a format string like "$#,###.00", the same format definition in the new JSON format would look like:

      {
          "currencyPosition": "prefix",
          "currencySymbol": "$",
          "thousandSeparated": true,
          "mantissa":2
      }

      Please note that the TDX service responsible for importing Measures.csv now expects the file to be in Excel format. This means you have to escape double quotes in strings. There's a Modeler migration that converts your existing measures to the new format.

      Example 9. 

      Here's how you can specify the new format in the Measures.csv file.

      Measure,Label,Intersection,DataType,DefaultValue,DefaultAgg,DefaultSpread,RecalcRuleName,PercentBase,PercentParentDimension,Format,HAlignment,Readonly,SpreadByMetric,DerivationType,EmptyValue,BaseAmbiguous
      ..
      Sales,Sales,sku_store_day,decimal,,total,ratioEven,,,,"{""currencyPosition"":""prefix"",""currencySymbol"":""$"",""thousandSeparated"":true,""mantissa"":2}",right,,,,,
      ..
      

    • The library containing the Modeler migration tool is included in the distribution under <LOGICBLOX_HOME>/lib/npm/modeler-migrations-<VERSION>.tgz and contains a command line tool for doing various transformations/migrations of modeler configurations.

      To use the npm module, you first need to install it by running npm install <LOGICBLOX_HOME>/lib/npm/modeler-migrations-<VERSION>.tgz. You will then be able to use the command line tool. The module contains a README.md file that lists all available transformations as well as detailed instructions for use.

      To automatically upgrade a Modeler application from one version to another, run the upgrade script to upgrade between versions:

      ./node_modules/.bin/migrate-modeler --fromVersion <prevVer> --toVersion <toVer> /path/to/my/modelerapp

      where the version numbers refer to LogicBlox releases (e.g. 4.11.0).

      NOTE: If you installed the npm module globally, you won't need the ./node_modules/.bin prefix on the command.
  • Measure service
    • The following has been changed in the CubiQL syntax: if you're writing intersection restriction with more than one label you use ...!{....} now instead of ...!<...>.

Release Information

  • Server requirements
    • Operating System: 64-bit Linux; Apple macOS 10.10+ is supported for local development
    • Java Runtime Environment 8, update 101 or higher
    • Python 2.7 or higher
  • Client requirements
    • Applications using LogicBlox Modeler: Modeler supports major browsers not older than 1 year. Google Chrome provides the best performance.
    • Requirements for applications using non-LogicBlox Modeler components may vary per application.