LogicBlox 4.3.11

Release Date: June 2nd 2016

What's New

Database
  • This release introduces a new implementation of the meta-engine (used for managing meta data such as predicates and logic rules). This implementation results in significant performance improvements and reduction in the memory consumption in the runtime overall, compared to the previous implementation.

    Key new features are:

    • Greatly improved latency of transactions that execute very large queries. Most measure service queries fall into this category and should benefit significantly.

    • Greatly improved latency of transactions that modify the schema and logic of the database (e.g. adding or removing blocks). Due to the incremental execution graph maintenance, full evaluation of the installed logic is now never necessary on logic additions or removal. Only the affected logic (that is in rules depending on the change) need to be recomputed. The impact is most significant for applications that have many predicates or rules.

    • Several longstanding bugs with removing blocks have been resolved. Removing blocks should now work and we ask users to actively start using this feature and report any problems. Note that it is still a requirement that any dependent blocks be removed manually. E.g. if block A contains declarations of a predicate used in block B, block B must be removed before block A is.

    • Rules for derived-only predicates may now be added or removed in any database lifetime block. In previous versions of 4.x, we required all rules for a single derived-only predicate to be added in the same block.

    • Significantly reduced memory usage. We have measured an improvement of about 20% in most applications, but the gain can be significantly bigger for workbook-based applications.

    • Significant reduction in the time needed to open workspaces, in particular when opening workspaces from EBS snapshots.

  • The lb-server process now uses static linking to include the various LB libraries. This has no impact on the user and in our experiments it improves performance by about 10% depending on the workload.

Developer Tools
  • The logging format has been improved to make it easier to correlate error messages to a workspace, branch or a transaction for concurrent transactions. We have changed the logging format to now include information on workspace, branch, and transaction on every line. As an example of the change:

    2016-05-26 13:54:52,611999+00:00 warning long-running "ExUnRule file:69(5)--79(45)#24BYO45A" (took 16.81 sec)

    is now logged as:

    2016-05-26 13:54:52,611999+00:00 warning [workspace] [txn:317e946be] long-running "ExUnRule file:69(5)--79(45)#24BYO45A" (took 16.81 sec) 

    Important

    Note that this change may break monitoring or log analysis tools!

Workbook Framework
  • Workbook framework performance and memory footprint has been dramatically improved due to reduction in installed block per workbook instantiation, and using more performant logic when logic does need to be installed.

    • The workbook framework no longer installs inactive blocks that are only needed for the TDX-based implementation, if branch transfer implementation is used. This results in improved memory usage and performance.

    • The workbook framework now utilizes a new runtime branch parameter feature, which drastically reduces the amount of logic installed into the database and consequently memory usage and time needed to open a workspace. The branch parameter feature allows the workbook framework to add inactive blocks with unbound branch parameters. The concrete branch to use is only specified when executing the inactive block.

Workflow
  • Implemented an optimization for the LogiQL rules related to restartable processes lifecycle. This makes the workflows more scalable in the presence of a large number of restartables.

  • By default, lb workflow status now avoids printing workflow instances that had no process instances selected. The new --show-empty-instances flag forces the previous behavior, where those instances were printed.

  • New workflow tasks to manipulate strings that represent URLs: lb.string.EncodeURL and lb.string.DecodeURL.

  • The workflow task lb.ReadCsv now accepts file URIs as parameters.

Corrected Issues

The issues listed below have been corrected since the 4.3.10 release.

  • The lb export-workspace previously did not correctly handle pending diskcommit transactions and could export a previous version of the database. This bug could cause the exported workspace to miss schema or data changes executed just before the export. This problem has been fixed.

  • Fixed a memory leak that would very slowly, but steadily cause memory usage to increase when executing many transactions (hours of short-running transactions). The growth is small enough that it is unlikely to have impacted any current users.

  • In very rare cases lines from a CSV file would be dropped when importing the file due to a bug in the implementation of parallel processing. This issue has been resolved.

  • Resolved an issue where in rare cases a deadlock could occur when restarting the measure service.

  • Resolved an issue with the measure service that would prevent an expression and the same expression with a functional annotation from being distinguished during code generation.

  • Internal improvements to the CubiQL optimizer yielding significant performance improvements to lengthy warm-ups and queries that have not yet been cached.

  • It is now possible to use @prev stages on IDB metrics in measure language inverses.

  • Resolved an issue with the workflow code generator that prevented multiple restartable cleanup sections to be installed in a workspace.

  • modeler-js User Interface Components:
    • Resolved an issue where the "clear filter" option was not available via the context menu

Installation and Upgrade information

Installation Instructions

Installing LogicBlox 4.3.11 is as simple as following the steps outlined below:

  1. Download the installation package.
  2. Extract the tarball in <YourPreferredInstallDirectory>
  3. Run the following command:
    source <YourPreferredInstallDirectory>/logicblox-4.3.11/etc/profile.d/logicblox.sh
    
    NOTE: this script will set all the necessary environment variables. You might want to add this command to your .bashrc.

Release Information

Server requirements
Operating System: 64 bit Linux; OSX 10.10+ is supported for local development.
Java Runtime Environment 7, update 11 or higher
Python 2.7 or higher
CPU compatible with the corei7 architecture
Client requirements
Applications using modeler-js User Interface Components: Google Chrome
Requirements for applications using non-modeler-js components may vary per application.