LogicBlox 4.3.1

Release Date: August 10th 2015

What's New

Database
  • Default Values: With LogicBlox 4.3.1 we re-introduce the ability to define default values for predicates. Default values make it possible to efficiently work with sparse data by physically storing only facts with values different from the default value (usually non-zero). Default values automatically handle the complications of sparse data representations in rules. For example, with default values, a computation for netsales from sales and returns is simply expressed as:

    net_sales[sk,st,d] = sales[sk,st,d] - returns[sk,st,d].

    The database will transparently handle the cases where only sales or returns have non-zero values. The evaluation is efficient in the sense that it ignores cases where both predicates logically have a zero value.

    This feature is based on an earlier capability in LogicBlox 3.x. Please refer to the Default Values chapter in the LogicBlox Reference Manual for an extensive overview of how to work with default values.

  • Database Branching: LogicBlox 4.3.1 officially introduces database branching as a novel feature of databases. The feature is similar to branching and tagging in source code management system (like Git and Mercurial), but is unique for a database system. In LogicBlox a database can now be branched in constant time, which means that creating a branch will be very quick, and independent of the size of the database. Branches are isolated from each other and can differ in data as well as schema.

    Branches can be used for what-if analysis or other tasks that need long transactions lasting hours, days, or weeks. It can also be used to execute application upgrades with a simple way to back out of the upgrade procedure. Versions of the database can also be tagged simply as checkpoints for recovery or to determine changes made during a time period. Branches are also great for executing testcases from a testsuite in isolation.

    Given that database branching is such a unique and novel feature in databases, we are looking forward to seeing how the community will use this capability.

    Please refer to the Database Branching chapter in the manual for an extensive overview.

  • Databases can now be imported from read-only files.

  • We have re-introduced a feature to log long-running rules. This is a convenient tool for finding performance issues without having to enable full performance logging, which is usually not appropriate for deployed applications. The limit can be configured in lb-server.config or using an environment variable.

    Please refer to the Performance Monitoring chapter in the LogicBlox Administration Manual for details on how to use this feature.

LogiQL Language
  • Added support for POSIX extended regular expression builtins:
    • string:match:
      string:match(s,pattern) -> string(s), string(pattern).

      True if the string s matches the pattern. A pattern is a regular expression in POSIX extended regular expression syntax.

      Example 53. 

      string:match("226-223-4921", """\d+-\d+-\d+""").  // true
      string:match("226-223-4921", """\l+-\l+-\l+""").  // false 
    • string:matches:
      string:matches[s,regex,n]=subexp -> string(s), string(regex), int(n), string(subexp).

      Matches the string s against the regular expression regex and returns the indexed (n) matching subexpressions.

      Example 54. 

      For example:

      matches(n, subexp) -> int(n), string(subexp).
      matches(n, subexp) <- string:matches["aaabc", "(a*)(b).", n]=subexp.

      results in:

      /--- start of matches facts ---\
        0, "aaabc"
        1, "aaa"
        2, "b"
      \---- end of matches facts ----/ 
    • string:rewrite:
      string:rewrite[s, regex, replace]=r -> string(s), string(regex), string(replace), string(r).

      Rewrite the string 's' using the regular expression regex and the replacement text replace.

      Example 55. 

      string:rewrite["afoobbarc", ".*(foo).*(bar).*", """_\1_\2_"""]="_foo_bar_". 

    Please refer to the LogicBlox Reference Manual for a complete overview of all the built-in predicates.

Services Framework
  • Asynchronous TDX: The Tabular Data Exchange (TDX) transaction service now supports an asynchronous variant, which is indicated by an async=true URL parameter to the transaction start request. In async mode, the requests to parts (e.g. TDX import or export requests) and the commit request all return immediately instead of blocking until the workspace transaction completes. Clients can then GET the transaction's poll URL to check the status of the workspace transaction, and GET the transaction URL to retrieve more detailed information. This functionality is also exposed by the lb-web's Java and Python APIs, and by the TdxImport and TdxExport lb-workflow tasks.

  • Generalized admission control: The admission control support in lb-web has been generalized. Previously the only supported policy was "exclusive writes", in which read-only service requests were executed in parallel but write requests were executed exclusively. We have now introduced a more extensible method where we initially support exclusive writes (the previous default), and executing all requests for services assigned to a queue serially. The configuration of the admission control has been moved to lb-web-server.config. Before, admission control queues used to be declared by simply using them in a service.

    Please refer to the LogicBlox Admin Manual for a complete overview of the feature and examples. Please note the backwards compatibility note in the upgrade instructions.

Measure Service

  • Aggregation fusion optimization that attempts to merge generated aggregations if they are aggregating along the same dimensions over the same source data.
  • Optimized ambig aggregation for both the default and non-default value cases.
  • Optimized the average aggregation for the default value case.
  • The notion of the "signature" has been introduced in several places to encapsulate the intersection and type of an expression, as well as whether it is functional or set-valued. This will replace the use of separate intersection and type fields throughout the protocol.
  • ParamExprs now use the order of the dimensions in their intersection to determine their key-order.

Developer Tools

  • cloud-store improvements:
    • cloud-store now automatically aborts any multipart upload that cannot be completed successfully.

    • The new command cloud-store list-pending-uploads will list all pending uploads whose associated keys start with the specified S3 prefix. The prefix should contain, at least, a bucket name.

    • New command cloud-store abort-pending-uploadsto abort a pending upload, either by id or by date/datetime.

      For more information on cloud-store, please refer to the LogicBlox Admin Manual.

Corrected Issues

The issues listed below have been corrected since the 4.3.0 release.

  • Resolved an issue where workspace disk usage would continue to grow quickly for certain workloads, due to continued allocation of unused pages.

  • We have addressed an issue where long-running applications with diverse and large queries could use a significant amount of memory caching query evaluation infrastructure.

  • Resolved a durability issue.

  • Resolved an issue with regard to the memory usage when running lb export-workspace.

  • Transaction aborts are now logged as "---TXN-ABORTED----", instead of an error, to support more accurate monitoring of logs.

  • The lb popcount command, similarly to the lb predinfo command, does not show internal predicates by default anymore.

  • Measure Service:
    • Ratio/even now uses the default aggregation when calculating the value of the OldFixed/NewFixed fixed positions.

Installation and Upgrade information

Installation Instructions

Installing LogicBlox 4.3.1 is as simple as following the steps outlined below:

  1. Download the installation package.
  2. Extract the tarball in <YourPreferredInstallDirectory>
  3. Run the following command:
    source <YourPreferredInstallDirectory>/logicblox-4.3.1/etc/profile.d/logicblox.sh
    
    NOTE: this script will set all the necessary environment variables. You might want to add this command to your .bashrc.

Upgrade Information

  • Admission queues now must be declared in lb-web-server.config before services can be assigned to them.

Release Information

Server requirements
Operating System: 64 bit Linux; OSX 10.10+ is supported for local development.
Java Runtime Environment 7, update 11 or higher
Python 2.7 or higher
Client requirements
Applications using modeler-js User Interface Components: Google Chrome
Requirements for applications using non-modeler-js components may vary per application.