LogicBlox 4.3.13

Release Date: August 5th 2016

What's New

modeler-js User Interface Components

See what's new in this release for modeler-js based applications in our LogicBlox 4.3.13 - New Features playlist!

  • End-User Charting: End-users can now turn any grid into a chart by selecting the Chart Mode option, available via the settings cogwheel. Users can choose from a number of chart types through a drop-down, and change the dimensions and measures used for the chart using the same axis configuration panels used for configuring the pivot grid. To return to grid mode, users can de-select the Chart Mode option via the settings cogwheel.

    Example 12. End-User Charting

    The figure below shows the new Chart Mode option that is available in the settings cogwheel:

    While in chart mode, a number of new options are available to users, as also illustrated below:

    • The drop-down box can be used to change the chart type on the fly.
    • The "Export as PNG" button allows users to save the chart in .png format.
    • The "View Source" button allows the user to see the Vega configuration of the chart. This can be very useful for debugging issues.
    • The "Open in Vega Editor" button can be used to inspect as well as edit the chart configuration using the Vega Editor.

    It is possible to configure via the modelingFeatures section of the application's html page, which of these options are visible to users. By default, only the "Export to PNG" and "View Source" options are visible.

     modelingFeatures: {
        ...
        chart: {
          exportAsPng: true,
          viewSource: true,
          openInVegaEditor: false
        },
        ...
    }

    The figure below shows the Vega Editor, where users can make changes to the Vega specification of a chart. This is mostly useful for debugging purposes.

  • Slice Synchronization: This feature replaces the sliceValueMeasure feature for slice synchronization. Whereas slices synchronized using a measure (through sliceValueMeasure) have their choices shared by all users on the same workspace, this new feature allows for slice choices to be separately maintained and synchronized for each individual user. Slice synchronization can be configured through the view JSON, or through the UI by end users.

    Example 13. End-User Slice Synchronization

    The following options are now available to users via the context menu of slice fields, as illustrated in the figure below:

    • Synch All: selecting this option causes the slice values of the selected level to be synchronized over all views that are available via the active navigation panel.
    • Synch All on Page: this option is only available on canvases containing multiple views. It allows users to synchronize all views on the canvas.
    • Synch with ...: this option opens up a pop-up window, displaying all the views with which the user can synchronize the slice value of the selected level. Only views that the user can access via the active navigation panel are listed here. The user can then choose to select either one or multiple views for slice synchronization.

    Once views are synchronized, the user also has the option to disable the synchronization, using the options Unsynch All, Unsynch All on Page, and Unsynch This:

    • Unsynch All: unsynchronizes all views that are synchronized with the initiating view, including from one another.
    • Unsynch All on Page: unsynchronizes the views on the initiating page.
    • Unsynch This: unsynchronizes the initiating view from the other views it is synchronized with, while the other views remain synchronized.

    Note

    Please review the Upgrade Information section on how to upgrade the application's sheet JSON configuration to use this new feature.

    Example 14. Example of the JSON configuration of Quick Slice Synch

    The example below illustrates the JSON configuration for synchronizing the slice selection of sheet1 and sheet2, as well as setting the initial value of the slice.

    // sheet1.json
    {
        "id": "sheet1",
        "title": "Sheet 1",
        "pivotConfig": {
            "axis": {...}
        },
        "signals": {
            "output": [
                {
                    "dataType": "POSITIONS",
                    "emitterId": "Slice@Calendar:day",
                    "eventName": "Slice Change",
                    "name": "<signal_name>",
                    "init": {
                        value: [{
                            "qualifiedName": "Calendar:day",
                            "measure": "SelectedDay" //configuration of the initial value via a measure
                        }]
                    }
                }
            ],
            "input": [
                {
                    "handler": "changeSliceSignalHandler",
                    "listenerId": "Slice@Calendar:day",
                    "signalName": "<signal_name>"
                }
            ]
        }
        ...
    }
    
    // sheet2.json
    {
        "id": "sheet2",
        "title": "Sheet 2",
        "pivotConfig": {
            "axis": {...}
        },
        "signals": {
            "output": [
                {
                    "dataType": "POSITIONS",
                    "emitterId": "Slice@Calendar:day",
                    "eventName": "Slice Change",
                    "name": "<signal_name>"
                }
            ],
            "input": [
                {
                    "handler": "changeSliceSignalHandler",
                    "listenerId": "Slice@Calendar:day",
                    "signalName": "<signal_name>"
                }
            ]
        }
        ...
    }
  • Session Recording Support via FullStory: It is often useful to allow users to record their sessions to provide information about issues they encounter in the application, or to provide information for developers and product owners on how the application is being used. We now offer an easy option for an application to embed session recording capabilities, through FullStory. FullStory, "DVR for all customer interactions", provides access to full fidelity recordings of user sessions, including JavaScript console outputs which can be extremely useful for diagnosing issues.

    To turn on recording, add the following segment to your AuthenticatedModelerController object:

    fullStory: {
        enabled: true,
        startByDefault: true,
        organizationKey: <ORG_KEY>
    },

    <ORG_KEY> should be arranged for your specific application. Reach out to your operations team to obtain on organization key.

    This will add a button to the menu pane of the application, that will allow users to start and stop recording. startByDefault can be set to false, which will allow webpage to be loaded with recording off by default.

  • View Configuration: The services for hosting sheets and canvases have changed. We now have the following services:

    # Upload a list of view configurations (sheets and canvases)
    POST /views
    
    # Get all view configurations
    GET /views
    
    # Get a particular view configuration
    GET /view/?id=abc
    
    # Upload a single view configuration
    POST /view
    
    # Get all view states for a particular user
    GET /viewstates/?user=user1
    
    # Get a single view state for a user
    GET /viewstate/?id=abc&user=user1
    
    # Put a view state for a user
    POST /viewstate/?user=user1

    The default view configuration is now stored and can be retrieved via the /view services. Previously, the /sheet and /canvas service would return either the default configuration, or if a user had ever edited it, the user's configuration. Hence, once a user made an edit, it was not possible to query the default configuration without deleting the user's edit in the database. User's changes are now stored separately in the /viewstate services allowing us to "Reset State" without retracting facts from the database.

    Note

    Please review the Upgrade Information section to learn about the changes that are required to your application.

  • Simplified HTML Configuration: Previously, applications were required to pass in a pageConfig object when constructing the AuthenticatedModelerController in their HTML pages. These were usually copied over from the reference implementation, where developers only changed one attribute, the navigationTreeId. Now it is possible to simply pass in the navigationTreeId as an attribute of the configuration and which will then be used as the default pageConfig to a view with the navigation tree as a sidebar.

    Note

    The change is backwards compatible and applications can continue to send pageConfigs but we recommend switching to the much simpler navigationTreeId instead. Take a look at the Upgrade Information section to learn about the changes that are required to your application to use this feature.

Database
  • The performance of decimal total aggregations has improved by about 25% for the most common use-cases (1, 2 or 3 key argument of type entity or integer). Note that the performance of an entire transaction may differ from 25% depending on the percentage of the workload that consists of total aggregations.

  • The auto_backup_mode is now set to none by default. Enable this setting can cause disk usage and performance issues, and our recommendations is for projects to leave this mode disabled. For projects that desire to experiment with enabling auto_backup_mode, this option can be configured in lb-server.config

Services Framework
  • Implemented a new handler to expose the connectblox remove block command. This can be used in the same way as other connectblox handlers, by declaring a remove_block service to be exposed in a certain URI prefix.

Developer Tools
  • To improve the error reporting of the cloud-store command when incorrect encryption keys are used, the download command now verifies a small checksum of the public key before encrypting. This allows the tool to give a high-level error about the incorrect key usage, as opposed to obscure internal decryption errors. This feature is only used for objects uploaded with this version or later.

  • The list command has internally been revised and the object listing now also shows more information about the objects. At the API-level the list command has changed as well, but as far as we know there are no API users that call list.

  • Cloud-store has improved handling of AWS S3 SlowDown responses. The improved retry code now avoids overwhelming the service when many objects are uploaded at the same time, improving overall throughput.

Workflow
  • Previously all parameters of a main workflow needed to be bound during its installation in a workspace. It is now possible to delay the binding until the time the instance is started, which provides more flexibility to how a workflow is used. It is also now possible to have set variables as main workflow parameters, as well as set literals.

    Example 15. 

    Given a main workflow foo(x, y="y", z*, w*={"1", "2"}) {...} it is now possible to choose to bind parameters when installing the workflow or when starting it. For example:

    • Trying to both install and then start the workflow foo without passing in any parameters causes an error, because both the x and z parameters are unbound:

      $lb workflow install -f file.wf --name foo --main foo
      $lb workflow start --name foo
      error: start command failed:
      	Parameter 'x' must be bound when starting workflow 'foo'.
      	Parameter 'z' must be bound when starting workflow 'foo'.

    • In the example below, we bind the parameters while installing the workflow. The parameters do not need to be bound anymore when starting it:

      $lb workflow install -f file.wf --name foo --main foo --param x="x" --param z="z"
      $lb workflow start --name foo
      Started new instance of workflow 'foo' rooted by process with id '10000000058'.

    • It's also possible to install the workflow without parameters, and only binding them when starting the workflow:

      $lb workflow install -f file.wf --name foo --main foo
      $lb workflow start --name foo --param x="x" --param z="z"
      Started new instance of workflow 'foo' rooted by process with id '10000000092'.

    • Main parameters that have default values can have their values overwritten on install:

      $ lb workflow install -f file.wf --name foo --main foo --param y="new_y"
      $ lb workflow start --name foo --param x="x" --param z="z"
      Started new instance of workflow 'foo' rooted by process with id '10000000063'.

    • However, if no value is passed on install, the default value is used, and cannot be overwritten on start:

      $ lb workflow install -f file.wf --name foo --main foo 
      $ lb workflow start --name foo --param x="x" --param z="z" --param y="new_y"
      error: start command failed:
          Parameter 'y' cannot be resolved when starting workflow 'foo'.
          Either the workflow's main method does not expose a parameter with this name,
          or the parameter has a default value, which can only be overwritten when
          installing the workflow.

  • The lb workflow action and purge commands now also support a --timeout option.

Workbook Framework
  • The workbook handler now does not bundle hierarchy files together by default as it previously did, because in certain situations that can lead to very poor performance. This change only impacts TDX-based transfers, often used for backup and restore, but does not affect cross-branch transfers usually used for commit and refresh.

Corrected Issues

The issues listed below have been corrected since the 4.3.12 release.

  • We identified that a low-level prefetching method was causing problems in out-of-core use-cases, in particular for workspaces that are fragmented (with free as well as used pages). This low-level prefetching method has been disabled in this release. Some applications will see improved out-of-core performance, in particular on less performant I/O devices.

  • We identified and fixed a rare low-level data structure problem that could manifest itself with various errors or incorrect results. This fix has not yet been backported to earlier releases. We encourage applications to update to 4.3.13 to obtain this fix.

  • We addressed a problem in the handling of aborted transactions that caused obscure errors (missing pageinfo or invalid pageid, followed by continued push_request errors). This is an important fix that we encourage applications to update for.

  • When trying to install a workflow that has the same name as an already installed workflow (either when using the lb workflow install or lb workflow run commands), the user is now presented with an error.

  • Fixed an issue in the TDX transaction API that could potentially lead to a thread being leaked if the client code would not shutdown the API properly.

  • Measure Service:

    • Admin restart requests now complete in isolation from non-admin requests, to avoid a possible race condition.

    • Resolved a caching issue when clearing all measure service installed logic.

    • Resolved and issue where the use of an average, mode, ambig or count-distinct aggregation anywhere deeper than the first position in a composite aggregation method would cause an exception.

    • The measure service can now be started with an incomplete model that mentions entities for levels that don't exist yet, however the entities still have to be declared manually before the level is used in a query.

Installation and Upgrade information

Installation Instructions

Installing LogicBlox 4.3.13 is as simple as following the steps outlined below:

  1. Download the installation package.
  2. Extract the tarball in <YourPreferredInstallDirectory>
  3. Run the following command:
    source <YourPreferredInstallDirectory>/logicblox-4.3.13/etc/profile.d/logicblox.sh
    
    NOTE: this script will set all the necessary environment variables. You might want to add this command to your .bashrc.

Upgrade Information

Upgrade information for applications using the modeler-js User Interface Components:
  • To upgrade the application sheet JSON configurations to use the new quick slice synchronization, you can simply run the following command:
    $ migrate-modeler 4.3.12 4.3.13
    Note that ModelerTestUtil currently does not support tests for slice synchronization. setSliceValue no longer results in the propagation of the set slice value to other sheets with which that slice is synchronized.
  • Update your html pages to use the simplified HTML configuration:

    Example 16. Simplified HTML Configuration

    Example of the pageConfig object that applications were required to provide prior to the 4.3.13 release:

    var pageConfig = {
        id: "page-with-sidebar",
        views: {
            navigationTree: {
                id: "sidebar",
                module: "ModelerNavigationTree",
                config: {
                    navigationTreeId: "unitplan-reporting"
                }
            }
        },
        layout: {
            shape: {
                columns: "200px,1",
                rows: "1,1px"   // 1px is for the LogView. We actually want to hide it,
                                // and this is the only way we can do that, for now
            },
            viewPositions: {
                navigationTree: { col: 0, row: 0, colspan: 1, rowspan: 1 }
            }
        }
    };
    
    React.render(React.createElement(TargetModeler, {
        pageConfig: pageConfig,
        actionButtons: [{
            actionId: "refresh-all", label: "Retrieve"
        }]
    }), contentEl);

    Starting this release, applications can simply pass in the navigationTreeId as an attribute of the configuration:

    React.render(React.createElement(AuthenticatedModelerController, {
        navigationTreeId: "your-navigation-tree-id",
        actionButtons: [{
            actionId: "refresh-all", label: "Retrieve"
        }]
    }), contentEl);

  • Authorize users the new /view and /viewstate services. For modeler-js applications, these are typically stored in src/config/authorization/:

    • Add the following lines to src/config/authorization/operations.csv, replacing <APPPREFIX> with your application's service prefix:

      /<APPPREFIX>/view|view service
      /<APPPREFIX>/views|view service
      /<APPPREFIX>/viewstate|view state service
      /<APPPREFIX>/viewstates|view states service
      /<APPPREFIX>/navtree|navigation tree service
    • Add the following lines to src/config/authorization/permission_operations.csv:

      general access permission|/<APPPREFIX>/view
      general access permission|/<APPPREFIX>/views
      general access permission|/<APPPREFIX>/viewstate
      general access permission|/<APPPREFIX>/viewstates
      general access permission|/<APPPREFIX>/navtree 

  • To migrate saved user preferences, follow the steps below:

    1. Export current user canvas and sheets preferences using the /user_preferences_canvases and /user_preferences_sheets.
    2. Modify the exported files by replacing CANVASID (or SHEETID) with VIEWID
    3. Import them into the new /user_preferences_viewstates service

  • Update performanceTestUtil to call the tearDown function in modelerTestUtil.destroy. This will clean up memory for each test run.

Release Information

Server requirements
Operating System: 64 bit Linux; OSX 10.10+ is supported for local development.
Java Runtime Environment 7, update 11 or higher
Python 2.7 or higher
CPU compatible with the corei7 architecture
Client requirements
Applications using modeler-js User Interface Components: Google Chrome
Requirements for applications using non-modeler-js components may vary per application.