After following our “LogiQL in 30 minutes” tutorial you may be wondering: “but are LogicBlox applications really developed entirely in a REPL?” The answer to this, of course, is: no. The REPL is a useful tool for learning and testing ideas, but is not generally used to develop full-blown applications. LogicBlox is different to what you may be used to in many ways, but you will be relieved to learn that “real” LogicBlox applications are developed similar to applications using most other technology stacks: by typing code into text files, structured neatly into directories. In this article we will explore how to structure and compile applications, and how to load compiled code into a workspace.
LogicBlox applications typically consist of various components, e.g.:
- Back-end: code that lives in the LogicBlox database.
- Web client: the code that lives in the browser and communicates with the back-end using web service (AJAX) calls.
- Data integration: scripts that pull data from various data sources and import them to your LogicBlox application for processing, usually using LogicBlox’ Data Exchange Services.
In this article we will focus on structuring your application’s back-end code base.
Back-end code is organized into projects, often just one, but sometimes more. A project in turn consists of one or more modules, which you can think of as packages in your application. A module in turn consists of one or more source files (called “blocks”), for instance .logic
files containing LogiQL, or .proto
files containing protocol buffer definitions.
You compile, run tests and package your application using the lb config tool, which will produce a standard Makefile based on an application specification. When the Makefile is generated you can compile your application by running make
, run tests using make check
and package using make dist
.
Let’s have a look at an example inspired by our ice cream emporium application. For this application we defined a number of predicates:
cost
: tracking the per ice cream costprice
: tracking the selling price per ice creamweek_sales
: tracking number of sales per ice cream per weekweek_revenue
: calculating the revenue per ice cream per week based on sales and price.profit
: calculating the profit per ice cream (based on cost and price)week_profit
: calculating the profit per ice cream per week
We will reimplement these predicates, previously added to a workspace in an ad-hoc way as a LogicBlox application. Since our project is small, we will put all predicates into a single module called core
. As we expand the example, we will refactor this structure as required.
Directory structure
The full source code of our example can be found in this bitbucket repository. The structure used in this project looks as follows:
config.py
: the LB config configuration fileapplication.project
: the project file for ourapplication
projectcore/
: the directory containing the logic for ourcore
moduleicecream.logic
: a LogiQL source file with predicates for tracking ice cream attributes likecost
andprice
.sales.logic
: a LogiQL source file with all sales related predicates.revenue.logic
: a LogiQL source file with all revenue related predicates.profit.logic
: a LogiQL source file with all profit related predicates.
data/
: a directory containing some initial test datainit_data.logic
: test values for our predicates
Before we look at the .logic
files, let’s first have a look at the config.py
and application.project
files.
Project files
LogicBlox project files have a very simple comma-separated-value format. Here’s the initial version of our project file:
application, projectname
core, module
data/init_data.logic, execute
The first value on each line represents the value of a project entry and the second represents the entry type. This particular project file defines:
application
is the name of the project;- it has a module named
core
; and - upon installing the project into a workspace, the data/init_data.logic file should be “exec”ed (like the
exec
REPL command) — this file contains some initial sample data. Incidentally, this is not how you should import large amounts of data, but for a couple of values it’s ok.
That wasn’t so hard. Let’s move on to config.py
.
config.py
Our project’s config.py
file describes the structure of a project, its dependencies, the libraries it consists of, unit tests to be run to verify its correct operation, and the workspaces to be created for testing. As you can probably tell from its file name, config.py
is indeed a Python script, but you do not to know much Python to read or extend it. Here’s the config.py
for our basic application:
//lang:python
from lbconfig.api import *
lbconfig_package('application', version='0.1',
default_targets=['lb-libraries'])
depends_on(logicblox_dep)
lb_library(name='application', srcdir='.')
check_lb_workspace(name='application', libraries=['application'])
In regular English: this config.py
describes how to build the LogicBlox application named application
version 0.1
and that the default make
target to run is lb-libraries
(that is: all libraries defined in this file). The package depends on LogicBlox (obviously), and consists of one library named application
whose source files are stored in the same directory as the config.py
file.
For convenience we’ve added a “check workspace” definition, which will automatically create a fresh workspace for you with the name application
and will load the application
library into it. This is useful for manually checking that it’s indeed possible to load your predicates into an actual workspace.
Logic
Once upon a time, LogicBlox applications were built simply by writing LogiQL definition after LogiQL definition in plain .logic
files, very similar to how we’ve seen in the REPL tutorial:
//lang:logiql
cost[icecream] = c ->
string(icecream), int(c).
price[icecream] = p ->
string(icecream), int(p).
While very simple, this approach has a few disadvantages:
- Potential name clashes. This is not an issue for small projects, but as projects get big (and many LogicBlox applications contain hundreds or even thousands of predicates) it becomes useful to organize predicates into their own namespaces, similar to packages in Java. So that you can have multiple
sales
predicates, for instance. - Compilation times. Adding code to a LogicBlox workspace using
addblock
is quick if you just have a few predicates, but the speed of anaddblock
quickly degrades as the number of predicates grows due to the complicated checks that the compiler performs to ensure the code is valid. Therefore, a mechanism was required to not have to recompile and reanalyze the entire code base every time you change a single line.
To solve these issues LogicBlox now has a proper module system with separate compilation. The module system provides namespacing and aliasing of names for convenience. For instance, if you have a module named core
, with a block named icecream
with the above two predicates in them, these predicates will be named core:icecream:cost
and core:icecream:price
.
Here’s what a LogiQL source file using the module syntax looks like:
//lang:logiql
block(`icecream) {
export(`{
cost[icecream] = c ->
string(icecream), int(c).
price[icecream] = p ->
string(icecream), int(p).
})
} <-- .
All definitions are enclosed in a block
whose name has to match the file name. So in this case, the file name should be icecream.logic
. A module can have alias
es, export
s and clauses
. This particular file only has export
s and exports two predicates: cost
and price
. The export
section of a file defines its external interface. The export
section can only contain predicate signatures, that is: the predicate name, its arguments and types for all arguments. Any other definitions, for instance rules that derive their values from other predicates as well as extra constraints, have to go into the clauses
section.
To show how this works, the profit.logic file uses all three sections:
//lang:logiql
block(`profit) {
alias_all(`core:icecream),
alias_all(`core:sales),
alias_all(`core:revenue),
export(`{
profit[icecream] = value -> string(icecream), int(value).
week_profit[icecream, week] = value ->
string(icecream),
int(week),
int(value).
agg_profit[week] = value -> int(week), int(value).
agg_revenue[week] = value -> int(week), int(value).
}),
clauses(`{
profit[icecream] =
price[icecream] - cost[icecream].
profit[_] = value -> value >= 0.
week_profit[icecream, week] =
profit[icecream] * week_sales[icecream, week].
agg_profit[week] = value <-
agg<<value=total(p)>> week_profit[_, week] = p.
agg_revenue[week] = value <-
agg<<value=total(r)>> week_revenue[_, week] = r.
})
} <-- .
The alias_all
constructs import all predicate names from the core:icecream
, core:sales
, core:revenue
namespaces so that they can be referenced without having to use their fully qualified names. If you prefer, you can also alias them with a custom prefix, e.g.:
//lang:logiql
alias(`core:icecream, `ic),
To be able to refer to them via ic:cost
and ic:price
.
We also see that the export
section contains signatures for all predicates, including the ones whose facts are derived. This is obligatory when using modules. The reason is explicitness: to find out the signature of a predicate, you can simply open its .logic
file and look at the export
section, without having to consider where its facts are derived from, what the types of those predicate’s arguments are and so on.
Finally, the clauses
section contain the implementation of the predicates. For instance, the facts in profit
are calculated by subtracting cost from price. Additional constraints — like that the profit on any ice cream should always be zero or more — are also placed here.
Building and testing
So, now we have our project properly structured, how do we compile and test it? This depends on how you installed LogicBlox.
If you chose to go the Vagrant route, you need a Vagrantfile
in your project. Our repository already contains one so you should be all set after copying or symlinking a LogicBlox release tarball into the checkout directory. To setup the Vagrant VM, simply run:
$ vagrant up
$ vagrant ssh
You should now get a Linux bash prompt inside of the VM with LogicBlox installed and running.
If you installed LogicBlox natively you’re all set already, just make sure that the lb services are running:
$ lb services status
If not, start them with:
$ lb services start
In either setup, to compile the project, simply run:
$ lb config
$ make
To load your code into a workspace for testing run:
$ make check check-ws-application
You can now open the command-line REPL (very similar tot he Web-based REPL) using:
$ lb
and then opening the test workspace (which we named application
) using:
lb> open application
From here we can list all predicates (note that this REPL also lists built-in predicates, not just custom ones) and print their contents. For instance:
lbi application> list
lbi application> print core:profit:agg_profit
To end the REPL session, simply run the exit
command:
lbi application> exit
This covers the bare basics of structuring and compiling your LogicBlox projects. To learn more about LogicBlox’ module system, read the reference manual chapter about it. More details about lb config are also available in the reference manual, and a full list of lb config commands is available as well.
Great post!
This post explains well the first element of the three elements: 1-Back-end 2-Web client 3-Data integration.
That would be great if you write a post about the second element (i.e. web client). Although there are some hints in the reference manual about this topic, I think explaining it all in one place using examples (similar to this post) would be much more helpful.
Thanks! Yes, a post about building a front-end to services is planned. Stay tuned!