Entity resolution — determining multiple distinct pieces of data as identifiers for the same real-world object — is a challenging problem in data integration. LogicBlox team member Nikolaos Vasiloglou, together with academic collaborators from Carlton University Zeinab Bahmani and Leopoldo Bertossi developed an effective method of entity resolution using matching dependencies. Their work was recently published at the 9th International Conference on Scalable Uncertain Management (SUM).
Abstract Entity resolution (ER), an important and common data cleaning problem, is about detecting data duplicate representations for the same external entities, and merging them into single representations. Relatively recently, declarative rules called matching dependencies (MDs) have been proposed for specifying similarity conditions under which attribute values in database records are merged. In this work we show the process and the benefits of integrating three components of ER: (a) Classifiers for duplicate/non-duplicate record pairs built using machine learning (ML) techniques, (b) MDs for supporting both the blocking phase of ML and the merge itself; and (c) The use of the declarative language LogiQL — an extended form of Datalog supported by the LogicBlox platform — for data processing, and the specification and enforcement of MDs.