24 Pages Posted: 25 Oct 2002
Date Written: November 2001
The integration of data from autonomous and heterogeneous sources calls for the prior identification and resolution of semantic conflicts that may be present. Unfortunately, this requires the system integrator to sift through the data from disparate systems in a painstaking manner. In this paper, we suggest that this process can be (at least) partially automated by presenting a methodology and techniques for the discovery of potential semantic conflicts as well as the underlying data transformation needed to resolve the conflicts. Our methodology begins by classifying data value conflicts into two categories: context independent and context dependent. While context independent conflicts are usually caused by unexpected errors, the context dependent conflicts are primarily a result of the heterogeneity of underlying data sources. To facilitate data integration, data value conversion rules are proposed to describe the quantitative relationships among data values involving context dependent conflicts. A general approach is proposed to discover data value conversion rules from the data. The approach consists of five major steps: relevant attribute analysis, candidate model selection, conversion function generation, conversion function selection and conversion rule formation. It is being implemented in a prototype system, DIRECT, for business data using statistics based techniques. Preliminary study indicated that the proposed approach is promising.
Keywords: Value Conflicts, Data Integration, Autonomous Database Systems
Suggested Citation: Suggested Citation
Lu, Hongjun and Fan, Weiguo and Goh, Cheng Hian and Madnick, Stuart and Cheung, David W., Discovering and Reconciling Value Conflicts for Data Integration (November 2001). MIT Sloan Working Paper No. 4153-01; CISL Working Paper No. 1999-03. Available at SSRN: https://ssrn.com/abstract=335580 or http://dx.doi.org/10.2139/ssrn.335580