Download this Paper Open PDF in Browser

Discovering and Reconciling Value Conflicts for Data Integration

24 Pages Posted: 25 Oct 2002  

Hongjun Lu

National University of Singapore (NUS) - School of Computing

Weiguo Fan

Virginia Polytechnic Institute & State University - Department of Accounting and Information Systems

Cheng Hian Goh

National University of Singapore (NUS) - School of Computing

Stuart Madnick

Massachusetts Institute of Technology (MIT) - Sloan School of Management

David W. Cheung

The University of Hong Kong - Department of Computer Science and Information Systems

Date Written: November 2001

Abstract

The integration of data from autonomous and heterogeneous sources calls for the prior identification and resolution of semantic conflicts that may be present. Unfortunately, this requires the system integrator to sift through the data from disparate systems in a painstaking manner. In this paper, we suggest that this process can be (at least) partially automated by presenting a methodology and techniques for the discovery of potential semantic conflicts as well as the underlying data transformation needed to resolve the conflicts. Our methodology begins by classifying data value conflicts into two categories: context independent and context dependent. While context independent conflicts are usually caused by unexpected errors, the context dependent conflicts are primarily a result of the heterogeneity of underlying data sources. To facilitate data integration, data value conversion rules are proposed to describe the quantitative relationships among data values involving context dependent conflicts. A general approach is proposed to discover data value conversion rules from the data. The approach consists of five major steps: relevant attribute analysis, candidate model selection, conversion function generation, conversion function selection and conversion rule formation. It is being implemented in a prototype system, DIRECT, for business data using statistics based techniques. Preliminary study indicated that the proposed approach is promising.

Keywords: Value Conflicts, Data Integration, Autonomous Database Systems

Suggested Citation

Lu, Hongjun and Fan, Weiguo and Goh, Cheng Hian and Madnick, Stuart and Cheung, David W., Discovering and Reconciling Value Conflicts for Data Integration (November 2001). MIT Sloan Working Paper No. 4153-01; CISL Working Paper No. 1999-03. Available at SSRN: https://ssrn.com/abstract=335580 or http://dx.doi.org/10.2139/ssrn.335580

Hongjun Lu

National University of Singapore (NUS) - School of Computing ( email )

3 Science Drive 2
Singapore 117543
Singapore

Weiguo Fan

Virginia Polytechnic Institute & State University - Department of Accounting and Information Systems ( email )

Pamplin College of Business
3007 Pamplin Hall
Blacksburg, VA 24061
United States
540-231-6588 (Phone)

HOME PAGE: http://www.cob.vt.edu/acis/faculty/wfan/

Cheng Hian Goh

National University of Singapore (NUS) - School of Computing ( email )

3 Science Drive 2
Singapore 117543
Singapore

Stuart E. Madnick (Contact Author)

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

E53-321
Cambridge, MA 02142
United States
617-253-6671 (Phone)
617-253-3321 (Fax)

David Wai-lok Cheung

The University of Hong Kong - Department of Computer Science and Information Systems ( email )

Room 301, Chow Yei Ching Building
Pokfulam Road
Hong Kong

Paper statistics

Downloads
148
Rank
167,811
Abstract Views
2,695