Knowledge Discovery from Databases: the Nyu Project

Abstract

More and more application domains, from financial market analysis to weatherprediction, from monitoring supermarket purchases to monitoring satellite images, arebecomingly increasingly data-intensive. The result is massive databases that are growingat a rapid rate - it has been estimated that the worldâ¬Â"s electronic data almostdoubles every year. With this rate of data explosion, there is a pressing need for computersto play an increasing role in analyzing these huge data repositories which areimpossible to penetrate manually. The challenge is to ferret out the regularities in thedata that will prove to be interesting to the user.A group in the Information Systems department at the NYU Business School hasbeen working in this area for a number of years. The focus of our project is now on thediscovery of patterns from time series data. In this paper we give an overview of thekinds of databases we are "miningâ¬Â? and the kinds of temporal patterns and rules whichwe are attempting to discover. In the first phase of this research, we have developed ataxonomy of patterns as a way to organize our research agenda. We wish to share thetaxonomy with the research community in the "knowledge discovery in databases" areasince we have found it useful in classifying the universe of regularities or patterns intodistinct types, that is, patterns which differ in terms of their structure and the amount6f search effort required to find them. Although the primary focus of our project ison time series data, and the examples we will present are chosen from this arena, thetaxonomy is general enough to apply to any type of data.