1、 Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Data Mining: IntroductionLecture Notes for Chapter 1Introduction to Data MiningbyTan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 2 l Lots of data is being collected and warehoused Web data, e-commerce purc
2、hases at department/grocery stores Bank/Credit Card transactionsl Computers have become cheaper and more powerfull Competitive Pressure is Strong Provide better, customized services for an edge (e.g. in Customer Relationship Management)Why Mine Data? Commercial ViewpointWhy Mine Data? Scientific Vie
3、wpointl Data collected and stored at enormous speeds (GB/hour) remote sensors on a satellite telescopes scanning the skies microarrays generating gene expression data scientific simulations generating terabytes of datal Traditional techniques infeasible for raw datal Data mining may help scientists
4、in classifying and segmenting data in Hypothesis Formation Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 4 Mining Large Data Sets - Motivationl There is often information “hidden” in the data that is not readily evidentl Human analysts may take weeks to discover useful informationl Much
5、 of the data is never analyzed at allThe Data GapTotal new disk (TB) since 1995Number of analystsFrom: R. Grossman, C. Kamath, V. Kumar, “Data Mining for Scientific and Engineering Applications” Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 5 What is Data Mining?lMany Definitions Non-tr
6、ivial extraction of implicit, previously unknown and potentially useful information from data Exploration & analysis, by automatic or semi-automatic means, of large quantities of data in order to discover meaningful patterns Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 6 What is (not)
7、Data Mining?l What is Data Mining? Certain names are more prevalent in certain US locations (OBrien, ORurke, OReilly in Boston area) Group together similar documents returned by search engine according to their context (e.g. Amazon rainforest, A,)l What is not Data Mining? Look up phone number in ph
8、one directory Query a Web search engine for information about “Amazon” Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 7 l Draws ideas from machine learning/AI, pattern recognition, statistics, and database systemsl Traditional Techniquesmay be unsuitable due to Enormity of data High dime
9、nsionality of data Heterogeneous, distributed nature of dataOrigins of Data MiningMachine Learning/Pattern RecognitionStatistics/AIData MiningDatabase systems Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 8 Data Mining TaskslPrediction Methods Use some variables to predict unknown or fu
10、ture values of other variables.lDescription Methods Find human-interpretable patterns that describe the data.From Fayyad, et.al. Advances in Knowledge Discovery and Data Mining, 1996 Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 9 Data Mining Tasks.lClassification PredictivelClustering
11、DescriptivelAssociation Rule Discovery DescriptivelSequential Pattern Discovery DescriptivelRegression PredictivelDeviation Detection Predictive Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 10 Classification: DefinitionlGiven a collection of records (training set ) Each record contains
12、 a set of attributes, one of the attributes is the class.lFind a model for class attribute as a function of the values of other attributes.lGoal: previously unseen records should be assigned a class as accurately as possible. A test set is used to determine the accuracy of the model. Usually, the given data set is divided into training and test sets, with training set used to build the model and test set used to validate it.
