What is Data Mining?
Data Mining is the process of analyzing large data sets in order to find
patterns that can help to isolate key variables to build predictive models for
management decision making. In essence, data mining helps businesses to
optimize their processes so that their customers receive the most relevant
services and the costs of serving them are proportionate to the value of the
profits earned from them, a company’s exposure to risk is proportionate to
premiums earned, etc. Data Mining enables companies to segment their customer
base and to tailor products and services to the needs and purchasing power of
individual groups of customers.
Who is Data Mining for?
Data Mining is for executives involved in strategic and tactical decision making
as well as operating managers responsible for cost reduction. Strategic
Managers use data mining for competitive intelligence, identifying market
opportunities, product launch decisions and product positioning. Managers
responsible for tactical decision making use similar tools for sales
forecasting, direct marketing, customer acquisition, retention and extension
purposes and marketing campaign analysis. Finally, operational managers can use
similar data for decisions such as the choice of sub-prime borrowers or supply
Things to consider when implementing Data Mining
It's worthwhile remembering the adage "Garbage in and Garbage out" when
implementing data mining. Poor quality of data can jeopardize any attempt to
use data analysis for decision making. This is especially true when data is
purchased from external vendors.
Size of the Database
Data Mining comes in a variety shapes and forms depending on primarily the size
of databases that have to be analyzed. Smaller companies cannot afford
packages, such as SAS, which are expensive to license. Excel, with its advanced
data functions, is adequate for small companies and Access for data storage.
Eventually, databases inevitably grow in size and companies have to plan for
migration to more complex data mining tools.
Nature of Application
The choice of data mining methods depends a great deal on the kind of decision
making that a company wants to achieve. Sales related decision making is best
undertaken with OLAP tools such as those offered by Cognos or Siebel. Companies
looking to do predictive modeling are better-off with SAS or SPSS. Similarly,
data processing can be done with a variety of databases ranging from Access for
small databases, MS SQL for medium size operations to more complex enterprises
types like Oracle and Teradata for data coming from networks like credit card
Data mining requires a great deal of ingenuity that highly educated people can
provide. Typically, the people hired for data mining have a graduate degree and
bigger companies in the financial services companies hire PhDs. Data Mining
requires a portfolio of skills in data management, statistical analysis and
business decision making which are hard to find.