Privacy Policy for dataminingtechniques.net

The privacy of our visitors to dataminingtechniques.net is important to us.

At dataminingtechniques.net, we recognize that privacy of your personal information is important. Here is information on what types of personal information we receive and collect when you use and visit dataminingtechniques.net, and how we safeguard your information. We never sell your personal information to third parties.

Log Files

As with most other websites, we collect and use the data contained in log files. The information in the log files include your IP (internet protocol) address, your ISP (internet service provider, such as AOL or Shaw Cable), the browser you used to visit our site (such as Internet Explorer or Firefox), the time you visited our site and which pages you visited throughout our site.

Cookies and Web Beacons

We do use cookies to store information, such as your personal preferences when you visit our site. This could include only showing you a popup once in your visit, or the ability to login to some of our features, such as forums.

We also use third party advertisements on dataminingtechniques.net to support our site. Some of these advertisers may use technology such as cookies and web beacons when they advertise on our site, which will also send these advertisers (such as Google through the Google AdSense program) information including your IP address, your ISP , the browser you used to visit our site, and in some cases, whether you have Flash installed. This is generally used for geotargeting purposes (showing New York real estate ads to someone in New York, for example) or showing certain ads based on specific sites visited (such as showing cooking ads to someone who frequents cooking sites).

DoubleClick DART cookies

We also may use DART cookies for ad serving through Google’s DoubleClick, which places a cookie on your computer when you are browsing the web and visit a site using DoubleClick advertising (including some Google AdSense advertisements). This cookie is used to serve ads specific to you and your interests (”interest based targeting”). The ads served will be targeted based on your previous browsing history (For example, if you have been viewing sites about visiting Las Vegas, you may see Las Vegas hotel advertisements when viewing a non-related site, such as on a site about hockey). DART uses “non personally identifiable information”. It does NOT track personal information about you, such as your name, email address, physical address, telephone number, social security numbers, bank account numbers or credit card numbers. You can opt-out of this ad serving on all sites using this advertising by visiting http://www.doubleclick.com/privacy/dart_adserving.aspx

You can chose to disable or selectively turn off our cookies or third-party cookies in your browser settings, or by managing preferences in programs such as Norton Internet Security. However, this can affect how you are able to interact with our site as well as other websites. This could include the inability to login to services or programs, such as logging into forums or accounts.

Deleting cookies does not mean you are permanently opted out of any advertising program. Unless you have settings that disallow cookies, the next time you visit a site running the advertisements, a new cookie will be added.

Introduction to Data mining Architecture

Data mining is described as a process of discover or extracting interesting knowledge from large amounts of data stored in multiple data sources such as file systems, databases, data warehouses and etc.  This knowledge contributes a lot of benefits to business strategies, scientific, medical research, governments and individual.

Data is collected explosively every minute through business transactions and stored in relational database systems. In order to provide insight about the business processes, data warehouse systems have been built to provide analytical reports for business users to make decisions. Data is now stored in database and/or data warehouse system so data mining system should be designed to decouple or couple with these systems. This question leads to four possible architectures of a data mining system as follows:

• No-coupling: in this architecture, data mining system does not utilize any functionality of a database or data warehouse system. A no-coupling data mining system retrieves data from a particular data sources such as file system, processes data using major data mining algorithms and stores results into file system. The no-coupling data mining architecture does not take any advantages of database or data warehouse that is already very efficient in organizing, storing, accessing and retrieving data. The no-coupling architecture is considered a poor architecture for data mining system however it is used for simple data mining processes.
• Loose Coupling: in this architecture, data mining system uses database or data warehouse for data retrieval. In loose coupling data mining architecture, data mining system retrieves data from database or data warehouse, processes data using data mining algorithms and stores the result in those systems. This architecture is mainly for memory-based data mining system that does not require high scalability and high performance.
• Semi-tight Coupling: in semi-tight coupling data mining architecture, beside linking to database or data warehouse system, data mining system uses several features of database or data warehouse systems to perform some data mining tasks including sorting, indexing, aggregation…etc. In this architecture, some intermediate result can be stored in database or data warehouse system for better performance.
• Tight Coupling: in tight coupling data mining architecture, database or data warehouse is treated as an information retrieval component of data mining system using integration. All the features of database or data warehouse are used to perform data mining tasks. This architecture provides system scalability, high performance and integrated information.

A Data Mining Architecture

Let’s examine a tight-coupling data mining architecture in a greater detail.

Data Mining Architecture

There are three tiers in the tight-coupling data mining architecture:

1. Data layer: as mentioned above, data layer can be database and/or data warehouse systems. This layer is an interface for all data sources. Data mining results are stored in data layer so it can be presented to end-user in form of reports or other kind of visualization.
2. Data mining application layer is used to retrieve data from database. Some transformation routine can be performed here to transform data into desired format. Then data is processed using various data mining algorithms.
3. Front-end layer provides intuitive and friendly user interface for end-user to interact with data mining system. Data mining result presented in visualization form to the user in the front-end layer.

In this article, we’ve discussed various data mining architectures, its advantages and disadvantages. And then we looked into a tight-couple data mining architecture – the most desired, high performance, high scalable data mining architecture.

Data mining is an important part of knowledge discovery process that analyzes large enormous set of data and gives us unknown, hidden and useful information and knowledge. Data mining has not only applied effectively in business environment but also in other fields such as weather forecast, medicine, transportation, healthcare, insurance, government and etc. Data mining brings a lot of advantages when using in a specific industry. Beside those advantages, data mining also has its own disadvantages as well such as privacy, security and misuse of information. We will examine the advantage and disadvantages of data mining in different industries.

Advantages of Data Mining

Marketing / Retail

Data mining helps marketing companies to build models based on historical data to predict who will respond to new marketing campaign such as direct mail, online marketing campaign and etc. Through this prediction, marketers can have appropriate approach to sell profitable products to targeted customers with high satisfaction.

Data mining brings a lot of benefit s to retail company in the same way as marketing. Through market basket analysis, the store can have an appropriate production arrangement in the way that customers can buy frequent buying products together with pleasant. In addition, it also help the retail company offers a certain discount for particular products what will attract customers.

Finance / Banking

Data mining gives financial institutions information about loan information and credit reporting. By building a model from previous customer’s data with common characteristics, the bank and financial can estimate what are the god and/or bad loans and its risk level. In addition, data mining can help banks to detect fraudulent credit card transaction to help credit card’s owner prevent their losses.

Manufacturing

By applying data mining in operational engineering data, manufacturers can detect faulty equipments and determine optimal control parameters. For example semi-conductor manufacturers had a challenge that even the conditions of manufacturing environments at different wafer production plants are similar, the quality of wafer are lot the same and some for unknown reasons even contain defects. Data mining has been applied to determine the ranges of control parameters that lead to the production of golden wafer. Then those optimal control parameters are used to manufacture wafers with desired quality.

Governments

Data mining helps government agency by digging and analyzing records of financial transaction to build patterns that can detect money laundering or criminal activity.

Disadvantages of data mining

Privacy Issues

The concerns about the personal privacy have been increasing enormously recently especially when internet is booming with social networks, e-commerce, forums, blogs…. Because of privacy issues, people are afraid of their personal information is collected and used in unethical way that potentially causing them a lot of trouble. Businesses collect information about their customers in many ways for understanding their purchasing behaviors trends. However businesses don’t last forever, some days they may be acquired by other or gone. At this time the personal information they own probably is sold to other or leak.

Security issues

Security is a big issue. Businesses owns information about their employee and customers including social security number, birthday, payroll and etc. However how properly this information is taken is still in questions. There have been a lot of cases that hackers were accesses and stole big data of customers from big corporation such as Ford Motor Credit Company, Sony… with so much personal and financial information available, the credit card stolen and identity theft become a big problem.

Misuse of information/inaccurate information

Information collected through data mining intended for marketing or ethical purposes can be misused. This information is exploited by unethical people or business to take benefit of vulnerable people or discriminate against a group of people.

In addition, data mining technique is not perfectly accurate therefore if inaccurate information is used for decision-making will cause serious consequence.

Conclusion

Data mining brings a lot of benefits to businesses, society, governments as well as individual. However privacy, security and misuse of information are the big problem if it is not address correctly.

Summary: This tutorial discusses about the data mining applications in various areas including sales/marketing, banking, insurance, health care, transportation and medicine.

Data mining is a process that analyzes the large amount of data to find the new and hidden information that improves business efficiency. Various industries have been adopt data mining to their mission-critical business processes to gain competitive advantages and help business grows. This tutorial illustrates some data mining applications in sale/marketing, banking/finance, health care and insurance, transportation and medicine.

Data Mining Applications in Sales/Marketing

Data mining enables the businesses to understand the patterns hidden inside past purchase transactions, thus helping in plan and launch new marketing campaigns in prompt and cost effective way. The following illustrates several data mining applications in sale and marketing.

• Data mining is used for market basket analysis to provides insight information on what product combinations were purchased, when they were bought and in what sequence by customers.  This information helps businesses to promote their most profitable products to maximize the profit. In addition, it encourages customers to purchase related products that they may have been missed or overlooked.
• Retails companies uses data mining to identify customer’s behavior buying patterns.

Data Mining Applications in Banking / Finance

• Several data mining techniques such as distributed data mining has been researched, modeled and developed to help credit card fraud detection.
• Data mining is used to identify customers loyalty by analyzing the data of customer’s purchasing activities such as the data of frequency of purchase in a period of time, total monetary value of all purchases and when was the last purchase. After analyzing those dimensions, the relative measure is generated for each customer. The higher of the score, the more relative loyal the customer is.
• To help bank to retain credit card customers, data mining is used.  By analyzing the past data, data mining can help banks to predict customers that likely to change their credit card affiliation so they can plan and launch different special offers to retain those customers.
• Credit card spending by customer groups can be identified by using data mining.
• The hidden correlation’s between different financial indicators can be discovered by using data mining.
• From historical market data, data mining enable to identify stock trading rules.

Data Mining Applications in Health Care and Insurance

The growth of the insurance industry is entirely depends on the ability of converting data into the knowledge, information or intelligence about customers, competitors and its markets. Data mining is applied in insurance industry lately but brought tremendous competitive advantages to the companies who have implemented it successfully. The data mining applications in insurance industry are listed below:

• Data mining is applied in claims analysis such as identifying which medical procedures are claimed together.
• Data mining enables to forecasts which customers will potentially purchase new policies.
• Data mining allows insurance companies to detect risky customers’ behavior patterns.
• Data mining helps detect fraudulent behavior.

Data Mining Applications in Transportation

• Data mining helps to determine the distribution schedules among warehouses and outlets and analyze loading patterns.

Data Mining Applications in Medicine

• Data mining enables to characterize patient activities to see coming office visits.
• Data mining help identify the patterns of successful medical therapies for different illnesses.

Data mining applications are continuously developing in various industries to provide more hidden knowledge that enable to increase business efficiency and grow businesses.

Summary: This tutorial discusses about data mining processes and describes the Cross-industry standard process for Data Mining (CRISP-DM).

Introduction to Data Mining Processes

Data mining is a promising and relatively new technology that is defined as a process of discovering hidden valuable and useful knowledge or information by analyzing large amounts of data storing in databases or data warehouse using different techniques such as machine learning, artificial intelligence(AI) and statistical.  A wide range of organizations in various industries are making use of data mining including manufacturing, marketing, chemical, aerospace, etc.,  to take advantages over their competitors. The needs for a standard data mining therefore increased dramatically. The data mining process must be reliable and repeatable by business people with little knowledge or no data mining background. In 1990, a cross-industry standard process for data mining (CRISP-DM) first published after going through a lot of workshops, and contributions from over 300 organizations. Let’s examine the cross-industry standard process for data mining in greater details.

The Cross-Industry Standard Process for Data Mining (CRISP-DM)

Cross-Industry Standard Process for Data Mining (CRISP-DM) consists of six phases intended as a cyclical process as the following figure:

Cross-Industry Standard Process for Data Mining (CRISP-DM)

1. Business understanding - In the business understanding phase, first it is a must to understand business objectives clearly and make sure to find out what the client really want to achieve. Next, we have to assess the current situation by finding about the resources, assumptions, constraints and other important factors which should be considered. Then from the business objectives and current situations, we need to create data mining goals to achieve the business objective and within the current situation. Finally a good data mining plan has to be established to achieve both business and data mining goals. The plan should be as details as possible that have step-by-step to perform during the project including the initial selection of data mining techniques and tools.
2. Data understanding - First, the data understanding phase starts with initial data collection that collects data from available sources to get familiar with data. Some important activities must be carried including data load and data integration in order to make the data collection successfully. Next, the “gross” or “surface” properties of acquired data need to be examined carefully and reported. Then, the data need to be explored by tackling the data mining questions, which can be addressed using querying, reporting and visualization. Finally, the data quality must be examined by answering some important questions such as “Is the acquired data complete?”, “Is there any missing values in the acquired data?”
3. Data preparation - The data preparation normally consumes about 90% of the time. The outcome of the data preparation phase is the final data set. Once data sources available are identified, they need to be selected, cleaned, constructed and formatted into the desired form. The data exploration task at a greater depth may be carried during this phase to notice the patterns based on business understanding.
4. Modeling - First, modeling techniques have to be selected to be used for the prepared dataset. Next, the test scenario must be generated to validate the models’ quality and validity. Then, one or more models are created by running the modeling tool on the prepared dataset. Last but not least, models need to be assessed carefully involving stakeholders to make sure that created models are meet business initiatives.
5. Evaluation - In the evaluation phase, the model results must be evaluated in the context of business objectives in the first phase. In this phase, new business requirements may be raised due to new patterns has been discovered in the model results or from other factors. Gaining business understanding is an iterative process in data mining. The go or no-go decision must be made in this step to move to the deployment phase.
6. Deployment - The knowledge or information that gain through data mining process needs to be presented in such a way that stakeholders can use it when they want it. Based on the business requirements, the deployment phase could be as simple as creating a report or as complex as a repeatable data mining process across the organization. In this phase, the deployment, maintained and monitoring plans have to be created for deployment and future supports. From project point of view, the final report of the project need to summary the project experiences and review the project to see what need to improved created learned lessons.

The CRISP-DM offers a uniform framework for experience documentation and guidelines. In addition the CRISP-DM can apply in different industry with different type of data.

In this tutorial, you’ve learned about data mining processes and examine the cross-industry standard process for data mining in details.

Summary: This article discusses about the data explosion, knowledge discovery and more importantly answer the question what is data mining by introducing some data mining definitions.

Data Explosion

Data Explosion

Nowadays corporate and organizations are accumulating data at an enormous rate and from a very broad variety of sources such as customer transactions, credit card transactions, bank cash withdrawal to hourly weather data. A lot of relational database servers have been built to store such massive quantities of data. To put the data into the database servers, online transactional process (OLTP) systems have been developed to help business run smoothly based on their own business processes. Those OLTP systems stores all the transactional data into the database for every transaction happens to the business in every second such as sale orders, purchase orders in sale to head count data in human capital management. To enables the top executives to make decisions faster based on facts, online analytical processing (OLAP) systems such as data warehouses have been developed rapidly recently. There are a vast amount of data is recorded in the OLTP systems and pushing to OLAP systems for reporting purpose. As the matter of fact, the data itself is critical to a company’s growth. It contains knowledge that could lead to important business decisions that bring business to the next level. These data is never been examined in a superficial manner. It is becoming data rich but knowledge poor.

We need information but what we have is a huge amount of data flooding around companies, organizations even individuals. Because of the amount of data is so enormous that human cannot process it fast enough to get the information out of it at the right time, the machine learning technology has been established to solve this problem potentially.

Knowledge Discovery

Knowledge discovery is a process that extracts implicit, potentially useful or previously unknown information from the data. The knowledge discovery process is described as follows:

Knowledge Discovery Process

Let’s examine the knowledge discovery process in the diagram above in details:

• Data comes from variety of sources is integrated into a single data store called target data
• Data then is pre-processed and transformed into standard format.
• The data mining algorithms process the data to the output in form of patterns or rules.
• Then those patterns and rules are interpreted to new or useful knowledge or information.

The ultimate goal of knowledge discovery and data mining process is to find the patterns that are hidden among the huge sets of data and interpret them to useful knowledge and information. As described in process diagram above, data mining is a central part of knowledge discovery process. Let answer the question “what is data mining?” by examining several data mining definitions.

What is Data Mining – Data Mining Definitions

The data mining definition appears on the first papers on commercial data mining is defined as:

The process of extracting previously unknown, comprehensible and actionable information from large databases and using it to make crucial business decisions  - Simoudis 1996.

This data mining definition has business flavor and for business environments. However, data mining is a process that can be applied to any type of data ranging from weather forecasting, electric load prediction, product design, etc.

Data mining also can be defined as the computer-aid process that digs and analyzes enormous sets of data and then extracting the knowledge or information out of it. By its simplest definition, data mining automates the detections of relevant patterns in database.

The data mining tutorial section gives you a brief introduction of data mining, its important concepts, architecture, process and applications. If you are new to the data mining area and looking for a good overview of data mining, this section is designed just for you.

What Data Mining Tutorial Covers

• What is data mining- discusses about data explosion, knowledge discovery and provides some definitions of data mining.
• Data mining architecture - covers various data mining architecture and discusses about its advantages and disadvantages of each.
• Advantages & disadvantages of data mining - discusses about the advantages and disadvantages of data mining using businesses, society, governments and individual.
• Data mining processes -  covers Cross Industry Standard Process for Data Mining  or CRSP-DM, one of the most well-known and practical data mining process.
• Applications of data mining – looks into data mining applications in various areas including sales/marketing, banking, insurance, health care, transportation and medicine.
• Data mining example – gives you a real world example of data mining.
• Data mining and warehousing – discusses about the data mining and data warehouse.

If you want to have more data mining tutorial that we don’t cover here, feel free to request through contact form.

We are trying our best to provide you all information related to data mining, tools and applications. If you have any suggestion or feedback to make our website better, feel free to drop us an email.

The data mining techniques website was launched to offer you a comprehensive information on data mining and data mining techniques. It also gives you review and reference to data mining tools and applications.

There are several major data mining techniques have been developed and used in data mining projects recently including association, classification, clustering, prediction and sequential patterns. We will briefly examine those data mining techniques with example to have a good overview of them.

Association

Association is one of the best known data mining technique. In association, a pattern is discovered based on a relationship of a particular item on other items in the same transaction. For example, the association technique is used in market basket analysis to identify what products that customers frequently purchase together. Based on this data businesses can have corresponding marketing campaign to sell more products to make more profit.

Classification

Classification is a classic data mining technique based on machine learning. Basically classification is used to classify each item in a set of data into one of predefined set of classes or groups. Classification method makes use of mathematical techniques such as decision trees, linear programming, neural network and statistics. In classification, we make the software that can learn how to classify the data items into groups. For example, we can apply classification in application that “given all past records of employees who left the company, predict which current employees are probably to leave in the future.” In this case, we divide the employee’s records into two groups that are “leave” and “stay”. And then we can ask our data mining software to classify the employees into each group.

Clustering

Clustering is a data mining technique that makes meaningful or useful cluster of objects that have similar characteristic using automatic technique. Different from classification, clustering technique also defines the classes and put objects in them, while in classification objects are assigned into predefined classes. To make the concept clearer, we can take library as an example. In a library, books have a wide range of topics available. The challenge is how to keep those books in a way that readers can take several books in a specific topic without hassle. By using clustering technique, we can keep books that have some kind of similarities in one cluster or one shelf and label it with a meaningful name. If readers want to grab books in a topic, he or she would only go to that shelf instead of looking the whole in the whole library.

Prediction

The prediction as it name implied is one of a data mining techniques that discovers relationship between independent variables and relationship between dependent and independent variables. For instance, prediction analysis technique can be used in sale to predict profit for the future if we consider sale is an independent variable, profit could be a dependent variable. Then based on the historical sale and profit data, we can draw a fitted regression curve that is used for profit prediction.

Sequential Patterns

Sequential patterns analysis in one of data mining technique that seeks to discover similar patterns in data transaction over a business period. The uncover patterns are used for further business analysis to recognize relationships among data.