Month: January 2017

Business Intelligence: Predictions

The future of …

  • Data mining:

o    Web structure mining (studying the web structure of web pages) and web usage analysis (studying the usage of web pages) will become more prominent in the future.  Victor and Rex (2016) stated that web mining differs from traditional data mining by scale (web information is much larger in number, making 10M web pages seem like it’s too small), access (web information is mostly public, whereas traditional data could be private), and structure (web pages have unstructured, and semi-structured data, whereas traditional data mining, has some explicit level of structure).  The structure of a website can contain: Page Rank, Page number, Damping factor, Number of pages, out-links, in-links, etc.  Your page is considered an authoritative piece if there are many in-links, or it can be considered a hub if it has many out-links, and this helps define page rank and structure of the website (Victor & Rex, 2016).  But, page rank is too trivial of calculation.  One day we will be able to not only know a page rank of a website, but learn its domain authority, page authority, and domain validity, which will help define how much value a particular site can bring to the person.  If Google were to adopt these measures, we could see

  • Data mining’s link to knowledge management (KM):

o    A move towards the away from KM tools and tool set to seeing knowledge as being embedded into as many processes and people as possible (Ferguson, 2016). KM relies on sharing, and as we move away from tools, processes will be setup to allow this sharing to happen.  Sharing occurs more frequently with an increase in interactive and social environments (Ferguson, 2016).  Thus, internal corporate social media platforms may become the central data warehouse, hosting all kinds of knowledge.  The issue and further research need to go into this, is how do we more people engaged on a new social media platform to eventually enable knowledge sharing. Currently, forums, YouTube, and blogs are inviting, highly inclusive environments that share knowledge, like how to solve a particular issue (evident by YouTube video tutorials).  In my opinion, these social platforms or methods of sharing, show the need for a more social, inclusive, and interactive environment needs to be for knowledge sharing to happen more organically.

o    IBM (2013), shows us a glimpse of how knowledge management from veteran police officers, crime data stored in a crime data warehouse, the power of IBM data mining, can be to identifying criminals.  Mostly criminals commit similar crimes with similar patterns and motives.  The IBM tools augment officer’s knowledge, by narrowing down a list of possible suspects of crime down to about 20 people and ranking them on how likely the suspects committed this new crime.  This has been used in Miami-Dade County, the 7th largest county in the US, and a tool like this will become more widespread with time.

  • Business Intelligence (BI) program and strategy:

o    Potential applications of BI and strategy will go into the health care industry.  Thanks to ObamaCare (not being political here), there will be more data coming in due to an increase in patients having coverage, thus more chances to integrate: hospital data, insurance data, doctor diagnosis, patient care, patient flow, research data, financial data, etc. into a data warehouse to run analytics on the data to create beneficial data-driven decisions (Yeoh, & Popovič, 2016; Topaloglou & Barone, 2015).

o    Potential applications of BI and strategy will affect supply chain management.  The Boeing Dreamliner 787, has outsourced 30% of its parts and components, and that is different to the current Boeing 747 which is only 5% outsourced (Yeoh, & Popovič, 2016).  As more and more companies increase their outsourcing percentages for their product mix, the more crucial is capturing data on fault tolerances on each of those outsourced parts to make sure they are up to regulation standards and provide sufficient reliability, utility, and warranty to the end customer.  This is where tons of money and R&D will be spent on in the next few years.

References

  • Ferguson, J. E. (2016). Inclusive perspectives or in-depth learning? A longitudinal case study of past debates and future directions in knowledge management for development. Journal of Knowledge Management, 20(1).
  • IBM (2013). Miami-Dade Police Department: New patterns offer breakthroughs for cold cases. Smarter Planet Leadership Series.  Retrieved from http://www.ibm.com/smarterplanet/global/files/us__en_us__leadership__miami_dade.pdf
  • Topaloglou, T., & Barone, D. (2015) Lessons from a Hospital Business Intelligence Implementation. Retrieved from http://www.idi.ntnu.no/~krogstie/test/ceur/paper2.pdf
  • Victor, S. P., & Rex, M. M. X. (2016). Analytical Implementation of Web Structure Mining Using Data Analysis in Educational Domain. International Journal of Applied Engineering Research, 11(4), 2552-2556.
  • Yeoh, W., & Popovič, A. (2016). Extending the understanding of critical success factors for implementing business intelligence systems. Journal of the Association for Information Science and Technology, 67(1), 134-147.

Business Intelligence: OLAP

Within a Business Intelligence (BI) program online analytical processing (OLAP) and customer relationship management (CRMs) are both applications have strategic uses for the company and are dependent on the data warehouse to help analyze multidimensional datasets stored in them to provide data-driven solutions to queries. They are both systems that require data analytics to turn all the multidimensional data into insightful information. OLAP’s multidimensional view of the data warehouse data sets can occur because it is mapped onto n-dimensional data cubes, where data can then be easily rolled up, drilled down, slice and dice, and pivot (Conolly & Begg, 2014). OLAP can have many applications outside of customer relationships.  Thus, OLAP is more versatile compared to CRMS, because CRMs are more targeted/focused with their approach, analysis of the customer relationship to the company/product.  CRMs main goal is to analyze internal and external data stored in the data warehouse, to come up with insights like “predicted affinity to buy” of a customer, the “cost or profit” of a customer, “prediction of future customer behavior”, etc. (Ahlemeyer-Stubbe & Shirley, 2014).  The information gained from the CRM can empower employees at the company on a customer’s affinity towards a product to either sell similar items or items of the result in a market basket analysis.

OLAP is the online analytical processing application, which allows people to examine data in real time from different points of view in aid to driving more data-driven decisions (McNurlin et al., 2008).  With OLAP, computers can now make what-if analysis and goal-based decisions using data. The key ability of OLAPs systems are to help answer the “Why?” question, as well as the typical “Who?” and “What?” questions (Conolly & Begg, 2014).  Connolly and Begg (2014) further explain that OLAP is a specialized implementation of SQL. Unfortunately, data queried is assumed to be static and unchanging.  Hence, the low volatile aspect of a data warehouse, with multidimensional databases is ideal for OLAP apps.  They value of the data warehouse does not come from just storing the right kind of data, but through making and conducting analysis, to solve queries that will in the end help make data driven decisions that are the best for the company.  According to Conolly & Begg (2014), OLAP tools have been used in studying the effectiveness of marking campaigns, product sales forecasting, and capacity planning.  However, it is of the opinion of Conolly & Begg (2014) that data mining tools can surpass the capabilities of OLAP tools.

CRMs, on the other hand, focuses a wide range of concepts revolving how companies store, capture and analyze customer, vendor, and partner relationship data. Information stored in CRMs could be interactions with customers, vendors or partners, which allow the company to gain insights based on previous interactions and could even be grouped/associated into different customer segments, market basket analysis, etc. (Ahlemeyer-Stubbe & Shirley, 2014). CRMs can assist in real time with making data-driven decisions with respects to a company’s customers (Mcnurlin, Sprague, & Bui, 2008).  The goal is to use the current data, to help the company build more optimal communications and relationships with it customers, vendors or partners.  Both internal and external data of the company is usually added to the data warehouse for the CRM. Through the use of the internet, companies can study more about their customers and their noncustomers, to aid a company to become more customer centric (McNurlin et al., 2008).  McNurlin et al. (2008) stated a case study with Wachovia Bank purchasing a pay-by-use CRM system from salesforce.com.  After the system was set up within six weeks, sales reps had 30 more hours to use on selling more bank services, and managers can use the data that was collected by the CRM to tell the sales reps which customers would have the highest yield.

References:

Business Intelligence: Decision Support Systems

Many years ago a measure of Business Intelligence (BI) systems was on how big the data warehouse was (McNurlin, Sprague,& Bui, 2008).   This measure made no sense, as it’s not all about the quantity of the data but the quality of the data.  A lot of bad data in the warehouse means that it will provide a lot of bad data-driven decisions. Both BI and Decision Support Systems (DSS) help provide data to support data-driven decisions.  However, McNurlin et al. (2008) state that a DSS is one of five principles of BI, along with data mining, executive information systems, expert systems, and agent-based modeling.

  • A BI strategies can include, but is not limited to data extraction, data processing, data mining, data analysis, reporting, dashboards, performance management, actionable decisions, etc. (Fayyad, Piatetsky-Shapiro, & Smyth, 1996; Padhy, Mishra, & Panigrahi, 2012; and McNurlin et al., 2008). This definition along with the fact the DSS is 1/5 principles to BI suggest that DSS was created before BI and that BI is a more new and holistic view of data-driven decision making.
  • A DSS helps execute the project, expand the strategy, improve processes, and improves quality controls in a quickly and timely fashion. Data warehouses’ main role is to support the DSS (Carter, Farmer, & Siegel, 2014).  The three components of a DSS are Data Component (comprising of databases, or data warehouse), Model Component (comprising of a Model base) and a dialog component (Software System, which a user can interact with the DSS) (McNurlin et al., 2008).

McNurlin et al (2008) state a case study, where Ore-Ida Foods, Inc. had a marketing DSS to support its data-driven decisions by looking at the: data retrieved (internal data and external market data), market analysis (was 70% of the use of their DSS, where data was combined, and relationships were discovered), and modeling (which is frequently updated).  The modeling offered great insight for the marketing management.  McNurlin et al. (2008), emphasizes that DSS tend to be defined, but heavily rely on internal data with little or some external data and that vibrational testing on the model/data is rarely done.

The incorporation of internal and external data into the data warehouse helps both BI strategies and DSS.  However, the one thing that BI strategies provide that DSS doesn’t is “What is the right data that should be collected and presented?” DSS are more of the how component, whereas BI systems generate the why, what, and how, because of their constant feedback loop back into the business and the decision makers.  This was seen in a hospital case study and was one of the main key reasons why it succeeded (Topaloglou & Barone, 2015).  As illustrated in the hospital case study, all the data types were consolidated to a unifying definition and type and had a defined roles and responsibilities assigned to it.  Each data entered into the data warehouse had a particular reason, and that was defined through interviews will all different levels of the hospital, which ranged from the business level to the process level, etc.

BI strategies can affect supply chain management in the manufacturing setting.  The 787-8, 787-9, and 787-10 Boeing Dreamliners have outsourced ~30% of its parts and components or more, this approach to outsourcing this much of a product mix is new since the current Boeing 747 is only ~5% outsourced (Yeoh, & Popovič, 2016).  As more and more companies increase their outsourcing percentages for their product mix, the more crucial it is to capture data on fault tolerances on each of those outsourced parts.  Other things that BI data could be used is to make decisions on which supplier to keep or not keep.  Companies as huge as Boeing can have multiple suppliers for the same part, if in their inventory analysis they find an unusually larger than average variance in the performance of an item: (1) they can either negotiate a lower price to overcompensate a larger than average variance, or (2) they could all together give the company a notice that if they don’t lower that variance for that part they will terminate their contract.  Same things can apply with the auto manufacturing plants or steel mills, etc.

Resources:

 

Business Intelligence: Data Mining

Data mining is just a subset of the knowledge discovery process (or concept flow of Business Intelligence), where data mining provides the algorithms/math that aid in developing actionable data-driven results (Fayyad, Piatetsky-Shapiro, & Smyth, 1996). It should be noted that success has much to do with the events that lead to the main event as it does with the main event.  Incorporating data mining processes into Business Intelligence, one must understand the business task/question behind the problem, properly process all the required data, analyze the data, evaluate and validate the data while analyzing the data, apply the results, and finally learn from the experience (Ahlemeyer-Stubbe & Coleman, 2014). Conolly and Begg (2014), stated that there are four operations of data mining: predictive modeling, database segmentation, link analysis, and deviation detection.  Fayyad et al. (1996), classifies data mining operations by their outcomes: prediction and descriptive.

It is crucial to understand the business task/question behind the problem you are trying to solve.  The reason why is because some types of business applications are associated with particular operations like marketing strategies use database segmentation (Conolly & Begg, 2014).  However, any of the data mining operations can be implemented for any business application, and many business applications can use multiple operations.  Customer profiling can use database segmentation first and then use predictive modeling next (Conolly & Begg, 2014). By thinking outside of the box about which combination of operations and algorithms to use, rather than using previously used operations and algorithms to help meet the business objectives, it could generate even better results (Minelli, Chambers, & Dhiraj, 2013).

A consolidated list (Ahlemeyer-Stubbe & Coleman, 2014; Berson, Smith, & Thearling 1999; Conolly & Begg, 2014; Fayyad et al., 1996) of the different types of data mining operations, algorithms and purposes are listed below.

  • Prediction – “What could happen?”
    • Classification – data is classified into different predefined classes
      • C4.5
      • Chi-Square Automatic Interaction Detection (CHAID)
      • Support Vector Machines
      • Decision Trees
      • Neural Networks (also called Neural Nets)
      • Naïve Bayes
      • Classification and Regression Trees (CART)
      • Bayesian Network
      • Rough Set Theory
      • AdaBoost
    • Regression (Value Prediction) – data is mapped to a prediction formula
      • Linear Regression
      • Logistic Regression
      • Nonlinear Regression
      • Multiple linear regression
      • Discriminant Analysis
      • Log-Linear Regression
      • Poisson Regression
    • Anomaly Detection (Deviation Detection) – identifies significant changes in the data
      • Statistics (outliers)
  • Descriptive – “What has happened?”
    • Clustering (database segmentation) – identifies a set of categories to describe the data
      • Nearest Neighbor
      • K-Nearest Neighbor
      • Expectation-Maximization (EM)
      • K-means
      • Principle Component Analysis
      • Kolmogorov-Smirnov Test
      • Kohonen Networks
      • Self-Organizing Maps
      • Quartile Range Test
      • Polar Ordination
      • Hierarchical Analysis
    • Association Rule Learning (Link Analysis) – builds a model that describes the data dependencies
      • Apriori
      • Sequential Pattern Analysis
      • Similar Time Sequence
      • PageRank
    • Summarization – smaller description of the data
      • Basic probability
      • Histograms
      • Summary Statistics (max, min, mean, median, mode, variance, ANOVA)
  • Prescriptive – “What should we do?” (an extension of predictive analytics)
    • Optimization
      • Decision Analysis

Finally, Ahlemeyer-Stubbe and Coleman (2014) stated that even though there are a ton of versatile data mining software available that would do any of the abovementioned operations and algorithms; a good data mining software would be deployable across different environments and include tools for data prep and transformation.

References