Artificial intelligence (AI) seems to be the latest term which is fostering great attention and publicity amongst the general public as demonstrated by the many articles in much of the mainstream media. For example, AI is the cover story in the most recent edition of The Economist .Yet, for experienced analytics practitioners in marketing and in particular direct marketing, AI is not necessarily new. The development of predictive models have always considered the latest mathematical advancements such as AI. However, the most recent breakthrough in significantly improved performance of AI solutions has occurred due to the ability in leveraging larger volumes of data as well as the ability to process it. As a result, marketers and data scientists need to more fully consider AI as a viable predictive analytics solution that can be deployed to solve the business problem at hand. With AI becoming more integrated into businesses, elimination of manual tasks and processes and ultimately increased automation will become the outcome. But first let’s explore a little bit of history concerning AI and increased automation.
No one doubts the fundamental changes caused by artificial intelligence (AI) that will emerge both long-term as well as short-term, particularly as the economy continues to move more towards increased automation. In the last twenty years, we have certainly witnessed this increasing level of automation. Think of the typical office in the 1980’s. In creating a document that was sent to another department, 3 people were involved which included the creator of the document, an admin assistant to type it, and a courier to transfer it to the appropriate area. With automation, all these tasks can now be done by one person. No advanced artificial intelligence was utilized here. Instead, the birth of the PC and the internet were the fundamental cornerstones in providing this increased level of office automation.
Another example of increasing automation are call center customer service areas which have developed automated voice messaging systems. These systems purport to achieve better customer service, which is arguably debateable, but from the organizational standpoint, they more importantly achieve significant cost savings. Despite these cost savings, most of us have experienced heightened levels of frustration in trying to talk to an actual live person.
With increasing automation, cost effectiveness has been the mantra of most organizations in the last 20 years. This enhanced focus on cost effectiveness has also resulted in organizations exploring options to outsource tasks to countries with a lower standard of living. Over the years, we have seen how outsourcing of tasks has evolved from the more routine mundane tasks to the more highly advanced knowledge-based tasks. I cannot tell you the number of calls that I have received about outsourcing the technical data science skills in building a predictive model. Under this paradigm, my company becomes the “managers” of predictive analytics solutions in applying outsourced solutions towards a given problem. However, our business model has always been to both manage and execute the development of predictive analytics solutions and at this point in time we have not bowed to the altar of outsourced analytics. No need to discuss the merits or failures of this approach as this debate is now becoming a mainstream political issue throughout the world.
With the evolution of artificial intelligence becoming more mainstream, one may ask about what will be the impact in automation and in particular to the more knowledge-intensive tasks such as the discipline of predictive analytics. Increasing levels of automation that continue to replace labor may result in outsourcing becoming a moot point as the technology costs become far inferior to even using lower-cost labor from third world countries. But what skills will artificial intelligence (AI) replace within the data scientist’s arsenal. In theory with AI, choosing the right mathematical algorithm becomes obsolete as the machine determines the right technique. The machine through its artificial intelligence algorithms outputs the solution which can be immediately applied to a given business problem. In AI, we hear such new concepts as deep learning, which utilize the mathematics of neural nets. As mentioned above, neural nets are not new to predictive analytics and have been used by practitioners for the last twenty years, particularly by marketing departments where analytics is a primary focus. In fact, much of the more recent developments in AI have been about enhancing the developments of neural net algorithms and the literature has provided many examples that have yielded superior results to what was developed fifteen years ago. This is particularly relevant in the areas of image and voice recognition. Yet, recognizing that these algorithms are indeed better than in the past, does that translate to improved performance in predicting consumer behaviour within a marketing program? The reason for this question and its specificity towards consumer behaviour is that much of the historical work done in predictive analytics has been in the area of marketing and credit card risk where we are dealing with consumers as the records of interest. In my experience, much of this behaviour is difficult to predict with a high level of accuracy due to the high degree of random error or variation. Under these kind of scenarios, simple solutions trying to generalize overall trends can work quite well with some of the obvious solutions being techniques such as logistic regression or decision trees. But this does not mean that AI should always be discarded when looking at consumer behaviour. In fact, one could simply broaden the definition of AI to look at all modelling techniques which include neural nets as well as the more traditional techniques. Software already exists and a few of the leading providers in this area are able to provide this kind of capability.
So where is the biggest impact of AI on data science and what skills of the data scientist can be done more quickly by the machine? If we look at the four step approach to building predictive analytics solutions as mentioned in my book: “Data Mining for Managers-How to Use Data (Big and Small) to Solve Business Challengers”, we can better understand where AI will have the biggest impact. Each of these four stages (identifying the business problem, creating the analytical file, applying the right analytical technique, implementation and measurement of the solution) are critical in achieving an end solution. Can AI completely replace all the skills that are required in each of these four stages or are there certain stages and tasks where AI will have the most relevance? Future articles will examine this in more detail.
There is no question that artificial intelligence is changing the predictive analytics landscape. We can either act as Luddites and refuse to acknowledge the technology or embrace it and try to better understand what this exactly means within the predictive analytics process. By leveraging the advantages of artificial intelligence, the vision of the future predictive analytics practitioner might be a shift towards solving more business problems and creating the right data environment for the specific business problem and less emphasis on some of the more technically demanding tasks. In subsequent blogs on this same topic, I will discuss this vision in more detail and outline what the future of a data scientist might look like in a world using the latest AI technologie