MANTHAN: Your personal journey in the field of big data analytics is an exciting one to read . What is the biggest surprise you’ve experienced in this rapidly growing, data driven world?
KIRK: The biggest surprise that I have experienced is how incredibly rapidly the world has woken up to and adopted the power of data and the power of algorithms. I worked in this field for many years and it was hard to get anyone to take it seriously 10-15 years ago. But in the past 5 years, the field has exploded, including startups, new businesses, new lines of business in old organizations, data science advocates in all types of organizations, the number of use cases across all sectors, and the demand for data scientists and data scientist training programs.
The growth of interest, applications, tools, startups, and people working in this field almost seems to be *faster* than exponential.
MANTHAN: You’ve shared an example in our round-up on the impact big dataanalytics has made on society. Could you give us an example of how analysis of big data has helped improve a business situation?
KIRK: A very large financial services company (we’ll call it ABC) was concerned about customer attrition. Whenever a customer took their investments to another business, ABC lost money. So, ABC decided to explore their customer engagement data to see if they could find a precursor signal in the data that indicated when a customer was perhaps likely to take their investments out and move their funds to another business.
ABC invested approximately one million dollars in a “proof of concept” project to search the data and build a model of customer attrition. They did find a signal in the data, which was simply the sudden increase in the frequency of customer logins to their online account in the month prior to withdrawing their investments. So, ABC deployed a friendly customer engagement program in which they reached out to those customers whose monthly login frequency suddenly increased — ABC sent information about a new online investment calculator, investment performance metrics, new investment strategies, updated FAQs, financial management advice, tools to simplify reinvesting in their other financial service products, etc. At the end of the 3-month “proof of concept” test period, ABC estimated that their one million dollar data science investment had saved the company one billion dollars in customer value!
That was an ROI of $1000 for every $1 investment in their program. So, the company decided to invest in a much larger permanent data science team for greater customer engagement and marketing insights.
“The growth of interest, applications, tools, startups, and people working in this field almost seems to be *faster* than exponential.”
MANTHAN: As the Principal Data Scientist at Booz Allen Hamilton, how do you feel perceptions have changed (for good or bad) towards big data analytics solutions?
KIRK: I believe that the worst of the “big data” hype is now behind us. A lot of growth in the data science and big data analytics fields has already taken place in the past few years, despite all of the negative hype. Now, with the hype being diminished, we are settling down to greater growth, investment, innovation, and value creation in the field. So, the perceptions are definitely changed for the good!
MANTHAN: What recent problem have you helped solve by applying a data driven scientific solution?
KIRK: I cannot discuss client-specific problems and solutions, but one case that we are developing for a future potential client is around a fairly well known and common problem in the field of talent analytics: using data and modeling techniques to predict employee performance, job satisfaction, and possible attrition.
We considered an ensemble of different predictive modeling techniques that gave greater insights into the problem than a single model approach, with most of our focus is on the employee attrition problem (which is much like the customer attrition problem mentioned above): what are the key signals in the data that might indicate when such an outcome is likely to occur?
“I believe that the worst of the “big data” hype is now behind us.”
MANTHAN: What do you imagine the future will be like when we have greater computing capabilities to fully harness the potential of our big data?
KIRK: There are 3 essential contributors to big data analytics solutions and data science success, and why big data science is so popular these days, are these:
- larger and more comprehensive data sets.
- more efficient and effective algorithms.
- faster more powerful computing capabilities.
As computing capabilities increase, then similarly will our abilities increase to explore even greater dimensions and combinations of dimensions of much larger data sets.
The insights to be gained will continue to grow in proportion to the computing power that we can apply to the problem. The combinatorial explosion of different combinations of diverse data sets to be explored will become even more enormous as the Internet of Things grows in diversity and ubiquity. The discovery potential from all of these new acquired data will be lost unless we also acquire greater computing capabilities.
So, I expect that the 3 essential contributors to big data science success will continue for a long time to come!
More Resources 
