Most companies are woefully inefficient at strategically using the vast amount of corporate data at their disposal. While data acquisition, data storage, and data management capabilities are accelerating exponentially, recent surveys on management trends note most executives feel they are lagging their own expectations for using this accumulating data for strategic planning and business decision making.

Data preparation remains a major hurdle. When speaking with corporate business analysts, they disparagingly refer to much of their data mining process as a tedious time sink with a Pareto-structured 80% of their time spent data sourcing and data cleansing and only only 20% spent on strategic analysis and developing business recommendations. They desperately want to flip this ratio.

High Hopes for Business Analytics Technology Advances

Business analytics trends indicate technology advances will further automate data acquisition, data integration, and data cleansing, but statistical analysis tools are integrating advanced analytics and predictive modeling automation as well. If you add in front-end natural language query interfaces for non-technical business users and embed decision rules for prescriptive analytics, the primary question involves the role business analysts will fill between data scientists and business decision makers.

Will machines replace business analysts? No, and here is why:

1. The huge expansion of captured information creates a surge in data volume that still needs sorting, cleansing, cataloguing, and data munging. Although more of this process will be automated, the shear increase in data volume greatly expands data discovery and data verification tasks.
2. The ability to analyze a wider variety of data sources, both structured and unstructured, requires creative thinking about what new information may be relevant for specific decisions.
3. Advanced analytics leveraging artificial intelligence, and machine learning in particular, require supervised iterative training to produce reliable predictive analytics models.
4. Most companies are still in an early discovery stage for identifying key indicators and predictive data sets. Business analysts, data scientists, and data engineers have a lot of work ahead before automation extends across companies.
5. The turnover of personnel and the plain whims of executives in the C-suite ensure a constantly changing set of business questions will arise in support of strategic directions.

Taking all of the emerging technology trends, rising data volumes, and changing business questions into account, the role of data analysts (and data engineers) are safe for the foreseeable future. It also appears the balance of data sourcing and data preparation versus analysis supporting organizational objectives and decision making will unfortunately remain the same.

Rather than altering the mix of data preparation and data analysis, the greatest change ahead for business analytics is overall productivity. Business analysts will be able to leverage a higher volume and variety of data and develop business recommendations at a faster speed. The expectations for higher analytical accuracy will, of course, rise as well.

Rising Data Analytics Productivity Gated Only by Organizational Skills

The strategic decision-making process will never be fully automated. SAS Business Analytics, IBM Business Analytics and Optimization, and other leading analytics solutions, are paving the way for automation, but they also lead with management consulting and training services. New strategic management processes may require significant considerations for organizational change and roles for data scientists, data engineers, business analysts, and front-line decision makers.

IBM’s C-suite research, “Analytics: The New Path to Value” indicates the greatest challenges to adopting widespread business analytics are organizational barriers rather than data or investment requirements. According to the study, the top obstacles include:

• Lack of understanding how to use analytics to improve the business
• Lack of bandwidth due to competing priorities
• Lack of skills internally in the line of business

As our technology solutions and data collection capabilities advance, demand for guidance from business management consultants and training providers will grow. Technology will improve the speed and reliability of advanced analytics and risk management to support strategic objectives, but business analysts still have plenty of work ahead on discovering, collecting, cleansing, and analyzing key data sets to feed into ongoing business analysis and decision-making processes.