Why AI & Machine Learning projects stall
Artificial Intelligence (AI), Machine Learning (ML) and data science have reached a level of awareness and hype not seen in the technology space for some years. Not a week passes where the issue of lack of specialist skills and capabilities is not pointed out as being the major impediment to projects or where examples of specific solutions are hailed as being the ultimate proof of concept. However, as with most new technology areas, moving from a proof-of-concept phase to scale implementations is proving hard, especially in AI & ML. The average large and mid-sized organization has yet to implement and manage more than a handful of algorithms or machine learning models. They have not yet broken out of the proof of concept mode. Why is that? Simply put it is lack of dedicated money, experience and skills coupled with the right constituencies of the technology provider and implementor not engaging with each other. And there is a simple way to understand this.
Most new technologies go through the challenge of moving from proof of concept phase to broader application. All of them deal with a relatively simple two-by two-challenge represented below. Let’s break this down
First dimension: Technology or software providers
Two main requirements must be in place to succeed (apart from the products and services, obviously). The first requirement is technical expertise such as skills and experience in machine learning, data science and programming. This is generally the strength of many small and midsized AI and machine learning-based technology providers. Even though they are generally challenged with finding experienced people in these areas. The second requirement is business expertise such as understanding the issues of different business functions like Supply Chain, Finance or HR or understanding, in-depth, the regulatory and business issues of different vertical industries. Here many AI providers either lack expertise and experience or are only experts in one narrow and specific area and generally not at depth. Note that in some instances, the internal IT organization acts as a technology provider to the rest of the business and could therefore be considered in this dimension.
Second dimension: Organizations implementing AI solutions.
The same two requirements are needed in organizations implementing the AI solution. First is the need for technical expertise, especially data science and machine learning expertise. Most organizations have only recently established this and few of these organizations have experience with large scale implementations. The second requirement is business expertise and experience. Most functional leadership in many organizations have limited understanding of AI and ML, let alone the requirements around data science and data management. To scale AI solutions, it is paramount that functional leadership or the business side of organizations acquire deeper knowledge of what is required to succeed with AI and ML.
In both technology providers and organizations implementing AI solutions, it is paramount that the business and technology part of the organization collaborates in each organization and between organizations. Solving this will result in the expected rapid growth of AI and ML.
Now that I have established this simple matrix, we can learn from the challenges currently experienced in scaling AI, by looking at the intersections between the technology provider (Provider) and the implementing organization (Implementor). Some of the observations, although not exhaustive, are:
Technical (Provider) to/from Technical (implementor):
Most AI technology providers engage their technical expertise with the client’s technical expertise. While critical to AI the project, a number of issues can arise that are good for both the provider and implementor to understand.
First, without having an established relationship with the business function and understanding of the business issues for the organization implementing the project, an increased risk of project failure arises.
Secondly, funding of AI and ML projects often come from the business functions budget but without the business function involved, projects remain small, become harder to scale and can get cancelled.
Third, without involvement of the business function, access to data can become a challenge thus limiting the ability to scale the project.
And lastly, given that most internal data science teams are newly established, using external technical expertise is often regarded as a threat by the internal team. Both the provider and the implementor need to appreciate this.
Technical (Provider) to/from Business (Implementor): In some instances, the technology providers technical expertise engages with the client’s business function. This can lead to two challenges.
First, is the obvious inability to understand each other given the different backgrounds and objectives. This causes misaligned project outcomes.
The second it’s simply not progressing a project at all.
Technical (Implementor) to/from Business (Provider): In the reverse situation of implementors, data science or development organization engaging the business experts of the technology provider similar challenges as in the reverse situation occur.
Business (Implementor) to Business (Provider): Lastly, the engagement between the business side of the implementor and provider is the area with the greatest opportunity. The providers business experts can give tangible use cases to the implementors business organization, thereby ensuring greater buy-in and ability to unlock the opportunities that AI provides to organizations from a perspective of revenue growth, cost optimization and risk management. Equally it provides an opportunity for the implementing organization’s business leadership to gain knowledge in how to manage and lead organizations in which AI and ML is an integral part of the business function.
Without clear engagement models between technology providers of AI and ML solutions and organizations with a desire to scale AI solutions, we will not move beyond a proof on concept phase. The development of the senior leadership in the implementing organization around the benefits of AI and ML solutions, as well as how AI and machine learning changes management and leadership skills, is critical. Stay tuned as the latter will be covered here in greater detail over the next couple of months.