(Originally published on VentureBeat “Why the hybrid AI compromise isn’t a solution at all”)
In recent years, AI has emerged as an undisputed competitive advantage, which means smart business leaders facing the digital transformation have some really familiar questions: How do you measure success? What questions do you need to ask to ensure that you’re making the right decisions right out of the gate? And most importantly, how do you implement AI in your business to deliver a positive impact?
The answers to those questions all fall under one umbrella: Do you choose a centralized approach or a decentralized approach to embedding AI into your business? There are pros and cons to both methods.
Centralized vs decentralized
The first step of any data science project is to define a problem within the business — often referred to as strategic problem formulation — and develop a data set for that problem. Once that is complete, you enter what Ople CEO Pedro Alves defines the data science technical chasm. The chasm is the point at which the data scientist finishes all the technical steps, such as feature engineering and algorithm selection. Once the AI model is built, the data scientist delivers the model and the end business user begins to use the solution.
It’s within this process the roadblocks happen, and the biggest problem with both centralized and decentralized AI is communication. For the centralized team, there’s great communication within the technical area, but not so much with the business team that’s actually going to use the product. With decentralized, it’s the opposite. Business users and data scientists will collaborate greatly, but data scientists would be isolated from other teams.
The challenges for each
In a centralized approach, the team is isolated from all other groups they’re serving within an organization. As a result, it requires extra efforts for the team to understand the nuances of the problems each internal business unit is facing. Due to the lack of collaboration, the effectiveness in building useful data science or useful AI might not be as good.
With the decentralized approach, every team is equipped with its own data scientists. However, data scientists now operate on their own teams. You gain the advantage of embedding AI capabilities directly within each team, increasing the Return on AI from each project. Still, you lose the cohesive approach toward data science for your entire organization.
The hiring dilemma
One of the eternal problems of implementing AI is hiring enough data scientists, of high enough quality, to fit either model. Addressing the centralized aspect, Pedro says that the volume of products that a core AI department can produce is low. “I’ve never seen a centralized data science or AI team that can serve all the entities within the organization that need it,” Pedro explains. “There are only so many people they can hire and so many projects they can do.”
With a decentralized team, departments can run into problems in their ability to hire the right people to meet their needs.
“It’s easier to get that super talent when you have a cutting edge AI project or goal,” Pedro says. “When you’re not a top-notch tech company, but you need a highly-skilled AI person, it’s less enticing, so then it’s harder to hire. The talent you can hire is less experienced, so the quality of the work is lesser. Ultimately, how quickly you can create a team to execute AI projects is substantially lower on the decentralized approach.
The hybrid approach – not the best of both worlds
The hybrid approach might sound like the best solution — but it’s essentially either a bandaid, or only feasible for some companies, and certainly not ideal.
A typical hybrid approach is where a company starts with a centralized team. In this case, the company hires data scientists into their center of excellence division. As mentioned earlier, it is easier to hire resources this way. After the team is formed, the company will spend years creating a strong structure around how AI projects will be executed at the company. Tools will be bought, steps will be defined, and most importantly, a strong communication line among data scientists will be established. As a result, the company will have built an internal system for data scientists and business users to communicate and carry out AI projects.
In other words, a company will try to solve the data science chasm problem by building a centralized team to be the driver of technology and innovation for the company. At the same time, using decentralized teams in the various departments within the company, in order to improve communication between each departmental team and the data scientists. In a way, the company will have created an internal platform for the embedded departmental AI teams to use and empowers them from a technical perspective.
The real solution
Whether you choose to go with a centralized or decentralized solution, the way to create processes and scientist-to-business communication is with a platform.
If you have a platform that automates the technical components of a data science project, which is where data scientists spend 99% of their time, suddenly the data scientists have time to have conversations within different teams. They can now better identify the core business problem and derive an approach that will fix the problem. Moreover, projects will be completed in hours to days instead of two or three months, letting a centralized department handle more volume.
In decentralized AI, the same platform allows more companies to adopt AI. Because all the technical elements are being automated by the platform, companies have a much easier time hiring a data scientist because the requirements for knowledge and experience are lower. Moreover, the best practices and processes can be achieved by sharing a platform throughout the company.
“No matter which method you choose, automation of the process will solve the problems one way or another,” Pedro says. “If you’re truly automating things in a way that’s useful, easy to use, and of high enough quality that a great data scientist would appreciate, you’re solving the problem.”
The additional benefit of having a platform that automates technical tasks is now anyone with access to data can now equip themselves with AI to find patterns and discover opportunities. Because the data science technical chasm is now handled by an AI Platform, like the Ople Platform, business analysts and subject matter experts who live very close to data are enabled to create business impact more than ever.
Request a demo today and see for yourself how easy and fast it is to find the business insight that you require.