Report: 79% of ML teams say they ‘lack access’ to data needed to identify bias and ethics issues

A new report from Arize AI illustrates that the day-to-day reality for data scientists and machine learning (ML) engineers is painful and slow, speaking to a distinct need for better ways to collaborate and understand where and why problems are emerging. In all, 48.6% of data scientists say their jobs are more difficult after COVID-19. More than one in four (26.2%) data scientists and ML engineers admit that it takes them one week or more to detect and resolve an issue with ML models. 

One of the survey’s biggest findings documents a concerning chasm between business and technical teams. Eighty-seven percent of data scientists and ML engineers report that they encounter issues with business executives not being able to quantify the ROI of ML initiatives, at least sometimes. Additionally, more than half (52.3%) also report that business executives simply don’t understand machine learning. Likely contributing to this disconnect is the fact that “sharing data with others on the team” and “convincing stakeholders when a new model is better” remain issues at least sometimes for over 80% of ML practitioners. To solve these issues, ML practitioners would likely be well-served by initiatives to increase internal ML literacy and quantify AI ROI by tying ML model performance metrics to key business metrics. 

Another important finding speaks to broader conversations around AI ethics, fairness and equity. Seventy-nine percent of ML teams report that they “lack access to protected data needed to root out bias or ethics issues” at least some of the time, likely demonstrating the need for modernized data policies that grant AI practitioners access to protected data across the ML lifecycle. Despite this, over one in three (38.5%) ML engineers say negative headlines about AI inspire them “to take action to fix systemic bias.”

In November and December 2021, Arize AI surveyed data scientists, machine learning engineers, software engineers, technical executives and others by promoting a poll within several technical communities and journals. The goal: understand the challenges facing MLOps professionals, who are building systems that are relied on in nearly every industry today to increase profitability, productivity, and even save lives.

Read the full report by Arize AI. 

Originally appeared on: TheSpuzz