New Survey Reveals Poor Visibility and Lack of Prescriptive Insights as Top Challenges for DataOps in 2022
Benchmarked Survey from 2022 DataOps Unleashed Event Highlights the Key Priorities and Pressing Challenges Facing the Burgeoning DataOps Market
Unravel Data, the only data operations platform providing full-stack visibility and AI-powered recommendations to drive more reliable performance in modern data applications, today released key findings from a survey administered to several hundred attendees at the most recent DataOps Unleashed event in order to gauge the priorities, challenges, and progress of leading data teams as they seek to modernize their big data management and analytics capabilities so they can fully realize their real-time data analytics ambitions.
Read More: SalesTechStar Interview with Chris Weber, COO and President at SaaSOptics
“Data is the lifeblood of the modern enterprise and those organizations who have dedicated the resources and budget to modernizing their data stacks are the ones who will be best positioned to drive innovations in the coming decade,” said Kunal Agarwal, co-founder and CEO of Unravel Data. “The results of this latest survey show just how complex the modern data stack has become and illustrates the many unanticipated challenges that come with efficiently managing and optimizing data pipelines across multiple public cloud providers and platforms. It also serves to validate that there is an obvious demand for a purpose-built solution that can help these teams gain the critical visibility they need to drive the most value from their data operations.”
Read More: SalesTechStar Interview with Michael Ramsey, VP of Customer Workflow Products at ServiceNow
Some of the key findings collected from the most recent survey, which benchmarks responses from the year prior, include:
- DataOps as a practice is hitting an inflection point: There was an almost 80% increase from the year prior of respondents who said they are in the active stage of adopting a formal DataOps approach to manage and optimize their data pipelines. This year, more than 41% of attendees reported they are actively employing DataOps methodologies, compared to just less than a quarter (24%) in 2021.
- Visibility into data pipelines remains the top challenge: For the second year in a row, when participants were asked what they viewed as the top challenge with operating their data stack, respondents cited the lack of visibility across their environment as their most significant obstacle. Whereas in the previous year respondents reported that “controlling runaway costs” was the second biggest, this year the “lack of proactive alerts” was noted as the second most challenging aspect.
- Complexity of cloud migrations is more time consuming than previously thought: Sixty percent of respondents from this year’s event estimated that their cloud migration project would take between 12-24 months, representing a 150% increase over the prior year’s projection. The challenge of forecasting the duration of these cloud migration initiatives reveals the vast amount of complexity and uncertainty that data teams face when attempting to map out these critical projects.
- Automation continues to be a key driver: When asked about the role of automation in managing their DataOps pipelines, three in four DataOps professionals in both years reported that the ability to “automatically test and verify before moving jobs/pipelines to production” was the most important automation priority when compared to other aspects such as automating troubleshooting of platform or pipeline issues.
- Data teams spend more time building than deploying/managing pipelines: For both years, data professionals reported that they spent the majority of their day building their data pipelines (39% in 2021 and 43% in 2022). In 2022, respondents reported spending slightly less time maintaining or troubleshooting their pipelines (30%) than the year prior (34%) while the time spent deploying data pipelines remained the same at 27% for both years.
Now in its second year, DataOps Unleashed is a one-day virtual conference underwritten by Unravel Data, that seeks to build a vibrant peer-based community for DataOps professionals to share best practices. Data professionals can productively collaborate with one another and deliver on the promise of what it means to be a data-led company. This past year’s event attracted more than 2,300 data professionals and featured presentations from data leaders at some of the most data-forward enterprise organizations in the world, including AWS, DBS, Google, Johnson & Johnson, and many others.
