At Subsurface LIVE 2023 Dremio Launches New Data Lakehouse Features and Sees Exponential Community Growth

Continued product adoption and new product features, along with rapid community growth, fuel Dremio’s momentum in 2023

Dremio, the easy and open data lakehouse, —at Subsurface LIVE 2023—announced the rollout of key new features. These features enable customers to more easily create their data lakehouses by performantly loading data into Apache Iceberg tables, query and federate across more data sources with Dremio Sonar, automatically format SQL queries in the Dremio SQL Runner, and securely connect Microsoft PowerBI using single sign-on (SSO).

“Every company we speak to is struggling to help their business move faster and self-serve while maintaining security and governance. Data meshes solve for these competing priorities, and we’ve been innovating to make it easier and easier for companies to create and operate data mesh architectures.”

Dremio also has added roll back and data optimization features for Apache Iceberg tables, making it even easier to manage data lakehouses using the open table format standard. Furthermore, customers can now use several new SQL functions for an even better SQL experience.

Read More: Rebuy Raises $17M Series A To Make Online Retail Personal

Dremio’s expanding functionality with Apache Iceberg now includes:

• Copying data into Apache Iceberg tables – Dremio’s new COPY INTO SQL command makes it even easier and faster to load data into Apache Iceberg tables, which are a foundational component of data lakehouses. With one command, customers can now copy data from CSV and JSON file formats stored in Amazon S3, Azure Data Lake Storage (ADLS), HDFS, and other supported data sources into Apache Iceberg tables using the columnar Parquet file format for performance. Dremio efficiently distributes the copy operation across the entire engine to load data more quickly.

• Optimizing Apache Iceberg tables – When using Dremio’s data manipulation (DML) commands to insert, update, and delete data from an Apache Iceberg table, additional files are created to represent these mutations to the table. Often, customers will have many small files as a result of these operations, which can impact read and write performance on that table and utilize excess storage. To improve the performance of Apache Iceberg tables, customers can now use the OPTIMIZE command in Dremio Sonar to consolidate these files into an optimal size. Customers running frequent DML operations can use OPTIMIZE at a regular interval to keep their Apache Iceberg tables efficient.

• Table roll back for Apache Iceberg – Customers can now restore their Apache Iceberg tables to a specific time or snapshot ID with Dremio’s new ROLLBACK command. This makes it easy to revert a table back to a previous state with a single command. When rolling back a table, Dremio will create a new Apache Iceberg snapshot from the prior state and use it as the new current table state.

Visit our documentation to learn more about building and operating your data lakehouse with COPY INTO, OPTIMIZE, and ROLLBACK commands for Apache Iceberg.

Dremio’s new functionality also includes new connectors for Microsoft PowerBI, Snowflake, and IBM Db2. Customers using Dremio and PowerBI can now use single sign-on (SSO) to access their Dremio Cloud and Dremio Software engines from PowerBI, simplifying access control and user management across their data architecture. The Snowflake and IBM Db2 connectors give customers the ability to quickly add Snowflake data warehouses and IBM Db2 databases as data sources for Dremio. This makes it easy to include data in these systems as part of the Dremio semantic layer, enabling customers to explore this data in their Dremio queries and views.

Read More: SalesTechStar Interview with Ben Calfee, VP Commercial Sales of Showpad

Additionally, customers can now add Dremio clusters as data sources, enabling query federation across these clusters. This feature set enables connectivity across Dremio environments, including hybrid environments where you have Dremio clusters running in a public cloud and on-premises. This makes it easy to extend Dremio semantic layers across clusters, giving analysts and BI engineers access to more datasets and insights. Along with popular connectors already operational like Tableau, ThoughtSpot, dbt, and Alteryx, these new connectors add more flexibility for customers.

“It’s been great to see the incredible growth of our product driving value for our customers,” said Tomer Shiran, co-founder and CPO of Dremio. “Every company we speak to is struggling to help their business move faster and self-serve while maintaining security and governance. Data meshes solve for these competing priorities, and we’ve been innovating to make it easier and easier for companies to create and operate data mesh architectures.”

Dremio’s growth and momentum continue to be recognized across the industry with key awards in 2022 and already in 2023. These include: CNBC’s 25 Top Startups for the Enterprise, Deloitte’s 2022 Technology Fast 500, CRN’s The 10 Hottest Big Data Tools Of 2022, DBTA’s Trend-Setting Products in Data and Information Management for 2023, and Tech Ascension’s 2022 Big Data Award Winners. In addition to these awards, Dremio has also received 11 badges from G2, including “Easiest to Use,” “High Performer,” and “Momentum Leader.”

“We are delighted to be recognized by trusted organizations like Deloitte and CNBC and are excited to see the growth of our community. I am proud of Team Dremio for their hard work last year and am looking forward to more growth in 2023 as we see a continued interest in open data lakehouse architectures,” said Read Maloney, CMO of Dremio.

Dremio’s continued community building has led to thousands of registrants at its flagship event, Subsurface LIVE, between its in-person events in San Francisco, New York and London, and its virtual event on day one. Subsurface LIVE is an event for Dremio customers, data industry leaders, and other industry experts.

For thirsty users, Dremio also launched its Data Hops user community events in Dallas, Santa Monica, Boston, Chicago, and London earlier this year. New cities are being added to the schedule throughout the year, enabling data professionals to network and learn more about self-service analytics in a comfortable, casual environment.

In addition to the event lineup for Dremio’s community in 2023, the community also has a thriving Slack channel with active users in the data and analytics space.