The Fifth Elephant 2024 Annual Conference (12th &13th July)
Maximising the Potential of Data — Discussions around data science, machine learning & AI
Jul 2024
8 Mon
9 Tue
10 Wed
11 Thu
12 Fri
13 Sat 09:00 AM – 06:05 PM IST
14 Sun
Maximising the Potential of Data — Discussions around data science, machine learning & AI
Jul 2024
8 Mon
9 Tue
10 Wed
11 Thu
12 Fri
13 Sat 09:00 AM – 06:05 PM IST
14 Sun
Vinish Reddy
@vinish
Submitted Jun 3, 2024
Apache Hudi, Delta Lake, and Iceberg are leading open-source projects that offer decoupled storage with transactional and metadata layers, known as table formats in cloud storage. These formats store data in open columnar formats like Parquet and include metadata for schema, commit history, partitions, and column statistics. Selecting a table format can be challenging due to the unique features of each project. Enter XTable—an open-source project that ensures seamless interoperability between table formats. Instead of creating a new format, XTable provides abstractions for translating metadata, enabling data to be written in any format and converted for use by various compute engines. This session will showcase XTable’s solution to the challenges of format selection and interoperability in lakehouse workloads, including a live demonstration of XTable in action.
https://github.com/apache/incubator-xtable
Speaker: Vinish Reddy
https://www.linkedin.com/in/vinish-reddy-pannala-868702108/
Who is the audience for this talk?
Data Engineers building data lake/lakehouse in their organisations.
What is the problem you are trying to solve ?
Inter-operability across table formats, catalogs and query engines.
What is the scope of this talk i.e., what content will you cover in this talk?
We will start with intro about table formats and inter-operability, then deep dive into XTable. This will be followed by a live demo. In the end future road map for the OSS project will be discussed.
How will participants benefit from your talk?
Particaptns can learn invaluable insights in building data lakes and table formats, hoping that it also excites them to contribute to Apache OSS projects.
More info can be found here.
https://xtable.apache.org/
https://siliconangle.com/2023/11/15/onehouse-open-sources-onetable-data-tool-support-google-microsoft/
Jul 2024
8 Mon
9 Tue
10 Wed
11 Thu
12 Fri
13 Sat 09:00 AM – 06:05 PM IST
14 Sun
Hosted by
Supported by
Gold Sponsor
Sponsor
Community Partner
Beverage Partner
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}