Skip to main content

[@ServeTheHomeVideo] Datalakes and Lake Houses - Open Storage Summit 2025 Session 7

· 5 min read
[@ServeTheHomeVideo] Summarizer
Video Bot

@ServeTheHomeVideo - "Datalakes and Lake Houses - Open Storage Summit 2025 Session 7"

Link: https://youtu.be/Zm8mNLHFu4w

Short Summary​

Number One Action Item/Takeaway: Start with a modular, open-standards-based architecture for storage and compute to ensure flexibility and avoid vendor lock-in as AI technology rapidly evolves.

Executive Summary: Data lakes and lakehouses are critical for enterprise AI, requiring robust storage and data management solutions. Building on open standards with a modular approach to both compute and storage allows enterprises to adapt to the fast-paced evolution of AI while optimizing performance and cost. Don't forget the financial angle!

Key Quotes​

Okay, here are five direct quotes from the provided transcript that I found to be particularly insightful:

  1. Brena Buke: "First layer is object storage. This is the data lake part of your data lakehouse. It has to be performant. It has to be secure. If your queries are slow, if your data is insecure, your data lakehouse is not going to last very long."
  2. Brena Buke: "One of the really amazing things that we've seen as an industry trend in the last few years is object storage as primary storage for both LLMs and for analytics use cases."
  3. Simon Lightstone: "...customers who want to just use Postgress with their Minow storage using a data lakeink and using Postgress to query that rather than learning some sort of other technology And that's one thing we I want to talk about here."
  4. Shiva Guram Murthy: "...at the center of data is actually storage is the critical enabler of all the data that's out there..."
  5. Paul Mloud: "You know the main thing is to listen to the customer. You know that that uh is very important right because the customer will have some very spec specific requirements and they may have preferences."

Detailed Summary​

Here's a detailed summary of the YouTube video transcript, presented in bullet points, focusing on key topics, arguments, and information, while excluding sponsor announcements:

Key Topics:

  • Data Lakes and Lakehouses for Enterprise AI: The core focus is on how data lakes and lakehouses are becoming essential for enterprises leveraging AI.
  • Importance of Data Ingestion (ETL/ELT): Emphasizes that robust data ingestion and ETL processes are vital for feeding high-quality, relevant data into AI models.
  • Object Storage: Discusses the role of object storage (like MinIO) in providing scalability and cost-effectiveness for data lakes and lakehouses.
  • Open Table Formats (Apache Iceberg, Delta Lake, Apache Hudi): Highlights the significance of open table formats in adding database-like features to data lakes (ACID compliance, time travel, schema evolution).
  • Separation of Storage and Compute: Explains how data lakehouse architecture separates storage and compute, allowing users to choose the best compute resources for their workloads.
  • Hardware Considerations: AMD and Super Micro's role in enabling the ecosystem.

Arguments and Information:

  • Data Lakes and Lakehouses are Revolutionizing AI: They provide a unified platform for managing diverse data and supporting both analytics and AI workloads at scale.
  • Without Proper Data Ingestion, AI Fails: Ineffective ETL processes lead to AI initiatives that are misaligned with organizational goals or simply ineffective.
  • Object Storage as Primary Storage: Object storage is becoming the primary storage layer for both LLMs and analytics use cases due to its performance and scalability.
  • MinIO's Perspective:
    • MinIO views object storage as the first layer of a data lakehouse, emphasizing the need for performance, security, and scalability.
    • Advocates for choosing object storage without a POSIX interface or metadata database to minimize latency.
    • MinIO boasts wide adoption by major enterprises and a strong open-source community.
    • Minio operates between a data lake built on standard hardware and consumer applications such as visualistion or LLM model
  • EDB (EnterpriseDB) and Postgress:
    • EDB is a major contributor to PostgreSQL, focusing on open-source database solutions.
    • EDB provides a solution called the "sovereign data and AI factory" to simplify the use of Postgress in conjunction with storage and compute resources.
    • The platform offers a single interface for managing the entire Postgress estate, simplifying monitoring and scaling.
    • EDB provides turn-key simplicity in moving data from operation database to the data lake.
    • Partnerships with Super Micro to make it easy to bring Postgress into the data center, and a six times return on investment.
  • AMD's Role:
    • AMD sees storage as a critical enabler for AI.
    • AMD offers a complete portfolio of CPU, GPU, and networking products to support the AI ecosystem.
    • AMD CPUs provide leadership in per-core performance, memory channels, and memory capacity.
    • AMD DPU products offer high bandwidth (up to 400 Gbps) for data transfers between nodes.
    • AMD benchmarks show strong performance in AI workloads compared to competitors.
  • Super Micro's Role:
    • Super Micro provides data center building blocks, including liquid cooling solutions.
    • Offers a wide range of SKUs, from CPUs to GPUs.
    • Storage solutions offer high capacity, supporting both flash and 3.5-inch drives.
    • Helps customers scale their AI workloads to meet specific requirements.
    • Addresses the needs to migrate legacy data sources and redirect streams to an AI stack.
  • Panel discussion
    • Each panellist gives advice for companies that are looking to explore AI, what considerations they should take, the need for patience and working to an open standard, the importance of the finance angle with consideration for how AI implementations are modular by nature.