Data sourced from Dakota Marketplace, the global LP and GP intelligence platform trusted by investment professionals. Learn More | Book a Demo
Watch The Replay | Post-Call Notes
Executive Summary
Dakota hosted a product webinar breaking down Databricks and Snowflake, the two data platforms that investment firms most often weigh when building the data foundation behind their AI efforts.
Head of Product Pat Tighe and President Rob Robertson were joined by CTO Priya Stanley, who explained the difference between a data warehouse, a data lake, and a data lakehouse, and where each platform started. The group covered which types of firms and teams fit each platform, why some of the largest firms run both, how Dakota delivers data through each, and how costs compare.
The takeaway: Snowflake is a data warehouse evolving toward AI, and Databricks is a data lake that built the lakehouse and is pushing further into AI. Same destination, different starting points. Dakota's data is already available directly through both.
In this article, we'll recap the platform fundamentals, how to choose between them, how Dakota shares data on each, the cost discussion, the FAQs covered, and the resources mentioned during the session.
Platforms and Topics Discussed in This Webinar
Warehouse, Lake, and Lakehouse: The Basics
Priya started with three definitions. A data warehouse stores structured data in rows and columns, runs fast and reliable SQL queries, and cleans data before it goes in (schema-on-write). Snowflake is the cleanest example, alongside Amazon Redshift and Google BigQuery. A data lake stores structured, semi-structured, and unstructured data such as text, images, and video at low cost and massive scale, and applies structure only when someone queries it (schema-on-read). Amazon S3 and Azure Data Lake Storage are basic examples, though a lake without discipline can turn into a data swamp.
The lakehouse combines both ideas, pairing the flexibility of a lake with the reliability and query speed of a warehouse on the same raw storage. Databricks is the cleanest example, and Microsoft Fabric spans both categories. In Pat's framing, structured data that is ready to use can go straight to a warehouse, while data that needs more cleaning and structuring goes to a lake first.
Book a demo to see how Dakota data fits into your data stack.
Snowflake for Analytics Teams, Databricks for Data Science and Engineering
Snowflake started as a warehouse built for structured data, fast SQL, and BI dashboards. Databricks started as a lakehouse built on Apache Spark for large, messy, unstructured data that can run machine learning directly. Priya's shorthand: Snowflake is for the analytics team, and Databricks is for the data science and engineering team.
Both have spent the last two years addressing the main criticism of their products, with a focus on AI. Snowflake answered the lock-in criticism, the concern that data is hard to pull out and use with other tools, by adding Apache Iceberg, an open format. Databricks answered the complexity criticism, that it required Spark and Python skills, by adding Databricks SQL and better BI tooling so regular analysts can use it. As a result, the two now compete for the same budget.
Book a demo to talk through which setup fits your team.
Why Some Firms Use Both
Priya walked through where each platform tends to win, based on use cases the session cited. Databricks tends to win on risk and model governance, with Morgan Stanley's regulatory risk calculations as one example, and on real-time work such as fraud detection, which Capital One was cited for. Snowflake tends to win on regulatory reporting and compliance, and BlackRock's Aladdin platform was the example, where one governed data set serves portfolio managers, risk, and compliance teams. Quant research is a toss-up: simpler SQL-based teams lean Snowflake, and heavier ML teams lean Databricks.
Governance also means different things on each. Databricks' Unity Catalog traces what data fed an AI model and what came out, which suits model risk and auditing. Snowflake focuses on what each user has access to. Pat noted that many of the largest firms use both, with the technology team in Databricks and the analytics and business teams in Snowflake. Rob added that most of what Dakota hears from its customers is Snowflake, though some use Databricks.
Book a demo to hear how other firms are using Dakota data in their platforms.
Private Sharing, Marketplaces, and the API
Dakota meets customers where they already work. Both platforms offer marketplace listings, but Dakota can also do a private share tailored to a firm. A standard marketplace listing has a predefined set of accounts and contacts, so a firm that only cares about RIAs can get just the RIA accounts and contacts. This works the same way on Snowflake and Databricks.
Compared with the raw API, a private share is far lighter. The API requires developers, custom coding, and every scenario to be identified and handled up front. If a firm's workflows already run on Snowflake or Databricks, Dakota's data simply shows up there. Pat compared the Snowflake integration to downloading an app onto your phone.
Book a demo to see Dakota data in Snowflake or Databricks.
Cost: It Depends on the Workload
Priya urged caution about quoting either vendor's numbers. Snowflake claims 50% to 70% savings versus Databricks, but she called that a marketing claim and not a neutral benchmark. Snowflake tends to look simpler and cheaper for straightforward BI at small scale, though it can carry a simplicity premium. Databricks can be more cost efficient at large scale for large, complex workloads. In both cases, an unoptimized deployment gets expensive, and how much you use the platform drives the bill.
Frequently Asked Questions
Which platform should we start with?
It depends on who will use it. Databricks suits firms with a technically advanced machine learning background that are ready to handle unstructured data systematically. Snowflake suits established firms that mainly want clean, accessible data for analytics. The largest firms often use both, with engineering teams in Databricks and analytics teams in Snowflake.
Do we need to choose just one?
No. Many firms run both, and Dakota's data is available directly through each. Dakota can also extend delivery to whichever platform a firm runs.
How is a private share different from a marketplace listing?
A standard marketplace listing comes with a set of predefined accounts, contacts, and records. A private share lets Dakota tailor the data set to a firm, such as only RIA accounts and contacts, on either Snowflake or Databricks.
Is a data share easier than using the Dakota API?
Yes, for firms already building on Snowflake or Databricks. The API needs developers and custom code built end to end, while a share places Dakota's data inside the platform your team already uses.
Want to see Dakota Marketplace data inside your own Snowflake or Databricks environment? Our team will walk you through it. Talk to an Expert →
