Suggestions
Liran Cohen
Founder and CEO, Lynk Semantic Layer
Liran Cohen is the Founder and CEO of Lynk.ai, a startup established in 2024 that focuses on leveraging artificial intelligence to transform data into actionable business insights. The company aims to provide data-driven teams with a comprehensive entity-based representation of their business, utilizing an AI-powered data modeling semantic engine known as Lynk Studio.1
In addition to his role at Lynk.ai, Liran Cohen has experience as the co-founder and CTO of Sosivio, a company specializing in cloud-native visibility through AI technology. His educational background includes studies at Bar-Ilan University.23
Highlights
#dataengineering #snowflake #cost
Snowflake is a great DWH platform, but as its not a cheap one. Here's one quick win you can apply in less than 5 minutes: Optimizing Warehouse Auto-Suspend Time in Snowflake
By default, Snowflake suspends a warehouse after 10 minutes of inactivity. This means you could be paying for an extra 10 minutes of runtime for each query, even when the warehouse isn’t actively being used. Fortunately, you can easily adjust these settings to minimize unnecessary credit consumption.
The Tradeoff:
Suspending a warehouse drops its cached data—the data Snowflake retrieves from storage to perform queries. However, this doesn't affect the query results cache, which remains available even if the warehouse is suspended.
When to Reduce Auto-Suspend Time:
• Infrequent Use: For warehouses that are not frequently running, such as those used for ad-hoc queries.
• Frequent Data Changes: When warehouse data cache changes often, for instance, due to running queries that require data from various sources.
Check the Impact:
You can monitor the effect of your changes via Snowflake’s UI:
Navigate to Admin > Cost Management > Consumption, and select the relevant warehouse to see credit usage over time.
How to Change Auto-Suspend Settings:
You can adjust the auto-suspend time either through a simple query or directly via Snowflake’s UI (see link in first comment)
Last week, I attended the fantastic #dataengineering meetup hosted by #dataqueens and #pagaya. The talks were impressing and very informative! Out of all the insights shared that evening, one thing stood out for me: Four out of three speakers had the term “Single Source of Truth” in the center of their presentations.
Why does this matter? Solving the single source of truth challenge will unlock the main challenges we face in data as also seen in dbt's 2024 state of Analytics Engineering report:
-> Trust: No more getting multiple different answers to the same question -> Velocity: with clarity, comes velocity - It should be clear how to consume and build business entities and metrics. There should be only one place where these are defined (for each business unit) -> Reduce Costs: By reducing spaghetti code and code duplications -> Enable AI: If we don’t understand what each term means and where to find what we need, how can AI? A well-documented, centralized, and governed single source of truth can unlock the potential of AI.
At Lynk.ai, we invest a lot in data modeling best practices to address those challenges - and build a data single source of truth for our customers in a quick and reliable way
