dataaaaa!

Semglot is an open-source semantic-layer transpiler that translates across dialects like dbt semantic models and Snowflake Cortex. By routing everything through one neutral intermediate representation, "adding a dialect adds every conversion into and out of it, not just one."

Snowflake has introduced several data engineering updates, including preview capabilities for DCM Projects. Notably, "VOLATILE scalar UDFs are supported with dynamic table incremental refresh" is now generally available, alongside general availability for cloning dynamic Iceberg tables.

dbt Core v1.12 introduces an opt-in Rust parser that is significantly faster than the Python parser, delivering 5-10x speedups on large projects. This release also adds native private package installation via SSH and supports JavaScript UDFs on Snowflake and BigQuery.

Google BigQuery integrates SQL and Python workflows using the %%bqsql cell magic in Jupyter notebooks. This allows data engineers to execute SQL queries directly within Python environments, seamlessly passing query results into pandas DataFrames for advanced data manipulation and analysis.

Omni launched Routines, allowing users to schedule recurring AI analyses via plain-text prompts. Instead of querying the warehouse directly, each run compiles the prompt through the semantic layer, inheriting modeled join paths and metric definitions to ensure safe, governed data outputs.

Current agentic databases are merely passive infrastructure for agents. An actually agentic database would act autonomously, modifying schemas or fixing race conditions. The database is the safest place to let an agent act because it has robust guardrails like transactions and constraints.

ingestr CDC introduces a CLI-based approach to Change Data Capture, supporting Microsoft SQL Server. It offers a batch mode that queries Log Sequence Numbers on a schedule, and a streaming mode using a --stream flag for continuous replication. The streaming mode is at-least-once.

As data artifact creation becomes faster and cheaper, the scarce resource is company-wide consensus. Data teams must build a singular reality to prevent globally inconsistent data. To scale self-serve safely, Ramp routes AI agents through an abstraction layer rather than the raw data lake.

To classify over a million columns during a Snowflake migration, a solo engineer built a three-tier engine. It uses cheap regex rules with data-type filters for the majority of columns, reserving native Snowflake classification and language models only for ambiguous cases.

Nao Labs released an open-source analytics agent with a harness UI designed for data professionals. While traditional agents focus on execution, Nao optimizes the context layer using static assets like schemas. If an agent is the "brain," the harness is everything that lets it actually operate.

Greybeam provides free, open-source Snowflake cost observability to track warehouse efficiency, AI spend, and storage down to the database. It secures data by running under a least-privilege, read-only role. Greysight only reads metadata by running SELECTs against Snowflake's usage views.

Cloudflare has acquired Arroyo to bring serverless SQL stream processing to its developer platform. Built in Rust, Arroyo provides stateful aggregations and joins. The engine will remain open-source and self-hostable.

To combat dashboard sprawl and costly, unused warehouse queries, data teams can implement a 90/90 rule. If an artifact has no views in 90 days, archive it; if silence persists for another 90 days, delete it. Roughly 9 in 10 data artifacts were dead.

DuckDB simplifies text analytics by enabling keyword, full-text, and semantic search directly in SQL. It queries Hugging Face datasets using the hf:// prefix and tokenizes text with regexpsplitto_table. DuckDB offers a high-performance SQL engine that simplifies and streamlines text analytics.

Production analytics agents succeed by focusing on context engineering rather than model upgrades. "Context engineering is mostly data engineering." For example, Anthropic reached 95% accuracy by teaching its agent procedural knowledge, while Vercel achieved 100% success by removing 80% of its

Modern data stacks are often fragmented, requiring complex integrations. A unified lakehouse simplifies this by providing a single foundation for warehouse-grade performance, storage economics, live federation, and agent-ready context.

Apitap is an open-source Rust-based transfer engine that moves 10 million rows from Postgres to ClickHouse in 9.9 seconds. By avoiding row decoding and using raw binary streams, it bypasses ORM overhead. The engine should be so cheap to run that the databases become your only cost.

Snowflake has announced that CREATE OR ALTER is generally available for 24 object types. This feature creates an object if it does not exist, or modifies an existing one to match the definition.

Snowflake has launched its Online Feature Store in public preview to serve the latest feature values with low-latency online retrieval backed by Snowflake Postgres. Designed for real-time ML applications like fraud detection, it supports batch, stream, and real-time feature views.

The duckdb-paimon extension allows DuckDB to query Apache Paimon format data directly without ETL pipelines or Spark clusters. Built on paimon-cpp, it features zero JVM dependency and uses Apache Arrow for zero-copy data exchange. It supports projection and predicate pushdown.

PostHog re-architected its data warehouse from ClickHouse to single-tenant DuckDB instances to improve data modeling flexibility. The new setup uses a custom extension for local compute and DuckLake for storage-compute separation.

Decathlon Chief Data and AI Officer Didier Mamma structures the company's global strategy around key data engineering initiatives. Their major technical projects focus on building a robust lakehouse architecture, improving data quality and governance, and deploying agentic workflows.

dlt is an open-source Python library designed to build, deploy, monitor, and scale data pipelines. It serves as an alternative for migrating away from proprietary tools like Fivetran by allowing engineers to load data from any REST API. dlt is the open-source Python library for data pipelines.

To document a complex Google Tag Manager estate, the author built a dependency graph of tags, triggers, and variables from a JSON workspace export. An AI agent used this graph to write concise documentation notes. The interesting context lives in the edges.

Self-hosting DuckDB involves a graduation path based on concurrent writers and readers. While local NVMe storage is highly optimized, moving data to cloud object storage like S3 introduces authorization overhead and network bandwidth bottlenecks. "Maintaining is, if anything, harder than before."