DuckDB OpenTelemetry Extension
The DuckDB OpenTelemetry Extension lets you ingest OpenTelemetry data from storage or multiple OpenTelemetry protocol standards (OTAP, OTLP) over gRPC/HTTP into DuckDB.
Do analysis on metrics, logs, or traces or export telemetry to anything DuckDB supports using a few lines of SQL statements: data lakes, databases, parquet, vortex, or object storage systems.
Start using duckdb-otlp
Section titled “Start using duckdb-otlp”- Get Started - install the extension, query bundled OTLP files, and POST one log over HTTP.
- WASM Demo - query OTLP JSON, JSONL, and protobuf samples in your browser.
- Live Ingest Quickstart - POST one OTLP/HTTP log record and query it from DuckDB.
Query and export telemetry
Section titled “Query and export telemetry”- How to analyze telemetry - query traces, logs, and metrics with a few practical starting points.
- How to stream to local DuckLake - land OTLP/HTTP rows in local Parquet files tracked by DuckLake.
- How to stream to remote DuckLake - land OTLP/HTTP rows in DuckLake with Neon Postgres metadata and R2 data files.
- How to stream to Parquet - land OTLP/HTTP rows as partitioned Parquet files on disk or S3.
- How to stream to Amazon S3 Tables - land OTLP/HTTP rows in Amazon S3 Tables as an Iceberg catalog.
- How to stream to Cloudflare R2 Data Catalog - land OTLP/HTTP rows in Cloudflare R2 Data Catalog as an Iceberg catalog.
- How to export telemetry to Parquet - write traces, logs, and metrics to reusable Parquet datasets.
- How to stream to Vortex - convert OTLP files to Vortex columnar files locally with
COPY ... (FORMAT vortex).
Set up data sources
Section titled “Set up data sources”- How to install the extension - install from community extensions or build locally.
- How to configure the OpenTelemetry Collector - write OTLP file exports for DuckDB analysis.
- How to point the OpenTelemetry Demo at local DuckLake - stream demo telemetry into a local DuckLake container.
Reference
Section titled “Reference”- API Reference - file readers and live ingest functions.
- Schema Reference - columns for traces, logs, gauges, sums, histograms, and exponential histograms.
- Live Ingest Reference -
otlp_serve,otlp_flush,otlp_stop,otlp_server_list, HTTP endpoints, auth, buffering, and durability. - Error Reference - file reader failures and live-ingest error responses.
- Performance Reference - file formats, projection, file sizes, materialization, and live ingest notes.
Project internals
Section titled “Project internals”- Architecture - how the file readers, Arrow bridge, schema layer, and HTTP ingest server work.
- Contributing - build, test, and contribute to the project.