Query with Quack

Run the duckdb-otlp Docker image in local-ducklake mode, enable the image’s Quack server, and connect to it from a DuckDB process outside Docker.

Use this setup when you want the container to handle ingestion and DuckLake writes while a local DuckDB shell, notebook, or script queries telemetry through Quack.

Quack is experimental in DuckDB v1.5.4. Run it on trusted local networks. For remote access, put it behind a TLS-terminating reverse proxy.

Configure

Create .env:

DUCKDB_MODE=local-ducklake
DUCKDB_OTLP_TOKEN=dev-token-123456

DUCKLAKE_NAME=lake

DUCKDB_QUACK_ENABLED=1
DUCKDB_QUACK_ADDR=0.0.0.0:9494
DUCKDB_QUACK_TOKEN=dev-quack-token-123456

DUCKDB_OTLP_TOKEN protects OTLP/HTTP ingest on port 4318. DUCKDB_QUACK_TOKEN protects DuckDB query access on port 9494.

Local DuckLake mode uses DUCKLAKE_CATALOG_PATH=/data/ducklake/catalog.duckdb and DUCKLAKE_DATA_PATH=/data/ducklake/storage by default. Set those variables when you want DuckLake metadata or Parquet files somewhere else inside the mounted /data volume.

Start the server

Prefer to run the same services manually in the DuckDB shell? See Run manually.

mkdir -p data

docker run --rm --name duckdb-otlp \
  --env-file .env \
  -p 4318:4318 \
  -p 9494:9494 \
  -v "$(pwd)/data:/data" \
  ghcr.io/smithclay/duckdb-otlp:latest

The container initializes DuckLake, starts the OTLP/HTTP ingest server, and starts Quack in the same DuckDB process. The server names the catalog lake and makes it the default catalog Quack exposes, so attached clients reach the signal tables (otlp_logs, otlp_traces, …) directly instead of through a fully qualified lake.main.* name.

Leave the container running while clients send OTLP/HTTP requests and local DuckDB clients query through Quack.

POST a log record

In another terminal:

curl -sS http://localhost:4318/v1/logs \
  -H 'Authorization: Bearer dev-token-123456' \
  -H 'Content-Type: application/json' \
  -d '{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"quack-local-ducklake-demo"}},{"key":"deployment.environment","value":{"stringValue":"docs"}}]},"scopeLogs":[{"scope":{"name":"duckdb-otlp-guide"},"logRecords":[{"timeUnixNano":"1704067200000000000","observedTimeUnixNano":"1704067200123456789","severityNumber":9,"severityText":"INFO","body":{"stringValue":"hello from local DuckDB over Quack"},"attributes":[{"key":"guide","value":{"stringValue":"query-with-quack"}}]}]}]}]}'

Response:

{"status":"buffered","rows":1,"batches":1}

Rows are accepted before they are durable. They commit automatically in the background, on graceful shutdown, or immediately after an explicit flush.

Connect from local DuckDB

In another terminal, start the DuckDB CLI on your host machine:

duckdb

Install and load Quack in the local DuckDB process:

INSTALL quack;
LOAD quack;

Create a Quack secret scoped to the container endpoint, then attach the remote DuckDB process:

CREATE SECRET duckdb_otlp_quack (
  TYPE quack,
  SCOPE 'quack:localhost:9494',
  TOKEN 'dev-quack-token-123456'
);

ATTACH 'quack:localhost:9494' AS otel (TYPE quack);

For localhost, Quack uses plain HTTP by default. The Docker port mapping forwards local port 9494 to quack:0.0.0.0:9494 inside the container.

If you connect to Quack on a remote host, protect the transport with TLS termination. A tunneling extension such as Query-farm/quackscale keeps the Quack listener off the public network.

Query telemetry like local tables

Point the whole session at the attached server, then query the telemetry tables directly — no SQL-in-a-string wrapping:

USE otel;

The daemon makes its telemetry catalog (named lake here) the default catalog that Quack exposes, so the signal tables — otlp_logs, otlp_traces, otlp_metrics_gauge, otlp_metrics_sum, otlp_metrics_histogram, otlp_metrics_exp_histogram — are reachable by their plain names. Quack runs each scan inside the container’s DuckDB process and streams the result back, pushing the selected columns down to the server.

The server accepts rows before it makes them durable. For a short local test, flush the ingest buffer first. otlp_flush is a server-side function rather than a table, so run it through the attached catalog’s query macro:

FROM query('SELECT * FROM otlp_flush(''otlp:0.0.0.0:4318'')');

Then query the signal tables directly:

SELECT time_unix_nano, service_name, severity_text, body
FROM otlp_logs
WHERE service_name = 'quack-local-ducklake-demo'
ORDER BY time_unix_nano DESC
LIMIT 5;

The same works for traces:

SELECT trace_id, name, service_name, duration_time_unix_nano
FROM otlp_traces
ORDER BY start_time_unix_nano DESC
LIMIT 20;

Run server-side SQL

Transparent scans stream rows to the client and project columns server-side, but they do not push down filters or aggregations. For heavy aggregations — or anything that is not a plain table scan, such as otlp_flush or DDL — run the SQL inside the server session so only the result crosses the wire. Use the attached catalog’s query macro:

FROM otel.query(
  $$
  SELECT service_name, count(*) AS rows
  FROM otlp_logs
  GROUP BY ALL
  ORDER BY rows DESC
  $$
);

Or quack_query for a one-shot call without attaching:

FROM quack_query(
  'quack:localhost:9494',
  'SELECT service_name, count(*) AS rows FROM otlp_logs GROUP BY ALL ORDER BY rows DESC',
  token = 'dev-quack-token-123456'
);

Use USE otel plus plain table names for interactive exploration; push aggregations into query / quack_query when you do not want to stream every row to the client.

Stop cleanly

docker stop duckdb-otlp

The image sends otlp_stop('otlp:0.0.0.0:4318') during shutdown, so remaining buffered rows are committed before the process exits.

Run manually

To run this configuration in a DuckDB 1.5.4+ shell instead of the daemon, execute the SQL below. Replace bracketed values with the corresponding values from this guide. Keep the shell open while clients send telemetry.

mkdir -p data/ducklake/storage
duckdb data/duckdb-otlp-control.duckdb

-- The daemon embeds otlp statically; the shell loads the extension explicitly.
INSTALL otlp FROM community;
LOAD otlp;

-- Attach the local DuckLake catalog used by the Docker guide.
INSTALL ducklake;
LOAD ducklake;
ATTACH 'ducklake:data/ducklake/catalog.duckdb' AS lake (
  DATA_PATH 'data/ducklake/storage'
);

-- Create the target schema before starting ingestion.
CREATE SCHEMA IF NOT EXISTS lake.main;

-- Match the daemon's default-catalog setting for unqualified queries.
USE lake;

-- Start OTLP/HTTP. Seal cadence, file sizes, and buffer limits use defaults;
-- override only as needed — see the Live Ingest Reference:
-- https://smithclay.github.io/duckdb-otlp/reference/serve/
SELECT listen_url, catalog_name, schema_name
FROM otlp_serve(
  'otlp:0.0.0.0:4318',
  catalog := 'lake',
  schema := 'main',
  token := 'dev-token-123456',
  allow_other_hostname := true
);

-- The guide's daemon configuration also enables Quack.
INSTALL quack;
LOAD quack;
SELECT listen_uri
FROM quack_serve(
  'quack:0.0.0.0:9494',
  token := 'dev-quack-token-123456',
  allow_other_hostname := true
);

Before closing DuckDB, stop both listeners cleanly:

-- Stop Quack, then commit buffered telemetry and stop OTLP.
CALL quack_stop('quack:0.0.0.0:9494');
SELECT status, dropped_rows
FROM otlp_stop('otlp:0.0.0.0:4318');

Clean up

After you stop the container, remove the local DuckLake files if you no longer need them:

rm -rf data