duck_net
Experimental DuckDB community extension for HTTP/HTTPS client access
Highlights
- First-class HTTP/HTTPS client functions inside DuckDB SQL
- Built on quack-rs, so the FFI surface is zero-unsafe Rust
- Explores the "SQL as a data pipeline" pattern without external glue
The premise
A sister project to quack-rs and
duckdb-behavioral, duck_net
asks a single question: what does a data pipeline look like if
HTTP fetch is a SQL function rather than an external
orchestration step?
SELECT http_get('https://example.com/api/users').body;Instead of wiring up a Python script that fetches, parses, and loads data into DuckDB, the fetch becomes a table-valued function and the rest of the transformation is a normal SQL query. A lot of real-world ETL jobs collapse into a single statement once that primitive exists.
What it’s really testing
The project is explicitly experimental. HTTP inside an
analytical database is a loaded design decision, with serious
questions about caching, determinism, retries, rate limiting,
and observability that a production version would need to
answer. duck_net isn’t trying to resolve those yet.
What it is doing is stress-testing quack-rs. Building a non-trivial extension that touches async runtimes, TLS, HTTP clients, and DuckDB’s replacement-scan API is the fastest way to find the places where the SDK’s abstractions leak. If the SDK makes this kind of extension approachable, the abstractions are earning their keep. If it doesn’t, those are the first places to iterate on.
In that sense duck_net is dogfooding rather than shipping,
and that’s the point.