Skip to content

HUGEINT/UHUGEINT max-value overflows decimal128(38, 0) on RecordBatch return path #32

@rustyconover

Description

@rustyconover

Symptom

Round-tripping a HUGEINT or UHUGEINT column through the example.echo
table-in-out fixture (or any other path that re-serialises an Arrow batch
on the way back to DuckDB) raises:

vgi_rpc.utils.IPCError: IPC batch validation failed: In column 1: Invalid:
Decimal value 170141183460469231731687303715884105727 does not fit in
precision of decimal128(38, 0)
[worker: uv run --project vgi-fixture-worker]

Reproduction

Discovered while writing pushdown coverage in
vgi/test/sql/integration/filter_pushdown/hugeint.test.
The HUGEINT max literal 170141183460469231731687303715884105727 (2^127 − 1, 39 digits) overflows the decimal128(38, 0) PyArrow type used on the echo
return path. Same issue with UHUGEINT max
340282366920938463463374607431768211455 (2^128 − 1, 39 digits).

SELECT * FROM example.echo((SELECT * FROM (VALUES
  (1, 170141183460469231731687303715884105727::HUGEINT)
) v(n, h)));

Where to look

The conversion happens on the worker side when echo's pass-through builds
a Record Batch from the input. HUGEINT is registered to round-trip through
the DuckDB.hugeint extension type
(vgi/duckdb/src/common/arrow/arrow_type_extension.cpp:558)
backed by fixed_size_binary(16), so the storage does fit; the
decimal128(38, 0) only appears on echo's return-path serialisation. Look
at how vgi-rpc's IPC validator infers a target Arrow type for the columns
emitted by the worker — decimal128(38, 0) is one digit short of what
HUGEINT/UHUGEINT can hold.

Workaround in tests

The new hugeint.test exercises a 38-digit value
(99999999999999999999999999999999999999) instead of the type max. That
covers the kernel-pair fix without tripping the return-path bug.

Priority

Low — only affects the absolute boundary literals, very rare in real data.
But quietly silent-failing on max values is bad enough to deserve a fix or
a clear error.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions