Root cause
ConstantEncodingDecoder.arrayFromScalar only special-cases the all-null dtype (DType.Null → NullArray). For a nullable primitive/bool dtype whose constant scalar value is null it falls through to constantPrimitive/decodeBool, which read the absent proto field as 0/false and produce an all-valid constant.
decodeBool line ~120: scalar.bool_value() != null && scalar.bool_value() → always false when field is absent
constantPrimitive: fills the output buffer with 0L bytes when the scalar number field is missing
Result: an all-null vortex.constant column exports 0.0/0/false instead of all-null — silent corruption.
Affected slugs (round 7 triage)
All confirmed against parquet oracle:
| Slug |
Affected columns |
| behavioral-risk-factor-surveillance-system |
TOLDCFS / HAVECFS / WORKCFS (all-null F64) |
| cardiovascular-diseases-risk-prediction-dataset |
11 all-null cols + COLGHOUS null constant chunk at row 393216 |
| diabetes-health-indicators-dataset |
PAINACT2 / QLMENTL2 / QLSTRES2 / QLHLTH2 (all-null F64) |
| fhv_tripdata_2025 |
SR_Flag (all-null I64) |
| green_tripdata_2025 |
ehail_fee (all-null F64) |
| heart-disease-health-indicators-dataset |
PAINACT2 / QLMENTL2 / QLSTRES2 / QLHLTH2 (all-null F64) |
| openpowerlifting |
Squat4Kg etc. (null constant in all-null chunks at rows 655360/1048576) |
| personal-key-indicators-of-heart-disease |
TOLDCFS / HAVECFS / WORKCFS (all-null F64) |
| stackoverflow-tags |
IsModeratorOnly / IsRequired (all-null bool) |
Fix
In arrayFromScalar, before dispatching on dtype, check whether the scalar value itself is null (all proto fields absent). If so, return a MaskedArray wrapping an all-zeros array with an all-invalid validity bitmap — or return a ConstantArray with a null constant marker.
Alternatively, treat it the same as DType.Null and return a NullArray of the right row count (simpler, but loses dtype information for schema-correct output).
Root cause
ConstantEncodingDecoder.arrayFromScalaronly special-cases the all-null dtype (DType.Null → NullArray). For a nullable primitive/bool dtype whose constant scalar value is null it falls through toconstantPrimitive/decodeBool, which read the absent proto field as0/falseand produce an all-valid constant.decodeBoolline ~120:scalar.bool_value() != null && scalar.bool_value()→ alwaysfalsewhen field is absentconstantPrimitive: fills the output buffer with0Lbytes when the scalar number field is missingResult: an all-null
vortex.constantcolumn exports0.0/0/falseinstead of all-null — silent corruption.Affected slugs (round 7 triage)
All confirmed against parquet oracle:
Fix
In
arrayFromScalar, before dispatching on dtype, check whether the scalar value itself is null (all proto fields absent). If so, return aMaskedArraywrapping an all-zeros array with an all-invalid validity bitmap — or return aConstantArraywith a null constant marker.Alternatively, treat it the same as
DType.Nulland return aNullArrayof the right row count (simpler, but loses dtype information for schema-correct output).