[core] Introduce id-based writtenFieldIds on DataFileMeta

[zhuxiangyi]

Purpose

This is the first refactoring PR toward moving the data-evolution read/write path from column names to field ids, as discussed in #8334 (so that top-level fields and nested fields are treated the same, per @JingsongLi's suggestion).

It records the columns written in a data file by field id (writtenFieldIds) in addition to the existing name-based writeCols. Field ids are stable across column renames and can address nested fields uniformly (every nested field has a globally unique id), which later lets the read/write/commit/compact path drop name-based resolution. This PR is groundwork only — behavior is unchanged.

Changes

• DataFileMeta: append a nullable _WRITTEN_FIELD_IDS ARRAY<INT> to SCHEMA; add writtenFieldIds() (default null); thread it through PojoDataFileMeta and the forAppend/create factories.
• DataFileMetaSerializer: serialize/deserialize the new field, isNullAt-guarded so old manifests read it as null.
• Version dispatch: add DataFileMetaWriteColsLegacySerializer freezing the previous 20-field layout, and bump DataSplit (8→9) and CommitMessageSerializer (11→12) to dispatch old binary streams to it.
• Writers: dual-write writtenFieldIds (derived from the writeCols field ids) next to writeCols, so old readers keep working via writeCols.
• DataEvolutionUtils.writtenFieldIds(file, schemaFetcher): resolve a file's written columns to ids (writtenFieldIds if present, else writeCols names → ids). Consumers switch to this in the follow-up PR.

Compatibility

• Backward (new reader, old data): the new field is a nullable appended field, so old manifests read it as null; old binary DataSplit/CommitMessage streams are dispatched to the frozen legacy serializer.
• Forward (old reader, new data): manifests are read by name (Avro), so an old reader drops the unknown field and keeps working via the dual-written writeCols. writeCols will only be dropped later, once all supported versions understand writtenFieldIds.

Tests

DataFileMetaWrittenFieldIdsCompatibilityTest: round-trip with/without ids, the frozen legacy layout still deserializes, new-stream/old-serializer forward read, and name→id resolution (incl. row-tracking system fields). Existing DataSplitCompatibleTest (v1–v8 fixtures), DataEvolutionTableTest and serializer tests remain green.

Part of the plan in #8334.

[JingsongLi]

Modify data file meta is too complex... Let's go back to your original plan, using field.field.

[zhuxiangyi]

Thanks @JingsongLi. Just to confirm the direction so I don't go back and forth:

• Close this PR ([core] Introduce id-based writtenFieldIds on DataFileMeta #8531) and drop the field-id / DataFileMeta changes;
• Go back to the original approach in [core][spark][flink] Support sub-field-level data evolution for nested columns #8334 — encode nested sub-fields as dotted paths (nest.a) inside the existing writeCols, with no change to DataFileMeta / manifest format;
• Continue the feature on that PR.

If that's right, I'll close #8531, reopen #8334 from draft and rebase it onto the latest master. Sound good?

[JingsongLi]

@zhuxiangyi Sounds good. The changes to the original PR are gradually being made and broken down into multiple small PRs for rapid progress.