Skip to content

【WIP】[common][filesystem] Add FileIO contract suite - #9161

Open
sundapeng wants to merge 10 commits into
apache:masterfrom
sundapeng:fileio-contract-suite
Open

【WIP】[common][filesystem] Add FileIO contract suite#9161
sundapeng wants to merge 10 commits into
apache:masterfrom
sundapeng:fileio-contract-suite

Conversation

@sundapeng

@sundapeng sundapeng commented Aug 11, 2026

Copy link
Copy Markdown
Member

What changed

This PR defines the FileIO behavior used by Paimon and runs the same contract tests against Local, Hadoop-local, HDFS, and S3.

  • Add a FileIO API page covering all 37 public methods and Paimon's current call patterns.
  • Test shared behavior, default helpers, and caller preconditions.
  • Correct the mkdirs Javadoc: an existing directory returns true.
  • Fix mismatches found in no-overwrite copy, atomic creation, two-phase cleanup and rollback, and S3 multipart initialization.

Scope

There are no public API signature changes, implementation flags, or Hadoop/POSIX compatibility claim. The existing FileIOBehaviorTestBase is unchanged. Ambiguous rename cases, listing order, pre-close visibility, exact timestamp precision, and backend request counts remain outside the shared contract.

Testing

  • LocalFileIOBehaviorTest and HadoopLocalFileIOBehaviorTest: 127 tests.
  • S3MultiPartUploadTest and S3AtomicWriteTest: 5 tests.
  • Spotless checks and the Docusaurus production build passed locally.

@sundapeng sundapeng changed the title [common] Define provider-neutral FileIO contract [common][core][s3] Add provider-neutral FileIO contract suite Aug 11, 2026
@sundapeng sundapeng changed the title [common][core][s3] Add provider-neutral FileIO contract suite 【wip】[common][core][s3] Add provider-neutral FileIO contract suite Aug 11, 2026
@sundapeng sundapeng changed the title 【wip】[common][core][s3] Add provider-neutral FileIO contract suite [common][filesystem] Add provider-neutral FileIO contract suite Aug 11, 2026
@sundapeng sundapeng changed the title [common][filesystem] Add provider-neutral FileIO contract suite 【WIP】[common][filesystem] Add provider-neutral FileIO contract suite Aug 11, 2026
@sundapeng sundapeng changed the title 【WIP】[common][filesystem] Add provider-neutral FileIO contract suite 【WIP】[common][filesystem] Define FileIO behavior Aug 11, 2026
@sundapeng sundapeng changed the title 【WIP】[common][filesystem] Define FileIO behavior 【WIP】[common][filesystem] Add FileIO contract suite Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant