S3 resources

An S3 resource exposes object operations within a bucket and optional prefix:

resources:
  artifacts:
    provider: s3
    bucket: example-artifacts
    prefix: ridge/runs
    region: us-west-2

Credentials and optional endpoint configuration come from Boto3's ambient environment and configuration. Region is resource-scoped, so one inventory can contain buckets in different regions.

Keys are relative to the configured prefix but are not normalized as POSIX paths. Listing is flat and paginated with an opaque continuation cursor. Writes replace complete objects. Large streamed uploads use bounded sequential multipart parts and attempt to abort unfinished uploads after ordinary failure or cancellation.

A delegated data_root appends a relative prefix to its parent's prefix with a / separator. It does not normalize interior key text or probe for a directory. Listing and metadata remain relative to the final view; backend-specific key restrictions still apply.

Exact read/stat/write/delete and copy endpoints use full-key lock footprints, including delegated prefixes. Different objects can overlap; aliases of the same object conflict on writes. Listing remains whole-domain; explicit sessions can reserve exact keys or omit paths for whole-domain protection. Incompatible resources sharing a lock key disable narrowing for that domain. See action-defined footprints.

When region is omitted, Boto3's ambient region selection applies. Reads and writes preserve bytes; direct reads/writes are buffered. Use copy for streamed file/object transfers, including S3-to-S3 through the Ridge host. Abrupt process termination or an abort failure can leave multipart uploads behind; inspect them with your storage tooling and manage their lifecycle outside Ridge.

Ridge does not create buckets, manage lifecycle rules, perform partial updates, or map directories implicitly to key prefixes.

Deletion

data.delete issues one native DeleteObject for the exact resource-relative key, without a version ID, HEAD preflight, or automatic SDK retry. Ambient AWS authority must permit s3:DeleteObject. Ridge returns acknowledged even when the key was missing; it does not claim prior existence or physical erasure. With bucket versioning enabled, deletion creates a delete marker and retains older versions; suspended versioning follows native null-version semantics. Repeated requests can create additional markers. Ridge never purges versions, expands prefixes, or deletes buckets, and rejects recursive=true. Use a background idempotency key to deduplicate submission; inspect uncertain outcomes before trying a new attempt. See shared deletion.

Streamed upload size

Ridge's S3 destination uses 8 MiB parts and permits at most 10,000 parts, limiting each streamed upload to 78.125 GiB (83,886,080,000 bytes). This is a Ridge implementation limit, not a statement of S3 service capacity. Larger streams fail when they need another part; size is not rejected in advance. Check artifact size before starting a long copy to S3. This ceiling does not limit S3 sources copied to filesystem destinations.