Oxygen Strata: keeping a repository on your own storage

A repository can be pushed to, cloned from and pulled from a bucket in your own cloud account, or a directory on a shared drive, with no server in between. The bucket is just another remote:

oxy remote add cloud s3://my-bucket/repos/project --profile work
oxy remote check cloud
oxy push cloud
oxy clone s3://my-bucket/repos/project?profile=work
oxy pull

It works because Oxygen's objects never change and are named by their hash, so any store that can keep write-once objects and update one small record by compare-and-swap can hold a complete repository. Nothing in the store is trusted: every object is re-hashed as it is read, changes are signed, and a bookmark's attestation is verified under your policy, as for any remote. The data stays in your account, in your region, under your keys; an operator of a hub need never see it.

Supported stores:

A store holds one repository per prefix. The first push binds the prefix to the repository; a push of another repository to the same prefix is refused. Use a prefix per repository, as repos/project above.

Setting up a bucket

Amazon S3

1. Create a bucket in the region you want the data kept in (for example eu-west-2, London), with public access blocked.

2. Give the identity oxy will use these permissions, and no more:

   {
     "Version": "2012-10-17",
     "Statement": [
       { "Effect": "Allow", "Action": "s3:ListBucket",
         "Resource": "arn:aws:s3:::my-bucket" },
       { "Effect": "Allow",
         "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
         "Resource": "arn:aws:s3:::my-bucket/repos/*" }
     ]
   }
   

   A read-only identity (s3:ListBucket and s3:GetObject) gives a working clone and pull.

3. Credentials come from the places the AWS tools use, in this order: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the environment; the profile named by --profile (or AWS_PROFILE, else default) in ~/.aws/credentials; what the AWS CLI can produce for that profile (aws sso login, aws login, assumed roles, credential processes), so anything that works for aws works for oxy; and the instance's or container's role.

4. The region is taken from --region, AWS_REGION, the profile, or asked of S3. --endpoint http://… points at an S3-compatible service.

Google Cloud Storage

1. Create a bucket in the region you want (for example europe-west2, London), with uniform access and public access prevention.
2. Create a service account and grant it Storage Object Admin on that bucket only (Storage Object Viewer for a read-only identity), or use your own account.
3. Credentials: a service account key file named by --credentials or GOOGLE_APPLICATION_CREDENTIALS; else the application default credentials that gcloud auth application-default login writes; else the instance's service account.

Azure Blob Storage

1. Create a storage account in the region you want (for example UK South), with public blob access off, and a container in it.
2. Create a service principal (az ad sp create-for-rbac) with Storage Blob Data Contributor scoped to that container (Storage Blob Data Reader for read-only), or use your own sign-in.
3. Credentials: AZURE_CLIENT_ID, AZURE_CLIENT_SECRET and AZURE_TENANT_ID for a service principal; AZURE_STORAGE_ACCOUNT_KEY or AZURE_STORAGE_SAS_TOKEN; else what az login left behind; else the machine's managed identity.

A directory

dir:///media/usb/project needs nothing. Several people on one machine, or one person from several checkouts, can share it; a reference is updated under a lock file beside it.

Using a store remote

oxy remote add NAME URL stores the URL; --profile, --region, --endpoint and --credentials are kept on it as query parameters, so a clone can name them too: oxy clone "gs://bucket/project?credentials=key.json".

oxy remote check NAME reaches the store with the credentials in hand and says how long that took, what the prefix holds (this repository, another one, or nothing yet), and then puts the store through the checks a repository depends on: object round trips, create-only writes, listing, deletion and compare-and-swap on a reference. It writes under a probe prefix of its own and leaves nothing behind.

Then push, pull, fetch and clone work as for any remote. Every push uploads one bundle holding what the store lacks, and moves the record of bookmarks and tags from the version it read. Two people pushing at once are safe: whoever lands second is applied over what landed, or told to pull first when the same bookmark moved.

oxy gc --remote NAME folds the bundles into one, so a fresh clone downloads one file. The superseded bundles are removed only seven days later (--now removes them at once), so a clone that was listing them at the time still completes.

What a store holds

PREFIX/
  oxygen.toml                     format version and repository id, written once
  bundles/00000001-<hash>.bundle  one per push: changes, signatures, seals, objects
  bundles/00000002-<hash>.bundle
  refs                            bookmarks with heads and attestations, tags, policy
  gc                              bundles a gc superseded, with when

Each client keeps a mirror of the store under .oxy/remotes/NAME/strata/, brought level before every use, and runs the same checks on a push that a hub runs. The mirror is a cache: delete it and the next use rebuilds it.

Troubleshooting

Every error says what to do next. The common ones:

• access denied — the identity cannot do this on the bucket: check its policy covers ListBucket on the bucket and GetObject, PutObject and DeleteObject under the prefix (or the equivalents on GCS and Azure).
• no usable credentials — nothing in the chain above produced any: set a profile, sign in with the provider's tool, or set the variables.
• profile "x" is not in ~/.aws/config or ~/.aws/credentials — the name on the URL is wrong, or the files live elsewhere (AWS_CONFIG_FILE, AWS_SHARED_CREDENTIALS_FILE).
• no region was given and S3 did not say — add --region, set AWS_REGION, or put region in the profile.
• holds a different repository — the prefix is taken: use another.
• a push may be in progress, so try again — a bundle another push uploaded is not yet named in the record; a moment later it is. If it never is, the store is missing changes: oxy remote check says what it holds.
• the remote kept moving while pushing — several pushes landed while yours was retried; oxy pull, then push again.

oxy remote check NAME is the first thing to run when something is wrong: it separates reachability, permissions and behaviour.