OXYGENHUB
# 454236d1108d oxygen / docs / strata.md
7.0 KiB · Markdown · last changed Strata: the user guide, a README section, and the design marked as built 1 month ago renderedsource blame history raw
8e119486 Andy McCall ↶ 1 
# Oxygen Strata: keeping a repository on your own storage
2
3 
A repository can be pushed to, cloned from and pulled from a bucket in
4 
your own cloud account, or a directory on a shared drive, with no server
5 
in between. The bucket is just another remote:
6
7 
```
8 
oxy remote add cloud s3://my-bucket/repos/project --profile work
9 
oxy remote check cloud
10 
oxy push cloud
11 
oxy clone s3://my-bucket/repos/project?profile=work
12 
oxy pull
13 
```
14
15 
It works because Oxygen's objects never change and are named by their
16 
hash, so any store that can keep write-once objects and update one small
17 
record by compare-and-swap can hold a complete repository. Nothing in the
18 
store is trusted: every object is re-hashed as it is read, changes are
19 
signed, and a bookmark's attestation is verified under your policy, as
20 
for any remote. The data stays in your account, in your region, under
21 
your keys; an operator of a hub need never see it.
22
23 
Supported stores:
24
25 
| URL | Store |
26 
|---|---|
27 
| `s3://BUCKET/PREFIX` | Amazon S3, and S3-compatible services with `--endpoint` |
28 
| `gs://BUCKET/PREFIX` | Google Cloud Storage |
29 
| `az://ACCOUNT/CONTAINER/PREFIX` | Azure Blob Storage |
30 
| `dir://PATH` | a directory: a USB stick, a network share, or for trying things out |
31
32 
A store holds one repository per prefix. The first push binds the prefix
33 
to the repository; a push of another repository to the same prefix is
34 
refused. Use a prefix per repository, as `repos/project` above.
35
36 
## Setting up a bucket
37
38 
### Amazon S3
39
40 
1. Create a bucket in the region you want the data kept in (for example
41 
   `eu-west-2`, London), with public access blocked.
42 
2. Give the identity oxy will use these permissions, and no more:
43
44 
   ```json
45 
   {
46 
     "Version": "2012-10-17",
47 
     "Statement": [
48 
       { "Effect": "Allow", "Action": "s3:ListBucket",
49 
         "Resource": "arn:aws:s3:::my-bucket" },
50 
       { "Effect": "Allow",
51 
         "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
52 
         "Resource": "arn:aws:s3:::my-bucket/repos/*" }
53 
     ]
54 
   }
55 
   ```
56
57 
   A read-only identity (`s3:ListBucket` and `s3:GetObject`) gives a
58 
   working clone and pull.
59 
3. Credentials come from the places the AWS tools use, in this order:
60 
   `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` in the environment; the
61 
   profile named by `--profile` (or `AWS_PROFILE`, else `default`) in
62 
   `~/.aws/credentials`; what the AWS CLI can produce for that profile
63 
   (`aws sso login`, `aws login`, assumed roles, credential processes), so
64 
   anything that works for `aws` works for `oxy`; and the instance's or
65 
   container's role.
66 
4. The region is taken from `--region`, `AWS_REGION`, the profile, or
67 
   asked of S3. `--endpoint http://…` points at an S3-compatible service.
68
69 
### Google Cloud Storage
70
71 
1. Create a bucket in the region you want (for example `europe-west2`,
72 
   London), with uniform access and public access prevention.
73 
2. Create a service account and grant it **Storage Object Admin** on that
74 
   bucket only (**Storage Object Viewer** for a read-only identity), or
75 
   use your own account.
76 
3. Credentials: a service account key file named by `--credentials` or
77 
   `GOOGLE_APPLICATION_CREDENTIALS`; else the application default
78 
   credentials that `gcloud auth application-default login` writes; else
79 
   the instance's service account.
80
81 
### Azure Blob Storage
82
83 
1. Create a storage account in the region you want (for example
84 
   `UK South`), with public blob access off, and a container in it.
85 
2. Create a service principal (`az ad sp create-for-rbac`) with
86 
   **Storage Blob Data Contributor** scoped to that container (**Storage
87 
   Blob Data Reader** for read-only), or use your own sign-in.
88 
3. Credentials: `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET` and
89 
   `AZURE_TENANT_ID` for a service principal; `AZURE_STORAGE_ACCOUNT_KEY`
90 
   or `AZURE_STORAGE_SAS_TOKEN`; else what `az login` left behind; else
91 
   the machine's managed identity.
92
93 
### A directory
94
95 
`dir:///media/usb/project` needs nothing. Several people on one machine,
96 
or one person from several checkouts, can share it; a reference is
97 
updated under a lock file beside it.
98
99 
## Using a store remote
100
101 
`oxy remote add NAME URL` stores the URL; `--profile`, `--region`,
102 
`--endpoint` and `--credentials` are kept on it as query parameters, so a
103 
clone can name them too: `oxy clone "gs://bucket/project?credentials=key.json"`.
104
105 
`oxy remote check NAME` reaches the store with the credentials in hand
106 
and says how long that took, what the prefix holds (this repository,
107 
another one, or nothing yet), and then puts the store through the checks
108 
a repository depends on: object round trips, create-only writes, listing,
109 
deletion and compare-and-swap on a reference. It writes under a probe
110 
prefix of its own and leaves nothing behind.
111
112 
Then `push`, `pull`, `fetch` and `clone` work as for any remote. Every
113 
push uploads one bundle holding what the store lacks, and moves the
114 
record of bookmarks and tags from the version it read. Two people
115 
pushing at once are safe: whoever lands second is applied over what
116 
landed, or told to pull first when the same bookmark moved.
117
118 
`oxy gc --remote NAME` folds the bundles into one, so a fresh clone
119 
downloads one file. The superseded bundles are removed only seven days
120 
later (`--now` removes them at once), so a clone that was listing them at
121 
the time still completes.
122
123 
## What a store holds
124
125 
```
126 
PREFIX/
127 
  oxygen.toml                     format version and repository id, written once
128 
  bundles/00000001-<hash>.bundle  one per push: changes, signatures, seals, objects
129 
  bundles/00000002-<hash>.bundle
130 
  refs                            bookmarks with heads and attestations, tags, policy
131 
  gc                              bundles a gc superseded, with when
132 
```
133
134 
Each client keeps a mirror of the store under `.oxy/remotes/NAME/strata/`,
135 
brought level before every use, and runs the same checks on a push that a
136 
hub runs. The mirror is a cache: delete it and the next use rebuilds it.
137
138 
## Troubleshooting
139
140 
Every error says what to do next. The common ones:
141
142 
- **access denied** — the identity cannot do this on the bucket: check
143 
  its policy covers `ListBucket` on the bucket and `GetObject`,
144 
  `PutObject` and `DeleteObject` under the prefix (or the equivalents on
145 
  GCS and Azure).
146 
- **no usable credentials** — nothing in the chain above produced any:
147 
  set a profile, sign in with the provider's tool, or set the variables.
148 
- **profile "x" is not in ~/.aws/config or ~/.aws/credentials** — the
149 
  name on the URL is wrong, or the files live elsewhere
150 
  (`AWS_CONFIG_FILE`, `AWS_SHARED_CREDENTIALS_FILE`).
151 
- **no region was given and S3 did not say** — add `--region`, set
152 
  `AWS_REGION`, or put `region` in the profile.
153 
- **holds a different repository** — the prefix is taken: use another.
154 
- **a push may be in progress, so try again** — a bundle another push
155 
  uploaded is not yet named in the record; a moment later it is. If it
156 
  never is, the store is missing changes: `oxy remote check` says what
157 
  it holds.
158 
- **the remote kept moving while pushing** — several pushes landed while
159 
  yours was retried; `oxy pull`, then push again.
160
161 
`oxy remote check NAME` is the first thing to run when something is
162 
wrong: it separates reachability, permissions and behaviour.