Switch from S3 to Backblaze B2 storage

- Update opam-publish script to use b2 CLI via uv
- Update README with B2 setup and Cloudflare CDN instructions
- Fix simdjsont checksum for new B2 upload

Changed files
+74 -25
bin
packages
simdjsont
simdjsont.0.1.0
+64 -15
README.md
··· 17 17 18 18 ``` 19 19 ┌──────────────┐ opam-publish ┌─────────────┐ 20 - │ Your Package │ ────────────────▶│ S3 Storage │◀── tarballs 21 - └──────────────┘ └─────────────┘ 20 + │ Your Package │ ────────────────▶│ Backblaze │◀── tarballs 21 + └──────────────┘ │ B2 │ 22 + └─────────────┘ 22 23 23 24 24 25 ┌─────────────┐ 25 - │ Cloudflare │◀── public reads 26 + │ Cloudflare │◀── CDN + custom domain 26 27 └─────────────┘ 27 28 28 29 ┌──────────────┐ opam-publish ┌─────────────┐ ··· 34 35 35 36 ### Requirements 36 37 37 - - `mc` (MinIO client, for S3-compatible uploads) 38 + - `uv` (for running b2 CLI tool) 38 39 - `git` (for pushing to this repo) 39 40 - `tar`, `sha256sum` (standard Unix tools) 40 41 - `opam` (optional, for linting validation) ··· 42 43 43 44 ### Configuration 44 45 45 - **1. Configure mc alias** (stores credentials securely in `~/.mc/`): 46 + **1. Authorize b2 CLI**: 46 47 47 48 ```bash 48 - mc alias set mys3 https://your-s3-endpoint ACCESS_KEY SECRET_KEY 49 + uv tool run b2 account authorize <applicationKeyId> <applicationKey> 49 50 ``` 51 + 52 + You can get your application key from the Backblaze B2 dashboard under "App Keys". 50 53 51 54 **2. Create opam-publish config** (`~/.config/opam-publish/config`): 52 55 ··· 55 58 chmod 700 ~/.config/opam-publish 56 59 57 60 cat > ~/.config/opam-publish/config << 'EOF' 58 - MC_ALIAS=mys3 59 - S3_BUCKET=your-bucket 61 + B2_BUCKET=your-bucket 60 62 PUBLIC_URL_BASE=https://packages.yourdomain.com 61 63 OPAM_REPO_URL=git@your-git-host:path/to/this/repo 62 64 EOF ··· 104 106 1. Validates `.opam` file (required fields, runs `opam lint` if available) 105 107 2. Builds a source tarball (excludes `.git`, `_build`, `_opam`) 106 108 3. Computes SHA256 checksum 107 - 4. Uploads tarball to S3 109 + 4. Uploads tarball to Backblaze B2 108 110 5. Clones this repo, adds package metadata with URL + checksum 109 111 6. Pushes commit to this repo 110 112 ··· 115 117 ok 116 118 ==> Building mylib.1.0.0.tbz... 117 119 42K 118 - ==> Uploading to S3... 119 - https://packages.example.com/mylib/mylib.1.0.0.tbz 120 + ==> Uploading to B2... 121 + https://packages.yourdomain.com/mylib/mylib.1.0.0.tbz 120 122 ==> Cloning opam repository... 121 123 ==> Pushing to opam repository... 122 124 ==> Published mylib 1.0.0 ··· 177 179 opam-publish 178 180 ``` 179 181 182 + ## Cloudflare CDN Setup (Optional) 183 + 184 + To serve packages via a custom domain with Cloudflare: 185 + 186 + ### 1. DNS Record 187 + 188 + Add a CNAME record pointing to your B2 bucket: 189 + 190 + ``` 191 + Type: CNAME 192 + Name: opam (or your subdomain) 193 + Target: f00X.backblazeb2.com # Replace X with your cluster number from `b2 account get` 194 + Proxy: ON (orange cloud) 195 + ``` 196 + 197 + ### 2. Transform Rule 198 + 199 + Create a URL rewrite rule in **Rules > Transform Rules > Rewrite URL**: 200 + 201 + ``` 202 + Name: B2 Bucket Rewrite 203 + When: (http.host eq "packages.yourdomain.com") 204 + Rewrite to: 205 + Path: Dynamic 206 + Expression: concat("/file/your-bucket", http.request.uri.path) 207 + ``` 208 + 209 + ### 3. SSL Settings 210 + 211 + Set SSL mode to **Full** in **SSL/TLS > Overview**. 212 + 213 + ### 4. Cache Settings (Recommended) 214 + 215 + Create a cache rule for better performance: 216 + 217 + ``` 218 + URL: packages.yourdomain.com/* 219 + Cache Level: Cache Everything 220 + Edge Cache TTL: 1 month 221 + ``` 222 + 180 223 ## Troubleshooting 181 224 182 225 ### "version not found" ··· 197 240 198 241 Each version can only be published once. Bump the version number. 199 242 200 - ### S3 upload fails 243 + ### B2 upload fails 201 244 202 - Check mc alias and bucket access: 245 + Check b2 authorization: 203 246 204 247 ```bash 205 - mc alias list 206 - mc ls mys3/your-bucket 248 + uv tool run b2 account get 249 + uv tool run b2 bucket list 250 + ``` 251 + 252 + Re-authorize if needed: 253 + 254 + ```bash 255 + uv tool run b2 account authorize <applicationKeyId> <applicationKey> 207 256 ``` 208 257 209 258 ### Git push fails
+9 -9
bin/opam-publish
··· 10 10 Create it with: 11 11 mkdir -p ~/.config/opam-publish 12 12 cat > ~/.config/opam-publish/config << 'EOF' 13 - MC_ALIAS=mys3 14 - S3_BUCKET=your-bucket 13 + B2_BUCKET=your-bucket 15 14 PUBLIC_URL_BASE=https://packages.yourdomain.com 16 15 OPAM_REPO_URL=git@your-git-host:your-repo 17 16 EOF 18 17 19 - Then configure mc: 20 - mc alias set mys3 https://your-s3-endpoint ACCESS_KEY SECRET_KEY" 18 + Then authorize b2: 19 + uv tool run b2 account authorize <applicationKeyId> <applicationKey>" 21 20 source "$CONFIG_FILE" 22 21 } 23 22 ··· 71 70 echo " $(du -h "${archive}" | cut -f1)" 72 71 } 73 72 74 - upload_s3() { 73 + upload_b2() { 75 74 local pkg="$1" archive="$2" 76 75 local filename 77 76 filename=$(basename "${archive}") 78 - echo "==> Uploading to S3..." 79 - mc cp --quiet "${archive}" "${MC_ALIAS}/${S3_BUCKET}/${pkg}/${filename}" 80 - echo " ${PUBLIC_URL_BASE}/${pkg}/${filename}" 77 + local b2_path="${pkg}/${filename}" 78 + echo "==> Uploading to B2..." 79 + uv tool run b2 file upload --no-progress "${B2_BUCKET}" "${archive}" "${b2_path}" 80 + echo " ${PUBLIC_URL_BASE}/${b2_path}" 81 81 } 82 82 83 83 publish_opam() { ··· 126 126 local checksum 127 127 checksum=$(sha256sum "/tmp/${archive}" | cut -d' ' -f1) 128 128 129 - upload_s3 "$pkg" "/tmp/${archive}" 129 + upload_b2 "$pkg" "/tmp/${archive}" 130 130 publish_opam "$pkg" "$version" "$archive" "$checksum" 131 131 132 132 rm -f "/tmp/${archive}"
+1 -1
packages/simdjsont/simdjsont.0.1.0/opam
··· 37 37 38 38 url { 39 39 src: "https://opam.qswarm.com/simdjsont/simdjsont.0.1.0.tbz" 40 - checksum: "sha256=abf633c49ed29642417347d8c457eb642e3e0dae07a4ce2ec6b91c32feaa7334" 40 + checksum: "sha256=dbbbebd1401fedba7542684b9a9ccb35f5c34e34a2e01a1480544e024b1bcff3" 41 41 }