← Back to list
r2-glacier-migration
by pollinations
Your Friendly Open-Source Gen-AI Platform
⭐ 3,819🍴 583📅 Jan 23, 2026
colaboratorycolaboratory-notebookdiffusiongangenerativehacktoberfesthacktoberfest-acceptedhacktoberfest2025
Run in ManusUse Cases
🧠
AI Model Integration
Integrate LLM and ML models into your application.
✨
Prompt Optimization
Improve prompts for better results.
📊
Automated Data Analysis
AI-powered data analysis and insight extraction.
SKILL.md
name: r2-glacier-migration description: Monitor and manage R2 to AWS Glacier Deep Archive migration. Use when checking transfer status, resuming transfers, or managing the archive migration.
R2 → AWS Glacier Deep Archive Migration
Migrate ~42.6 TB from Cloudflare R2 to AWS S3 Glacier Deep Archive.
Cost Savings
- R2: ~$638/month → Glacier: ~$42/month
- Savings: ~$596/month
Buckets
| Bucket | Size | Objects | Status |
|---|---|---|---|
| pollinations-text | 16.96 TB | 1.86B | 🔄 In progress |
| pollinations-images | 25.64 TB | 221M | ⏳ Pending |
Quick Commands
Check Status
# Is transfer running?
ssh ninon "screen -ls"
# View recent logs
ssh ninon "tail -50 ~/r2-glacier/transfer-text.log"
# Check checkpoint
ssh ninon "cat /tmp/r2-glacier-pollinations-text-checkpoint.json | jq ."
Attach to Screen (Interactive)
ssh -t ninon "screen -r r2-text"
# Detach: Ctrl+A, D
Resume if Crashed
ssh ninon "screen -dmS r2-text bash -c '~/r2-glacier/run-text-transfer.sh; exec bash'"
File Locations
| Location | Path | Purpose |
|---|---|---|
| Skill | scripts/r2-to-glacier-streaming.js | Main transfer script |
| Skill | scripts/package.json | Dependencies |
| ninon | ~/r2-glacier/ | Deployed script + deps |
| ninon | ~/r2-glacier/run-text-transfer.sh | Wrapper with credentials |
| ninon | ~/r2-glacier/transfer-text.log | Transfer log |
| ninon | /tmp/r2-glacier-*-checkpoint.json | Resume checkpoint |
Script Usage
# Test (small batch)
node r2-to-glacier-streaming.js -b pollinations-text --batch-size 100 --max-batches 1
# Full transfer
node r2-to-glacier-streaming.js -b pollinations-text --batch-size 50000 --concurrency 30
# Resume after interruption
node r2-to-glacier-streaming.js -b pollinations-text --resume
# Dry run (count only)
node r2-to-glacier-streaming.js -b pollinations-text --dry-run
AWS Setup
- Bucket:
s3://pollinations-archive - Region: us-east-1
- Storage Class: DEEP_ARCHIVE
Verify Archives
aws s3 ls s3://pollinations-archive/ --recursive --human-readable | head -20
Important Notes
- ⚠️ 180-day minimum on Glacier Deep Archive - don't delete R2 until verified!
- Some objects fail with "Header overflow" (very large responses) - acceptable loss
- Text bucket (1.86B objects) takes several days
- R2 egress is free, AWS ingress is free
Start Images Bucket (After Text Complete)
-
Update
~/r2-glacier/run-text-transfer.shon ninon:# Change bucket name node r2-to-glacier-streaming.js -b pollinations-images --batch-size 50000 --concurrency 30 -
Start new screen:
ssh ninon "screen -dmS r2-images bash -c '~/r2-glacier/run-images-transfer.sh; exec bash'"
Tracking
- Issue: #5860
- PR: #5861
Score
Total Score
80/100
Based on repository quality metrics
✓SKILL.md
SKILL.mdファイルが含まれている
+20
✓LICENSE
ライセンスが設定されている
+10
○説明文
100文字以上の説明がある
0/10
✓人気
GitHub Stars 1000以上
+15
✓最近の活動
1ヶ月以内に更新
+10
✓フォーク
10回以上フォークされている
+5
○Issue管理
オープンIssueが50未満
0/5
✓言語
プログラミング言語が設定されている
+5
✓タグ
1つ以上のタグが設定されている
+5
Reviews
💬
Reviews coming soon
