lhaig / pack-garage

Published by Lance Haig | View Source

pack
0 stars
Description
S3-compatible storage cluster
License
MPL-2.0
Tags
#storage #s3
Versions
Select a version to see its documentation and run command.
Quick Run (v0.0.1)
ramble pack run lhaig/pack-garage@v0.0.1
README
# Garage Nomad Pack Deploy [Garage](https://garagehq.deuxfleurs.fr/), an S3-compatible distributed object storage service, to HashiCorp Nomad. ## Features - S3-compatible API for object storage - Distributed architecture with configurable replication - Nomad native service discovery for peer connection - Static website hosting capability - Admin API for cluster management ## Prerequisites 1. **Host volumes** configured on Nomad clients: ```hcl # In your Nomad client configuration client { host_volume "garage-data" { path = "/var/lib/garage/data" read_only = false } host_volume "garage-meta" { path = "/var/lib/garage/meta" read_only = false } } ``` 2. **RPC secret** generated: ```bash openssl rand -hex 32 ``` ## Usage ### Basic deployment (3-node cluster) ```bash nomad-pack run garage \ --var rpc_secret="<your-32-byte-hex-secret>" ``` ### Single-node development setup ```bash nomad-pack run garage \ --var rpc_secret="$(openssl rand -hex 32)" \ --var count=1 \ --var replication_factor=1 ``` ### Production deployment with static ports ```bash nomad-pack run garage \ --var rpc_secret="<your-secret>" \ --var count=3 \ --var replication_factor=3 \ --var s3_api_port=3900 \ --var admin_token="<admin-token>" ``` ## Post-deployment Setup After deployment, you need to configure the cluster layout: ```bash # Get allocation ID ALLOC=$(nomad job allocs -json garage | jq -r '.[0].ID') # Check node status nomad alloc exec $ALLOC garage status # Assign capacity to each node (repeat for each node ID shown in status) nomad alloc exec $ALLOC garage layout assign -z dc1 -c 10G <node-id> # Apply the layout nomad alloc exec $ALLOC garage layout apply --version 1 # Create a bucket nomad alloc exec $ALLOC garage bucket create my-bucket # Create an access key nomad alloc exec $ALLOC garage key create my-key # Grant access nomad alloc exec $ALLOC garage bucket allow --read --write my-bucket --key my-key ``` ## Variables | Variable | Description | Default | |----------|-------------|---------| | `job_name` | Override job name | `garage` | | `region` | Nomad region | `""` | | `datacenters` | Target datacenters | `["*"]` | | `count` | Number of instances | `3` | | `garage_version` | Docker image tag | `v1.0.1` | | `rpc_secret` | 32-byte hex RPC secret | **required** | | `replication_factor` | Data replication factor | `3` | | `db_engine` | Metadata DB engine | `lmdb` | | `register_service` | Register with Nomad SD | `true` | | `service_name` | Base service name | `garage` | | `service_tags` | S3 service tags | `["s3", "storage"]` | | `s3_api_port` | S3 API port (0=dynamic) | `0` | | `rpc_port` | RPC port (0=dynamic) | `0` | | `web_port` | Web hosting port (0=dynamic) | `0` | | `admin_port` | Admin API port (0=dynamic) | `0` | | `cpu` | CPU allocation (MHz) | `500` | | `memory` | Memory allocation (MB) | `512` | | `admin_token` | Admin API bearer token | `""` | ## Services The pack registers these services with Nomad service discovery: | Service | Port | Description | |---------|------|-------------| | `garage-rpc` | 3901 | Inter-node RPC communication | | `garage-s3` | 3900 | S3-compatible API | | `garage-web` | 3902 | Static website hosting | | `garage-admin` | 3903 | Admin API (if token set) | ## Architecture ``` ┌─────────────────┐ │ Nomad Service │ │ Discovery │ └────────┬────────┘ │ ┌─────────────────┼─────────────────┐ │ │ │ ▼ ▼ ▼ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Garage 1 │◄──│ Garage 2 │──►│ Garage 3 │ │ (RPC mesh) │ │ (RPC mesh) │ │ (RPC mesh) │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ ▼ ▼ ▼ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Host Volume │ │ Host Volume │ │ Host Volume │ │ data/meta │ │ data/meta │ │ data/meta │ └─────────────┘ └─────────────┘ └─────────────┘ ``` Nodes discover each other via Nomad service discovery and form a peer-to-peer mesh for data replication. ## License MPL-2.0