128 lines
4.3 KiB
Markdown
128 lines
4.3 KiB
Markdown
# ep-zpool-balancer
|
|
|
|
A Bash script that rebalances data across all vdevs of a ZFS pool after new
|
|
disks (or vdevs) have been added to a `raidz2`/`raidz3` (or any other) pool.
|
|
|
|
## Why
|
|
|
|
ZFS only spreads **new** writes across all vdevs in a pool. Blocks written
|
|
before new vdevs were added stay exactly where they are, so old data keeps
|
|
living on the original vdevs while newly added vdevs stay mostly empty. ZFS
|
|
has no built-in rebalance operation.
|
|
|
|
`ep-zpool-balancer.sh` works around this by rewriting every file in place:
|
|
|
|
1. Copy the file to a temporary file (`cp --preserve=all`) in the same
|
|
directory, so it stays on the same dataset.
|
|
2. Verify the copy with a SHA-256 checksum comparison against the original.
|
|
3. Atomically replace the original with the verified copy (`mv`).
|
|
|
|
Because the copy is a fresh write, ZFS allocates its blocks across the whole
|
|
pool — including the new vdevs — which gradually rebalances pool usage.
|
|
|
|
## Requirements
|
|
|
|
- Linux/BSD host with the pool imported (the script must run where the ZFS
|
|
pool is mounted, not from Windows).
|
|
- `bash`, plus standard GNU utilities: `zfs`, `zpool`, `find`, `stat`,
|
|
`sha256sum`, `awk`, `df`, `numfmt`, `realpath`.
|
|
- Enough free space headroom on the pool to temporarily hold a second copy
|
|
of the largest file being rewritten (see `--min-free`).
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
chmod +x ep-zpool-balancer.sh
|
|
```
|
|
|
|
## Usage
|
|
|
|
```
|
|
ep-zpool-balancer.sh -p PATH [PATH ...] [options]
|
|
ep-zpool-balancer.sh -P POOLNAME [options]
|
|
```
|
|
|
|
### Target selection (one required)
|
|
|
|
| Option | Description |
|
|
| --- | --- |
|
|
| `-p, --path PATH` | Directory (inside a ZFS dataset) to rebalance. Repeatable. |
|
|
| `-P, --pool POOLNAME` | Rebalance every mounted dataset belonging to this pool. |
|
|
|
|
### Options
|
|
|
|
| Option | Description |
|
|
| --- | --- |
|
|
| `-m, --min-free PERCENT` | Skip/abort rewriting a file if pool free space would drop below this percentage. Default: `10`. |
|
|
| `-s, --state-file PATH` | File tracking already-rebalanced files, so a run can be resumed. Default: `<target>/.ep-zpool-balancer-state`. |
|
|
| `-r, --reset-state` | Clear the state file and start fresh. |
|
|
| `-e, --exclude PATTERN` | Shell glob pattern to exclude (matched against full file path). Repeatable. |
|
|
| `-H, --include-hardlinks` | Also rewrite files with more than one hardlink (skipped by default, since rewriting breaks the hardlink group). |
|
|
| `-l, --log-file PATH` | Also append log output to this file. |
|
|
| `-n, --dry-run` | Show what would be rebalanced without changing anything. |
|
|
| `-c, --clean-tmp` | Remove leftover `*.ep-zpool-balancer.tmp.*` files from a previous interrupted run before starting. |
|
|
| `-v, --verbose` | Verbose/debug output. |
|
|
| `-h, --help` | Show usage help. |
|
|
|
|
## Examples
|
|
|
|
Dry-run a single dataset first to see what would happen:
|
|
|
|
```bash
|
|
./ep-zpool-balancer.sh -p /tank/data -n
|
|
```
|
|
|
|
Rebalance it for real:
|
|
|
|
```bash
|
|
./ep-zpool-balancer.sh -p /tank/data
|
|
```
|
|
|
|
Rebalance every dataset in pool `tank`, keeping at least 15% free space:
|
|
|
|
```bash
|
|
./ep-zpool-balancer.sh -P tank -m 15
|
|
```
|
|
|
|
Resume an interrupted run (already-rewritten files are skipped via the state
|
|
file):
|
|
|
|
```bash
|
|
./ep-zpool-balancer.sh -p /tank/data
|
|
```
|
|
|
|
Exclude a directory and log to a file:
|
|
|
|
```bash
|
|
./ep-zpool-balancer.sh -p /tank/data -e '/tank/data/no-touch/*' -l rebalance.log
|
|
```
|
|
|
|
## How it decides what to skip
|
|
|
|
- Files already recorded in the state file (already rebalanced this run/series of runs).
|
|
- Symlinks and non-regular files.
|
|
- Files matching an `--exclude` pattern.
|
|
- Hardlinked files (`nlink > 1`), unless `-H/--include-hardlinks` is given.
|
|
- Any file whose rewrite would push pool free space below `--min-free`.
|
|
|
|
## Safety notes
|
|
|
|
- The original file is only replaced after the copy is verified
|
|
byte-for-byte via SHA-256; on any mismatch or error, the original is left
|
|
untouched and the temp file is removed.
|
|
- `Ctrl-C` finishes the current file safely, then stops; progress already
|
|
made is preserved in the state file so the next run resumes where it left
|
|
off.
|
|
- Exit codes: `0` success, `2` completed with errors, `130` interrupted.
|
|
- Always test with `--dry-run` first, and confirm `zpool status` shows the
|
|
pool healthy before running against production data.
|
|
|
|
## Checking rebalance progress
|
|
|
|
Use standard ZFS tooling to observe per-vdev allocation before/after a run:
|
|
|
|
```bash
|
|
zpool list -v tank
|
|
zpool iostat -v tank
|
|
```
|