# rule34vault-downloader

Python script to download public bookmarks from `rule34vault.com` for a username. No authorization is required for public bookmark pages.

## Setup

```bash
python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
```

## Usage

Download bookmarks for a user:

```bash
python r34vault_downloader.py <user> --out downloads
```

Download only a page range:

```bash
python r34vault_downloader.py <user> --start-page 1 --end-page 3 --out downloads
```

Process only the first N bookmark items, useful for testing:

```bash
python r34vault_downloader.py <user> --start-page 6 --end-page 6 --limit 1 --out downloads
```

## Output

Files are grouped by Rule34Vault post prefix:

```text
downloads/
  1263/
    1263322.jpg
  996/
    996379.mp4
  download_order.txt
  seen.txt
  failed.txt
  skipped.txt
```

`download_order.txt` records successful downloads in bookmark order:

```text
000001 1263322 downloads/1263/1263322.jpg https://rule34vault.com/posts/1263/1263322/1263322.jpg
```

## Behavior

- Uses browser-like headers and post-page `Referer` headers.
- Uses streaming `GET`; it does not rely on `HEAD` requests.
- Parses bookmark data from `script#ng-state` and preserves bookmark order.
- Tries original image URLs first, for example `1263322.jpg`.
- If a page image is only `1263303.small.jpg`, it tries `1263303.jpg` before falling back to the small file.
- For videos, parses real `<video>` and `<source src="...">` elements from the post page.
- Retries transient failures with exponential backoff.
- Uses plain text logs only; no SQLite database.

## Tests

```bash
. .venv/bin/activate
pip install pytest
pytest -q
```