# Zero-Bandwidth Duplicati API

Thin `Node.js + Express` metadata service for Duplicati restores. The backend owns only SQLite metadata and enhancement jobs; file bytes stay on WebDAV and are never proxied through the runtime restore APIs.

## What changed

- Multiple uploaded Duplicati task databases can now coexist at the same time.
- Every uploaded task library becomes its own `sourceId`.
- A single uploaded `Duplicati-server.sqlite` can be stored separately and used to map friendly task names onto task database files.
- Directory browsing and file metadata APIs are now `sourceId`-scoped.
- The frontend is now split into three pages:
  - `/` for source management
  - `/source.html?sourceId=...` for a source workbench
  - `/file.html?sourceId=...&id=...` for video preview and browser-direct downloads
- Video preview and downloads now pull encrypted dblocks directly from WebDAV in the browser, decrypt them locally, and never proxy the file stream through the backend.
- The backend can build an enhanced SQLite copy per source by fetching remote `*.dblock.zip.aes` files, decrypting them, and writing:
  - `archive_entry_index`
  - `volume_crypto_cache`
  - `volume_scan_inventory`
  - `enhancement_meta`

## Endpoints

- `GET /api/sources`
- `GET /api/sources/:sourceId`
- `POST /api/sources/upload`
- `GET /api/server-db`
- `POST /api/server-db/upload`
- `PUT /api/sources/:sourceId/secrets`
- `POST /api/sources/:sourceId/enhance`
- `GET /api/sources/:sourceId/enhance/status`
- `GET /api/ls?sourceId=...&path=/&snapshot=latest`
- `GET /api/file-info?sourceId=...&id=...`
- `GET /healthz`

Compatibility routes:

- `POST /api/source/upload`
- `GET /api/source/current`

`/api/source/current` only works when the system currently contains exactly one source. If there are multiple sources, it returns `409 SOURCE_ID_REQUIRED`.

## Quick start

```bash
npm install
copy .env.example .env
npm start
```

Environment variables:

- `PORT`: HTTP port, default `3000`
- `APP_DB_PATH`: service-owned SQLite path, default `./data/app.sqlite`
- `UPLOAD_DIR`: source storage root, default `./data/sources`
- `PREVIEW_MAX_BYTES`: max file size flagged as safe for in-memory preview, default `16777216`

## Multi-source model

Each upload is stored under its own directory:

- `data/sources/<sourceId>/raw.sqlite`
- `data/sources/<sourceId>/enhanced.sqlite`
- `data/sources/<sourceId>/enhanced.sqlite.tmp`
- `data/sources/<sourceId>/work/*`

Duplicate uploads are deduplicated by SHA-256. If the same task database is uploaded again, the service returns the existing `sourceId` instead of creating a second copy.

The optional `Duplicati-server.sqlite` upload is stored separately under the upload root and never appears in `/api/sources`. It is only used to resolve friendly task names by matching `path.basename(Backup.DBPath)` against each uploaded task database filename.

When a server DB is available:

- source cards prefer `Backup.Name` as `displayName`
- `displayNameSource` becomes `server-db`
- `matchedBackupName` returns the matched task name

If no match exists, the source keeps using its original filename as `displayName`.

## Upload model

There are now two upload paths:

1. Task DB upload: `POST /api/sources/upload`
2. Server DB upload: `POST /api/server-db/upload`

Use the task DB upload for random-name Duplicati job databases such as `TMQRJYNADS.sqlite`.

Use the server DB upload only for `Duplicati-server.sqlite`. Uploading that file to the task DB endpoint is rejected with `422 INVALID_SOURCE_SCHEMA`.

## Enhancement flow

Raw Duplicati task databases can browse files immediately, but `/api/file-info` stays gated until archive indexes exist.

To enhance a source:

1. Upload a raw task database.
2. Save WebDAV base URL, auth mode, username/password, and backup passphrase.
3. Start enhancement for that `sourceId`.
4. The backend serially downloads each `*.dblock.zip.aes`, decrypts the AES Crypt v2 container, scans the ZIP central directory, and writes supplemental tables into an enhanced SQLite copy.
5. After success, the source flips to `ready` and `/api/file-info` starts returning ordered segments and dblock mappings.

Current runtime assumptions:

- WebDAV auth supports `basic` and `anonymous`
- AES enhancement currently supports AES Crypt v2 volumes
- enhancement jobs run serially in-process
- credentials are stored in `app.sqlite` and are not returned by the API

If enhancement fails, source browsing still works, and `/api/file-info` returns `409 ENHANCEMENT_FAILED`.

## Frontend restore flow

The runtime restore path stays "thick frontend, thin backend":

1. `source.html` browses the selected source with `/api/ls`
2. `file.html` fetches `/api/file-info`
3. the browser asks WebDAV directly for the required `*.dblock.zip.aes`
4. the browser decrypts AES Crypt v2 locally
5. the browser caches plain ZIP volumes in OPFS
6. a root-scope `Service Worker` serves pseudo-Range responses for `<video>`
7. downloads write restored bytes directly to disk with `showSaveFilePicker()`

Important constraints:

- the backend still does not proxy file bytes
- browser-side WebDAV credentials are entered separately on `file.html`
- the current v1 range model fetches each needed dblock as a whole encrypted file, then decrypts it in the browser
- remote random access is therefore "logical range over cached dblocks", not true O(1) random access inside `.aes`

Supported browser target:

- desktop Chromium with `Service Worker`, `OPFS`, `IndexedDB`, `showSaveFilePicker()`, `Web Crypto`, and `DecompressionStream`

## Supplemental schema

The enhancement job writes the SQL shape described in [sql/supplemental-schema.sql](C:/Users/nanxun/Documents/jiami/sql/supplemental-schema.sql).

If you already have a pre-enhanced SQLite file, uploading it is also supported. In that case the source is immediately marked `ready`.

## Runtime note

This implementation uses Node 24's built-in `node:sqlite` module. In this Windows environment, native `sqlite3` compilation is not reliable enough, so the service uses the built-in adapter instead.