# Uniform Siphon - Sitecore Download commands Reference

> Command-line reference for the Uniform Siphon Sitecore download (extract) commands

Source: https://docs.uniform.app/docs/guides/migration/sitecore/download-reference

This page is the command-line reference for the **download (extract) phase** of a Sitecore migration — the commands of `Siphon.Download.exe` that pull items, media, and presentation out of a live Sitecore instance.

They reach Sitecore two different ways, and it matters where each one points. `download-items` and `download-media` go through the deployed `UniformMigrationService.aspx` endpoint (`--uniformServiceUrl`) on a **CD** instance; `download-presentation` drives a browser against the Experience Editor (`--host`) on the **CM**. See [Deploy to Sitecore](https://docs.uniform.app/docs/guides/migration/sitecore/download#deploy-to-sitecore).

The executables, how configuration and environment variables are resolved, logging, and the [common options](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference#common-options) shared by every command are documented once on the [common reference](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference). Each switch below lists its **command-scoped** environment variable; the global `SIPHON_{PARAMETER}` form works too.

If you are migrating for the first time, start with the [Sitecore migration walkthrough](https://docs.uniform.app/docs/guides/migration/sitecore/walkthrough) — it explains _when_ and _why_ to run each command.

---

## configure-download

Interactive wizard (part of the `siphon` wrapper) that walks you through the download-phase settings and writes a ready-to-use `.env`. Run it once at the start of a migration instead of assembling environment variables by hand — it saves progress after each answer, so you can re-run it to resume or edit.

```powershell
PS C:\migration> siphon configure-download
```

It prompts for the work directory, the common connection settings (database, service URL, secret), and per-command options for `download-items`, `download-media`, and (optionally) `download-presentation`, then writes keys such as `SIPHON_DOWNLOAD_ITEMS_ROOT` and `SIPHON_DOWNLOAD_ITEMS_OUTPUTDIR`. See the [download guide](https://docs.uniform.app/docs/guides/migration/sitecore/download#configure-siphon) for a full sample session. (For a blank template covering every key instead of a guided wizard, use `siphon generate-dot-env`.)

---

## download-items

Download Sitecore content items (and, optionally, their rendering datasources and field-referenced items) through the migration service. This is the backbone of the migration — every later phase reads from these files.

```powershell
PS C:\migration> siphon download download-items `
  --database web `
  --uniformServiceUrl https://cd.example.com/layouts/UniformMigrationService.aspx `
  --secret my_secret `
  --root "DAC24EDD-44FB-42EF-9ECD-1E8DAF706386" `
  --outputDir "C:\migration\data\items"
```

### Required parameters

**`SIPHON_DOWNLOAD_ITEMS_ROOT`** = `<guid>` (`--root`)  
Root item whose subtree (item + all descendants) is downloaded. For a single site pass its **Home** item; for a multi-site solution either download sites one by one, or pass the all-content root `11111111-1111-1111-1111-111111111111` to grab everything.

**`SIPHON_DOWNLOAD_ITEMS_DATABASE`** = `<string>` (`--database`)  
The Sitecore database to download from. Use **`web`** — the database that serves **published** content (i.e. your publishing target) — for a normal migration; if your solution publishes to a different database, use that one instead. You _can_ run a **second** pass against **`master`** to also pull **unpublished / draft** content, but this is **not recommended**: reconciling draft and published versions adds significant complexity. Prefer publishing all required content before the content freeze and migrating from `web` only.

**`SIPHON_DOWNLOAD_ITEMS_UNIFORMSERVICEURL`** = `<url>` (`--uniformServiceUrl`)  
Full URL to the `UniformMigrationService.aspx` you deployed to the Sitecore **CD**, e.g. `https://cd.example.com/layouts/UniformMigrationService.aspx`. Deploy it to **every** load-balanced CD instance — the load balancer then spreads `--threads` across the whole farm, which is the biggest lever on extraction speed, and a node without the file fails requests intermittently. Point this at the **CM** only for a `--database master` pass, since a CD has no `master` database. See [Deploy to Sitecore](https://docs.uniform.app/docs/guides/migration/sitecore/download#deploy-to-sitecore).

**`SIPHON_DOWNLOAD_ITEMS_SECRET`** = `<string>` (`--secret`)  
The secret phrase configured inside `UniformMigrationService.aspx`.

**`SIPHON_DOWNLOAD_ITEMS_OUTPUTDIR`** = `<path>` (`--outputDir`)  
Local folder the downloaded items are written to.

### Optional parameters

**`SIPHON_DOWNLOAD_ITEMS_DOWNLOADRENDERINGSDATASOURCES`** = `true|false` (`--downloadRenderingsDatasources`)  
Also download the rendering **datasource** items referenced from each page's Shared/Final Renderings layout, even when they live outside `--root`. Recommended, so components keep their content. Default: `false`.

**`SIPHON_DOWNLOAD_ITEMS_DOWNLOADFIELDSREFERENCES`** = `true|false` (`--downloadFieldsReferences`)  
Also download items referenced from **fields** (links, droptrees, multilists, …) when they fall outside `--root`. Recommended, so references resolve during migration. Default: `false`.

**`SIPHON_DOWNLOAD_ITEMS_DEVICEID`** = `(empty)|<guid>` (`--deviceId`)  
Sitecore device whose layout is read (default: the Default device `{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}`).

**`SIPHON_DOWNLOAD_ITEMS_EXCLUDEITEMSFILE`** = `(empty)|<path>` (`--excludeItemsFile`)  
Skip specific items. Point at a JSON file (or pass its contents) shaped as `{ "item-guid-1": "any comment", ... }` — the keys are the item IDs to exclude.

**`SIPHON_DOWNLOAD_ITEMS_SKIPLOCALDATASOURCES`** = `true|false` (`--skipLocalDatasources`)  
Do not walk a page's **local datasource folder**: a child of a page named `_local` or `data` is not downloaded, and neither is anything below it. Only a child of a **page** (an item that has a layout) is skipped — a shared, site-wide `Data` folder that does not sit under a page is still downloaded in full. Default: `false`. See [skipping page-local datasources](https://docs.uniform.app/docs/guides/migration/sitecore/download#skipping-page-local-datasource-folders).

**`SIPHON_DOWNLOAD_ITEMS_LOCALDATASOURCEFOLDERNAMES`** = `(empty)|<string>` (`--localDatasourceFolderNames`)  
Replace the folder names `--skipLocalDatasources` looks for. Compared case-insensitively; delimiters `,;|`. A value here **replaces** the defaults rather than adding to them. Default: `_local|data`.

**`SIPHON_DOWNLOAD_ITEMS_FORCE`** = `true|false` (`--force`)  
Re-download items that already exist locally instead of skipping them. Use it to refresh a stale download. Default: `false`.

**`SIPHON_DOWNLOAD_ITEMS_INPUTFILE`** = `(empty)|<path>` (`--inputFile`)  
Download only the items listed in a `_missing-datasources-and-media-{timestamp}.json` file (produced by an earlier run) — used to fetch items that were reported missing.

**`SIPHON_DOWNLOAD_ITEMS_UNSUCCESSFULDOWNLOADSFILENAME`** = `(empty)|<string>` (`--unsuccessfulDownloadsFileName`)  
Override the name of the `_unsuccessful_downloads-{timestamp}.json` report file.

**`SIPHON_DOWNLOAD_ITEMS_DISABLEKEEPALIVECHECK`** = `true|false` (`--disableKeepAliveCheck`)  
Stop polling `/sitecore/service/keepalive.aspx`. Use it when that endpoint is restricted or removed on the target instance. Default: `false`.

**`SIPHON_DOWNLOAD_ITEMS_THREADS`** = `(empty)|<int>` (`--threads`)  
Number of parallel download workers. Default: `4`.

**`SIPHON_DOWNLOAD_ITEMS_STATSINTERVALSECONDS`** = `(empty)|<int>` (`--statsIntervalSeconds`)  
How often, in seconds, to log the throughput report and re-read `--tuningFile`. `0` turns both off. Default: `10`. See [monitoring and tuning a running download](#monitoring-and-tuning-a-running-download).

**`SIPHON_DOWNLOAD_ITEMS_TUNINGFILE`** = `(empty)|<path>` (`--tuningFile`)  
The JSON file re-read while the run is in flight, so the thread count can be changed — or the run [paused](#pausing-a-download) — without restarting it. Default: `settings-override.json` in the current directory.

**`SIPHON_DOWNLOAD_ITEMS_AUTOTUNE`** = `true|false` (`--autotune`)  
Search for a good thread count while the run is in flight: add or remove one worker every `--autotuneinterval`, keep the change if throughput improved, revert it otherwise. Default: `false`.

**`SIPHON_DOWNLOAD_ITEMS_AUTOTUNEMAX`** = `(empty)|<int>` (`--autotunemax`)  
Upper bound for `--autotune`; the lower bound is always `1`. Default: `32`.

**`SIPHON_DOWNLOAD_ITEMS_AUTOTUNEINTERVAL`** = `(empty)|<int>` (`--autotuneinterval`)  
Seconds between `--autotune` steps — each step is measured over exactly one of these. Default: `10`.

**`SIPHON_DOWNLOAD_ITEMS_RETRYCOUNT`** = `(empty)|<int>` (`--RetryCount`)  
Retry attempts for each item before it is recorded as unsuccessful. Default: `3`.

**`SIPHON_DOWNLOAD_ITEMS_TIMEOUT`** = `(empty)|<int>` ms (`--Timeout`)  
Timeout for each request to the migration service, in milliseconds. Default: `60000` (must be > 100).

**`SIPHON_DOWNLOAD_ITEMS_STOPONERROR`** = `true|false` (`--stopOnError`)  
Abort the download on the first error instead of collecting errors and continuing. Default: `false`.

**`SIPHON_DOWNLOAD_ITEMS_SKIPCLEARINGFOLDERS`** = `true|false` (`--skipClearingFolders`)  
Do not clear the download output folders before writing — use it when combining several runs into one folder. Default: `false`.

Plus the [common options](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference#common-options) (`--languages`, logging).

> **Download `en` first:**
>
> If you restrict `--languages`, the first pass must be `en` and it must cover `/sitecore/templates` and `/sitecore/system/Languages` — those items normally have an `en` version only, and without them the migration has no content types and no locales to build from. This holds even when all of your content lives in other languages. See [Always download `en` first](https://docs.uniform.app/docs/guides/migration/sitecore/download#always-download-en-first).

> **`--skipLocalDatasources` prunes the tree walk only:**
>
> A datasource that lives inside a skipped folder but is pointed at **explicitly** — from a rendering's datasource or from a reference field — is still downloaded when `--downloadRenderingsDatasources` or `--downloadFieldsReferences` is on. Those switches follow an item **id**, and an item's path is not known until it has been fetched. So the switch removes the bulk of a page-local subtree (everything nothing points at), not every item in it.

> **Resuming a download:**
>
> `download-items` is resumable. Items already present on disk (`{outputDir}/{language}/{item-id}.json`) are skipped, so an interrupted download can simply be re-run. Note that content created _between_ attempts may not be picked up automatically — pass `--force` to refresh everything.

---

## download-media

Download Sitecore media library items and their binary files. Run it after `download-items` so `--skipUnusedMedia` can limit the download to the media your content actually references.

```powershell
PS C:\migration> siphon download-media `
  --database web `
  --uniformServiceUrl https://cd.example.com/layouts/UniformMigrationService.aspx `
  --secret my_secret `
  --outputDir "C:\migration\data\items" `
  --mediaRootDir "C:\migration\data\media" `
  --skipUnusedMedia
```

### Required parameters

**`SIPHON_DOWNLOAD_MEDIA_DATABASE`** = `<string>` (`--database`)  
The Sitecore database — normally **`web`** (the publishing target); match whatever you used for `download-items`. See [`download-items`](#download-items) for when a second `master` pass makes sense.

**`SIPHON_DOWNLOAD_MEDIA_UNIFORMSERVICEURL`** = `<url>` (`--uniformServiceUrl`)  
URL to the deployed `UniformMigrationService.aspx` on the **CD** — same endpoint as [`download-items`](#download-items).

**`SIPHON_DOWNLOAD_MEDIA_SECRET`** = `<string>` (`--secret`)  
Service secret.

**`SIPHON_DOWNLOAD_MEDIA_OUTPUTDIR`** = `<path>` (`--outputDir`)  
Destination folder for the media item JSON (typically the same items folder used by `download-items`, so references line up).

### Optional parameters

**`SIPHON_DOWNLOAD_MEDIA_SKIPUNUSEDMEDIA`** = `true|false` (`--skipUnusedMedia`)  
Download only the media files referenced by previously-downloaded content. Usually a large reduction in volume. Cannot be combined with `--root`. Default: `false`.

**`SIPHON_DOWNLOAD_MEDIA_ROOT`** = `(empty)|<guid>` (`--root`)  
Media library subtree to download from (or, with `--skipUnusedMedia` off, to filter by). Not supported together with `--skipUnusedMedia`.

**`SIPHON_DOWNLOAD_MEDIA_MEDIAROOTDIR`** = `(empty)|<path>` (`--mediaRootDir`)  
Folder the binary files are written to (default: `{outputDir}/media`). Keep it separate from the items folder.

**`SIPHON_DOWNLOAD_MEDIA_MEDIAPREFIX`** = `(empty)|<string>` (`--mediaPrefix`)  
The media URL prefix to recognise/strip (default `/-/media/`). Change it only if your instance serves media under a different prefix.

**`SIPHON_DOWNLOAD_MEDIA_EXCLUDEUNSUPPORTEDMIMETYPES`** = `true|false` (`--excludeUnsupportedMimeTypes`)  
Skip files whose MIME type Uniform does not yet support, instead of downloading them. Default: `false`.

**`SIPHON_DOWNLOAD_MEDIA_DEVICEID`** = `(empty)|<guid>` (`--deviceId`)  
Sitecore device whose layout is read (default: the Default device `{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}`).

**`SIPHON_DOWNLOAD_MEDIA_EXCLUDEITEMSFILE`** = `(empty)|<path>` (`--excludeItemsFile`)  
Exclude specific media items via a JSON file shaped as `{ "item-guid": "comment", ... }`.

**`SIPHON_DOWNLOAD_MEDIA_FORCE`** = `true|false` (`--force`)  
Re-download media that already exists locally. Default: `false`.

**`SIPHON_DOWNLOAD_MEDIA_INPUTFILE`** = `(empty)|<path>` (`--inputFile`)  
Fetch only the media listed in a `_missing-…json` file from an earlier run.

**`SIPHON_DOWNLOAD_MEDIA_UNSUCCESSFULDOWNLOADSFILENAME`** = `(empty)|<string>` (`--unsuccessfulDownloadsFileName`)  
Override the name of the unsuccessful-downloads report file.

**`SIPHON_DOWNLOAD_MEDIA_DISABLEKEEPALIVECHECK`** = `true|false` (`--disableKeepAliveCheck`)  
Stop polling `keepalive.aspx`. Default: `false`.

**`SIPHON_DOWNLOAD_MEDIA_THREADS`** = `(empty)|<int>` (`--threads`)  
Number of parallel media-download workers. Default: `4`.

**`SIPHON_DOWNLOAD_MEDIA_STATSINTERVALSECONDS`** = `(empty)|<int>` (`--statsIntervalSeconds`)  
How often, in seconds, to log the throughput report and re-read `--tuningFile`. `0` turns both off. Default: `10`. See [monitoring and tuning a running download](#monitoring-and-tuning-a-running-download).

**`SIPHON_DOWNLOAD_MEDIA_TUNINGFILE`** = `(empty)|<path>` (`--tuningFile`)  
The JSON file re-read while the run is in flight, so the thread count can be changed — or the run [paused](#pausing-a-download) — without restarting it. Default: `settings-override.json` in the current directory.

**`SIPHON_DOWNLOAD_MEDIA_AUTOTUNE`** = `true|false` (`--autotune`)  
Search for a good thread count while the run is in flight: add or remove one worker every `--autotuneinterval`, keep the change if throughput improved, revert it otherwise. Default: `false`.

**`SIPHON_DOWNLOAD_MEDIA_AUTOTUNEMAX`** = `(empty)|<int>` (`--autotunemax`)  
Upper bound for `--autotune`; the lower bound is always `1`. Default: `32`.

**`SIPHON_DOWNLOAD_MEDIA_AUTOTUNEINTERVAL`** = `(empty)|<int>` (`--autotuneinterval`)  
Seconds between `--autotune` steps — each step is measured over exactly one of these. Default: `10`.

**`SIPHON_DOWNLOAD_MEDIA_RETRYCOUNT`** = `(empty)|<int>` (`--RetryCount`)  
Retry attempts for each media file before it is recorded as unsuccessful. Default: `3`.

**`SIPHON_DOWNLOAD_MEDIA_TIMEOUT`** = `(empty)|<int>` ms (`--Timeout`)  
Timeout for each request to the migration service, in milliseconds. Default: `60000` (must be > 100).

**`SIPHON_DOWNLOAD_MEDIA_STOPONERROR`** = `true|false` (`--stopOnError`)  
Abort the download on the first error instead of collecting errors and continuing. Default: `false`.

**`SIPHON_DOWNLOAD_MEDIA_SKIPCLEARINGFOLDERS`** = `true|false` (`--skipClearingFolders`)  
Do not clear the download output folders before writing — use it when combining several runs into one folder. Default: `false`.

Plus the [common options](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference#common-options) (`--languages`, logging).

> **Versioned media:**
>
> To download **unversioned** media files, set `Media.RequestProtection.Enabled` to `false` and clear the cache via `/sitecore/admin/cache.aspx` on the source instance first.

---

## download-presentation

Download the Sitecore presentation layer — each page's rendered markup captured in **Experience Editor mode** (its special markup carries the hierarchical placeholder/rendering structure Siphon needs). This lets the migration reconstruct the **component hierarchy** of compositions and, experimentally, code-generate React components. It is optional: skip it for a content-only migration.

The command logs into Sitecore with Playwright and finds every previously-downloaded item under `--startPath` that has presentation assigned (a non-empty Layout or Page Design field). Install Playwright first with `.\playwright.ps1 install` (see the [installation guide](https://docs.uniform.app/docs/guides/migration/sitecore/install#3-install-playwright)).

```powershell
PS C:\migration> siphon download-presentation `
  --inputDir C:\migration\data\items `
  --outputDir C:\migration\data\presentation `
  --host https://sc.dev `
  --productionHost https://www.example.com `
  --startPath /sitecore/content/Habitat/Home `
  --loginUrl https://sc.dev/sitecore/login `
  --username admin `
  --password b `
  --database web
```

### Required parameters

**`SIPHON_DOWNLOAD_PRESENTATION_INPUTDIR`** = `<path>` (`--inputDir`)  
Folder of the previously downloaded Sitecore items (the `download-items` output).

**`SIPHON_DOWNLOAD_PRESENTATION_OUTPUTDIR`** = `<path>` (`--outputDir`)  
Destination for the presentation files (`index.html` + `index.json` per page, under `<outputDir>/<language>/<item path>/`). With `--assets`, downloaded assets go into `<outputDir>/<language>/_assets/` instead of the page folders.

**`SIPHON_DOWNLOAD_PRESENTATION_HOST`** = `<url>` (`--host`)  
The Sitecore **CM** URL, e.g. `https://cm.example.com/`. A bare host is upgraded to `https://` (or `http://` for `localhost`/IPs). This must be a Content Management instance: the command captures Experience Editor markup (`?sc_mode=edit`), which a CD cannot render — point it at a CD and every page comes back a server error page instead of content. Note this is the opposite of `--uniformServiceUrl`, which belongs on the CD.

**`SIPHON_DOWNLOAD_PRESENTATION_PRODUCTIONHOST`** = `<url>` (`--productionHost`)  
The production/CD host used for production-HTML comparison, e.g. when `--host` is `https://cm.example.com` the production host is `https://www.example.com`.

**`SIPHON_DOWNLOAD_PRESENTATION_STARTPATH`** = `<path>` (`--startPath`)  
Site root item, e.g. `/sitecore/content/Habitat/Home`.

**`SIPHON_DOWNLOAD_PRESENTATION_LOGINURL`** = `<url>` (`--loginUrl`)  
Sitecore login page used, with `--username`/`--password`, to authenticate the Playwright session.

**`SIPHON_DOWNLOAD_PRESENTATION_USERNAME`** = `<string>` (`--username`)  
Sitecore user for the Experience Editor session.

**`SIPHON_DOWNLOAD_PRESENTATION_PASSWORD`** = `<string>` (`--password`)  
Password for that user.

**`SIPHON_DOWNLOAD_PRESENTATION_DATABASE`** = `<string>` (`--database`)  
The Sitecore database — normally **`web`** (the publishing target), matching your `download-items` run; defaults to the logged-in user's context database. See [`download-items`](#download-items) for the `master`/draft-content note.

### Optional parameters

**`SIPHON_DOWNLOAD_PRESENTATION_VIRTUALPATH`** = `(empty)|<path>` (`--virtualPath`)  
Capture only a sub-tree of the site, e.g. `/about-habitat`.

**`SIPHON_DOWNLOAD_PRESENTATION_SITENAME`** = `(empty)|<string>` (`--siteName`)  
The Sitecore `<site>` name (from `<sites>` in showconfig, or the SXA Site item) to render under in Experience Editor, e.g. `website`.

**`SIPHON_DOWNLOAD_PRESENTATION_ASSETHOST`** = `(empty)|<url>` (`--assetHost`)  
Asset host used to make media URLs absolute, e.g. `https://sc.blob.core.windows.net/assets`.

**`SIPHON_DOWNLOAD_PRESENTATION_ASSETS`** = `true|false` (`--assets`)  
Also download the presentation assets (stylesheets, scripts, images) referenced in the production HTML of each page, plus the site's `sitemap.xml`. Default: `false`.

Assets are per-site, not per-page, so they are written once into `<outputDir>/<language>/_assets/`, kept out of the page folders. Inside `_assets` every file keeps its site-root-relative path — an asset served from `https://www.example.com/styles/main.css` lands in `_assets/styles/main.css` — so the folder can be published as the web root of a static mirror as-is. Sitecore media (`/-/media/...`) is skipped here; use [`download-media`](#download-media) for it, and files already on disk are not re-downloaded.

**`SIPHON_DOWNLOAD_PRESENTATION_SKIPDESCENDANTS`** = `true|false` (`--skipDescendants`)  
Capture only the matched items themselves, not their descendants. Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_EXCLUDETEMPLATES`** = `(empty)|<list>` (`--excludeTemplates`)  
Pipe-separated list of templates (names or IDs) to skip downloading.

**`SIPHON_DOWNLOAD_PRESENTATION_RESTOREINITIALCOOKIES`** = `true|false` (`--restoreInitialCookies`)  
After a successful login, save the cookies and restore them before every subsequent request (helps with instances that rotate session state). Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_FORCE`** = `true|false` (`--force`)  
Re-download the Experience Editor HTML and regenerate the `index.json` files even if they are already cached. Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_WHATIF`** = `true|false` (`--whatIf`)  
Dry run: make no web requests, only log the intended actions. Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_DISABLEKEEPALIVECHECK`** = `true|false` (`--disableKeepAliveCheck`)  
Stop polling `keepalive.aspx`. Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_THREADS`** = `(empty)|<int>` (`--threads`)  
Number of pages captured in parallel. Each worker logs in and drives a browser session of its own, and loads that page's production HTML through a second one. Default: `4`.

**`SIPHON_DOWNLOAD_PRESENTATION_STATSINTERVALSECONDS`** = `(empty)|<int>` (`--statsIntervalSeconds`)  
How often, in seconds, to log the throughput report and re-read `--tuningFile`. `0` turns both off. Default: `10`. See [monitoring and tuning a running download](#monitoring-and-tuning-a-running-download).

**`SIPHON_DOWNLOAD_PRESENTATION_TUNINGFILE`** = `(empty)|<path>` (`--tuningFile`)  
The JSON file re-read while the run is in flight, so the thread count can be changed — or the run [paused](#pausing-a-download) — without restarting it. Default: `settings-override.json` in the current directory.

**`SIPHON_DOWNLOAD_PRESENTATION_AUTOTUNE`** = `true|false` (`--autotune`)  
Search for a good thread count while the run is in flight: add or remove one browser session every `--autotuneinterval`, keep the change if throughput improved, revert it otherwise. Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_AUTOTUNEMAX`** = `(empty)|<int>` (`--autotunemax`)  
Upper bound for `--autotune`; the lower bound is always `1`. Default: `32`. Worth lowering here — every session is a browser context against the CM, and every captured page also loads the production page.

**`SIPHON_DOWNLOAD_PRESENTATION_AUTOTUNEINTERVAL`** = `(empty)|<int>` (`--autotuneinterval`)  
Seconds between `--autotune` steps — each step is measured over exactly one of these. Default: `10`.

**`SIPHON_DOWNLOAD_PRESENTATION_RETRYCOUNT`** = `(empty)|<int>` (`--RetryCount`)  
Retry attempts for each page before it is recorded as unsuccessful. Default: `3`.

**`SIPHON_DOWNLOAD_PRESENTATION_TIMEOUT`** = `(empty)|<int>` ms (`--Timeout`)  
Timeout for each request to Sitecore, in milliseconds. Default: `60000` (must be > 100).

**`SIPHON_DOWNLOAD_PRESENTATION_STOPONERROR`** = `true|false` (`--stopOnError`)  
Abort the download on the first error instead of collecting errors and continuing. Default: `false`.

**`SIPHON_DOWNLOAD_PRESENTATION_SKIPCLEARINGFOLDERS`** = `true|false` (`--skipClearingFolders`)  
Do not clear the output folders before writing — use it when combining several runs into one folder. Default: `false`.

Plus the [common options](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference#common-options) (`--languages`, logging).

---

## Monitoring and tuning a running download

A download of a large tree runs for hours, so every `Siphon.Download.exe` command reports what it is achieving while it works, and all three — `download-items`, `download-media` and `download-presentation` — can be re-tuned, or paused and resumed, **without being restarted**.

### Throughput reporting

Every `--statsIntervalSeconds` (default `10`) a download command logs one line: what the last window transferred, what the run has averaged since it started, and how many workers are running. A final line reports the run as a whole.

```
[stats] last 10s: 152 items, 673.8 KB, 336.2 KB/s, 76 items/s | total 00:16: 725 items, 3.3 MB, 213.5 KB/s, 45 items/s | threads 4
[stats] finished in 00:21: 1,111 items, 5 MB, 241.2 KB/s, 52 items/s
```

Only what was actually fetched is counted. An item, media file or page that was already on disk and skipped is not a download, so a resumed run reports the speed of the work it is really doing rather than the speed of skipping files it already has.

### Changing the thread count while the download runs

On the same interval, a download command re-reads a small JSON file — `settings-override.json` in the **current directory** by default, or wherever `--tuningFile` points:

```json
{
  "threads": 8
}
```

All three commands resize their worker pool within one interval. Workers are added straight away; when the number goes down, workers stop **after the item they are on** — never in the middle of a request — so nothing in flight is lost. Delete the file, or just the `"threads"` key, to hand control back to `--threads`. Any other keys in the file are left alone.

> **Tune against the [stats] lines:**
>
> The right thread count depends on the source instance rather than on the size of the tree: too few and the download crawls, too many and Sitecore starts timing out. Raise the number in the tuning file while you watch the `[stats]` lines, and back it off as soon as throughput stops improving or errors start appearing — an eight-hour download no longer has to be killed and restarted to try a different value.

> **What a thread costs on `download-presentation`:**
>
> A worker here is not a socket, it is a **browser session**: it logs into Sitecore separately, holds an Experience Editor session of its own, and loads each page's production HTML through a second browser context. All of them share one Chromium, but `--threads 4` still means four concurrent Experience Editor renders on the CM and four concurrent hits on the **production** site. Start low, watch the `[stats]` line, and keep `--autotunemax` modest on this command — a production host behind a WAF or rate limiter will notice.

### Pausing a download

`"pause": true` in the same file holds the run where it is:

```json
{
  "pause": true,
  "threads": 8
}
```

Workers finish the item they are already fetching and then park — nothing new is requested from Sitecore until the pause is lifted. The run itself stays alive, with its queue, its index and everything already on disk intact (on `download-presentation` the workers keep their browser sessions too, so resuming costs no logins), so this is how you get out of the way of a content freeze, a backup or a CM restart without throwing away hours of progress.

While a run is paused the file is re-read **every second** instead of once per `--statsIntervalSeconds`, so it picks up again promptly. Removing the `"pause"` key, setting it to `false`, or deleting the file altogether all resume the run.

A pause is otherwise silent — no downloads, no errors — so the run keeps saying that it is holding, once per `--statsIntervalSeconds`, and the throughput report marks the same windows. A paused download is never left looking like a hung one:

```
[stats] last 10s: 0 items, 0 B, 0 B/s, 0 items/s | total 02:14: 580 items, 2.7 MB, 198.1 KB/s, 41 items/s | threads 4 | PAUSED
[tuning] still PAUSED after 00:02:00 - nothing is being downloaded. Remove "pause" from settings-override.json, set it to false, or delete the file to resume.
```

A `"threads"` value alongside the pause is applied straight away, so `{ "pause": true, "threads": 8 }` holds the run **and** brings it back at eight workers. `--autotune` sits out a paused window and takes a fresh baseline once the download is moving again.

> **A pause holds work, not a finished run:**
>
> Pausing parks the workers, but it cannot keep a run that has nothing left to do from ending: if the queue drains while the download is paused, the command completes normally. Pause a download that is still working, not the last seconds of one.

### Letting Siphon find the thread count

`--autotune` runs that search for you. Every `--autotuneinterval` seconds (default `10`) it moves the pool by one worker, measures the next window, and keeps the change only if throughput improved by more than a small noise margin (3%) — otherwise it reverts and tries the other direction next. It never goes below `1` or above `--autotunemax` (default `32`), and it sits out the window a pause was lifted in, which is part hold and part work and would read as a collapse in throughput.

```
[autotune] baseline at 2 thread(s): 246.5 KB/s.
[autotune] 3 thread(s): 298.6 KB/s beats 246.5 KB/s at 2 thread(s) - keeping it.
[autotune] 4 thread(s): 369.8 KB/s beats 298.6 KB/s at 3 thread(s) - keeping it.
[autotune] 5 thread(s): 471.6 KB/s beats 369.8 KB/s at 4 thread(s) - keeping it.
[autotune] 4 thread(s): 378.3 KB/s is no better than 471.6 KB/s at 5 thread(s) - reverting to 5 and trying the other direction next.
```

The count it is trying is written back into the tuning file, so that file always shows the value in force. Editing the file yourself still wins — the tuner notices the change, re-measures from your number and carries on from there.

> **Which value applies:**
>
> `--threads` is the starting point. The tuning file overrides it for as long as the file has a `"threads"` key, and `--autotune` writes into that same file. A run with no tuning file and no `--autotune` behaves exactly as before: the pool stays at `--threads` from beginning to end.

---

## Utility commands

These auxiliary `Siphon.Download.exe` commands are not part of the normal download flow but are handy for troubleshooting and for reprocessing already-downloaded data. Each also accepts the [common options](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference#common-options).

### debug-get-item

Fetch a single downloaded item and print its JSON — useful when diagnosing why a specific item migrated the way it did.

**`SIPHON_DEBUG_GET_ITEM_INPUTDIR`** = `<path>` (`--inputDir`)  
required; the downloaded items folder.

**`SIPHON_DEBUG_GET_ITEM_ITEM`** = `<guid>|<path>` (`--item`)  
required; the item to dump, as an ID (`{110D559F-…}`) or a path (`/sitecore/content/Home`).

**`SIPHON_DEBUG_GET_ITEM_LANGUAGE`** = `(empty)|<string>` (`--language`)  
optional; language version to read (defaults to `--defaultLanguage`).

**`SIPHON_DEBUG_GET_ITEM_THREADS`** = `(empty)|<int>` (`--threads`)  
Worker count (rarely relevant for a single item). Default: `4`.

**`SIPHON_DEBUG_GET_ITEM_RETRYCOUNT`** = `(empty)|<int>` (`--RetryCount`)  
Retry attempts for each fetch before it is recorded as unsuccessful. Default: `3`.

**`SIPHON_DEBUG_GET_ITEM_TIMEOUT`** = `(empty)|<int>` ms (`--Timeout`)  
Timeout for the request, in milliseconds. Default: `60000` (must be > 100).

**`SIPHON_DEBUG_GET_ITEM_STOPONERROR`** = `true|false` (`--stopOnError`)  
Abort the run on the first error instead of collecting errors and continuing. Default: `false`.

**`SIPHON_DEBUG_GET_ITEM_SKIPCLEARINGFOLDERS`** = `true|false` (`--skipClearingFolders`)  
Do not clear the output folders before writing — use it when combining several runs into one folder. Default: `false`.

### reindex

Rebuild the item index (`index.json`) from the items already on disk — run it after manually adding, removing, or editing downloaded item files so later stages see a consistent index. (The media index is **not** rebuilt.) Also available from `Siphon.Migration.exe`.

**`SIPHON_REINDEX_INPUTDIR`** = `<path>` (`--inputDir`)  
required; the downloaded items folder.

**`SIPHON_REINDEX_THREADS`** = `(empty)|<int>` (`--threads`)  
Number of parallel reindex workers. Default: `4`.

### rebuild-presentation

Regenerate the presentation `index.json` files from the cached `index.html` files **without re-downloading them from Sitecore** — useful after a layout-parser change to reprocess captured markup quickly.

`--productionHost` is the exception: the command requests every page from the production site again to derive the component selectors, so a re-run still costs a full crawl. Set `--useCachedProductionHtml` to reuse the `prod.html` saved beside each `index.html` instead and make the whole re-run offline — the fast loop when you are iterating on selector generation rather than on the capture itself.

**`SIPHON_REBUILD_PRESENTATION_INPUTDIR`** = `<path>` (`--inputDir`)  
required; the downloaded items folder.

**`SIPHON_REBUILD_PRESENTATION_PRESENTATIONDIR`** = `<path>` (`--presentationDir`)  
required; the folder holding the cached `index.html` files.

**`SIPHON_REBUILD_PRESENTATION_STARTPATH`** = `<path>` (`--startPath`)  
required; the site root item.

**`SIPHON_REBUILD_PRESENTATION_OUTPUTDIR`** = `(empty)|<path>` (`--outputDir`)  
where the regenerated `index.json` files go (defaults to `--presentationDir`).

**`SIPHON_REBUILD_PRESENTATION_PRODUCTIONHOST`** = `(empty)|<url>` (`--productionHost`)  
production/CD host used for the component-HTML comparison.

**`SIPHON_REBUILD_PRESENTATION_USECACHEDPRODUCTIONHTML`** = `true|false` (`--useCachedProductionHtml`)  
Reuse the `prod.html` already saved next to each `index.html` instead of requesting the page from `--productionHost` again — turns a re-run into an offline re-parse. A cached copy that is not a real page (an intermittent outage response, an empty body) is requested again anyway, so a bad capture is never frozen in; a cached sign-in wall counts as a hit, because requesting it anonymously returns the same wall every time and that page simply yields no selectors. Default: `false`.

**`SIPHON_REBUILD_PRESENTATION_ASSETHOST`** = `(empty)|<url>` (`--assetHost`)  
asset host used to make media URLs absolute.

**`SIPHON_REBUILD_PRESENTATION_SKIPEXISTING`** = `true|false` (`--skipExisting`)  
skip pages whose `index.json` already exists. Default: `false`.

**`SIPHON_REBUILD_PRESENTATION_THREADS`** = `(empty)|<int>` (`--threads`)  
Number of parallel page workers. Default: `4`.

**`SIPHON_REBUILD_PRESENTATION_RETRYCOUNT`** = `(empty)|<int>` (`--RetryCount`)  
Retry attempts for each page before it is recorded as unsuccessful. Default: `3`.

**`SIPHON_REBUILD_PRESENTATION_TIMEOUT`** = `(empty)|<int>` ms (`--Timeout`)  
Timeout for each request, in milliseconds. Default: `60000` (must be > 100).

**`SIPHON_REBUILD_PRESENTATION_STOPONERROR`** = `true|false` (`--stopOnError`)  
Abort the run on the first error instead of collecting errors and continuing. Default: `false`.

**`SIPHON_REBUILD_PRESENTATION_SKIPCLEARINGFOLDERS`** = `true|false` (`--skipClearingFolders`)  
Do not clear the output folders before writing — use it when combining several runs into one folder. Default: `false`.

---

## File structure after download

When all downloads complete, your work directory holds the raw Sitecore export:

```
C:\migration\
├── items\
│   ├── _languages.json                 # Available languages
│   ├── .folders.json                   # The --itemFolders rule set this export was written with
│   └── {folder}\                       # content | media | core | other — see --itemFolders
│       └── {language}\
│           ├── {0-f}\
│           |   └── {0-f}\
│           |       └── {item-id}.json  # Individual items
│           ├── _index.json             # This folder's index, for this language
│           └── _map.json               # Path → id lookup for the same
├── media\
│   └── ...Binary media files
└── presentation\
    ├── test-results                    # For azure devops reporting tool
    └── {language}\
        ├── {first-level-children}\
        │   ├── {second-level-children}\
        │   │   ├── ....
        │   │   ├── index.html          # page raw output via Experience Editor
        │   │   ├── index.json          # page structure parsed from index.html
        │   │   └── prod.html           # page raw output from production website
        │   ├── index.html
        │   ├── index.json
        │   └── prod.html
        ├── index.html
        ├── index.json
        └── prod.html    
```

Each item folder keeps **its own index**, so the commands that write different trees no longer contend over one file — see [`--itemFolders`](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference#item-folders) for the rules, and for how to read or convert an export that predates the split.

An export downloaded before `--itemFolders` existed has no `{folder}` level — items sit directly under `items\{language}\` — and is still read exactly as it was. Nothing needs converting.

## Related pages

- [Common reference](https://docs.uniform.app/docs/guides/migration/sitecore/common-reference) — executables, configuration, environment variables, logging, and the common options.
- [Migrate commands reference](https://docs.uniform.app/docs/guides/migration/sitecore/migrate-reference) — the transform commands (`uniform-canvas`, `uniform-content`).
- [Push commands reference](https://docs.uniform.app/docs/guides/migration/sitecore/push-reference) — the load-phase commands.
- [Sitecore migration walkthrough](https://docs.uniform.app/docs/guides/migration/sitecore/walkthrough) — the end-to-end, explained migration process.