Downloading Indexes with BitTorrent

Gateways that publish shared indexes offer each band as a torrent, so you don't need a gateway to take a copy. Any BitTorrent client can download one, and peers don't charge for what they send. This guide downloads a band from turbo-gateway.com with qBittorrent and with aria2 on the command line, then checks it against the publisher's signature. To read single IDs rather than whole bands, see Reading Shared Indexes.

The torrent only delivers the bytes. What makes them trustworthy is the signed publication. Always finish with the check.

What You Need

Disk. About 21 GB for all five bands turbo-gateway.com publishes, or 8 MB to try the tip band. aria2 needs up to 1 GB more per band while it downloads (see below).
Network. Nothing to open to download: your client connects out to the peers. To seed well, forward or open your client's listening port, so that peers can connect to you as well. Some networks and hosting providers block or restrict BitTorrent traffic.

Find the Torrents

A publisher lists its bands in one signed JSON document at /ar-io/indexes. Each band that is seeded has a torrent entry, and its magnet link is part of what the publisher signed:

curl -s https://turbo-gateway.com/ar-io/indexes \
  | jq -r '.indexes[].bands[] | "\(.id)\n\(.torrent.magnet)\n"'

b0-h2010500-tip-20260930
magnet:?xt=urn:btih:d140717762cfa2d7696d7b8b907f8e1a5d05c46b&xt=urn:btmh:1220b47ad0895458ab839a0a940dedfc27749aee0543f7fe94d182594dec9d7d98ca&dn=6cd021a690e25fac&tr=http%3A%2F%2F167.235.37.218%3A6969%2Fannounce

b1-h1950000-tip-20260929
magnet:?xt=urn:btih:658d73729ff9da45323a679c46bc2b499dd76b54&xt=urn:btmh:1220f6ea7f0ed9613865001b75b74b1ece4110ea621e16cdc63cd5373d3bf39741b4&dn=0a27456b192621a7&tr=http%3A%2F%2F167.235.37.218%3A6969%2Fannounce
...

The band id shows the block heights it covers: b0-h2010500-tip runs from height 2,010,500 to the chain tip, and b4-h0-1349999 covers the start of the chain. turbo-gateway.com publishes five bands, about 21 GB in all. The tip band is small (8 MB) and rebuilt often. The older bands are 2 to 8 GB and rarely change. Start with b0 to try it out.

The magnets on this page were current when it was written. Bands are rebuilt and replaced, so take yours from the publication.

Prefer the magnet link. A .torrent file, served at the band's torrentUrl, is not signed. If you use one, check that the client shows the same info hash as the band's signed infohashV1.

Download with qBittorrent

Add the Magnet Link

In qBittorrent, choose File → Add Torrent Link, paste the band's magnet link, and click Download. Pick a save folder when asked.

The client first fetches the torrent's metadata from peers and checks it against the info hash in the link. Then it downloads the files.

Know What You Are Seeing

Name. The torrent is named with 16 hex characters, such as 6cd021a690e25fac for b0, not with the band id. The name is derived from the band's files, so two gateways publishing the same bytes share one swarm. It is also the name of the folder the files are saved in.
Info hash. The General tab shows Info Hash v1 and Info Hash v2. These should match the band's infohashV1 and infohashV2. The torrents are hybrid v1 and v2, so any current client can join.
Size. qBittorrent can show a size far larger than the band. b0 showed about 1 GB, but only 8 MB lands on disk. The difference is BEP 47 pad files, which line each file up with a piece boundary and are never written. The folder holds only the band's real files: manifest.json and 256 partitions, 00.cdb to ff.cdb.
Trackers. The Trackers tab lists the publisher's tracker as Working. It only answers for the publisher's own bands. Peers are also found through the DHT and peer exchange.

Let It Finish

In our test run, b0 finished in under 20 seconds from 4 seeders. The 2 GB b1 band took about 5 minutes from 2 seeders. Speed depends on how many gateways are seeding a band when you download it.

Download with aria2

aria2 downloads a magnet link from the command line and stops once the download finishes:

aria2c --seed-time=0 --bt-save-metadata=false \
  'magnet:?xt=urn:btih:d140717762cfa2d7696d7b8b907f8e1a5d05c46b&xt=urn:btmh:1220b47ad0895458ab839a0a940dedfc27749aee0543f7fe94d182594dec9d7d98ca&dn=6cd021a690e25fac&tr=http%3A%2F%2F167.235.37.218%3A6969%2Fannounce'

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
ece690|OK  |       0B/s|[MEMORY][METADATA]6cd021a690e25fac
710e97|OK  |    12MiB/s|/data/6cd021a690e25fac/00.cdb (513more)

Unlike qBittorrent, aria2 (1.37) writes the pad files, into a hidden .pad/ folder inside the download. Each pads a file up to the next 4 MiB piece boundary, so a band's 257 files can add up to 1 GB. For b0 that was 958 MB of zeros beside 7.9 MB of band. Delete it once the download is done:

rm -r 6cd021a690e25fac/.pad

To seed after downloading, which helps the next person, drop --seed-time=0.

Check the Download

The client checked every piece against the info hash. That proves you got the torrent the link names. It doesn't prove the files are the publisher's index. For that, check the publication's signature against the key the gateway registry holds for the publisher, then check each file's size and SHA-256 against the publication.

This script does all three. It uses Node.js 20 or later and one dependency (npm install json-canonicalize). The signature check is the same one explained in Verify It.

verify-band.mjs

// Verify a band downloaded over BitTorrent against the signed publication.
// Usage: node verify-band.mjs <publisher gateway> <band directory> [registry gateway]
import crypto from "node:crypto";
import { createReadStream } from "node:fs";
import { stat } from "node:fs/promises";
import path from "node:path";
import { canonicalize } from "json-canonicalize";

const [, , publisherUrl, bandDir, registryUrl = "https://vilenarios.com"] = process.argv;

const B58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz";
function base58Decode(s) {
  let n = 0n;
  for (const c of s) n = n * 58n + BigInt(B58.indexOf(c));
  const bytes = [];
  while (n > 0n) {
    bytes.unshift(Number(n % 256n));
    n /= 256n;
  }
  for (const c of s) {
    if (c !== "1") break;
    bytes.unshift(0);
  }
  return new Uint8Array(bytes);
}

async function sha256File(file) {
  const hash = crypto.createHash("sha256");
  for await (const chunk of createReadStream(file)) hash.update(chunk);
  return hash.digest("hex");
}

// 1. The publication, and the key the registry says must have signed it.
const doc = await fetch(`${publisherUrl}/ar-io/indexes`).then((r) => r.json());
const peers = await fetch(`${registryUrl}/ar-io/peers`).then((r) => r.json());
const registered = Object.values(peers.gateways).find((g) => g.wallet === doc.publisher);
if (registered === undefined) throw new Error(`${doc.publisher} is not in the registry`);

const { signature, ...unsigned } = doc;
if (signature.keyId !== registered.observerAddress) throw new Error("not the registered key");
const key = await crypto.subtle.importKey("raw", base58Decode(signature.keyId), { name: "Ed25519" }, false, ["verify"]);
const signed = await crypto.subtle.verify(
  "Ed25519",
  key,
  Buffer.from(signature.sig, "base64"),
  new TextEncoder().encode("ar-io-index-publication/v1\n" + canonicalize(unsigned)),
);
if (!signed) throw new Error("signature does not verify");
console.log(`publication sequence ${doc.sequence} signed by ${signature.keyId}: ok`);

// 2. The band whose files are in bandDir: matched by torrent name, which is
// the directory name a torrent client downloads into.
const bands = doc.indexes.flatMap((index) => index.bands);
const torrentName = (b) => new URLSearchParams(b.torrent?.magnet.split("?")[1]).get("dn");
const band = bands.find((b) => torrentName(b) === path.basename(bandDir));
if (band === undefined) throw new Error(`no band in the publication for ${path.basename(bandDir)}`);
console.log(`band ${band.id}, infohash v1 ${band.torrent.infohashV1}`);

// 3. Every file: size and SHA-256 as signed.
let bad = 0;
for (const file of band.files) {
  const p = path.join(bandDir, file.name);
  const size = await stat(p).then((s) => s.size, () => undefined);
  if (size !== file.size || (await sha256File(p)) !== file.sha256) {
    bad += 1;
    console.log(`MISMATCH ${file.name}`);
  }
}
console.log(bad === 0 ? `all ${band.files.length} files match: ok` : `${bad} files do not match`);
process.exit(bad === 0 ? 0 : 1);

node verify-band.mjs https://turbo-gateway.com ./6cd021a690e25fac

publication sequence 20 signed by 34LYvMptiDvBP5sqfh1oAd6Q4qFsy4PWaZ1HTFmML7h5: ok
band b0-h2010500-tip-20260930, infohash v1 d140717762cfa2d7696d7b8b907f8e1a5d05c46b
all 257 files match: ok

A changed or missing file is reported and the script exits with status 1:

MISMATCH 00.cdb
MISMATCH manifest.json
2 files do not match

The third argument is any gateway to read the registry from. Its /ar-io/peers lists every gateway's wallet and observerAddress. You can also read the registry with the ar.io SDK. The script checks the publication as it is now, so check a band soon after you download it. Once the publisher replaces a band, it drops out of the publication.

What to Do with It

Keep seeding. Leave the torrent running in your client. Every seeder makes the next download faster, for gateways as well as for people.
Look up IDs. A band is a partitioned CDB64 index from data item ID to root transaction. See Look Up One ID to read it.
Let a gateway do it. A gateway can subscribe to a publisher, download and check every band by itself, keep them current, and seed them:
```
./tools/index-swarm-setup --subscribe 34LYvMptiDvBP5sqfh1oAd6Q4qFsy4PWaZ1HTFmML7h5 --torrent --restart
```
See Index Sharing for the full setup.

How is this guide?

Downloading Indexes with BitTorrent

On this page