# Admin (/apis/ar-io-node/admin) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Access several password protected features and functions specific to your ar.io Gateway. # ArNS (/apis/ar-io-node/arns) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Get data from the ar.io Gateway Ar.io Name System # Blocks (/apis/ar-io-node/blocks) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Get current or historical Arweave block information # Chunks (/apis/ar-io-node/chunks) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Upload Arweave data chunks or get existing chunk offset information # Data (/apis/ar-io-node/data) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Core data retrieval operations for accessing transaction and data item content. Supports manifest resolution, range requests, caching, and verification status. These endpoints serve as the primary interface for retrieving data from the Permaweb. # Farcaster Frames (/apis/ar-io-node/farcaster-frames) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Retrieve and interact with Farcaster Frames using Arweave transactions. # Gateway (/apis/ar-io-node/gateway) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Operations related to the ar.io Gateway server itself, including health checks, metrics, and gateway-specific information # Index Querying (/apis/ar-io-node/index-querying) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Get data from the ar.io Gateway index using GQL # ar.io Gateway APIs (/apis/ar-io-node) import { Server, Network, Route, Database, Search, FileText, } from "lucide-react"; The ar.io Gateway is the core software for the ar.io network, serving the essential responsibility of gateways for accessing, caching, and querying data stored on Arweave. It provides a robust, decentralized infrastructure for interacting with the permanent web. ## Core Responsibilities The ar.io Gateway handles fundamental operations for the Arweave ecosystem: - **Data Access** - Retrieve transaction data, files, and metadata from Arweave - **Caching** - Intelligent caching strategies for improved performance and availability - **Data Querying** - Powerful search and indexing capabilities for Arweave data - **ArNS Resolution** - Resolve human-readable names to Arweave transaction IDs - **Network Management** - Coordinate with other gateways in the ar.io network ## Advanced Features Beyond basic gateway functionality, ar.io Gateway includes sophisticated capabilities: - **Parquet Generation** - Convert Arweave data into optimized Parquet format for analytics - **Data Verification** - Cryptographic verification of data integrity and authenticity - **Index Querying** - Advanced search and filtering across Arweave datasets - **Farcaster Frames** - Support for Farcaster protocol integration - **Admin Controls** - Comprehensive gateway management and configuration ## APIs Categories } title="Data Access" description="Retrieve transaction data, files, and metadata from Arweave" href="/apis/ar-io-node/data" /> } title="ArNS Resolution" description="Resolve human-readable names to Arweave transaction IDs" href="/apis/ar-io-node/arns" /> } title="Transactions & Blocks" description="Access transaction details, block information, and network data" href="/apis/ar-io-node/transactions" /> } title="Index Querying" description="Advanced search and filtering capabilities across Arweave data" href="/apis/ar-io-node/index-querying" /> } title="Network & Gateway" description="Gateway status, network information, and peer coordination" href="/apis/ar-io-node/network" /> } title="Admin & Management" description="Gateway configuration, pricing, and administrative controls" href="/apis/ar-io-node/admin" /> ## Get Involved with ar.io Gateways } title="Run a Gateway" description="Join the ar.io network by operating your own gateway and earn rewards" href="/build/run-a-gateway/quick-start" /> } title="Leverage Gateways with Wayfinder" description="Use Wayfinder SDK to access data through the distributed gateway network" href="/sdks/wayfinder" /> } title="Join the Network" description="Learn about the ar.io network and how to participate in the ecosystem" href="https://ar.io/network" /> ## Getting Started 1. **Explore the APIs endpoints** - Review the comprehensive APIs documentation 2. **Test with sample requests** - Try out the interactive examples 3. **Choose your integration approach** - Direct APIs calls or SDK usage 4. **Consider running a gateway** - Contribute to the network infrastructure The ar.io Gateway APIs provide the foundation for building robust, decentralized applications on Arweave with reliable data access and advanced querying capabilities. # Network (/apis/ar-io-node/network) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Get Arweave node info, peers and nework status # Pricing (/apis/ar-io-node/pricing) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Get the price (in winston) for an amount of bytes # Transactions (/apis/ar-io-node/transactions) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Submit a new Arweave transaction or get existing transaction information # Wallets (/apis/ar-io-node/wallets) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Get Arweave wallet balance and last transaction information # APIs Reference (/apis) Explore the REST APIs available in the ar.io ecosystem. Our services are built with a commitment to open source principles, and all repositories are publicly available under AGPL-3 licenses. ## Available Services } title="ar.io Gateway" description="The core gateway software providing access to data on Arweave. Includes data retrieval, ArNS resolution, and network management." href="/apis/ar-io-node" /> } title="Turbo" description="Upload and payment services providing fast, reliable data uploads to Arweave with instant confirmation and transparent pricing." href="/apis/turbo" /> ## ar.io Gateway APIs The ar.io Gateway serves as the primary interface for accessing Arweave data through the ar.io network. Key endpoints include: - **Data Access** - Retrieve transaction data and files from Arweave - **ArNS Resolution** - Resolve human-readable names to Arweave transaction IDs - **Network Information** - Query gateway health, pricing, and network status - **Transaction Management** - Submit and track transactions - **Admin Functions** - Gateway administration and configuration ## Turbo APIs Turbo provides high-performance upload services for the Arweave network with additional features: - **Data Upload** - Fast, reliable uploads with instant confirmation - **Payment Processing** - Transparent pricing and payment management - **Upload Tracking** - Monitor upload status and metadata - **Credit Management** - Handle payment credits and billing ## Open Source Commitment We believe strongly in open source development. All ar.io services are: - **Publicly Available** - Source code is open and accessible - **AGPL-3 Licensed** - Ensuring software freedom and transparency - **Community Driven** - Built with input from the developer community - **Auditable** - Code can be reviewed and verified by anyone ## Getting Started 1. **Choose your service** - Select the APIs that fit your needs 2. **Review the documentation** - Each service has comprehensive APIs documentation 3. **Test endpoints** - Use the interactive examples to explore functionality 4. **Integrate** - Implement the APIs in your applications For SDK alternatives to these REST APIs, visit our [SDK documentation](/sdks). ## Explore More } title="SDK Documentation" description="Use our TypeScript SDKs for easier integration and development" href="/sdks" /> } title="Quick Start - Upload" description="Start uploading data to Arweave with our upload guides" href="/build/upload" /> } title="Quick Start - Access" description="Learn how to retrieve and query data from Arweave" href="/build/access" /> } title="Run a Gateway" description="Deploy your own ar.io gateway and access these APIs directly" href="/build/run-a-gateway" /> # Turbo APIs (/apis/turbo) Turbo provides high-performance upload and payment services for the Arweave network, offering fast, reliable data uploads with instant confirmation and transparent pricing. ## Services } title="Upload Service" description="Fast, reliable data uploads to Arweave with instant confirmation and metadata management" href="/apis/turbo/upload-service/upload" /> } title="Payment Service" description="Transparent pricing, payment processing, and credit management for Turbo uploads" href="/apis/turbo/payment-service/payments" /> ## Upload Service The Turbo Upload Service provides high-performance data uploads to the Arweave network with features including: - **Fast Uploads** - Optimized upload processing for quick data submission - **Instant Confirmation** - Immediate upload confirmations and transaction IDs - **Metadata Management** - Comprehensive data tagging and organization - **Account Management** - User account and upload history tracking - **Service Information** - Real-time service status and capabilities Key endpoints include account management, upload processing, pricing information, and transaction data retrieval. ## Payment Service The Turbo Payment Service handles all financial aspects of data uploads with transparent and flexible payment options: - **Transparent Pricing** - Clear, upfront costs for all upload operations - **Multiple Currencies** - Support for various payment methods and currencies - **Credit Management** - Prepaid credits and balance tracking - **Payment Processing** - Secure payment handling and transaction management - **Approval Workflows** - Payment authorization and confirmation flows Key endpoints include balance management, payment processing, pricing calculations, and credit redemption. ## Getting Started with Turbo 1. **Choose your service** - Upload for data submission, Payment for financial operations 2. **Review the APIs documentation** - Detailed endpoint specifications and examples 3. **Test with sample data** - Try uploads and payment flows with test data 4. **Integrate into your application** - Implement the APIs in your workflow ## Use the Turbo SDK For a more convenient integration experience, consider using the Turbo SDK instead of direct API calls: } title="Interact with Turbo via the SDK" description="Use the Turbo SDK for simplified integration with built-in error handling, retries, and TypeScript support" href="/sdks/turbo-sdk" /> The SDK provides a higher-level interface with built-in error handling, automatic retries, and full TypeScript support, making it easier to integrate Turbo services into your applications. # Approvals (/apis/turbo/payment-service/approvals) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Credit sharing and approval management # Balance (/apis/turbo/payment-service/balance) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Account balance and credit management # Currencies (/apis/turbo/payment-service/currencies) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Supported currencies and exchange rates # Info (/apis/turbo/payment-service/info) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Service information and metadata # Payments (/apis/turbo/payment-service/payments) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Payment processing and top-up operations # Pricing (/apis/turbo/payment-service/pricing) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Pricing and cost calculation endpoints # Redemption (/apis/turbo/payment-service/redemption) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Credit redemption and gift processing # Account (/apis/turbo/upload-service/account) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Account balance and wallet information # Pricing (/apis/turbo/upload-service/pricing) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Pricing calculation endpoints # Service Info (/apis/turbo/upload-service/service-info) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Service information and health endpoints # Transaction Data (/apis/turbo/upload-service/transaction-data) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Transaction status and metadata retrieval # Upload (/apis/turbo/upload-service/upload) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Data item upload endpoints (single and multi-part) # Ar.io Name System (ArNS) (/build/access/arns) ArNS provides **human-readable URLs** for your Arweave data, making it easy to share and remember permanent addresses. ## What is ArNS? ArNS is a naming system that allows you to register human-readable names that point to your Arweave transactions. Instead of sharing long transaction IDs, you can use memorable URLs. **Example:** - **Before:** `https://turbo-gateway.com/bVLEkL1SOPFCzIYi8T_QNnh17VlDp4RylU6YTwCMVRw` - **After:** `https://myapp.ar.io` **Learn More:** For detailed information about ArNS architecture and how it works, see our [ArNS Documentation](/learn/arns). ## Get an ArNS Name The easiest way to get an ArNS name is via [arns.ar.io](https://arns.ar.io), which supports multiple payment methods: - **Fiat payments** - Credit cards and bank transfers - **Turbo Credits** - Use existing Turbo credits - **ARIO tokens** - Pay with ARIO cryptocurrency **Alternative registration method:** - **ar.io SDK** - Programmatic registration using the `buyRecord` API ### Using the ar.io SDK For developers, you can register ArNS names programmatically: ```js const ario = ARIO.mainnet(); // Buy a record with Turbo Credits or ARIO tokens const result = await ario.buyRecord({ name: 'my-domain', years: 1, // Payment method: 'turbo-credits' or 'ario-tokens' }); console.log('Record purchased:', result); ``` **Learn More:** For a complete list of ar.io SDK APIs, see the [ArNS SDK Documentation](/sdks/ar-io-sdk/ar-io-name-system-arns). ## Fetching Data via ArNS Once you've set up your ArNS name, fetch data using standard HTTP requests: ```js // Fetch content from your ArNS name const response = await fetch("https://my-data.ar.io"); if (!response.ok) { throw new Error(`HTTP error! status: ${response.status}`); } const data = await response.text(); console.log(data); ``` ## Why Use ArNS? ArNS provides significant advantages for accessing data on Arweave: **Decentralized Data Index** - ArNS creates a decentralized index of data accessible through any gateway in the ar.io network - No single point of failure - names resolve across all participating gateways - Censorship-resistant access to your content **Flexible Data Management** - **Permanent references** - Keep stable URLs even when updating underlying data - **Replaceable data** - Point names to new transaction IDs as content evolves - **Undernames** - Organize related content under a single name using underscores (e.g., `v2_myapp.ar.io`, `docs_myapp.ar.io`) **Supporting Network Decentralization** - ArNS purchases contribute to the protocol balance - Fees reward ar.io gateway operators for participating in the network - This economic model preserves decentralized access to data on Arweave - Your name registration helps maintain the infrastructure that serves your content ## Next Steps } > Register your own human-readable name on ArNS. } > Learn how to participate in the ar.io ecosystem. } > Advanced gateway routing for production applications. # Fetch Data (via REST API) (/build/access/fetch-data) The simplest way to access data on Arweave is through **HTTP requests** to gateways. This method works in any web browser and requires no additional setup. ## Fetching Data from Gateways Gateways are the most performant way to fetch data from Arweave, providing significant advantages over accessing Arweave nodes directly. **Why Gateways Are Faster:** - **Content Caching** - Pre-cached data for instant retrieval - **Data Indexing** - Fast search and query capabilities - **Network Optimization** - Distributed infrastructure for better performance - **Content Delivery** - Optimized serving with compression and CDN features ## REST APIs for Fetching Data Gateways support multiple API endpoints for accessing data: ### Standard Endpoint Access any transaction using this URL structure: ``` https:/// ``` **Examples:** - `https://turbo-gateway.com/bVLEkL1SOPFCzIYi8T_QNnh17VlDp4RylU6YTwCMVRw` - `https://turbo-gateway.com/FguFk5eSth0wO8SKfziYshkSxeIYe7oK9zoPN2PhSc0` ### Raw Data Endpoint For raw data access that bypasses manifest path resolution: ``` https:///raw/ ``` This endpoint returns the raw data bytes without resolving manifest paths, useful when you need the exact stored data. **Learn More:** For complete API documentation and testing, see the [ar.io Node Data APIs](/apis/ar-io-node/data). ## Sandboxing Ar.io gateways implement security measures by redirecting requests to sandbox subdomains for enhanced browser security. **Why Redirects Happen:** - **Security Isolation** - Content is served from isolated sandbox environments - **CSP Protection** - Prevents cross-site scripting attacks - **Resource Isolation** - Limits potential security vulnerabilities - **Browser Sandboxing** - Leverages same-origin policy for enhanced security **What to Expect:** - Initial request: `https://turbo-gateway.com/transaction-id` - Redirects to: `https://sandbox.turbo-gateway.com/transaction-id` (or similar) - Final content served from sandbox subdomain **Important:** Always follow redirects in your applications - the final sandbox URL contains the actual content. **Learn More:** For detailed information about how browser sandboxing works and why it's important for security, see our [Browser Sandboxing](/build/advanced/sandboxing) documentation. ## Using in Applications **JavaScript Example with Fetch:** ```js // Fetch data from Arweave (follows redirects automatically) const response = await fetch("https://turbo-gateway.com/your-transaction-id", { redirect: "follow", // Follow redirects automatically }); if (!response.ok) { throw new Error(`HTTP error! status: ${response.status}`); } const data = await response.text(); console.log(data); ``` **HTML Example:** ```html ``` ## Manifests For organized file collections, use manifests to create friendly path-based URLs: ``` https://turbo-gateway.com//path/to/file ``` **Example:** - `https://turbo-gateway.com/X8Qm…AOhA/index.html` - `https://turbo-gateway.com/X8Qm…AOhA/styles.css` - `https://turbo-gateway.com/X8Qm…AOhA/assets/logo.png` [Learn more about manifests](/build/upload/manifests) ## Next Steps } > Discover data by searching with tags, metadata, and filters. }> Set up a gateway to serve and cache your specific data. }> Start uploading your data to Arweave's permanent storage. } > Automatically route requests to the best performing gateway. # Find Data (via GraphQL) (/build/access/find-data) Use **GraphQL** to **find and identify** Arweave data with powerful search and filtering capabilities. GraphQL is used for discovery - you query to get transaction IDs, then use those IDs to fetch the actual data. **GraphQL is for Discovery, Not Direct Access** GraphQL finds data, it doesn't access it directly. Use GraphQL to get transaction IDs, then use those IDs with the REST API to fetch the actual data. ## How GraphQL Works GraphQL on Arweave follows a two-step process: 1. **Find** - Query GraphQL to discover transactions by tags, metadata, owner, or other criteria 2. **Fetch** - Use the transaction IDs from your query results to retrieve the actual data via the REST API This separation allows for powerful data discovery while keeping data retrieval fast and efficient. ## GraphQL Providers - **turbo-gateway.com** - `https://turbo-gateway.com/graphql` - Comprehensive indexing of all Arweave data - **Goldsky** - `https://arweave-search.goldsky.com/graphql` - High-performance GraphQL service with full data coverage **ar.io Gateways:** ar.io gateways support the `/graphql` endpoint, but they only return data they've indexed. If you're uploading data and want it unbundled and indexed, you can run a gateway and configure it to unbundle your data, or post data items/bundles via the gateway's APIs (recommended). [Learn more](/build/run-a-gateway/manage/filters). ## Quick Start The easiest way to get started is using the interactive GraphQL playground: 1. Navigate to [https://turbo-gateway.com/graphql](https://turbo-gateway.com/graphql) in your browser 2. Enter your GraphQL query in the interface 3. Press the "play" button to execute and see results ## Basic Query Structure Try this example query in the playground - it fetches the most recent 10 HTML pages from "MyApp": ```graphql query { transactions( tags: [ { name: "Content-Type", values: ["text/html"] } { name: "App-Name", values: ["MyApp"] } ] sort: HEIGHT_DESC first: 10 ) { edges { node { id tags { name value } data { size } } } } } ``` ## Example Queries Here's how to find videos using GraphQL: ```js const query = ` query { transactions( tags: [{ name: "Content-Type", values: ["video/mp4"] }] first: 10 ) { edges { node { id tags { name value } data { size } } } } } `; const response = await fetch("https://turbo-gateway.com/graphql", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query }), }); const data = await response.json(); const videos = data.data.transactions.edges; // This returns transaction IDs that you can use with HTTP requests console.log( "Found video IDs:", videos.map((v) => v.node.id) ); ``` ```js const query = ` query { transactions(owners: ["your-wallet-address"], first: 10) { edges { node { id tags { name value } } } } } `; const response = await fetch("https://turbo-gateway.com/graphql", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query }), }); const data = await response.json(); const transactions = data.data.transactions.edges; console.log("Found transactions:", transactions.map(t => t.node.id)); ``` ```js const query = ` query { transactions(block: { min: 1000000, max: 1100000 }, first: 10) { edges { node { id block { height } } } } } `; const response = await fetch("https://turbo-gateway.com/graphql", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query }), }); const data = await response.json(); const transactions = data.data.transactions.edges; console.log("Found transactions in block range:", transactions.map(t => ({ id: t.node.id, height: t.node.block.height }))); ``` ```js // First page const query = ` query { transactions( tags: [{ name: "App-Name", values: ["MyApp"] }] first: 10 ) { pageInfo { hasNextPage } edges { cursor node { id tags { name value } } } } } `; const response = await fetch("https://turbo-gateway.com/graphql", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query }), }); const data = await response.json(); const { edges, pageInfo } = data.data.transactions; // Next page using cursor if (pageInfo.hasNextPage) { const nextQuery = ` query($cursor: String) { transactions( tags: [{ name: "App-Name", values: ["MyApp"] }] after: $cursor first: 10 ) { pageInfo { hasNextPage } edges { cursor node { id tags { name value } } } } } `; const nextResponse = await fetch("https://turbo-gateway.com/graphql", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query: nextQuery, variables: { cursor: edges[edges.length - 1].cursor }, }), }); const nextData = await nextResponse.json(); console.log(nextData.data); } ``` ## Pagination Different GraphQL providers may enforce varying limits on the maximum number of items returned per query. For larger datasets, use cursor-based pagination techniques to navigate through the complete set of results. **How Pagination Works:** - Use `first` parameter to specify page size - Use `pageInfo.hasNextPage` to check if more results exist - Use `cursor` from the last item with `after` parameter for the next page ```js let allTransactions = []; let hasNextPage = true; let cursor = null; while (hasNextPage) { const query = ` query($cursor: String) { transactions( tags: [{ name: "App-Name", values: ["MyApp"] }] first: 100 ${cursor ? "after: $cursor" : ""} ) { pageInfo { hasNextPage } edges { cursor node { id tags { name value } } } } } `; const response = await fetch("https://turbo-gateway.com/graphql", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query, variables: cursor ? { cursor } : {}, }), }); const data = await response.json(); const { edges, pageInfo } = data.data.transactions; allTransactions.push(...edges); hasNextPage = pageInfo.hasNextPage; cursor = edges.length > 0 ? edges[edges.length - 1].cursor : null; console.log( `Loaded ${edges.length} transactions. Total: ${allTransactions.length}` ); } console.log(`Found ${allTransactions.length} total transactions`); ``` ## Query Optimization Tips Follow these guidelines for optimal performance: **Specificity:** - Use the most precise tags possible to narrow search scope - Query with essential tags only to reduce processing time **Schema Design:** - Design your app's schema to reflect query patterns - Use tags that encapsulate frequent combinations of criteria **Include Non-tag Fields:** - Add fields like `owner` to refine your search - This makes queries more efficient and targeted **Order Your Tags:** - Arrange tags from most specific to most general - This leverages Arweave's indexing more effectively **Example Optimized Query:** ```js // Well-optimized query with specific tags and useful fields const query = ` query { transactions( tags: [ { name: "App-Name", values: ["MyApp"] } { name: "Content-Type", values: ["application/json"] } { name: "Version", values: ["1.0"] } ] owners: ["your-wallet-address"] first: 20 ) { edges { node { id data { size type } tags { name value } block { height timestamp } owner { address } } } } } `; ``` ## Next Steps } > Learn how to retrieve the actual data using transaction IDs. }> Set up a gateway to index and serve your specific data. }> Start uploading your data to Arweave's permanent storage. } > Automatically route requests to the best performing gateway. # Access Data (/build/access) Once data is stored on Arweave, it's permanently available. Here's how to access it efficiently for your applications. ## Access Methods Different methods serve different needs. Each provides unique capabilities for retrieving data from Arweave. Search and discover data on Arweave Query by tags and metadata Filter by app, owner, timestamp Get transaction IDs for fetching } href="/build/access/find-data" icon={} /> Retrieve data bytes from Arweave REST API endpoints GET turbo-gateway.com/[txId] Returns raw data/files } href="/build/access/fetch-data" icon={} /> Assign names to data and apps Create names like ardrive.ar.io Point to any Arweave data Update targets as needed } href="/build/access/arns" icon={} /> ## Common Access Patterns **Finding Data** - Search for data by tags, owner, or timestamp - Discover content from specific applications - Get transaction IDs for data retrieval **Fetching Data** - Retrieve the actual files/data using transaction IDs - Access data via REST API: `GET turbo-gateway.com/[txId]` - Stream large files efficiently **Naming with ArNS** - Register memorable names for your apps and data - Create permanent links like `ardrive.ar.io` - Update where names point without changing the URL ## Quick Example: Find and Fetch ### Find Data Use GraphQL to search for data and get transaction IDs: ```graphql query { transactions( tags: [{ name: "App-Name", values: ["ArDrive"] }] first: 1 ) { edges { node { id } } } } ``` ### Fetch Data Use the transaction ID to retrieve the actual data: ```bash curl https://turbo-gateway.com/[transaction-id-from-above] ``` ## Additional Access Options # Wayfinder (/build/access/wayfinder) Wayfinder is a client-side routing and verification protocol that provides **decentralized, cryptographically verified access** to data stored on Arweave via ar.io. ## What is Wayfinder? Wayfinder solves the challenge of reliable data access on the permaweb by: - **Intelligent Routing** - Automatically selects the best gateway for each request - **Data Verification** - Cryptographically verifies data integrity - **Decentralized Access** - Eliminates single points of failure - **Seamless Integration** - Works behind the scenes for fast, reliable access **Learn More:** For detailed information about Wayfinder architecture and how it works, see our [Wayfinder Documentation](/learn/wayfinder). ## Get Started **Installation:** ```npm npm install @ar.io/wayfinder-core @ar.io/sdk ``` **Basic Usage:** ```js // Create wayfinder with default settings const wayfinder = createWayfinderClient({ ario: ARIO.mainnet(), }); // Fetch data using ar:// protocol try { const response = await wayfinder.request("ar://transaction-id"); const data = await response.text(); console.log("Data:", data); } catch (error) { console.error("Failed to fetch data:", error); } ``` **Full API Reference:** For complete documentation of all Wayfinder core APIs, see the [Wayfinder Core SDK Reference](/sdks/wayfinder/wayfinder-core). ## React Integration For React applications, use the wayfinder-react package: ```npm npm install @ar.io/wayfinder-react @ar.io/sdk ``` ```jsx function App() { return ( ); } function YourComponent() { const request = useWayfinderRequest(); const [data, setData] = useState(null); useEffect(() => { (async () => { const response = await request(`ar://${txId}`, { verificationSettings: { enabled: true, strict: true, }, }); const data = await response.arrayBuffer(); setData(data); })(); }, [request, txId]); return {data && {data}}; } ``` **Full API Reference:** For complete documentation of all Wayfinder React APIs, see the [Wayfinder React SDK Reference](/sdks/wayfinder/wayfinder-react). ## Why Use Wayfinder? Wayfinder eliminates centralized points of failure by distributing data access across ar.io, reducing dependency on arweave.net and providing advanced capabilities for production applications: **Maximum Reliability** - Intelligent gateway selection eliminates single points of failure - Automatic failover ensures data is always accessible - Built-in retry mechanisms handle network issues gracefully **Data Verification** - Cryptographic verification ensures data integrity - Multiple verification strategies protect against tampering - Trust but verify approach validates all responses **Performance Optimization** - Fastest ping routing selects optimal gateways - Round-robin distribution balances load across the network - Caching strategies reduce latency for frequently accessed data **Production Ready** - Developer-friendly APIs with React integration - Comprehensive error handling and logging - Configurable routing and verification strategies ## Next Steps } > Start building with the Wayfinder SDK. } > Use REST API for basic data retrieval. }> Use GraphQL to search for data. } > Create memorable names for your Arweave data. # Creating Drives (/build/advanced/arfs/creating-drives) To properly create a new drive, two new entities need to be created: a new Drive entity and a new Folder entity to serve as the root folder of that drive. ## New Drive Entity - The user must specify a `name` of the drive which is stored within the Drive Entity's metadata JSON. - ArDrive generates a new unique uuidv4 for the drive entity's `Drive-Id`. - ArDrive also generates a new unique uuidv4 for the drive entity's `rootFolderId`, which will refer to the `Folder-Id` of the new folder entity that will be created. - This `rootFolderId` is stored within the Drive Entity's metadata JSON. - Drive Entity Metadata transactions must have `Entity-Type: "drive"`. - ArDrive will that the current local system time as seconds since Unix epoch for the Drive Entity's `Unix-Time`. - The Drive Entity's `Drive-Privacy` must also be set to `public` or `private` in order for its subfolders and files to have the correct security settings. - If the drive is private: - Its `Cipher` tag must be filled out with the correct encryption algorithm (currently `AES256-GCM`). - Its `Cipher-IV` tag must be filled out with the generated Initialization Vector for the private drive. - The ArFS client must derive the Drive Key and encrypt the Drive Entity's metadata JSON using the assigned `Cipher` and `Cipher-IV`. ## New Root Folder Entity - The `name` of the drive and folder entities must be the same. - This `name` is stored within the Folder Entity's metadata JSON. - The Folder Entity's `Folder-Id` must match the `rootFolderId` previously created for the Drive Entity. - The Folder Entity's `Drive-Id` must match the `Drive-Id` previously created for the Drive Entity. - The Folder Entity must not include a `Parent-Folder-Id` tag. - This is how it is determined to be the root folder for a drive. - Folder Entity metadata transactions must have `Entity-Type: 'folder'`. - The client gets the user's local time for the `Unix-Time` tag, represented as seconds since Unix Epoch. - Public folders must have the content type `Content-Type: "application/json"`. - If the folder is private - Its `Cipher` tag must be filled out with the correct encryption algorithm (currently `AES256-GCM`). - Its `Cipher-IV` tag must be filled out with the generated Initialization Vector for the private folder. - Its content type must be `Content-Type: "application/octet-stream"`. - The ArFS client must encrypt the Drive Entity's metadata JSON using the assigned `Cipher` and `Cipher-IV`. ## Creating Files Files in ArFS require two separate transactions: 1. **File Metadata Transaction** - Contains file information and references 2. **File Data Transaction** - Contains the actual file data ### File Metadata Transaction ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Entity-Type: "file", File-Id: "", Parent-Folder-Id: "", Unix-Time: "" Metadata JSON { "name": "", "size": , "lastModifiedDate": , "dataTxId": "", "dataContentType": "", "isHidden": false, "pinnedDataOwner": "" } ``` ### File Data Transaction ```json Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", { File Data - Encrypted if private } ``` ## Creating Folders Folders are simpler than files as they only require a metadata transaction: ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Entity-Type: "folder", Folder-Id: "", Parent-Folder-Id?: "", Unix-Time: "" Metadata JSON { "name": "", "isHidden": false } ``` ## Creating Snapshots Snapshots provide a way to quickly synchronize drive state by rolling up all metadata into a single transaction: ```json ArFS: "0.15", Drive-Id: "", Entity-Type: "snapshot", Snapshot-Id: "", Content-Type: "", Block-Start: "", Block-End: "", Data-Start: "", Data-End: "", Unix-Time: "" ``` ## Implementation Example Here's a practical example of creating a complete drive structure: ```mermaid sequenceDiagram participant User participant Client participant Wallet participant Arweave User->>Client: Create drive "My Project" Client->>Client: Generate drive UUID Client->>Client: Generate root folder UUID Client->>Wallet: Request signature (if private) Wallet->>Client: Return signature Client->>Client: Derive drive key (if private) Client->>Client: Encrypt metadata (if private) Client->>Arweave: Upload drive entity Client->>Arweave: Upload root folder entity User->>Client: Create folder "Documents" Client->>Client: Generate folder UUID Client->>Client: Encrypt folder metadata (if private) Client->>Arweave: Upload folder entity User->>Client: Upload file "readme.txt" Client->>Client: Generate file UUID Client->>Client: Encrypt file metadata (if private) Client->>Client: Encrypt file data (if private) Client->>Arweave: Upload file metadata Client->>Arweave: Upload file data ``` ## Best Practices ### Naming Conventions - Use descriptive names for drives, folders, and files - Avoid special characters that might cause issues - Keep names under 255 characters - Use consistent casing ### Organization - Create logical folder structures - Use meaningful folder names - Implement proper versioning - Document your structure ### Performance - Batch operations when possible - Use efficient queries - Implement caching - Consider file sizes ### Security - Use strong passwords for private drives - Implement proper key management - Follow encryption best practices - Regular security audits ## Error Handling When creating ArFS entities, handle these common scenarios: ### Transaction Failures - Implement retry logic for failed uploads - Validate data before uploading - Check transaction confirmation status ### Validation Errors - Verify required tags are present - Check data format compliance - Validate UUID formats ### Network Issues - Implement timeout handling - Provide user feedback - Graceful degradation ## Next Steps Now that you know how to create ArFS entities, learn how to work with them: - [Reading Data](/build/advanced/arfs/reading-data) - Query and retrieve your ArFS data - [Privacy & Encryption](/build/advanced/arfs/privacy) - Secure your data with private drives - [Upgrading Private Drives](/build/advanced/arfs/upgrading-drives) - Update legacy drives to v0.15 # Data Model (/build/advanced/arfs/data-model) Because of Arweave's permanent and immutable nature, traditional file structure operations such as renaming and moving files or folders cannot be accomplished by simply updating onchain data. ArFS works around this by defining an append-only transaction data model based on the metadata tags found in the Arweave [Transaction Headers.](https://docs.arweave.org/developers/server/http-api#transaction-format) This model uses a bottom-up reference method, which avoids race conditions in file system updates. Each file contains metadata that refers to the parent folder, and each folder contains metadata that refers to its parent drive. A top-down data model would require the parent model (i.e. a folder) to store references to its children. These defined entities allow the state of the drive to be constructed by a client to look and feel like a file system: - Drive Entities contain folders and files - Folder Entities contain other folders or files - File Entities contain both the file data and metadata - Snapshot entities contain a state rollups of all entities' (such as drive, folder, file and snapshot) metadata within a drive ## Entity Relationships The following diagram shows the high level relationships between drive, folder, and file entities, and their associated data. More detailed information about each Entity Type can be found in the ArFS specification documentation. ```mermaid graph TD A[Drive Entity] --> B[Root Folder] B --> C[Subfolder 1] B --> D[Subfolder 2] B --> E[File 1] C --> F[File 2] C --> G[File 3] D --> H[File 4] D --> I[Subfolder 3] I --> J[File 5] A --> K[Drive Metadata] B --> L[Folder Metadata] C --> M[Folder Metadata] D --> N[Folder Metadata] I --> O[Folder Metadata] E --> P[File Metadata + Data] F --> Q[File Metadata + Data] G --> R[File Metadata + Data] H --> S[File Metadata + Data] J --> T[File Metadata + Data] U[Snapshot Entity] --> V[Complete Drive State] V --> A V --> B V --> C V --> D V --> I V --> E V --> F V --> G V --> H V --> J ``` As you can see, each file and folder contains metadata which points to both the parent folder and the parent drive. The drive entity contains metadata about itself, but not the child contents. So clients must build drive states from the lowest level and work their way up. ## Metadata Format Metadata stored in any Arweave transaction tag will be defined in the following manner: ```json { "name": "Example-Tag", "value": "example-data" } ``` Metadata stored in the Transaction Data Payload will follow JSON formatting like below: ```json { "exampleField": "exampleData" } ``` Fields with a `?` suffix are optional. ```json { "name": "My Project", "description": "This is a sample project.", "version?": "1.0.0", "author?": "John Doe" } ``` Enumerated field values (those which must adhere to certain values) are defined in the format "value 1 | value 2". All UUIDs used for Entity-Ids are based on the [Universally Unique Identifier](https://en.wikipedia.org/wiki/Universally_unique_identifier) standard. There are no requirements to list ArFS tags in any specific order. ## Building Drive State To construct the current state of a drive, clients must: 1. **Query for all entities** associated with a specific `Drive-Id` 2. **Sort by block height** to establish chronological order 3. **Process entities bottom-up** starting with files and folders 4. **Build the hierarchy** by following parent-child relationships 5. **Handle conflicts** by using the most recent entity version ### Example Drive State Construction ```mermaid sequenceDiagram participant Client participant Gateway participant Arweave Client->>Gateway: Query Drive-Id: abc123 Gateway->>Client: Return all entities Client->>Client: Sort by block height Client->>Client: Process files first Client->>Client: Process folders Client->>Client: Process drive metadata Client->>Client: Build hierarchy tree Client->>Client: Resolve conflicts Client->>Client: Return complete drive state ``` ## Entity Lifecycle Each ArFS entity follows a specific lifecycle pattern: ### Creation 1. Generate unique UUID for entity 2. Create metadata transaction with required tags 3. For files: create separate data transaction 4. Upload to Arweave network ### Updates 1. Create new entity with same ID 2. Update metadata as needed 3. Upload new transaction 4. Client processes both versions and uses latest ### Deletion 1. Mark entity as hidden (`isHidden: true`) 2. Upload new transaction 3. Entity remains in history but hidden from UI ## Data Integrity ArFS ensures data integrity through: - **Immutable transactions** - Once uploaded, data cannot be modified - **Cryptographic signatures** - All transactions are signed by the owner - **Version tracking** - Multiple versions of entities can exist - **Conflict resolution** - Clients use block height and timestamps to resolve conflicts ## Performance Considerations For large drives, consider these optimization strategies: - **Use snapshots** for quick state reconstruction - **Implement caching** for frequently accessed data - **Batch operations** when possible - **Query by date ranges** to limit data transfer ## Next Steps Now that you understand the ArFS data model, learn how to work with it: - [Privacy & Encryption](/build/advanced/arfs/privacy) - Secure your data with private drives - [Creating Drives](/build/advanced/arfs/creating-drives) - Start building with ArFS - [Reading Data](/build/advanced/arfs/reading-data) - Query and retrieve your data # Entity Types (/build/advanced/arfs/entity-types) ## Overview Arweave transactions provide for a separation between data and metadata about that data via the use of headers. Key-value tags in the headers provide for expressive description about the data as well as searchability via gateway GraphQL APIs. ArFS adds an additional layer of separation between data and metadata by using separate transactions for ArFS metadata and, where applicable, ArFS file data. But it also makes use of tags and data separation within an ArFS metadata transaction by including data critical to tracking drive composition in the tags space of ArFS metadata transactions and having most of the other metadata encoded as JSON in the data body of the metadata transaction. In the case of private entities, JSON data and file data payloads are always encrypted according to the protocol processes defined below. - Drive entities require a single metadata transaction, with standard Drive tags and encoded JSON with secondary metadata. - Folder entities require a single metadata transaction, with standard Folder tags and an encoded JSON with secondary metadata. - File entities require a metadata transaction, with standard File tags and an encoded Data JSON with secondary metadata relating to the file. - File entities also require a second data transaction, which includes a limited set of File tags and the actual file data itself. - Snapshot entities require a single transaction, which contains a Data JSON with all of the Drive's rolled up ArFS metadata and standard Snapshot GQL tags that identify the Snapshot. ArFS v0.14 introduces the `isHidden` property. `isHidden` is a boolean (true/false) that tells clients if they should display the file or folder. Hidden files still exist and will be included in [snapshots](#snapshot), but should not be rendered by clients. If `isHidden` is not present, its value should be assumed false. ArFS v0.15 introduces the `Signature-Type` metadata property on Drive entities, and a new entity type `DriveSignature`. ## Drive A drive is the highest level logical grouping of folders and files. All folders and files must be part of a drive, and reference the Drive ID of that drive. When creating a Drive, a corresponding "root" folder must be created as well. This separation of drive and folder entity enables features such as folder view queries, renaming, and linking. ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Drive-Privacy: "", Drive-Auth-Mode?: "password", Entity-Type: "drive", Signature-Type?: "1", Unix-Time: "" Metadata JSON { "name": "", "rootFolderId": "", "isHidden": false } ``` ## Drive-Signature ArFS versions prior to v0.15 applied encryption to drive contents with a signing scheme that, while secure, is now deprecated in modern Arweave software wallets. ArFS v0.15 introduces an updated signing scheme compatible with these wallets and as well as "Drive Signatures", a new entity type to help bridge the signature derivation schemes across ArFS versions. A drive signature uses the v0.15 encryption scheme to encrypt and store the pre-v0.15 wallet signature for a private drive that is necessary for deriving the "drive key" for that drive. This allows for continued access of historical drive contents into the future. ```json ArFS: "0.15", Entity-Type: "drive-signature", Signature-Format: "1", Cipher?: "AES256-GCM", Cipher-IV: "" {data: } ``` The encrypted "type 1" signature for the drive must be provided in the `data` field of the transaction creating the drive-signature entity. ## Folder A folder is a logical grouping of other folders and files. Folder entity metadata transactions without a parent folder id are considered the Drive Root Folder of their corresponding Drives. All other Folder entities must have a parent folder id. Since folders do not have underlying data, there is no Folder data transaction required. ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Entity-Type: "folder", Folder-Id: "", Parent-Folder-Id?: "", Unix-Time: "" Metadata JSON { "name": "", "isHidden": false } ``` ## File A File contains uploaded data, like a photo, document, or movie. In the Arweave File System, a single file is broken into 2 parts - its metadata and its data. A File entity metadata transaction does not include the actual File data. Instead, the File data must be uploaded as a separate transaction, called the File Data Transaction. The File JSON metadata transaction contains a reference to the File Data Transaction ID so that it can retrieve the actual data. This separation allows for file metadata to be updated without requiring the file itself to be reuploaded. It also ensures that private files can have their JSON Metadata Transaction encrypted as well, ensuring that no one without authorization can see either the file or its metadata. ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Entity-Type: "file", File-Id: "", Parent-Folder-Id: "", Unix-Time: "" Metadata JSON { "name": "", "size": , "lastModifiedDate": , "dataTxId": "", "dataContentType": "", "isHidden": false, "pinnedDataOwner": "", # Optional } ``` ### Pinning Files Since the version v0.13, ArFS supports Pins. Pins are files whose data may be any transaction uploaded to Arweave, that may or may not be owned by the wallet that created the pin. When a new File Pin is created, the only created transaction is the Metadata Transaction. The `dataTxId` field will point it to any transaction in Arweave, and the optional `pinnedDataOwner` field is gonna hold the address of the wallet that owns the original copy of the data transaction. ### File Data Transaction Example The File Data Transaction contains limited information about the file, such as the information required to decrypt it, or the Content-Type (mime-type) needed to view in the browser. ```json Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", { File Data - Encrypted if private } ``` ### File Metadata Transaction Example The File Metadata Transaction contains the GQL Tags necessary to identify the file within a drive and folder. Its data contains the JSON metadata for the file. This includes the file name, size, last modified date, data transaction id, and data content type. ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Entity-Type: "file", File-Id: "", Parent-Folder-Id: "", Unix-Time: "", { File JSON Metadata - Encrypted if private } ``` ## Snapshot ArFS applications generate the latest state of a drive by querying for all ArFS transactions made relating to a user's particular `Drive-Id`. This includes both paged queries for indexed ArFS data via GQL, as well as the ArFS JSON metadata entries for each ArFS transaction. For small drives (less than 1000 files), a few thousand requests for very small volumes of data can be achieved relatively quickly and reliably. For larger drives, however, this results in long sync times to pull every piece of ArFS metadata when the local database cache is empty. This can also potentially trigger rate-limiting related ArWeave Gateway delays. Once a drive state has been completely, and accurately generated, in can be rolled up into a single snapshot and uploaded as an Arweave transaction. ArFS clients can use GQL to find and retrieve this snapshot in order to rapidly reconstitute the total state of the drive, or a large portion of it. They can then query individual transactions performed after the snapshot. This optional method offers convenience and resource efficiency when building the drive state, at the cost of paying for uploading the snapshot data. Using this method means a client will only have to iterate through a few snapshots instead of every transaction performed on the drive. ### Snapshot Entity Tags Snapshot entities require the following tags. These are queried by ArFS clients to find drive snapshots, organize them together with any other transactions not included within them, and build the latest state of the drive. ```json ArFS: "0.15", Drive-Id: "", Entity-Type: "snapshot", Snapshot-Id: "", Content-Type: "", Block-Start: "", Block-End: "", Data-Start: "" ``` ### Snapshot Entity Data A JSON data object must also be uploaded with every ArFS Snapshot entity. This data contains all ArFS Drive, Folder, and File metadata changes within the associated drive, as well as any previous Snapshots. The Snapshot Data contains an array `txSnapshots`. Each item includes both the GQL and ArFS metadata details of each transaction made for the associated drive, within the snapshot's start and end period. A `tsSnapshot` contains a `gqlNode` object which uses the same GQL tags interface returned by the Arweave Gateway. It includes all of the important `block`, `owner`, `tags`, and `bundledIn` information needed by ArFS clients. It also contains a `dataJson` object which stores the correlated Data JSON for that ArFS entity. For private drives, the `dataJson` object contains the JSON-string-escaped encrypted text of the associated file or folder. This encrypted text uses the file's existing `Cipher` and `Cipher-IV`. This ensures clients can decrypt this information quickly using the existing ArFS privacy protocols. ```json { "txSnapshots": [ { "gqlNode": { "id": "bWCvIc3cOzwVgquD349HUVsn5Dd1_GIri8Dglok41Vg", "owner": { "address": "hlWRbyJ6WUoErm3b0wqVgd1l3LTgaQeLBhB36v2HxgY" }, "bundledIn": { "id": "39n5evzP1Ip9MhGytuFm7F3TDaozwHuVUbS55My-MBk" }, "block": { "height": 1062005, "timestamp": 1669053791 }, "tags": [ { "name": "Content-Type", "value": "application/json" }, { "name": "ArFS", "value": "0.11" }, { "name": "Entity-Type", "value": "drive" }, { "name": "Drive-Id", "value": "f27abc4b-ed6f-4108-a9f5-e545fc4ff55b" }, { "name": "Drive-Privacy", "value": "public" }, { "name": "App-Name", "value": "ArDrive-App" }, { "name": "App-Platform", "value": "Web" }, { "name": "App-Version", "value": "1.39.0" }, { "name": "Unix-Time", "value": "1669053323" } ] }, "dataJson": "{\"name\":\"november\",\"rootFolderId\":\"71dfc1cb-5368-4323-972a-e9dd0b1c63a0\", \"isHidden\":false}" } ] } ``` ## Schema Diagrams The following diagrams show complete examples of Drive, Folder, and File entity Schemas. ### Public Drive ```mermaid graph TD A[Drive Entity] --> B[Drive Metadata JSON] A --> C[Drive Tags] C --> D[ArFS: 0.15] C --> E[Entity-Type: drive] C --> F[Drive-Id: uuid] C --> G[Drive-Privacy: public] C --> H[Unix-Time: timestamp] B --> I[name: string] B --> J[rootFolderId: uuid] B --> K[isHidden: boolean] ``` ### Private Drive ```mermaid graph TD A[Drive Entity] --> B[Encrypted Drive Metadata JSON] A --> C[Drive Tags] C --> D[ArFS: 0.15] C --> E[Entity-Type: drive] C --> F[Drive-Id: uuid] C --> G[Drive-Privacy: private] C --> H[Drive-Auth-Mode: password] C --> I[Signature-Type: 1] C --> J[Cipher: AES256-GCM] C --> K[Cipher-IV: base64] C --> L[Content-Type: application/octet-stream] C --> M[Unix-Time: timestamp] B --> N[Encrypted JSON with name, rootFolderId, isHidden] ``` ## Next Steps Now that you understand the different ArFS entity types, explore how they work together: - [Data Model](/build/advanced/arfs/data-model) - Learn how entities relate to each other - [Privacy & Encryption](/build/advanced/arfs/privacy) - Understand how private entities work - [Creating Drives](/build/advanced/arfs/creating-drives) - Start building with ArFS # ArFS Protocol (/build/advanced/arfs) Arweave File System, or "ArFS" is a data modeling, storage, and retrieval protocol designed to emulate common file system operations and to provide aspects of mutability to your data hierarchy on [Arweave](/learn/what-is-arweave)'s otherwise permanent, immutable data storage blockweave. Due to Arweave's permanent, immutable and public nature traditional file system operations such as permissions, file/folder renaming and moving, and file updates cannot be done by simply updating the onchain data model. ArFS works around this by implementing a privacy and encryption pattern and defining an append-only transaction data model using tags within [Arweave Transaction headers](https://docs.arweave.org/developers/server/http-api#transaction-format). ## Key Features ### File Structure ArFS organizes files and folders using a hierarchical structure. Files are stored as individual transactions on the Arweave blockchain, while folders are metadata that reference these file transactions. ### Metadata Each file and folder has associated metadata, such as the name, type, size, and modification timestamp. ArFS leverages Arweave's tagging system to store this metadata in a standardized format, which allows for easy querying and organization. ### File Permissions ArFS supports public and private file permissions. Public files can be accessed by anyone on the network, while private files are encrypted using the owner's private key, ensuring only they can decrypt and access the content. ### File Versioning ArFS supports versioning of files, allowing users to store multiple versions of a file and access previous versions at any time. This is achieved by linking new file transactions to previous versions through the use of metadata tags. ### Search and Discovery ArFS enables users to search and discover files based on their metadata, such as file names, types, and tags. This is made possible by indexing the metadata stored within the Arweave blockchain. ### Interoperability ArFS is designed to be interoperable with other decentralized applications and services built on the Arweave network. This allows for seamless integration and collaboration between different applications and users. ## Getting Started To start using ArFS, you'll need to familiarize yourself with the Arweave ecosystem, acquire AR tokens to cover storage costs, and choose a compatible client or library to interact with the ArFS protocol. ## ArFS Version History | Version | Date | Release Notes | | ------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | 0.10 | August 2020 | The brief, beta version that was in use during initial testing of ArDrive across Web (Dart) and legacy CLI (Typescript). | | 0.11 | September 2020 | Includes all of the major functionality supporting file systems on Arweave including new drives, folders, files, renames, moves and privacy. | | 0.12 | December 2022 | Added Snapshot entities to support quick synchronization of drive state. | | 0.13 | August 2023 | Added pins | | 0.14 | January 2024 | Added `isHidden` property to file and folder metadata to enable clients to "hide" content from end users. | | 0.15 | May 2025 | Added `Drive-Signature` entity type and `Signature-Type` metadata property on Drive entities. | ## Next Steps Ready to dive deeper into ArFS? Here's what you should explore next: - [Entity Types](/build/advanced/arfs/entity-types) - Understand the different ArFS entities and their structure - [Data Model](/build/advanced/arfs/data-model) - Learn how ArFS organizes data hierarchically - [Privacy & Encryption](/build/advanced/arfs/privacy) - Secure your data with private drives - [Creating Drives](/build/advanced/arfs/creating-drives) - Get started with your first ArFS drive - [Reading Data](/build/advanced/arfs/reading-data) - Query and retrieve your ArFS data ## Resources For more information, documentation, and community support, refer to the following resources: - [Arweave Official Website](https://www.arweave.org/) - [Arweave Developer Documentation](https://docs.arweave.org/) - [Arweave Community Forums](https://community.arweave.org/) # Privacy & Encryption (/build/advanced/arfs/privacy) The Arweave blockweave is inherently public. But with apps that use ArFS, like ArDrive, your private data never leaves your computer without using military grade (and [quantum resistant](https://blog.boot.dev/cryptography/is-aes-256-quantum-resistant/#:~:text=Symmetric%20encryption%2C%20or%20more%20specifically,key%20sizes%20are%20large%20enough)) encryption. This privacy layer is applied at the Drive level, and users determine whether a Drive is public or private when they first create it. Private drives must follow the ArFS privacy model. With ArDrive specifically, every file within a Private Drive is symmetrically encrypted using [AES-256-GCM](https://iopscience.iop.org/article/10.1088/1742-6596/1019/1/012008/pdf) (for small files and metadata transactions) or [AES-256-CTR](https://xilinx.github.io/Vitis_Libraries/security/2020.1/guide_L1/internals/ctr.html) (for large files, over 100MiB). Every Private drive has a master "Drive Key" which uses a combination of the user's Arweave wallet signature, a user defined drive password, and a unique drive identifier ([uuidv4](https://en.wikipedia.org/wiki/Universally_unique_identifier)). Each file has its own "File Key" derived from the "Drive Key". This allows for single files to be shared without exposing access to the other files within the Drive. Once a file is encrypted and stored on Arweave, it is locked forever and can only be decrypted using its file key. **NOTE**: Usable encryption standards are not limited to AES-256-GCM or AES-256-CTR. Any Encryption method may be used so long as it is clearly indicated in the `cipher` tag. ## Deriving Keys Private drives have a global drive key, `D`, and multiple file keys `F`, for encryption. This enables a drive to have as many uniquely encrypted files as needed. One key is used for all versions of a single file (since new file versions use the same File-Id) `D` is used for encrypting both Drive and Folder metadata, while `F` is used for encrypting File metadata and the actual stored data. Having these different keys, `D` and `F`, allows a user to share specific files without revealing the contents of their entire drive. `D` is derived using HKDF-SHA256 with an [unsalted]() RSA-PSS signature of the drive's id and a user provided password. `F` is also derived using HKDF-SHA256 with the drive key and the file's id. ```mermaid graph TD A[User Password] --> B[Drive Key Derivation] C[Wallet Signature] --> B D[Drive ID] --> B B --> E[Drive Key D] E --> F[File Key Derivation] G[File ID] --> F F --> H[File Key F] E --> I[Encrypt Drive Metadata] E --> J[Encrypt Folder Metadata] H --> K[Encrypt File Metadata] H --> L[Encrypt File Data] style A fill:#e1f5fe style C fill:#e1f5fe style D fill:#e1f5fe style G fill:#e1f5fe style E fill:#c8e6c9 style H fill:#c8e6c9 ``` Other wallets (like [ArConnect](https://www.arconnect.io/)) integrate with this Key Derivation protocol just exposing an API to collect a signature from a given Arweave Wallet in order to get the SHA-256 signature needed for the [HKDF](https://en.wikipedia.org/wiki/HKDF) to derive the Drive Key. An example implementation, using Dart, is available [here](https://github.com/ardriveapp/ardrive-web/blob/187b3fb30808bda452123c2b18931c898df6a3fb/docs/private_drive_kdf_reference.dart), with a Typescript implementation [here](https://github.com/ardriveapp/ardrive-core-js/blob/f19da30efd30a4370be53c9b07834eae764f8535/src/utils/crypto.ts). ## Private Drives Drives can store either public or private data. This is indicated by the `Drive-Privacy` tag in the Drive entity metadata. ``` Drive-Privacy: "" ``` If a Drive entity is private, an additional tag `Drive-Auth-Mode` must also be used to indicate how the Drive Key is derived. ArDrive clients currently leverage a secure password along with the Arweave Wallet private key signature to derive the global Drive Key. ``` Drive-Auth-Mode?: 'password' ``` On every encrypted Drive Entity, a `Cipher` tag must be specified, along with the public parameters for decrypting the data. This is done by specifying the parameter with a `Cipher-*` tag. eg. `Cipher-IV`. If the parameter is byte data, it must be encoded as Base64 in the tag. ArDrive clients currently leverage AES256-GCM for all symmetric encryption, which requires a Cipher Initialization Vector consisting of 12 random bytes. ``` Cipher?: "AES256-GCM" Cipher-IV?: "" ``` Additionally, all encrypted transactions must have the `Content-Type` tag `application/octet-stream` as opposed to `application/json` Private Drive Entities and their corresponding Root Folder Entities will both use these keys and ciphers generated to symmetrically encrypt the JSON files that are included in the transaction. This ensures that only the Drive Owner (and whomever the keys have been shared with) can open the drive, discover the root folder, and continue to load the rest of the children in the drive. ## Private Files When a file is uploaded to a private drive, it by default also becomes private and leverages the same drive keys used for its parent drive. Each unique file in a drive will get its own set of file keys based off of that file's unique `FileId`. If a single file gets a new version, its `File-Id` will be reused, effectively leveraging the same File Key for all versions in that file's history. These file keys can be shared by the drive's owner as needed. Private File entities have both its metadata and data transactions encrypted using the same File Key, ensuring all facets of the data is truly private. As such, both the file's metadata and data transactions must both have a unique `Cipher-IV` and `Cipher` tag: ``` Cipher?: "AES256-GCM" Cipher-IV?: "" ``` Just like drives, private files must have the `Content-Type` tag set as `application/octet-stream` in both its metadata and data transactions: ``` Content-Type: "application/octet-stream" ``` ## Encryption Process Here's how the encryption process works for private drives: ```mermaid sequenceDiagram participant User participant Client participant Wallet participant Arweave User->>Client: Create private drive Client->>Wallet: Request signature Wallet->>Client: Return signature Client->>Client: Derive drive key Client->>Client: Encrypt drive metadata Client->>Arweave: Upload encrypted drive User->>Client: Upload file to private drive Client->>Client: Derive file key Client->>Client: Encrypt file metadata Client->>Client: Encrypt file data Client->>Arweave: Upload encrypted metadata Client->>Arweave: Upload encrypted data ``` ## Security Best Practices When working with private drives, follow these security guidelines: ### Password Management - Use strong, unique passwords for each drive - Consider using a password manager - Never share passwords in plain text ### Key Storage - Never store drive keys in plain text - Use secure key derivation functions - Implement proper key rotation if needed ### Access Control - Share file keys only with authorized users - Implement proper access logging - Regularly audit drive access ### Data Handling - Encrypt data before transmission - Use secure communication channels - Implement proper error handling ## Drive Signature (ArFS v0.15) ArFS v0.15 introduces a new `Drive-Signature` entity type to help bridge signature derivation schemes across ArFS versions. This is particularly important for maintaining access to private drives created with older wallet signing methods. The drive signature entity stores an encrypted version of the pre-v0.15 wallet signature that's necessary for deriving the drive key. This allows continued access to historical drive contents while using modern wallet signing APIs. ```mermaid graph TD A[Legacy Wallet Signature] --> B[Encrypt with v0.15 scheme] B --> C[Drive-Signature Entity] C --> D[Store on Arweave] D --> E[Retrieve when needed] E --> F[Decrypt signature] F --> G[Use for drive key derivation] style A fill:#ffecb3 style C fill:#c8e6c9 style G fill:#e1f5fe ``` ## Next Steps Ready to implement privacy in your ArFS applications? - [Creating Private Drives](/build/advanced/arfs/creating-drives) - Learn how to create secure drives - [Upgrading Private Drives](/build/advanced/arfs/upgrading-drives) - Update legacy drives to v0.15 - [Reading Data](/build/advanced/arfs/reading-data) - Query and decrypt your private data # Reading Data (/build/advanced/arfs/reading-data) Clients can perform read operations to create a timeline of entity write transactions which can then be replayed to construct the Drive state. This is done by querying an Arweave GraphQL index for the user's respective transactions. [Arweave GraphQL Guide](https://gql-guide.vercel.app/) can provide more information on how to use Arweave GraphQL. If no GraphQL index is available, drive state can only be generated by downloading and inspecting all transactions made by the user's wallet. This timeline of transactions should be grouped by the block number of each transaction. At every step of the timeline, the client can check if the entity was written by an authorized user. This also conveniently enables the client to surface a trusted entity version history to the user. To determine the owner of a Drive, clients must check for who created the first Drive Entity transaction using that `Drive-Id`. Until a trusted permissions or ACL system is put in place, any transaction in a drive created by any wallet other than the one who created the first Drive Entity transaction could be considered spam. The `Unix-Time` defined on each transaction should be reserved for tie-breaking same entity updates in the same block and should not be trusted as the source of truth for entity write ordering. This is unimportant for single owner drives but is crucial for multi-owner drives with updateable permissions (currently undefined in this spec) as a malicious user could fake the `Unix-Time` to modify the drive timeline for other users. - Drives that have been updated many times can have a long entity timeline which can be a performance bottleneck. To avoid this, clients can cache the drive state locally and sync updates to the file system by only querying for entities in blocks higher than the last time they checked. - Not checking for Drive Ownership could result in seeing incorrect drive state and GraphQL queries. ## Folder/File Paths ArweaveFS does not store folder or file paths along with entities as these paths will need to be updated whenever the parent folder name changes which can require many updates for deeply nested file systems. Instead, folder/file paths are left for the client to generate from the folder/file names. ## Folder View Queries Clients that want to provide users with a quick view of a single folder can simply query for an entity timeline for a particular folder by its id. Clients with multi-owner permissions will additionally have to query for the folder's parent drive entity for permission based filtering of the timeline. ## Basic Query Patterns ### Query All Drive Entities ```graphql query { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Entity-Type", values: ["drive"] } { name: "Drive-Id", values: ["your-drive-id"] } ] ) { edges { node { id block { height timestamp } tags { name value } } } } } ``` ### Query Folder Contents ```graphql query ($parentFolderId: String!) { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Parent-Folder-Id", values: [$parentFolderId] } ] ) { edges { node { id block { height timestamp } tags { name value } } } } } ``` ### Query File Entities ```graphql query ($fileId: String!) { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Entity-Type", values: ["file"] } { name: "File-Id", values: [$fileId] } ] ) { edges { node { id block { height timestamp } tags { name value } } } } } ``` ## Building Drive State The process of building drive state involves several steps: ```mermaid graph TD A[Query Drive Entities] --> B[Sort by Block Height] B --> C[Process Files First] C --> D[Process Folders] D --> E[Process Drive Metadata] E --> F[Build Hierarchy Tree] F --> G[Resolve Conflicts] G --> H[Return Complete State] style A fill:#e3f2fd style H fill:#c8e6c9 ``` ### Step-by-Step Process 1. **Query for all entities** associated with a specific `Drive-Id` 2. **Sort by block height** to establish chronological order 3. **Process entities bottom-up** starting with files and folders 4. **Build the hierarchy** by following parent-child relationships 5. **Handle conflicts** by using the most recent entity version ### Example Implementation ```javascript async function buildDriveState(driveId) { // Query all entities for the drive const entities = await queryDriveEntities(driveId); // Sort by block height entities.sort((a, b) => a.block.height - b.block.height); // Process entities const driveState = { drive: null, folders: new Map(), files: new Map(), }; for (const entity of entities) { const entityType = getTagValue(entity.tags, "Entity-Type"); switch (entityType) { case "drive": driveState.drive = processDriveEntity(entity); break; case "folder": driveState.folders.set( getTagValue(entity.tags, "Folder-Id"), processFolderEntity(entity) ); break; case "file": driveState.files.set( getTagValue(entity.tags, "File-Id"), processFileEntity(entity) ); break; } } return driveState; } ``` ## Using Snapshots For large drives, snapshots can significantly improve performance: ```mermaid sequenceDiagram participant Client participant Gateway participant Arweave Client->>Gateway: Query for latest snapshot Gateway->>Client: Return snapshot data Client->>Client: Process snapshot data Client->>Gateway: Query for newer transactions Gateway->>Client: Return newer entities Client->>Client: Merge with snapshot data Client->>Client: Return complete drive state ``` ### Snapshot Query ```graphql query ($driveId: String!) { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Entity-Type", values: ["snapshot"] } { name: "Drive-Id", values: [$driveId] } ] sort: HEIGHT_DESC first: 1 ) { edges { node { id block { height timestamp } tags { name value } } } } } ``` ## Performance Optimization ### Caching Strategies - **Local caching** - Store frequently accessed data locally - **Incremental updates** - Only fetch new transactions since last sync - **Snapshot usage** - Use snapshots for large drives - **Batch queries** - Combine multiple queries when possible ### Query Optimization - **Use specific tags** - Narrow down queries with relevant tags - **Limit results** - Use pagination for large result sets - **Filter by date** - Query specific time ranges - **Index utilization** - Leverage GraphQL indexes effectively ## Error Handling ### Common Issues - **Network timeouts** - Implement retry logic - **Invalid data** - Validate entity structure - **Missing entities** - Handle incomplete data gracefully - **Decryption errors** - Proper error handling for private data ### Best Practices - **Validate ownership** - Check drive ownership before processing - **Handle conflicts** - Resolve entity version conflicts - **Graceful degradation** - Provide fallbacks for missing data - **User feedback** - Inform users of sync status ## Security Considerations ### Data Validation - **Verify signatures** - Check transaction signatures - **Validate ownership** - Ensure drive ownership - **Check timestamps** - Validate entity timestamps - **Sanitize data** - Clean user-provided data ### Privacy Protection - **Decrypt carefully** - Handle private data securely - **Key management** - Protect encryption keys - **Access control** - Implement proper permissions - **Audit logging** - Track data access ## Next Steps Now that you understand how to read ArFS data, explore these related topics: - [Privacy & Encryption](/build/advanced/arfs/privacy) - Secure your data with private drives - [Upgrading Private Drives](/build/advanced/arfs/upgrading-drives) - Update legacy drives to v0.15 - [Creating Drives](/build/advanced/arfs/creating-drives) - Start building with ArFS # Upgrading Private Drives (/build/advanced/arfs/upgrading-drives) ## Overview Private drives rely on a combination of user-set password and a wallet signature for encryption and decryption. [Wander](https://www.wander.app/), formerly ArConnect, is a popular Arweave wallet that is deprecating its `signature()` method in favor of `signDataItem()` or `signMessage()`. In order to preserve access to private drive contents that were secured via drive keys created via 'signature()', ArFS v0.15 introduces a new drive key derivation scheme that both utilizes the modern signing APIs and bridges historical drive keys for usage with it. Because private drive entities exist on chain and their encryption cannot be altered, an upgrade is required to allow continued access to "V1" private drives. This upgrade essentially takes a signature from the drive owner wallet, encrypts it using the required signature structure for V2 private drives, and places it on Arweave as a new "Drive-Signature" entity. This allows the signature to be fetched and decrypted using the latest methods before using it to decrypt the private drive in the V1 format. The below instructions for upgrading a private drive will work during the deprecation period for the `signature()` method from Wanter. Once this period is over, and `signature()` loses all support, additional steps will be required to obtain the correct signature format to decrypt V1 private drives in order to upgrade them. There is, at this time, no set date for when the deprecation period will end. ## The Upgrade Process The upgrade process involves creating a new `Drive-Signature` entity that contains an encrypted version of the legacy signature needed to decrypt the private drive. ```mermaid sequenceDiagram participant User participant Client participant Wallet participant Arweave User->>Client: Initiate drive upgrade Client->>Wallet: Request legacy signature Wallet->>Client: Return signature Client->>Client: Encrypt signature with v0.15 scheme Client->>Arweave: Upload Drive-Signature entity Client->>Client: Update drive with Signature-Type tag Client->>Arweave: Upload updated drive entity Client->>User: Upgrade complete ``` ### Drive-Signature Entity The `Drive-Signature` entity stores the encrypted legacy signature: ```json ArFS: "0.15", Entity-Type: "drive-signature", Signature-Format: "1", Cipher?: "AES256-GCM", Cipher-IV: "" {data: } ``` ### Updated Drive Entity The drive entity is updated with a new `Signature-Type` tag: ```json ArFS: "0.15", Cipher?: "AES256-GCM", Cipher-IV?: "", Content-Type: "", Drive-Id: "", Drive-Privacy: "", Drive-Auth-Mode?: "password", Entity-Type: "drive", Signature-Type?: "1", Unix-Time: "" Metadata JSON { "name": "", "rootFolderId": "", "isHidden": false } ``` ## Using ArDrive The upgrade process has been made simple by using the [ArDrive app](https://app.ardrive.io/). ### Step 1: Log into ArDrive If the connected wallet has V1 private drives that need to be updated, a banner will appear at the top of the screen. ![ArDrive Upgrade Banner](https://turbo-gateway.com/kJzzrYY4KIHLTC9VOECfzvVAjaO1_FOkejWgsRNbLx4) ### Step 2: Click "Update Now!" This will open a modal listing the drives that need to be updated, and linking to more information about the upgrade process. ![ArDrive Upgrade Modal](https://turbo-gateway.com/Qa-qeKkr1flXl1-fdLapjKNmme0lKA43YQbjmMvnF-U) ### Step 3: Click "Update" The process of upgrading the private drives will begin, and involve signing messages depending on how many drives are being upgraded. When the process is complete, a new modal will appear listing the drives that have been successfully updated. ![ArDrive Upgrade Complete](https://turbo-gateway.com/qEwT3oFZbFDpmRQpw9j1c_okN5pIkFqRuoSwBzhc3HQ) ## Manual Upgrade Process If you need to upgrade drives programmatically, here's the process: ### 1. Identify V1 Drives Query for drives that don't have the `Signature-Type` tag: ```graphql query { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Entity-Type", values: ["drive"] } { name: "Drive-Privacy", values: ["private"] } ] ) { edges { node { id tags { name value } } } } } ``` ### 2. Create Drive-Signature Entity ```javascript async function createDriveSignature(driveId, legacySignature) { // Encrypt the legacy signature const encryptedSignature = await encryptSignature(legacySignature); // Create the drive signature entity const driveSignature = { data: encryptedSignature, tags: [ { name: "ArFS", value: "0.15" }, { name: "Entity-Type", value: "drive-signature" }, { name: "Signature-Format", value: "1" }, { name: "Cipher", value: "AES256-GCM" }, { name: "Cipher-IV", value: cipherIV }, ], }; // Upload to Arweave return await uploadTransaction(driveSignature); } ``` ### 3. Update Drive Entity ```javascript async function updateDriveEntity(driveId) { // Get existing drive entity const driveEntity = await getDriveEntity(driveId); // Add Signature-Type tag const updatedTags = [ ...driveEntity.tags, { name: "Signature-Type", value: "1" }, ]; // Create updated drive entity const updatedDrive = { data: driveEntity.data, tags: updatedTags, }; // Upload to Arweave return await uploadTransaction(updatedDrive); } ``` ## Verification After upgrading, verify the process was successful: ### Check Drive-Signature Entity ```graphql query ($driveId: String!) { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Entity-Type", values: ["drive-signature"] } { name: "Drive-Id", values: [$driveId] } ] ) { edges { node { id block { height timestamp } tags { name value } } } } } ``` ### Check Updated Drive Entity ```graphql query ($driveId: String!) { transactions( tags: [ { name: "ArFS", values: ["0.15"] } { name: "Entity-Type", values: ["drive"] } { name: "Drive-Id", values: [$driveId] } { name: "Signature-Type", values: ["1"] } ] ) { edges { node { id block { height timestamp } tags { name value } } } } } ``` ## Troubleshooting ### Common Issues - **Signature not found** - Ensure the wallet supports the required signing methods - **Encryption errors** - Verify the encryption parameters are correct - **Upload failures** - Check network connectivity and retry - **Permission denied** - Ensure you own the drive being upgraded ### Error Handling ```javascript async function upgradeDrive(driveId) { try { // Get legacy signature const legacySignature = await getLegacySignature(driveId); // Create drive signature entity await createDriveSignature(driveId, legacySignature); // Update drive entity await updateDriveEntity(driveId); console.log("Drive upgraded successfully"); } catch (error) { console.error("Upgrade failed:", error); // Handle error appropriately } } ``` ## Best Practices ### Before Upgrading - **Backup your data** - Ensure you have access to your drive contents - **Test with one drive** - Start with a single drive to verify the process - **Check wallet compatibility** - Ensure your wallet supports required methods - **Verify ownership** - Confirm you own the drives being upgraded ### During Upgrading - **Monitor progress** - Keep track of upgrade status - **Handle errors gracefully** - Implement proper error handling - **Batch operations** - Upgrade multiple drives efficiently - **User feedback** - Provide clear status updates ### After Upgrading - **Verify functionality** - Test drive access and operations - **Update clients** - Ensure all clients support v0.15 - **Monitor performance** - Check for any performance issues - **Document changes** - Keep track of upgraded drives ## Migration Timeline ```mermaid gantt title ArFS v0.15 Migration Timeline dateFormat YYYY-MM-DD section Phase 1 Legacy Support :active, legacy, 2024-01-01, 2024-06-30 section Phase 2 Migration Period :migration, 2024-07-01, 2024-12-31 section Phase 3 Legacy Deprecation :deprecation, 2025-01-01, 2025-06-30 section Phase 4 Full v0.15 :full, 2025-07-01, 2025-12-31 ``` ## Next Steps After upgrading your drives, explore these related topics: - [Privacy & Encryption](/build/advanced/arfs/privacy) - Understand the new encryption scheme - [Reading Data](/build/advanced/arfs/reading-data) - Query your upgraded drives - [Creating Drives](/build/advanced/arfs/creating-drives) - Create new v0.15 drives # EthAReum Protocol (/build/advanced/ethareum) The **EthAReum protocol** enables the generation of private keys for an Arweave wallet using a signature from an Ethereum or Solana wallet. This allows users to create an Arweave wallet directly through popular wallet providers like MetaMask, providing seamless cross-chain wallet management. Generated private keys provide a fully functional Arweave wallet, equipped to perform all standard operations, including holding AR tokens and Turbo Credits, and uploading data to the Arweave network. ## How It Works EthAReum uses a deterministic key derivation process that combines: - **Ethereum/Solana wallet signature** - Provides the cryptographic foundation - **User-generated password** - Adds additional entropy and security - **Standardized derivation algorithm** - Ensures reproducible results The protocol generates a unique Arweave wallet that is cryptographically linked to your Ethereum or Solana wallet but remains completely independent. ## Browser Compatibility **Recommended Browser**: For optimal performance, use **Chrome** when working with EthAReum and MetaMask. While EthAReum functions correctly in most browsers, there are ongoing efforts to resolve some edge case compatibility issues in other environments. ## Password Security The EthAReum protocol incorporates a user-generated password in the wallet derivation process. This password provides an extra layer of security by contributing additional entropy to the wallet's derivation and serves as a critical verification step for wallet access. **Permanent Password**: The password used during the derivation of private keys is **permanent and cannot be changed or recovered** by any administrator. ArDrive is a decentralized platform with no account administration. It is crucial to keep this password secure. ### Password Requirements - Must be set during initial wallet creation - Used for all subsequent logins - Required for encrypting private uploads - Cannot be recovered if forgotten ## Wallet Addresses The public address of the generated Arweave wallet is derived from its public key and will be **different** from the public address of the Ethereum or Solana wallet used to generate it. ### Viewing Your Address The exact steps to obtain your generated wallet's public address depend on the dApp interface: - **ArDrive**: Click the user profile icon in the top right when logged in - **Other dApps**: Check the wallet settings or profile section ## Key Management ### Keyfiles vs Seed Phrases The Arweave ecosystem primarily uses **keyfiles** rather than seed phrases for wallet access: - **Keyfile**: JSON file containing a Json Web Key (JWK) that acts as private keys - **Seed Phrase**: Supported but not universally implemented across all dApps ### Accessing Your Keys Both keyfile and seed phrase are available for download in most dApps: - **ArDrive**: Click the user profile icon in the top right when logged in - **Other dApps**: Check wallet settings or ## Security Considerations ### One-Way Control EthAReum generates Arweave wallet private keys using a signature from your Ethereum/Solana wallet, ensuring that control only extends in one direction: - ✅ **EthAReum can generate** Arweave wallets from Ethereum/Solana signatures - ❌ **EthAReum cannot access** your Ethereum/Solana wallet or assets - ✅ **Your Ethereum/Solana assets remain** completely secure and independent ### Signature Security **Beware of Malicious dApps**: Some malicious dApps or websites may disguise high-risk authorization transactions as simple signature requests. Always ensure that you only provide signatures to reputable and trusted dApps like ArDrive. ### Best Practices 1. **Verify dApp authenticity** before providing signatures 2. **Use strong, unique passwords** for wallet derivation 3. **Backup your keyfile** in a secure location 4. **Never share your password** or keyfile with anyone 5. **Test with small amounts** before committing to large transactions ## Implementation Examples ### Basic Wallet Generation ```javascript // Example: Generate Arweave wallet from Ethereum signature async function generateArweaveWallet(ethereumSignature, password) { // This is a conceptual example - actual implementation // would use the EthAReum protocol specification const derivedKey = await deriveKeyFromSignature( ethereumSignature, password, "arweave" // derivation context ); return { address: getAddressFromKey(derivedKey), keyfile: createKeyfile(derivedKey), seedPhrase: generateSeedPhrase(derivedKey), }; } ``` ### Integration with MetaMask ```javascript // Example: Request signature from MetaMask async function requestEthereumSignature() { const accounts = await ethereum.request({ method: "eth_requestAccounts", }); const message = "Sign this message to generate your Arweave wallet"; const signature = await ethereum.request({ method: "personal_sign", params: [message, accounts[0]], }); return signature; } ``` ## Use Cases ### Cross-Chain dApp Development - **Unified wallet experience** across Ethereum and Arweave - **Simplified onboarding** for users familiar with Ethereum - **Reduced friction** in multi-chain applications ### Data Storage Solutions - **Decentralized file storage** using existing Ethereum wallets - **NFT metadata storage** on Arweave with Ethereum wallet access - **Cross-chain data management** for DeFi applications ### Developer Benefits - **Familiar wallet interfaces** for users - **Reduced development complexity** for multi-chain apps - **Enhanced user experience** with single wallet management ## Next Steps Learn about structured data storage on Arweave using your generated wallet. Upload data efficiently using Turbo Credits with your EthAReum wallet. Learn how to find and access data stored with your generated wallet. # Advanced (/build/advanced) ## Overview Explore advanced topics and specialized guides for building on Arweave and ar.io. These resources are designed for developers and operators who need deeper technical knowledge and advanced configuration options. ## Advanced Topics **Advanced ArFS documentation** for structured data storage **Understanding wallet address normalization** across different networks **Security mechanisms** in ar.io gateways **Key topics:** - Same-origin policy **Generate Arweave wallets** from Ethereum or Solana wallets ## Ready to Go Advanced? **New to Arweave?** Start with our [Getting Started guide](/build) to understand the basics. **Building dApps?** Check out [ArFS Protocol](/build/advanced/arfs) for structured data storage solutions. # Normalized Addresses (/build/advanced/normalized-addresses) ## Overview Different blockchains use different formats for the [public keys](/glossary) of wallets, and the [native addresses](/glossary) for those wallets. In most cases, when a system in the Arweave ecosystem needs to display the wallet address of a wallet from a different blockchain, for instance in the `Owner.address` value of a data item signed by an ETH or Solana wallet, that address will be normalized into the format recognized by Arweave. Specifically, a 43 character base64url representation of the sha256 hash of the public key. This is done to prevent potential errors by systems in the Arweave ecosystem that expect these values to be a certain size and conform to a specific format. Essentially, normalized addresses are a way to represent public keys and wallet addresses from other blockchains in a way that is familiar to systems in the Arweave ecosystem. A tool for easily obtaining a normalized addresses from public keys can be found at [ar://normalize-my-key](https://normalize-my-key.ar.io/) ## At A Glance | | Arweave | ETH/POL | Solana | | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | -------------------------------------------- | | **Native Address** | 9ODOd-\_ZT9oWoRMVmmD4G5f9Z6MjvYxO3Nen-T5OXvU | 0x084af408C8E492aC52dc0Ec76514A7deF8D5F03f | Cd5yb4mvbuQyyJgAkriFZbWQivh2zM68KGZX8Ksn1L85 | | **base64url Encoded Public Key** | 0jkGWDFYI3DHEWaXhZitjTg67T-enQwXs50lTDrMhy2qb619_91drv_50J5PwrOYJiMmYhiEA5ojMvrrAFY-Dm1bJbJfVBU1kIsPho2tFcXnbSOa2_1bovAys0ckJU07wkbmIUpzp3trdxYReB4jayMMOXWw9B8xS0v81zFmK3IbCtL9N6WNTMONOSMATHFQrGqtDhDUqKyIsQZCBPFvfGykRWaLWzbtAUrApprqG9hfExQzppNsw0gsftNSHZ1emC5tC2fuib6FhQw9TE2ge9tUjEZNALcVZvopTtTX0H2gEfnRJ48UNeV3SKggjXcoPVeivmqXuPBGncXWWq1pHR-Xs4zSLA5Mgcw_tQJc4FIER0i7hUlZXoc991ZHyOvAC-GlHWzQwvrlY11oD38pB47NkHN2WVPtUCAtyYQe5TE6Xznd9kPgqqvVUkV0s0suh5vINGoiPEnMjyhYEN7eOmJRIJ_A87IJesbdPRV4ZzBsqPbd02RG3ZuVpc3gI1xKvwH1WS05XI8eWK-BbvB3oxB7WjaQTWcfBWhMEULiwx-SucuyAzPAw3i6Wjtq61TcL9SdWhmOf9_yo-Np052tj7MQ66nmgdOH_MEKYjAdFypxTsRQoSLbv28HEcSjwx8u3pY0q0gKMK_5X2XKJrp2i2GB_fVgbcpH9YsgrYxh1Q8 | 2W5VMzNKYwr51QsiYBHUS5h5wxZf_uBgG7C6xiHgBHwwLUty5LHKFFBDlAxTCTAhglcmys2_HQoOj_LnCkA3 | rK8XXxd8JqsZFPXVOwkSWS5Gh1SJzftfCOLpLk4i1FY | | **Normalized Address** | 9ODOd-\_ZT9oWoRMVmmD4G5f9Z6MjvYxO3Nen-T5OXvU | 5JtuS4yOFtUX2Rg3UU7AgBaUqh4s8wyyNTZk9UrzI-Q | K8kpPM1RID8ZM2sjF5mYy0rP4gXSRDbrwPUd9Qths64 | ## Public Keys and Addresses Crypto wallets consist of two separate components. The public keys, which are public knowledge and can be seen by anyone, and the private keys, which only the owner of a wallet should have access to. Crypto wallet addresses are derived from the public key. {" "} It is important to note that all crypto wallet public and private keys are binary data. The values provided below for Arweave and Ethereum/Polygon public keys are base64url and hex encoded representations of that binary data respectively. ### Arweave The public key for an Arweave wallet is the `n` field of the JWK json file. 0jkGWDFYI3DHEWaXhZitjTg67T-enQwXs50lTDrMhy2qb619_91drv_50J5PwrOYJiMmYhiEA5ojMvrrAFY-Dm1bJbJfVBU1kIsPho2tFcXnbSOa2_1bovAys0ckJU07wkbmIUpzp3trdxYReB4jayMMOXWw9B8xS0v81zFmK3IbCtL9N6WNTMONOSMATHFQrGqtDhDUqKyIsQZCBPFvfGykRWaLWzbtAUrApprqG9hfExQzppNsw0gsftNSHZ1emC5tC2fuib6FhQw9TE2ge9tUjEZNALcVZvopTtTX0H2gEfnRJ48UNeV3SKggjXcoPVeivmqXuPBGncXWWq1pHR-Xs4zSLA5Mgcw_tQJc4FIER0i7hUlZXoc991ZHyOvAC-GlHWzQwvrlY11oD38pB47NkHN2WVPtUCAtyYQe5TE6Xznd9kPgqqvVUkV0s0suh5vINGoiPEnMjyhYEN7eOmJRIJ_A87IJesbdPRV4ZzBsqPbd02RG3ZuVpc3gI1xKvwH1WS05XI8eWK-BbvB3oxB7WjaQTWcfBWhMEULiwx-SucuyAzPAw3i6Wjtq61TcL9SdWhmOf9_yo-Np052tj7MQ66nmgdOH_MEKYjAdFypxTsRQoSLbv28HEcSjwx8u3pY0q0gKMK_5X2XKJrp2i2GB_fVgbcpH9YsgrYxh1Q8 The public wallet address for that wallet is `9ODOd-_ZT9oWoRMVmmD4G5f9Z6MjvYxO3Nen-T5OXvU`, this is obtained by decoding the public key from base64url to normalize padding, sha256 hashing the result, and then base64url encoding that. ### Ethereum/Polygon The public key for an EVM wallet (Ethereum, Polygon/Matic) is derived from its private key, using the [Elliptic Curve Digital Signature Algorithm](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm), or ECDSA. `0xb5d96e5533334a630af9d50b226011d44b9879c3165ffee0601bb0bac621e0047c302d4b72e4b1ca145043940c53093021825726cacdbf1d0a0e8ff2e70a4037` The public wallet address is `0x084af408C8E492aC52dc0Ec76514A7deF8D5F03f`, this is obtained by removing the first byte from the public key, Keccak-256 hashing the remainder, taking the the last 20 bytes (40 hexadecimal characters) and prepending `0x` to it. ### Solana A Solana wallet is an array of 64 bytes. The first 32 bytes are the private key, and the last 32 bytes are the public key. Below is the public key portion of a Solana wallet: `[172, 175, 23, 95, 23, 124, 38, 171, 25, 20, 245, 213, 59, 9, 18, 89, 46, 70, 135, 84, 137, 205, 251, 95, 8, 226, 233, 46, 78, 34, 212, 86]` The public wallet address for this wallet is `Cd5yb4mvbuQyyJgAkriFZbWQivh2zM68KGZX8Ksn1L85`, this is derived by base58 encoding the public key bytes. ## Normalizing Addresses As shown in the above examples, the format of public keys, and the resulting derived wallet addresses, vary widely between blockchains. Arweave manages this by applying the same derivation methods that Arweave uses for its own wallets to the public keys from other chains. ### Ethereum/Polygon The leading `0x` and uncompressed flag `04` (if present) is removed from the public key of an EVM wallet, and then the remainder is base64url encoded to obtain the Arweave normalized public key. Continuing with the same public key in the above example, the normalized public key would be: `2W5VMzNKYwr51QsiYBHUS5h5wxZf_uBgG7C6xiHgBHwwLUty5LHKFFBDlAxTCTAhglcmys2_HQoOj_LnCkA3` This value is what is used as the GraphQL tag `owner` value for data items being uploaded to Arweave using an EVM wallet. The normalized address is then derived from this value by sha256 hashing it, and then base64url encoding the result: `5JtuS4yOFtUX2Rg3UU7AgBaUqh4s8wyyNTZk9UrzI-Q` ### Solana The normalized public key for Solana wallets are derived similarly. The 32 byte public key is base64url encoded: `rK8XXxd8JqsZFPXVOwkSWS5Gh1SJzftfCOLpLk4i1FY` Again, this value is used for the GraphQl tag `owner` when uploading data. It can then be sha256 hashed, and base64url encoded again to derive the normalized address: `K8kpPM1RID8ZM2sjF5mYy0rP4gXSRDbrwPUd9Qths64` ## Discovering Public Keys from On-Chain Transactions In many cases, you may only have a wallet's native address and need to derive its normalized address. Since normalized addresses are derived from public keys, and public keys are not always readily available, you can recover them from onchain transaction signatures. The [ar://normalize-my-key](https://normalize-my-key.ar.io/) tool implements this discovery process, allowing you to look up an address or transaction hash/signature to automatically recover the public key and derive the normalized address. ### EVM Chains (Ethereum, Polygon, Base) EVM chains use the Elliptic Curve Digital Signature Algorithm (ECDSA) for transaction signing. A unique property of ECDSA is that the public key can be mathematically recovered from a valid signature and the signed message hash. The recovery process works as follows: 1. **Fetch a transaction** where the target address is the sender (the `from` field) 2. **Reconstruct the unsigned transaction** by extracting the transaction fields (nonce, to, value, gas, data, chainId, etc.) and determining the transaction type (legacy, EIP-2930, or EIP-1559) 3. **Serialize the transaction** according to its type and compute the Keccak-256 hash of the serialized data 4. **Extract the signature components** (r, s, v) from the transaction 5. **Recover the public key** using ECDSA recovery with the message hash and signature The `v` value encodes the recovery bit needed to identify which of the two possible public keys is correct: - For EIP-1559/EIP-2930 transactions: `v` is already 0 or 1 - For pre-EIP-155 legacy transactions: `v` is 27 or 28, subtract 27 to get the recovery bit - For EIP-155 transactions: `v = chainId * 2 + 35 + recoveryBit` Once the 65-byte uncompressed public key is recovered, the normalized address is derived by removing the `0x` prefix and uncompressed flag (`04`), base64url encoding the remaining 64 bytes, then sha256 hashing and base64url encoding the result. Public key recovery only works with transactions where the target address is the **sender**. If an address has only received funds but never sent any, there will be no signature from which to recover the public key. ### Solana Solana transactions include the signer's public key directly in the transaction's account keys array. The first account key in any transaction is always the fee payer/signer. The discovery process is simpler: 1. **Fetch a transaction** by its signature using the Solana RPC 2. **Extract the first account key** from the transaction's account keys (this is the signer's public key) 3. **Convert to normalized format** by base64url encoding the 32-byte public key, then sha256 hashing and base64url encoding the result Unlike EVM chains, Solana does not require cryptographic recovery since the public key is explicitly included in the transaction data. # Browser Sandboxing (/build/advanced/sandboxing) ## Overview Browser sandboxing allows data requests to a gateway node to benefit from the security advantages of using a browser's same-origin policy by redirecting the requests to a pseudo-unique subdomain of the gateway's apex domain. For example, an attempt to access `https://turbo-gateway.com/gnWKBqFXMJrrksEWrXLQRUQQQeFhv4uVxesHBcT8i6o` would redirect to `https://qj2yubvbk4yjv24syelk24wqivcbaqpbmg7yxfof5mdqlrh4rova.turbo-gateway.com/gnWKBqFXMJrrksEWrXLQRUQQQeFhv4uVxesHBcT8i6o` Two DNS records are required to link a domain to an Arweave transaction on a gateway node. For example, `www.mycustomsite.com` would need the following records to link it to `www.arweave-gateway.net`: - A DNS CNAME record pointing to an Arweave gateway: www CNAME `arweave-gateway.net`, - A DNS TXT record linking the domain with a specific transaction ID: arweavetx TXT `kTv4OkVtmc0NAsqIcnHfudKjykJeQ83qXXrxf8hrh0S` When a browser requests `www.mycustomsite.com` the user's machine will (through the usual DNS processes) resolve this to the IP address for the gateway node `arweave-gateway.net`. When the gateway receives an HTTP request with a non-default hostname, e.g. `www.mycustomsite.com` instead of `www.arweave-gateway.net`, the gateway will query the DNS records for `www.mycustomsite.com` and the 'arweavetx' TXT record will tell the node which transaction to serve. ## TLS and its Role in Browser Sandboxing Transport Layer Security (TLS) is a cryptographic protocol designed to provide communications security over a computer network. In the context of Arweave applications and browser sandboxing, TLS plays a critical role in ensuring secure data transmission and enabling the effective use of browser security features. When Arweave applications are accessed without TLS, most browsers restrict the use of native cryptographic functions. These functions, which include hashing, signing, and verification, are essential for the secure operation of Arweave permaweb apps. Without TLS, not only are these functions unavailable, but the applications also become susceptible to various security threats, notably man-in-the-middle (MITM) attacks. Although Arweave transactions are signed, making direct MITM attacks challenging, the absence of encryption can expose other vulnerabilities. For instance, attackers could intercept and alter the `/price` endpoint, potentially causing transaction failures or leading to overcharging. To address these concerns, gateway operators are responsible for generating and maintaining TLS certificates for their gateways. This can be achieved through various systems, such as ACME for Let's Encrypt. An important step in setting up a gateway is obtaining a wildcard TLS certificate for the gateway's domain. This certificate secures traffic on both the apex domain and its single-level subdomains (e.g., `gateway.com` and `subdomain.gateway.com`). The integration of TLS is crucial for the implementation of browser sandboxing. When a browser requests a transaction from a gateway, the gateway issues a 301 redirect to a subdomain of the gateway, using a Base32 pseudo-unique address derived from the transaction ID. This redirection, secured by TLS, invokes the browser's same-origin policy. As a result, the requested web page is confined within a secure sandbox environment, isolated from other domains. This isolation is vital for maintaining the integrity and security of transactions and interactions within Arweave's permaweb applications. ## Deriving Sandbox Value Ar.io nodes generate browser sandbox values deterministically. Because of this, it is possible to calculate ahead of time what that value will be for a particular transaction id. Sandbox values are a Base32 encoding of the transaction ID. Ar.io gateways use the following code snippet to accomplish the encoding: ```typescript const expectedTxSandbox = (id: string): string => { return toB32(fromB64Url(id)) } ``` Example: ```typescript const id = 'gnWKBqFXMJrrksEWrXLQRUQQQeFhv4uVxesHBcT8i6o' const expectedTxSandbox = (id): string => { return toB32(fromB64Url(id)) } console.log(expectedTxSandbox) ``` Example Output: ```console qj2yubvbk4yjv24syelk24wqivcbaqpbmg7yxfof5mdqlrh4rova ``` View the full code for generating browser sandbox values [here](https://github.com/ar-io/arweave-gateway/blob/719f43f8d6135adf44c87701e95f58105638710a/src/gateway/middleware/sandbox.ts#L69). # AI Agents & LLMs (/build/agents) These docs are built to be consumed by **AI coding agents**, not just read by humans. Everything below is a plain-text file you can fetch, paste, or point an agent at — no scraping required. **Fastest path:** point your agent at `https://docs.ar.io/llms.txt`. It's a short index that links everything else, following the [llmstxt.org](https://llmstxt.org) convention agents already look for. ## Machine-readable entry points | File | What it is | Reach for it when | |---|---|---| | `/llms.txt` | A short **index** of the documentation with links and quick facts | Discovery — the entry point | | `/SKILL.md` | A **procedural agent skill**: SDK patterns, code recipes, URL conventions, constants | Generating working code | | `/llms-full.txt` | The **entire docs corpus** as one plain-text file (~1.2 MB) | Deep context, or RAG ingestion | | `/sdks//llm.txt` | **Per-SDK reference** text (see table below) | Working inside a single SDK | ### `llm.txt` vs `SKILL.md` — what's the difference? They solve different problems and are meant to be used together: - **`llms.txt` / `llms-full.txt` / `llm.txt` are *reference*.** They're documentation flattened to plain text so a model has the facts in context. Passive — they describe *what exists*. - **`SKILL.md` is *procedural*.** It follows the [Agent Skills](https://code.claude.com/docs/en/skills) convention: task-oriented instructions, known-good code recipes, and constants that teach an agent *how to actually build* on ar.io — which SDK call to make, the correct ArNS URL shape, what to avoid. A useful rule of thumb: load **`SKILL.md`** when the agent needs to *write code*, and **`llms-full.txt`** when it needs to *answer questions* about the wider platform. ## Per-SDK reference text Each SDK ships its own flattened reference, which is much cheaper to load than the full corpus: | SDK | Plain text | |---|---| | ar.io SDK | `/sdks/ar-io-sdk/llm.txt` | | Turbo SDK | `/sdks/turbo-sdk/llm.txt` | | Wayfinder | `/sdks/wayfinder/llm.txt` | | ArDrive Core JS | `/sdks/ardrive-core-js/llm.txt` | | CLIs (ArDrive CLI, ARIO Deploy) | `/sdks/(clis)/llm.txt` | ## Give your agent somewhere safe to run Agents learn fastest by *doing*, but you don't want an agent spending real money or writing permanent data while it figures things out. The **[Testnet Sandbox](/build/testnet)** is built for exactly this: - Runs the **full ar.io stack** — upload, payment, ArNS, and gateway — on Solana **devnet**. - Funded by a **faucet**, so there's no real value at risk. - **Nothing is permanent**: data is purged after ~3 days and never reaches mainnet Arweave. **One step needs a human.** The ARIO faucet is GitHub-gated and its OAuth consent can't be completed headlessly. Have a person claim once to the wallet your agent will use — after that, uploading, funding, and buying names are all scriptable. See [Agents and CI](/build/testnet/funds-and-faucet#agents-and-ci). ## Skills and integrations Several ar.io repositories ship agent tooling: - **ARIO Deploy** — a Claude Code skill for deploying apps to the permaweb and updating ArNS. See [what the skill does](/sdks/ario-deploy/what-the-skill-does) and [add it to your project](/sdks/ario-deploy/add-the-skill-to-your-project) for setup. - **`ar-io-gateway-operator`** — a conformant [Agent Skill](https://code.claude.com/docs/en/skills) for operating an ar.io node: health, indexing lag, ArNS resolution, data retrieval, and common pitfalls. Ships in the [ar-io-node](https://github.com/ar-io/ar-io-node) repo under `.claude/skills/ar-io-gateway-operator/`. - **`ario-testnet-faucet`** — the faucet claim API and flow, for agents working against the [Testnet Sandbox](/build/testnet). Ships in the `ar-io-faucet` repo. `/SKILL.md` above is itself a conformant Agent Skill — it carries `name` and `description` frontmatter, so you can drop it straight into a project as `.claude/skills/ario/SKILL.md`. ## Tools built into these docs Every page carries agent-friendly affordances: - **Open in AI** — send the current page straight to ChatGPT, Claude, Scira, or T3 Chat. - **Copy as Markdown** — grab the raw source of any page for pasting into a prompt. - **Ask Arie** — an in-docs AI assistant that answers from this documentation with citations. ## Pointing an agent at ar.io ### Load the index Fetch `https://docs.ar.io/llms.txt` so the agent knows what's available and where. ### Load the skill for code generation Fetch `https://docs.ar.io/SKILL.md` for known-good patterns, constants, and URL conventions. ### Narrow to an SDK Pull the relevant `llm.txt` (table above) instead of the full corpus — smaller context, better answers. ### Run against the sandbox Build and test on the [Testnet Sandbox](/build/testnet) before touching mainnet. ## Related # Bundler (/build/extensions/bundler) ## Overview A [Turbo ANS-104](https://github.com/ardriveapp/turbo-upload-service/) data item bundler can be run alongside an ar.io gateway. This allows gateways the ability to accept data items to be submitted to the Arweave blockweave. The bundler service can be easily run inside Docker in the same way that the gateway is. It utilizes a separate docker compose file for configuration and deployment, which also allows for the use of a separate file for environmental variables specific to the bundler service. Additionally, the separation allows operators to spin their bundler service up or down at any time without affecting their core gateway service. Despite the use of separate docker compose files, the bundler service shares a docker network with the ar.io gateway, and so is able to directly interact with the gateway service and data. For more information on ANS-104 Bundles, see the [ANS-104 Bundles](/learn/ans-104-bundles) page. ## Getting Started **NOTE**: The bundler service relies on GraphQL indexing of recently bundled and uploaded data to manage its pipeline operations. The ar.io gateway should have its indexes synced up to Arweave's current block height before starting the bundler's service stack. ### Configure Environmental Variables Environmental variables must be provided for the bundler to function and integrate properly with an existing ar.io gateway. The gateway repository provides a `.env.bundler.example` file that can be renamed to `.env.bundler` and used as a starting point. It contains the following: ```bash BUNDLER_ARWEAVE_WALLET='Stringified JWK wallet. e.g: '{ "n": "...", ... }' BUNDLER_ARWEAVE_ADDRESS='Address for above wallet' APP_NAME='ar.io bundler service' # Use localstack s3 bucket for shared data source between ar.io gateway and bundler AWS_S3_BUCKET=ar.io AWS_S3_PREFIX='data' AWS_ACCESS_KEY_ID='test' AWS_SECRET_ACCESS_KEY='test' AWS_REGION='us-east-1' AWS_ENDPOINT='http://localstack:4566' ``` - `BUNDLER_ARWEAVE_WALLET` must be the entire jwk of an Arweave wallet's keyfile, stringified. All uploads of bundled data items to Arweave will be signed and paid for by this wallet, so it must maintain a balance of AR tokens sufficient to handle the uploads. - `BUNDLER_ARWEAVE_ADDRESS` must be the [normalized public address](/glossary) for the provided Arweave wallet. - `APP_NAME` is a GraphQL tag that will be added to uploaded bundles. The remaining lines in the `.env.bundler.example` file control settings that allow the bundler service to share data with the ar.io gateway. Data sharing of contiguous data between a bundler and a gateway allows the gateway to serve optimistically cached data without waiting for it to fully settle on chain. ### Configure Optimistic Indexing By default, the bundler will only accept data items uploaded by data item signers whose [normalized wallet addresses](/glossary) are in the `ALLOW_LISTED_ADDRESSES` list. This is an additional environmental variable that can be added to your `.env.bundler` file, and must be a comma separated list of normalized public wallet addresses for wallets that should be allowed to bundle and upload data through your gateway. ```bash ALLOW_LISTED_ADDRESSES=, ``` The following permissioning configurations schemes are also possible: | Scheme | ALLOW_LISTED_ADDRESSES | SKIP_BALANCE_CHECKS | ALLOW_LISTED_SIGNATURE_TYPES | PAYMENT_SERVICE_BASE_URL | | -------------------------- | ------------------------------------------- | ------------------- | ---------------------------- | ------------------------ | | **Allow Specific Wallets** | Comma-separated normalized wallet addresses | false | EMPTY or supplied | EMPTY | | **Allow Specific chains** | EMPTY or supplied | false | arbundles sigtype int | EMPTY | | **Allow All** | n/a | true | n/a | n/a | | **Allow None** | EMPTY | false | EMPTY | EMPTY | | **Allow Payers** | EMPTY or supplied | false | EMPTY or supplied | Your payment service url | ### Set Up Indexing Bundlers submit data to the Arweave network as an [ANS-104 data item bundle](https://github.com/ArweaveTeam/arweave-standards/blob/master/ans/ANS-104.md). This means it is several transactions wrapped into one. A gateway will need to unbundle these transactions in order to index them. A gateway should include the following ANS-104 filters in order to unbundle and index transactions from a particular bundler: ```bash ANS104_INDEX_FILTER={ "always": true } ANS104_UNBUNDLE_FILTER={ "attributes": { "owner_address": "$BUNDLER_ARWEAVE_ADDRESS" } } ``` `$BUNDLER_ARWEAVE_ADDRESS` should be replaced with the [normalized public wallet address](/glossary) associated with the bundler. **NOTE**: The above filters must be placed in the `.env` file for the core gateway service, not the bundler. Gateways handle data item indexing asynchronously. This means they establish a queue of items to index, and work on processing the queue in the background while the gateway continues with its normal operations. If a gateway has broad indexing filters, there can be some latency in indexing data items from the bundler while the gateway works through its queue. ### Configure Optimistic Indexing Gateway operators control access to their [optimistic data item indexing](/glossary) API via an admin key that must be supplied by all bundling clients in order for their requests to be accepted. This key should be made available in the environment configuration files for BOTH the core gateway, and the bundler, and should be provided as `AR_IO_ADMIN_KEY`: ```bash AR_IO_ADMIN_KEY="Admin password" ``` **NOTE**: If a gateway is started without providing the admin key, a random string will be generated to protect the gateway's admin endpoints. This can be reset by restarting the gateway with the admin key provided in the `.env` file. ## Starting and Stopping the Bundler ### Starting The bundler service is designed to run in conjunction with an ar.io gateway, and so relies on the `ar-io-network` network created in Docker when the core gateway services are spun up. It is possible to spin up the bundler while the core services are down, but the network must exist in Docker. To start the bundler, specify the env and docker-compose files being used in a `docker compose up` command: ```bash docker compose --env-file ./.env.bundler --file docker-compose.bundler.yaml up -d ``` The `-d` flag runs the command in "detached" mode, so it will run in the background without requiring the terminal to remain active. ### Stopping To spin the bundler service down, specify the docker-compose file in a `docker compose down` command: ```bash docker compose --file docker-compose.bundler.yaml down ``` ### Logs While the bundler service is running in detached mode, logs can be checked by specifying the docker-compose file in a `docker compose logs` command: ```bash docker compose --file docker-compose.bundler.yaml logs -f --tail=0 ``` - `-f` runs the command in "follow" mode, so the terminal will continue to watch and display new logs. - `--tail=` defines the number of logs to display that existed prior to running the command. `0` displays only new logs. ## Useful Docker Commands Monitor and manage your bundler service with these commands: ```bash # View all running services docker ps # Start bundler service in background docker compose --env-file ./.env.bundler --file docker-compose.bundler.yaml up -d # Stop bundler service docker compose --file docker-compose.bundler.yaml down # Pull latest bundler images docker compose --file docker-compose.bundler.yaml pull # Follow bundler logs docker compose --file docker-compose.bundler.yaml logs -f --tail=10 # Check bundler service status docker compose --file docker-compose.bundler.yaml ps # Restart bundler service docker compose --file docker-compose.bundler.yaml restart ``` ## Next Steps Now that you have a bundler set up to accept data uploads, continue building your gateway infrastructure: } title="Set Up Monitoring" description="Deploy Grafana to visualize your gateway's performance metrics" href="/build/extensions/grafana" /> } title="Add ClickHouse" description="Improve query performance with ClickHouse and Parquet integration" href="/build/extensions/clickhouse" /> } title="Buy an ArNS Name" description="Get a human-readable name for your gateway and start serving the permanent web" href="/learn/arns/name-registration" /> # ClickHouse & Parquet (/build/extensions/clickhouse) ClickHouse is an optional analytical database that runs alongside your ar.io gateway to accelerate GraphQL queries on large datasets. It stores stable historical data in a compressed columnar format while SQLite continues handling real-time writes and recent data. ClickHouse requires version **24.8 or later**. Earlier versions have known issues with projections on ReplacingMergeTree tables. The default Docker image uses ClickHouse 26.3. ## How It Works The gateway uses a speed/batch layer architecture: - **SQLite (speed layer)** — Handles all real-time writes, recent/unstable data, and non-GraphQL operations (data retrieval, chunk fetches, admin APIs) - **ClickHouse (batch layer)** — Stores stable historical data exported as Parquet files, optimized for analytical GraphQL queries |writes| B[SQLite] B -->|stable data export| C[Parquet Files] C -->|batch import| D[ClickHouse] E[GraphQL Query] -->|height > boundary| B E -->|height ≤ boundary| D `} /> ### Query Routing When ClickHouse is enabled, GraphQL queries are split by block height: - **ClickHouse** handles blocks up to `(max_imported_height - 10)` — the bulk of historical data - **SQLite** handles blocks above that boundary — recent, potentially unstable data Results from both backends are merged and deduplicated before being returned to the client. If either backend is unavailable, a circuit breaker degrades gracefully rather than failing the entire query. ### When to Enable ClickHouse Enable ClickHouse if your gateway: - Serves heavy GraphQL traffic on historical data - Indexes millions of data items (large unbundling filters) - Needs faster bootstrapping than a full SQLite reindex (import pre-built Parquet snapshots) - Would benefit from 3-5x storage compression on historical data Keep it disabled for small deployments, gateways that only serve data retrieval (not GraphQL), or single-node low-resource environments. ## Quick Start ### Configure Environment Add ClickHouse settings to your `.env` file: ```bash CLICKHOUSE_URL=http://clickhouse:8123 CLICKHOUSE_PASSWORD=your-password CLICKHOUSE_SQLITE_MIN_HEIGHT_ENABLED=true ADMIN_API_KEY=your-admin-key ``` If using a non-default username: ```bash CLICKHOUSE_USER=your-username ``` ### Configure Unbundling Filters ClickHouse stores data items exported from SQLite. The more you unbundle, the more data flows into ClickHouse. At minimum, configure filters that match the pre-built snapshot you plan to import. The ArDrive snapshot includes all ArDrive-uploaded data items. To match this: ```bash ANS104_UNBUNDLE_FILTER='{ "and": [ { "not": { "or": [ { "tags": [ { "name": "Bundler-App-Name", "value": "Warp" } ] }, { "tags": [ { "name": "Bundler-App-Name", "value": "Redstone" } ] }, { "tags": [ { "name": "Bundler-App-Name", "value": "AO" } ] }, { "attributes": { "owner_address": "-OXcT1sVRSA5eGwt2k6Yuz8-3e3g9WJi5uSE99CWqsBs" } }, { "attributes": { "owner_address": "ZE0N-8P9gXkhtK-07PQu9d8me5tGDxa_i4Mee5RzVYg" } }, { "attributes": { "owner_address": "6DTqSgzXVErOuLhaP0fmAjqF4yzXkvth58asTxP3pNw" } } ] } }, { "tags": [ { "name": "App-Name", "valueStartsWith": "ArDrive" } ] } ] }' ANS104_INDEX_FILTER='{ "tags": [ { "name": "App-Name", "value": "ArDrive-App" } ] }' ``` See the [Filters guide](/build/run-a-gateway/manage/filters) for building custom filter configurations. ### Start ClickHouse ClickHouse runs as a Docker profile alongside your gateway: ```bash docker compose --profile clickhouse up -d ``` This starts both the ClickHouse server and the auto-import daemon. ### Import a Parquet Snapshot Download and import a pre-built snapshot to bootstrap your ClickHouse instance: ```bash # Download the ArDrive snapshot (~3.5GB, current to April 23, 2025) curl -L https://turbo-gateway.com/JVmsuD2EmFkhitzWN71oi9woADE4WUfvrbBYgremCBM -o ardrive-parquet.tar.gz # Extract tar -xzf ardrive-parquet.tar.gz # Move into the data directory mkdir -p data/parquet mv 2025-04-23-ardrive-ans104-parquet/* data/parquet/ # Run the import script ./scripts/clickhouse-import ``` The import takes 10-20 minutes depending on hardware. ### Verify Check the transaction count: ```bash docker compose exec clickhouse clickhouse-client \ --password your-password \ -q 'SELECT COUNT(DISTINCT id) FROM transactions' ``` Expected result: `32712311` (for the ArDrive snapshot). Test a GraphQL query: ```bash curl -g -X POST \ -H "Content-Type: application/json" \ -d '{"query":"query { transactions(ids: [\"YSNwoYB01EFIzbs6HmkGUjjxHW3xuqh-rckYhi0av4A\"]) { edges { node { block { height } bundledIn { id } } } } }"}' \ http://localhost:3000/graphql ``` ## Auto-Import The `clickhouse-auto-import` container runs a continuous loop that exports new stable data from SQLite to Parquet and imports it into ClickHouse. This keeps ClickHouse up to date without manual intervention. Each cycle: 1. Reloads TTL rules from `config/clickhouse-ttl-rules.yaml` 2. Advances the 3. Exports new Parquet partitions from SQLite 4. Imports Parquet into ClickHouse staging tables 5. Migrates staging to the final `transactions` table (applying TTL rules) ### Auto-Import Configuration | Variable | Default | Description | |----------|---------|-------------| | `CLICKHOUSE_AUTO_IMPORT_SLEEP_INTERVAL` | `3600` | Seconds between import cycles | | `CLICKHOUSE_AUTO_IMPORT_HEIGHT_INTERVAL` | `1000` | Blocks per Parquet partition | | `CLICKHOUSE_AUTO_IMPORT_MAX_ROWS_PER_FILE` | - | Maximum rows per Parquet file | The default 1-hour interval is conservative. For gateways with high unbundling throughput, reduce to 600-900 seconds to keep ClickHouse more current. ## TTL Rules (Data Retention) TTL rules control how long data items remain in ClickHouse before automatic expiration. Use them to manage storage costs by expiring ephemeral or low-value data while retaining important content indefinitely. ### Configuration Copy the example template and edit: ```bash cp config/clickhouse-ttl-rules.example.yaml config/clickhouse-ttl-rules.yaml ``` Example configuration: ```yaml # Keep L1 transactions forever, expire unmatched data items after 30 days default_ttl_seconds: 2592000 l1_never_expires: true rules: # Keep ArDrive content forever - tag_name: App-Name tag_value: ArDrive match: prefix never_expire: true # Expire ephemeral chat messages after 1 day - tag_name: App-Name tag_value: ephemeral-chat ttl_seconds: 86400 # Expire test uploads after 1 hour - tag_name: App-Name tag_value: test- match: prefix ttl_seconds: 3600 # Expire specific owner's data after 7 days - field: owner_address value: abcDEF0123xyz ttl_seconds: 604800 ``` ### Rule Precedence When multiple rules could apply, the first matching branch wins: 1. `l1_never_expires: true` AND row is L1 transaction → kept forever 2. Any matching rule with `never_expire: true` → kept forever 3. One or more matching TTL rules → shortest `ttl_seconds` wins 4. `default_ttl_seconds` set → applied to all unmatched rows 5. Otherwise → kept indefinitely (no expiry) ### Match Modes - **`exact`** (default) — O(1) dictionary lookup. Use for specific tag values. - **`prefix`** — Matches the beginning of the value. Use for `Content-Type` with parameters (e.g., `image/gif` catches `image/gif; charset=...`) or app name families. TTL rules apply only to rows imported after the rules are loaded. Previously imported rows keep their existing expiration. To apply new rules retroactively, re-import affected Parquet partitions. ### Normalization - Tag names are lower-cased and trimmed - Tag values are trimmed but case-preserving - Owner values use base64url format (as displayed on Arweave explorers) ## Deployment Topologies ### Single Gateway (Default) One gateway runs ClickHouse locally. The auto-import daemon handles continuous data ```bash docker compose --profile clickhouse up -d ``` ### Shared ClickHouse Cluster Multiple gateways share a replicated ClickHouse cluster. Only **one** gateway runs `clickhouse-auto-import` as the writer; others query the same cluster as readers. ```bash # Writer gateway CLICKHOUSE_URL=http://clickhouse-writer:8123 # runs clickhouse-auto-import # Reader gateways CLICKHOUSE_URL=http://clickhouse-reader:8123 # do NOT run clickhouse-auto-import ``` Reader gateways still need `START_WRITERS=true` because SQLite indexing is required for data retrieval, chunk fetches, and other non-GraphQL operations. Only GraphQL queries are routed to ClickHouse. ### Independent ClickHouse per Gateway Each gateway runs its own ClickHouse instance, bootstrapped from a shared Parquet snapshot (downloaded from Arweave, S3, or BitTorrent). No shared state between gateways. This is the simplest topology for operators who want ClickHouse performance without cluster management. ## Configuration Reference ### Connection | Variable | Default | Description | |----------|---------|-------------| | `CLICKHOUSE_URL` | - | ClickHouse HTTP endpoint (e.g., `http://clickhouse:8123`) | | `CLICKHOUSE_USER` | `default` | ClickHouse username | | `CLICKHOUSE_PASSWORD` | - | ClickHouse password | ### Query Routing | Variable | Default | Description | |----------|---------|-------------| | `CLICKHOUSE_SQLITE_MIN_HEIGHT_ENABLED` | `false` | Enable height-based query split between ClickHouse and SQLite | | `CLICKHOUSE_SQLITE_MIN_HEIGHT_BUFFER` | `10` | Number of blocks below ClickHouse max height still served by SQLite | | `CLICKHOUSE_MAX_HEIGHT_CACHE_TTL_SECONDS` | `60` | Cache duration for ClickHouse max-height lookup | | `CLICKHOUSE_QUERY_TIMEOUT_SECONDS` | `3` | Query timeout (both client and server side) | | `CLICKHOUSE_GQL_MAX_ROWS_TO_READ` | `10000000` | Safety guardrail: max rows scanned per query | | `CLICKHOUSE_GQL_DEDUPE_HEADROOM` | `4` | Pagination headroom multiplier for deduplication | ### Circuit Breaker The circuit breaker protects against cascading failures when the SQLite leg is unhealthy. When tripped, queries degrade to ClickHouse-only results with a partial result warning. | Variable | Default | Description | |----------|---------|-------------| | `CLICKHOUSE_SQLITE_CIRCUIT_BREAKER_TIMEOUT_MS` | `5000` | SQLite query timeout before counting as error | | `CLICKHOUSE_SQLITE_CIRCUIT_BREAKER_ERROR_THRESHOLD_PERCENTAGE` | `80` | Error rate percentage that trips the breaker | | `CLICKHOUSE_SQLITE_CIRCUIT_BREAKER_ROLLING_COUNT_TIMEOUT_MS` | `60000` | Rolling window for error rate calculation | | `CLICKHOUSE_SQLITE_CIRCUIT_BREAKER_RESET_TIMEOUT_MS` | `30000` | Wait time before attempting half-open trial | ## Tuning ### Short Pages in GraphQL Pagination If GraphQL queries return fewer results than expected per page, unmerged duplicates in ClickHouse may be consuming pagination slots. Increase the deduplication headroom: ```bash CLICKHOUSE_GQL_DEDUPE_HEADROOM=8 # default is 4 ``` This makes the inner query fetch more rows to compensate for duplicates that get collapsed. ### Query Timeouts For queries over very large datasets or complex tag filters, increase the timeout: ```bash CLICKHOUSE_QUERY_TIMEOUT_SECONDS=10 # default is 3 ``` ### Max Rows Safety Guardrail If queries hit the `max_rows_to_read` limit (returns an error), you can raise it — but do so with caution as it removes the protection against runaway full-table scans: ```bash CLICKHOUSE_GQL_MAX_ROWS_TO_READ=50000000 # default is 10M ``` ## Upgrading ### Schema Evolution Schema changes are applied automatically on each import cycle via idempotent `ALTER TABLE` statements. No manual migration steps are needed for routine upgrades. ### Projection Rebuild (One-Time) Gateways upgrading from pre-24.8 ClickHouse deployments may need to rebuild the owner projection: ```sql ALTER TABLE transactions MATERIALIZE PROJECTION owner_projection; ``` This runs in the background and may take time on large tables. ### Rollback If you need to roll back a ClickHouse upgrade: 1. Drop the `transactions` table 2. Re-import from your Parquet files using `./scripts/clickhouse-import` Parquet files are the source of truth — ClickHouse is always rebuildable from them. ## Troubleshooting ### Import Failures Failed imports leave files in `data/etl/staging/` for inspection. Check the auto-import container logs: ```bash docker compose --profile clickhouse logs clickhouse-auto-import -f ``` Fix the underlying issue (disk space, permissions, ClickHouse connectivity) and restart the container — the next cycle will retry. ### Disk Usage Growing If ClickHouse disk usage is higher than expected: - Check your TTL rules are configured and expiring data as intended - Verify background merges are running (rows are only deleted on merge): ```bash docker compose exec clickhouse clickhouse-client \ --password your-password \ -q 'SELECT partition, count() FROM system.parts WHERE table = '\''transactions'\'' GROUP BY partition' ``` - Force a merge on a specific partition if needed (heavyweight operation): ```bash OPTIMIZE TABLE transactions PARTITION 14 FINAL ``` ### Checking Import Status View the current max imported height: ```bash docker compose exec clickhouse clickhouse-client \ --password your-password \ -q 'SELECT max(height) FROM transactions' ``` Compare this to the gateway's current height (`/height` endpoint) — the difference indicates how far behind ClickHouse is. ## Docker Commands ```bash # Start gateway with ClickHouse docker compose --profile clickhouse up -d # Stop gateway with ClickHouse docker compose --profile clickhouse down # View ClickHouse logs docker compose --profile clickhouse logs clickhouse -f # View auto-import logs docker compose --profile clickhouse logs clickhouse-auto-import -f # Restart auto-import after config change docker compose --profile clickhouse restart clickhouse-auto-import # Pull latest images docker compose --profile clickhouse pull ``` ## Related } title="Filters" description="Configure unbundling and indexing filters that control what data flows into ClickHouse" href="/build/run-a-gateway/manage/filters" /> } title="Grafana Monitoring" description="Visualize gateway and ClickHouse performance metrics" href="/build/extensions/grafana" /> } title="Environment Variables" description="Full configuration reference for all gateway services" href="/build/run-a-gateway/manage/environment-variables" /> } title="Gateway Architecture" description="Understand the data layer and worker system architecture" href="/learn/gateways/architecture" /> # Grafana (/build/extensions/grafana) ## Overview Ar.io gateways track extensive performance and operational metrics using [Prometheus](https://prometheus.io/). A [Grafana](https://grafana.com/) sidecar can be deployed to visualize these metrics, providing an easy way to monitor gateway health and performance. The Grafana sidecar is deployed as a separate Docker container that uses the same network as the gateway, making it simple to integrate with your existing setup. ![Grafana Dashboard](/content/grafana.png) ## Quick Start ### Deploy Grafana Deploy the Grafana sidecar using the provided Docker Compose file: ```bash docker compose -f docker-compose.grafana.yaml up -d ``` This command assumes you're running from the root directory of the gateway. If running from a different directory, adjust the path to the docker-compose file accordingly. ### Verify Deployment Check that Grafana is running properly: ```bash docker compose -f docker-compose.grafana.yaml logs -f --tail=25 ``` Press `Ctrl+C` to exit the logs. Look for any error messages or permission issues. ### Access Grafana Navigate to `http://localhost:1024` in your browser to access Grafana. **Default credentials:** - Username: `admin` - Password: `admin` Updated credentials may be lost if the Grafana sidecar is restarted. Be sure to log into Grafana immediately after every startup to ensure Grafana cannot be accessed with the default credentials. ## Exposing Dashboard Publicly To expose your Grafana dashboard externally through your domain, you'll need to configure nginx as a reverse proxy. This requires DNS setup and SSL certificates as covered in the [gateway installation guide](/build/run-a-gateway/quick-start). This setup assumes you've already configured DNS, SSL certificates, and nginx as described in the [Installation & Setup guide](/build/run-a-gateway/quick-start). ### Deploy Grafana Sidecar First, ensure your Grafana container is running: ```bash docker compose -f docker-compose.grafana.yaml up -d ``` Verify it's accessible locally at `http://localhost:1024`. ### Update Nginx Configuration Edit your existing nginx configuration file (`/etc/nginx/sites-available/default`) to add the Grafana location block: ```nginx # Add this block inside your existing HTTPS server block (port 443) location /grafana/ { proxy_pass http://localhost:1024/grafana/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } ``` Your complete nginx configuration should look like this: ```nginx # Force redirects from HTTP to HTTPS server { listen 80; listen [::]:80; server_name .com *..com; location / { return 301 https://$host$request_uri; } } # Forward traffic to your node and provide SSL certificates server { listen 443 ssl; listen [::]:443 ssl; server_name .com *..com; ssl_certificate /etc/letsencrypt/live/.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/.com/privkey.pem; location / { proxy_pass http://localhost:3000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_http_version 1.1; # Forward ar.io headers if present in the request proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; } # Grafana dashboard access location /grafana/ { proxy_pass http://localhost:1024/grafana/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } ``` ### Test and Reload Nginx Validate your nginx configuration: ```bash sudo nginx -t ``` If the configuration is valid, reload nginx: ```bash sudo systemctl reload nginx ``` ### Access Your Dashboard Navigate to `https://.com/grafana/` in your browser to access your Grafana dashboard externally. **Default credentials:** - Username: `admin` - Password: `admin` ## Troubleshooting ### Fix Permission Issues ### Method 1: Modify Directory Permissions The simplest solution is to modify the permissions of the Grafana data directory: ```bash sudo chmod -R 777 ./data/grafana ``` This command assumes you're running from the root directory of the gateway. Adjust the path if running from a different directory. ### Method 2: Change Grafana User Alternatively, modify the `docker-compose.grafana.yaml` file to use root user: ```yaml grafana: image: grafana/grafana:latest user: root ports: - "3000:3000" ``` ### Verify Fix Restart Grafana and check logs: ```bash docker compose -f docker-compose.grafana.yaml restart docker compose -f docker-compose.grafana.yaml logs -f ``` ### Resolve Connection Problems ### Check Container Status Verify Grafana is running: ```bash docker compose -f docker-compose.grafana.yaml ps ``` ### Check Port Availability Ensure the port isn't already in use: ```bash netstat -tulpn | grep :1024 # or lsof -i :1024 ``` ### Review Logs Check for specific error messages: ```bash docker compose -f docker-compose.grafana.yaml logs --tail=50 ``` ### Fix Configuration Issues ### Validate Nginx Configuration Test your Nginx configuration: ```bash sudo nginx -t ``` ### Check Proxy Settings Ensure proxy headers are correctly configured: ```nginx location /grafana/ { proxy_pass http://localhost:1024/grafana/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } ``` ### Restart Services Restart both Nginx and Grafana: ```bash sudo systemctl restart nginx docker compose -f docker-compose.grafana.yaml restart ``` ## Security Considerations Always change the default admin credentials immediately after first login. Default credentials are publicly known and pose a security risk. ### Best Practices 1. **Change Default Password** - Use a strong, unique password 2. **Enable HTTPS** - Use SSL certificates for external access 3. **Restrict Access** - Use firewall rules to limit access 4. **Regular Updates** - Keep Grafana updated to latest version 5. **Backup Configuration** - Export and backup dashboard configurations ## Documentation & Support - **Grafana Documentation** - [Official Grafana docs](https://grafana.com/docs/) - **Prometheus Metrics** - [Understanding gateway metrics](https://prometheus.io/docs/concepts/metric_types/) - **Community Support** - Join the [ar.io Discord](https://discord.com/invite/HGG52EtTc2) for help ## Useful Docker Commands Monitor and manage your Grafana sidecar with these commands: ```bash # View all running services docker ps # Start Grafana sidecar docker compose -f docker-compose.grafana.yaml up -d # Stop Grafana sidecar docker compose -f docker-compose.grafana.yaml down # Pull latest Grafana images docker compose -f docker-compose.grafana.yaml pull # Follow Grafana logs docker compose -f docker-compose.grafana.yaml logs -f --tail=25 # Check Grafana container status docker compose -f docker-compose.grafana.yaml ps # Restart Grafana sidecar docker compose -f docker-compose.grafana.yaml restart # View Grafana logs without following docker compose -f docker-compose.grafana.yaml logs --tail=50 # Start Grafana in foreground (for debugging) docker compose -f docker-compose.grafana.yaml up # Check port availability netstat -tulpn | grep :1024 ``` ## Next Steps Now that you have monitoring set up, continue building your gateway infrastructure: } title="Optimize Performance" description="Learn advanced gateway optimization techniques for better performance" href="/build/run-a-gateway/manage/filters" /> } title="Add ClickHouse" description="Improve query performance with ClickHouse and Parquet integration" href="/build/extensions/clickhouse" /> } title="Deploy Bundler" description="Accept data uploads directly through your gateway" href="/build/extensions/bundler" /> # Extensions & Sidecars (/build/extensions) ## What are Extensions? Extensions are additional scripts and tools you can run alongside your gateway to expand its capabilities or enhance the operator experience. The full list of community extensions can be found at [gateways.ar.io/#/extensions](https://gateways.ar.io/#/extensions). ## What are Sidecars? Sidecars are dockerized services that add additional functionality, APIs, and services to ar.io gateways. They run as separate containers alongside your gateway, providing specialized capabilities. ## Getting Started with Team-Supported Sidecars The following sidecars are developed and maintained by the ar.io team, designed to run alongside your gateway as separate containers. }> Visualize gateway metrics with comprehensive dashboards and performance monitoring. } > Improve query performance for large datasets using columnar storage and analytical optimization. } > Accept and process ANS-104 data item uploads with multiple payment methods and access control. **Ready to enhance your gateway?** Click any sidecar above to get started with detailed setup guides. ## Explore More } title="Monitor your gateway with Grafana" description="Set up comprehensive monitoring and analytics for your gateway infrastructure" href="/build/extensions/grafana" /> } title="Performance Optimization" description="Optimize your gateway for large datasets and high-performance queries" href="/build/run-a-gateway/manage/filters" /> } title="Gateway Operations" description="Learn advanced gateway management, troubleshooting, and configuration" href="/build/run-a-gateway/manage" /> } title="Developer SDKs" description="Integrate ar.io services into your applications with our SDKs" href="/sdks" /> # Application Distribution with ArNS + Manifests (/build/guides/application-distribution) **Overview:** This guide shows how to distribute software permanently using Arweave manifests and ArNS. You can publish binaries once and serve them from stable, human-readable URLs that keep working as releases change. Files are stored immutably on Arweave, old versions remain available, and links never break. **What you get:** - Permanent storage for all releases - Stable URLs via ArNS (for example, `releases_yourapp.ar.io`) - Simple versioned paths for latest and pinned releases - Automated CI/CD with GitHub Actions and Turbo - No centralized registries or CDNs The case study that follows walks through a real implementation for the Harlequin CLI and shows how to apply this pattern to your own software. ## Architecture Overview ### System Components B[GoReleaser] B --> C[Multi-Platform Binaries] C --> D[Turbo SDK Upload] D --> E[Arweave Storage] E --> F[Manifest Creation] F --> G[ArNS Routing] G --> H[Global Distribution] I[Install Script] --> J[Platform Detection] J --> K[Binary Download] K --> L[Gzip Decompression] L --> M[Installation]`} /> ### Core Technologies 1. **Arweave**: Permanent data storage blockchain 2. **ArNS**: Decentralized naming service for content routing 3. **Turbo SDK**: Efficient data upload and payment handling 4. **GoReleaser**: Multi-platform binary compilation 5. **GitHub Actions**: Automated CI/CD pipeline ## Implementation Deep Dive ### 1. Binary Build Pipeline The system uses GoReleaser for creating optimized, multi-platform binaries: ```yaml # .goreleaser.yaml builds: - id: harlequin main: ./main.go binary: harlequin goos: [linux, darwin, windows] goarch: [amd64, arm64] flags: [-trimpath] ldflags: - -s -w # Strip debug info for smaller binaries - -X main.version={{.Version}} ``` **Platform Matrix:** - Linux: AMD64, ARM64 - macOS: AMD64 (Intel), ARM64 (Apple Silicon) - Windows: AMD64, ARM64 ### 2. Arweave Storage Strategy #### Compression Optimization All binaries are compressed with gzip before upload, achieving significant storage savings: ```typescript // Compress binary before upload const binaryData = readFileSync(binary); const compressedData = gzipSync(binaryData); const compressionRatio = ((1 - compressedData.length / binaryData.length) * 100).toFixed(1); // Upload with proper content headers const upload = await turboClient.upload({ data: compressedData, dataItemOpts: { tags: [ {name: "Content-Type", value: "application/gzip"}, {name: "Content-Encoding", value: "gzip"}, {name: "Original-Content-Type", value: "application/octet-stream"}, {name: "Original-Size", value: binaryData.length.toString()}, {name: "Compressed-Size", value: compressedData.length.toString()} ] } }); ``` **Storage Efficiency Results:** - Average compression ratio: ~70% - 10MB binary → ~3MB storage cost - Significant cost savings at scale #### Data Item Tagging Each upload includes comprehensive metadata for discoverability and management: ```typescript const dataItemOptions = { tags: [ {name: 'Type', value: 'release'}, {name: 'App-Name', value: 'Harlequin-CLI'}, {name: 'App-Version', value: version}, {name: 'Content-Type', value: 'application/gzip'}, {name: 'Platform', value: platform}, {name: 'Architecture', value: arch} ] } ``` ### 3. Manifest-Based Routing The system creates an Arweave manifest that provides structured routing for all binaries and metadata: ```typescript const manifest: ArweaveManifest = { manifest: 'arweave/paths', version: '0.1.0', index: { path: 'install_cli.sh' // Default route }, paths: { // Version-specific binaries 'releases/1.2.3/linux/amd64': { id: 'arweave_tx_id_1' }, 'releases/1.2.3/darwin/arm64': { id: 'arweave_tx_id_2' }, // Latest symlinks for convenience 'releases/latest/linux/amd64': { id: 'arweave_tx_id_1' }, 'releases/latest/darwin/arm64': { id: 'arweave_tx_id_2' }, // API endpoints 'releases': { id: 'releases_json_tx_id' }, 'install_cli.sh': { id: 'install_script_tx_id' } } }; ``` ### 4. ArNS URL Structure Combining ArNS with manifests creates a permanent, human-readable API for your releases. The ArNS undername points to the manifest, and the manifest paths define the URL structure: ```typescript // Update ArNS record to point to new manifest await ant.setRecord({ undername: 'install_cli', // Subdomain transactionId: manifestId, // New manifest TX ID ttlSeconds: 60 // Cache TTL }); ``` This creates a complete URL API: ``` https://install_cli_harlequin.ar.io/ ├── install_cli.sh # Installation script (default) ├── releases # JSON API with version metadata └── releases/ ├── 1.2.3/ # Version-specific binaries │ ├── linux/amd64 │ ├── darwin/arm64 │ └── windows/amd64 └── latest/ # Latest version aliases ├── linux/amd64 └── darwin/arm64 ``` **Key Benefits:** - `install_cli_harlequin.ar.io` provides a permanent, friendly URL - Manifest paths create a logical REST-like API structure - Updating the ArNS record points to new releases while preserving the URL ### 5. Intelligent Installation Script The installation script provides a sophisticated user experience with platform detection, version management, and error handling: #### Platform Detection ```bash # Detect OS and architecture OS="$(uname -s)" ARCH="$(uname -m)" case $OS in Darwin) PLATFORM="darwin" ;; Linux) PLATFORM="linux" ;; CYGWIN*|MINGW*|MSYS*) PLATFORM="windows" ;; esac case $ARCH in x86_64) ARCH="amd64" ;; aarch64|arm64) ARCH="arm64" ;; armv7*) ARCH="armv7" ;; esac ``` #### Version Management ```bash # Interactive version selection get_available_versions() { curl -fsSL "${BASE_URL}/releases" -o /tmp/releases.json if has_jq; then jq -r '.[].tag_name' /tmp/releases.json | sed 's/^v//' else parse_version "$TEMP_FILE" ``` ## CI/CD Integration ### GitHub Actions Workflow The deployment is fully automated through GitHub Actions: ```yaml name: Nx Release on: push: branches: [main, develop] paths: ['cli/**'] jobs: release: runs-on: ubuntu-latest steps: - name: Setup Go & GoReleaser uses: goreleaser/goreleaser-action@v6.4.0 - name: Release (Main - Stable) if: github.ref == 'refs/heads/main' run: npx nx release patch --yes env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ARWEAVE_WALLET_JWK: ${{ secrets.ARWEAVE_WALLET_JWK }} - name: Test Installation run: | curl -fsSL https://install_cli_harlequin.daemongate.io | \ DRYRUN=true VERSION=0.1.0 sh ``` ### Nx Integration The system leverages Nx for orchestrating the complex release pipeline: ```json { "nx-release-publish": { "executor": "nx:run-commands", "options": { "command": "goreleaser release --clean && cd scripts && yarn deploy" }, "configurations": { "dry-run": { "command": "goreleaser release --skip=publish && yarn deploy:dryrun" } } } } ``` **Pipeline Stages:** 1. **Build**: GoReleaser creates multi-platform binaries 2. **Upload**: Turbo SDK uploads compressed binaries to Arweave 3. **Manifest**: Creates routing manifest with all binary paths 4. **ArNS**: Updates domain to point to new manifest 5. **Verification**: Tests installation script functionality ## Security & Reliability ### Decentralization Benefits - **No Single Point of Failure**: Distributed across the entire ar.io network - **Censorship Resistance**: No central authority can block access - **Geographic Redundancy**: Data replicated globally ### Access Control ```typescript // Wallet-based deployment authorization const signer = new ArweaveSigner(wallet); const turboUploader = TurboFactory.authenticated({signer}); // Only authorized wallet can update ArNS records await ant.setRecord({ undername: 'install_cli', transactionId: manifestId }); ``` ## Conclusion This implementation demonstrates that Arweave manifests combined with ArNS provide a powerful alternative to traditional application distribution: - **Automated Release Pipeline**: CI/CD integration enables seamless multi-platform builds and deployment - **Decentralized and Permanent Storage**: Applications are stored immutably across the Arweave network - **Zero Reliance on Centralized Registries**: No dependency on npm, GitHub Releases, or traditional CDNs - **Permanent Friendly Names**: ArNS provides human-readable URLs with unbreakable links to your releases ## Next Steps } > Learn the fundamentals of Arweave manifest structure and creation. } > Register and manage ArNS names for your distribution endpoints. } > Explore advanced Turbo SDK features for optimized uploads. } > Understand security considerations for permanent data. # ArNS Marketplace (/build/guides/arns-marketplace) **ArNS tokens** can be traded and sold on standard Solana NFT marketplaces. ANTs (Ar.io Name Tokens) are Metaplex Core NFTs, making them tradeable on **Tensor**, **Magic Eden**, and other compatible platforms. ## Trading ANTs on NFT Marketplaces ANTs are standard Metaplex Core NFTs on Solana. This means they are natively supported by the Solana NFT ecosystem: - **[Tensor](https://tensor.trade)** — Solana's leading NFT marketplace with advanced trading features - **[Magic Eden](https://magiceden.io)** — Popular cross-chain NFT marketplace with Solana support - **Direct transfers** — ANTs can be sent directly between Solana wallets ### Lazy Reconciliation When an ANT is sold on a marketplace (outside the ar.io protocol), the existing controllers are not immediately cleared. Instead, **lazy reconciliation** occurs on the next write operation — controllers are cleared automatically, ensuring the new owner gets clean control of the name. ## What Are ANTs? **Ar.io Name Tokens (ANTs)** are: - **Metaplex Core NFTs** — Standard Solana NFTs with onchain state in PDAs - **Transferable assets** — Can be bought, sold, and traded on any compatible marketplace - **Digital assets** — Represent ownership and control of ArNS domains - **DNS-like routing** — Each ANT stores records pointing to Arweave transaction IDs ## How Trading Works ### 1. Token Ownership **When you own an ANT:** - You control the domain name and its records - You can update where it points (Arweave TX IDs or IPFS CIDs) - You can transfer ownership via direct transfer or marketplace sale - You can list it on Tensor, Magic Eden, or other Solana NFT marketplaces - You can add up to 10 controllers who can manage records on your behalf ### 2. Marketplace Trading **Trading mechanisms:** - **Marketplace listings** — List your ANT on Tensor or Magic Eden with a fixed price or auction - **Direct transfers** — Send the NFT directly to another Solana wallet - **Collection offers** — Accept collection-wide offers from buyers - **Instant sales** — Sell into existing bids on marketplaces ### 3. Name Characteristics **What makes ANTs desirable:** - **Domain length** - Shorter names are more memorable - **Memorability** - Easy-to-remember names are more useful - **Brand potential** - Names that could become recognizable - **Uniqueness** - Creative and distinctive names - **Content attached** - Domains with established content ## Potential Trading Examples ### Popular Domain Types **Short names:** - `ar://ai` - Single letter domains - `ar://web3` - Industry keywords - `ar://nft` - Popular terms **Brandable names:** - `ar://crypto` - Industry terms - `ar://decentralized` - Descriptive names - `ar://permanent` - Arweave-related terms ### Potential Use Cases **Personal branding** - Use memorable names for your identity **Project organization** - Create names for different projects **Content management** - Organize content under specific names **Community building** - Create recognizable names for communities ## Getting Started ### 1. Acquire ANTs **Ways to get ANTs:** - **Register new domains** - Create your own primary names - **Buy from others** - Purchase existing domains - **Participate in auctions** - Bid on available names - **Trade with others** - Exchange domains you own ### 2. Choose Names **Consider these factors:** - **Domain length** - Shorter names are more memorable - **Memorability** - Easy to remember and type - **Brand potential** - Could become recognizable - **Current content** - What's already attached to the domain - **Personal preference** - What fits your needs and style ### 3. Trade Safely **Best practices:** - **Verify ownership** - Confirm the seller owns the domain - **Check domain status** - Ensure it's not expired or locked - **Use escrow services** - Protect both buyer and seller - **Document transfers** - Keep records of all transactions ## Benefits - **Transferable ownership** - Move domains between wallets - **Creative expression** - Own and manage creative domain names - **Community participation** - Engage with the ArNS ecosystem - **Content organization** - Structure your permanent web presence - **Identity management** - Use names for personal or project identity ## Ready to Trade? } > Learn about ArNS Primary Names for domain creation. } > Learn how to deploy permanent apps and websites. } > Explore the ArNS documentation for advanced features. # Storing DePIN Data on Arweave Using Turbo (/build/guides/depin) DePIN networks require **scalable and cost-effective storage solutions** they can trust. With vast amounts of data generated by decentralized physical infrastructure networks, traditional onchain storage is prohibitively expensive, yet networks need reliable, long-term access to their device data. Arweave via ar.io provides **chain-agnostic, permanent and immutable storage** for a one-time fee, ensuring networks can access any device data previously stored and verify it has not been tampered with. ## Getting Started with DePIN Data Storage ### Prepare Your Data Structure Organize your DePIN device data in a consistent format. Here's an example for environmental sensor data: ```json { "device_id": "airmon-007", "timestamp": "2025-09-22T14:31:05Z", "location": { "lat": 51.5098, "lon": -0.118 }, "pm25": 16, "co2_ppm": 412, "noise_dB": 41.2 } ``` **Best Practices:** - Use consistent field names across all devices - Include timestamps in ISO format - Add device identifiers for tracking - Consider data compression for large datasets ### Tag Your Data for Discovery Proper tagging is essential for [finding your data](/build/access/find-data) later. Consider these tags for DePIN data: ```json { "name": "App-Name", "value": "AirQuality-DePIN-v1.0" }, { "name": "Device-ID", "value": "airmon-007" }, { "name": "Device-Type", "value": "Environmental-Sensor" }, { "name": "Network-Name", "value": "AirQuality-Network" }, { "name": "Data-Category", "value": "Air-Quality" }, { "name": "Location", "value": "London-UK" }, { "name": "Device-Timestamp", "value": "2025-09-22T14:31:05Z" } ``` **Tagging Strategy:** - Use consistent naming conventions - Include geographic identifiers - Add device type classifications - Include data categories for filtering For more detailed information on tagging, see our [Tagging documentation](/build/upload/tagging) ### Upload to Arweave Select the best method for your DePIN network's needs: ```typescript // Initialize with your wallet const turbo = await TurboFactory.authenticated({ privateKey: jwk, // Your Arweave wallet token: 'arweave' }) // Upload device data const result = await turbo.upload({ data: JSON.stringify(deviceData), dataItemOpts: { tags: [ { name: "Content-Type", value: "application/json" }, { name: "App-Name", value: "AirQuality-DePIN-v1.0" }, { name: "Device-ID", value: "airmon-007" }, { name: "Device-Type", value: "Environmental-Sensor" }, { name: "Network-Name", value: "AirQuality-Network" }, { name: "Data-Category", value: "Air-Quality" }, { name: "Location", value: "London-UK" }, { name: "Device-Timestamp", value: "2025-09-22T14:31:05Z" } ] } }) ``` ```bash # Install Turbo CLI npm install -g @ardrive/turbo-sdk # Upload a single file turbo upload-file --file-path sensor-data.json \ --tag "Content-Type:application/json" \ --tag "App-Name:AirQuality-DePIN-v1.0" \ --tag "Device-Type:Environmental-Sensor" \ --tag "Network-Name:AirQuality-Network" \ --tag "Data-Category:Air-Quality" \ --tag "Location:London-UK" \ --tag "Device-Timestamp:2025-09-22T14:31:05Z" # Upload entire folder turbo upload-folder --folder-path ./sensor-data \ --tag "App-Name:AirQuality-DePIN-v1.0" \ --tag "Network-Name:AirQuality-Network" \ --tag "Data-Category:Air-Quality" \ --index-file index.json ``` For more advanced uploading options, see our [Advanced Uploading with Turbo](/build/upload/advanced-uploading-with-turbo) guide, or the [Turbo SDK documentation](/sdks/turbo-sdk) directly. ## Querying Your DePIN Data ### Find Your Data Use GraphQL to search for your DePIN data by tags and criteria: ```graphql # Find all data for a specific device, most recent results first query { transactions( tags: [ { name: "App-Name", values: ["AirQuality-DePIN-v1.0"] } { name: "Device-ID", values: ["airmon-007"] } ] first: 100 sort: HEIGHT_DESC ) { edges { node { id tags { name value } data { size } } } } } ``` ```graphql # Find data by location query { transactions( tags: [ { name: "App-Name", values: ["AirQuality-DePIN-v1.0"] } { name: "Location", values: ["London-UK"] } ] first: 50 ) { edges { node { id tags { name value } } } } } ``` For more advanced querying options, see our [Find Your Data](/build/access/find-data) documentation. ### Access and Use Your Data Once you have transaction IDs from your queries, choose how to fetch and process the data: **Direct data fetching:** ```javascript // Example: Process air quality data async function processAirQualityData(transactionIds) { const results = [] for (const txId of transactionIds) { const response = await fetch(`https://turbo-gateway.com/${txId}`) const data = await response.json() // Process the data const processed = { Device_ID: data.device_id, timestamp: data.timestamp, location: data.location, pm25: data.pm25, co2_ppm: data.co2_ppm, noise_dB: data.noise_dB } results.push(processed) } return results } ``` For more information on fetching data, see our [Fetch Data](/build/access/fetch-data) documentation. **Verified data with optimized routing:** ```javascript import { createWayfinderClient, PreferredWithFallbackRoutingStrategy, FastestPingRoutingStrategy, HashVerificationStrategy } from "@ar.io/wayfinder-core"; const wayfinder = createWayfinderClient({ ario: ARIO.mainnet(), routingStrategy: new PreferredWithFallbackRoutingStrategy({ preferredGateway: 'https://your-gateway.com', fallbackStrategy: new FastestPingRoutingStrategy({ timeoutMs: 500 }), }), verificationStrategy: new HashVerificationStrategy({ trustedGateways: ['https://turbo-gateway.com'], }), telemetrySettings: { enabled: true, clientName: 'AirQuality-DePIN-v1.0', }, }); // Fetch and verify data using ar:// protocol async function processVerifiedAirQualityData(transactionIds) { const results = [] for (const txId of transactionIds) { const response = await wayfinder.request(`ar://${txId}`) const data = await response.json() // Process the verified data const processed = { Device_ID: data.device_id, timestamp: data.timestamp, location: data.location, pm25: data.pm25, co2_ppm: data.co2_ppm, noise_dB: data.noise_dB, noise_level: data.noise_dB, verified: true // Data is cryptographically verified } results.push(processed) } return results } ``` Learn more about data verification with [Wayfinder](/build/access/wayfinder). ## Next Steps In production, teams have several options to take this further to provide significantly more value to the network and its users including: } > Pay in different Tokens and organise device data files with folders or manifests. }> Operate a gateway optimised to index and serve your device data fast. }> Create mutable data structures for permanent device data and decentralised apps. These approaches can make your DePIN data even more resilient and useful. See more detailed guides about this below and or join our discord to find out more. ## Need Help? If you're interested in exploring these advanced features for your DePIN network, join our [Discord community](https://discord.com/invite/HGG52EtTc2) or reach out to our team. # Encrypted Data Storage with Nillion Blind Computing (/build/guides/encrypted-data-nillion) Store **encrypted data permanently** on Arweave while controlling access through **Nillion's nilDB private storage**. This approach ensures your data remains encrypted on Arweave, decryption keys are securely stored in **nilDB** (Private Storage), and only authorized users can access and decrypt the data. Arweave provides **permanent, immutable storage** for encrypted data, while Nillion's **nilDB** stores decryption keys encrypted and split across multiple nodes. Users can encrypt files using their private keys stored in nilDB, upload the encrypted data to Arweave, and later retrieve and decrypt the data using the same private key—all while maintaining complete control over access permissions. ## Prerequisites Before starting, you'll need to complete Nillion's setup process: ### Create Builder and Get Tokens **Create a Test Builder:** - Visit [Nillion Subscription Portal](https://subscription.nillion.com) - Create a testnet public/private key pair for network access - Use two distinct keys: one for network access, one for subscription payments **Get Testnet NIL Tokens:** - Visit [NIL Faucet](https://faucet.testnet.nillion.com/) - Fund your account with testnet NIL tokens ### Subscribe to nilDB Service **Activate nilDB Subscription:** - Use your subscription wallet to pay for nilDB service - Save your private key in hex format for authentication - You'll need this key to access Nillion's Private Storage services ### Set Up Development Environment **Create a New Project:** ```bash mkdir nillion-arweave-demo cd nillion-arweave-demo npm init -y ``` **Install Required Dependencies:** ```bash npm install @nillion/secretvaults @nillion/nuc @nillion/blindfold @ardrive/turbo-sdk arweave dotenv @noble/curves consola npm install --save-dev typescript @types/node ts-node @tsconfig/node20 ``` **Set Up TypeScript:** ```bash npx tsc --init ``` **Update tsconfig.json:** ```json { "extends": "@tsconfig/node20/tsconfig.json", "compilerOptions": { "outDir": "dist/src", "baseUrl": ".", "module": "esnext", "moduleResolution": "bundler", "allowUnusedLabels": false, "allowImportingTsExtensions": true, "noEmit": true, "allowUnreachableCode": false, "noFallthroughCasesInSwitch": true, "noUncheckedIndexedAccess": true, "noUnusedLocals": true, "noUnusedParameters": true, "forceConsistentCasingInFileNames": true, "declaration": true, "sourceMap": true, "resolveJsonModule": true }, "include": ["src"], "exclude": ["dist", "bundle", "node_modules"] } ``` **Configure Environment Variables:** Create a `.env` file with your configuration: ``` NIL_BUILDER_PRIVATE_KEY=your_hex_private_key_here NIL_BUILDER_COLLECTION_ID=your_collection_id_here # If it exists, one will be created otherwise NILCHAIN_URL=http://rpc.testnet.nilchain-rpc-proxy.nilogy.xyz NILAUTH_URL=https://nilauth.sandbox.app-cluster.sandbox.nilogy.xyz NILDB_NODES=https://nildb-stg-n1.nillion.network,https://nildb-stg-n2.nillion.network,https://nildb-stg-n3.nillion.network ``` ## Understanding the Implementation Let's build the application step by step, explaining what each part does: ### Set Up Project Structure and Imports First, create the main application file `src/index.ts` and understand the imports: ```typescript import "dotenv/config"; ``` **What these imports do:** - **Nillion SDKs**: `@nillion/secretvaults` and `@nillion/nuc` for private storage and access control - **Arweave**: `arweave` and `@ardrive/turbo-sdk` for permanent data storage - **Crypto**: Node.js built-in crypto functions for AES-256-GCM encryption - **Utilities**: Helper functions for key conversion and UUID generation ### Define Configuration and Types Set up the application configuration with proper TypeScript types: ```typescript NILCHAIN_URL: string; NILAUTH_URL: string; NILDB_NODES: string[]; NIL_PAYER_PRIVATE_KEY: string; NIL_BUILDER_PRIVATE_KEY: string; NIL_BUILDER_COLLECTION_ID: string; }; const config: AppConfig = { NILCHAIN_URL: process.env.NILCHAIN_URL || "http://rpc.testnet.nilchain-rpc-proxy.nilogy.xyz", NILAUTH_URL: process.env.NILAUTH_URL || "https://nilauth.sandbox.app-cluster.sandbox.nilogy.xyz", NILDB_NODES: process.env.NILDB_NODES ? process.env.NILDB_NODES.split(",") : [ "https://nildb-stg-n1.nillion.network", "https://nildb-stg-n2.nillion.network", "https://nildb-stg-n3.nillion.network", ], NIL_BUILDER_PRIVATE_KEY: process.env.NIL_BUILDER_PRIVATE_KEY, NIL_BUILDER_COLLECTION_ID: process.env.NIL_BUILDER_COLLECTION_ID, }; ``` **What this does:** - **Type safety**: Defines the structure of our configuration object - **Environment variables**: Loads settings from `.env` file with fallback defaults - **Network endpoints**: Points to Nillion testnet infrastructure - **Multiple nodes**: Uses 3 nilDB nodes for redundancy and security ### Create Encryption and Decryption Functions Implement AES-256-GCM encryption using the user's private key: ```typescript // Utility functions const downloadFile = async (url, location) => { try { const response = await fetch(url); const data = await response.text(); await fs.promises.writeFile(location, data); console.log(`💾 Downloaded file saved to: ${location}`); } catch (error) { console.error("❌ Download file failed:", error); } }; const encryptContent = (content, encryptionKey) => { // Convert hex private key to buffer const privateKeyBuffer = Buffer.from(encryptionKey, "hex"); // Derive a 32-byte encryption key from your private key using SHA-256 const derivedKey = createHash("sha256").update(privateKeyBuffer).digest(); // Validate derived key length (should be 32 bytes for AES-256) if (derivedKey.length !== 32) { throw new Error("Derived encryption key must be exactly 32 bytes for AES-256"); } // Generate secure encryption parameters const iv = randomBytes(16); // Encrypt using AES-256-GCM const cipher = createCipheriv("aes-256-gcm", derivedKey, iv); let encrypted = cipher.update(content); encrypted = Buffer.concat([encrypted, cipher.final()]); // Get the authentication tag (16 bytes for GCM) const authTag = cipher.getAuthTag(); // Combine IV + authTag + encrypted data into a single buffer return Buffer.concat([iv, authTag, encrypted]); }; ``` **What this does:** - **Key derivation**: Uses SHA-256 to create a 32-byte encryption key from the user's private key - **AES-256-GCM**: Provides both confidentiality and authenticity - **Random IV**: Each encryption uses a unique initialization vector - **Auth tag**: Prevents tampering with the encrypted data - **Combined format**: Stores IV, auth tag, and encrypted data together ### Create Decryption Function Implement the corresponding decryption function: ```typescript const decryptContent = (encryptedData, encryptionKey, outputLocation) => { // Convert hex private key to buffer const privateKeyBuffer = Buffer.from(encryptionKey, "hex"); // Derive the same 32-byte encryption key using SHA-256 const derivedKey = createHash("sha256").update(privateKeyBuffer).digest(); // Validate derived key length if (derivedKey.length !== 32) { throw new Error("Derived encryption key must be exactly 32 bytes for AES-256"); } // Extract IV, auth tag, and encrypted content if (encryptedData.length ### Create Token Generation Function Implement delegation token creation for secure access control: ```typescript const generateToken = async (parentToken, command, audience, tokenExpirySeconds, privateKey, body) => { const token = NucTokenBuilder.extending(parentToken) .command(command) .audience(audience) .expiresAt(Math.floor(Date.now() / 1000) + tokenExpirySeconds); if (body) token.body(body); return token.build(privateKey); }; ``` **What this does:** - **Delegation tokens**: Creates time-limited access tokens for users - **Command-based**: Specifies exactly what operations the token allows - **Audience-specific**: Only works for the intended user - **Time-limited**: Automatically expires after the specified duration - **Cryptographically signed**: Uses the builder's private key for authenticity ### Create Utility Files Create supporting utility files for Arweave operations and logging: **Create `src/arweave.ts`:** ```typescript const ARWEAVE_HOST = "turbo-gateway.com"; try { const arweave = Arweave.init({ host: ARWEAVE_HOST, port: 1984, protocol: 'http', timeout: 20000, logging: true, }); // Generate a new wallet const wallet = await arweave.wallets.generate(); // Get the wallet address const address = await arweave.wallets.jwkToAddress(wallet); return { wallet, address }; } catch (error) { logger.error('Error creating wallet:', error); } } try { const signer = new ArweaveSigner(wallet); const turbo = TurboFactory.authenticated({ signer }); const result = await turbo.upload({ data, dataItemOpts: { tags: [ { name: "Content-Type", value: "text/plain" }, { name: "Title", value: "My demo file" }, ], }, }); return result; } catch (error) { logger.error('Error uploading securely encrypted file:', error); } } ``` **Create `src/logger.ts`:** ```typescript ``` **What these do:** - **Arweave utilities**: Handle wallet creation and file uploads using Turbo - **Consola logging**: Provides a beautiful, consistent logging experience with colors and formatting - **Error handling**: Gracefully handle and log errors ### Implement the Main Workflow Now let's implement the main application logic that ties everything together: ```typescript async function main() { try { let collectionId; const dataId = randomUUID(); // Builder Client const builderKeypair = Keypair.from(config.NIL_BUILDER_PRIVATE_KEY); const builder = await SecretVaultBuilderClient.from({ keypair: builderKeypair, urls: { chain: config.NILCHAIN_URL, auth: config.NILAUTH_URL, dbs: config.NILDB_NODES, }, blindfold: { operation: "store" }, }); await builder.refreshRootToken(); // Check if builder is set up correctly const existingProfile = await builder.readProfile(); ``` **What this does:** - **Builder initialization**: Creates a builder client with your private key - **Network connection**: Connects to Nillion testnet infrastructure - **Token refresh**: Gets a fresh authentication token - **Profile validation**: Ensures the builder is properly set up ### Set Up Data Collection Create or validate the collection for storing user private keys: ```typescript // Create the Owned Collection if it doesn't exist if (!config.NIL_BUILDER_COLLECTION_ID) { collectionId = randomUUID(); const schema = { "$schema": "http://json-schema.org/draft-07/schema#", type: "array", items: { type: "object", properties: { _id: { "type": "string", "format": "uuid" }, private_key: { "type": "object", "properties": { "%share": { "type": "string" } }, required: ["%share"] } }, required: ["_id", "private_key"] } }; const collection = { _id: collectionId, type: "owned", name: "Nillion / Arweave Demo App User Profiles", schema, }; const collectionResult = await builder.createCollection(collection); console.log(`✅ Created Owned Collection with ID: ${collectionId}`); console.log(`Make sure to update your .env file with this NIL_BUILDER_COLLECTION_ID to proceed.`); process.exit(0); } else { if (existingProfile.data.collections.indexOf(config.NIL_BUILDER_COLLECTION_ID) === -1) { throw new Error(`Builder does not have collection ${config.NIL_BUILDER_COLLECTION_ID} registered. Please check your .env configuration.`); } else { console.log(`✅ Builder is set up correctly with DID: ${builder.did}`); console.log(`✅ Using Owned Collection ID: ${config.NIL_BUILDER_COLLECTION_ID}`); collectionId = config.NIL_BUILDER_COLLECTION_ID; } } ``` **What this does:** - **Collection creation**: Creates a new collection if none exists - **Schema definition**: Defines the structure for storing user private keys - **Validation**: Ensures the builder has access to the specified collection - **Environment setup**: Prompts you to update your `.env` file with the collection ID ### Create User and Store Private Key Generate a new user and securely store their private key in nilDB: ```typescript // Create user and store its private key in nilDB const secretKey = await SecretKey.generate( { nodes: config.NILDB_NODES.map(url => ({ url })) }, { store: true } ); const userKeypair = Keypair.from(bytesToHex(secretKey.material)); const userDid = userKeypair.toDid().toString(); const user = await SecretVaultUserClient.from({ baseUrls: config.NILDB_NODES, keypair: userKeypair, blindfold: { operation: "store" } }); // Grant write access to the user const delegationToken = await generateToken( builder.rootToken, new Command(["nil", "db", "data", "create"]), userKeypair.toDid(), 3600, // 1 hour builder.keypair.privateKey() ); console.log(`🗝️ Delegation token created`); // User creates profile with Private Key await user.createData(delegationToken, { owner: userDid, acl: { grantee: builder.did.toString(), read: false, write: false, execute: true, }, collection: collectionId, data: [ { _id: dataId, private_key: { "%allot": userKeypair.privateKey() }, } ], }); console.log(`✅ User profile created: ${userDid}`); ``` **What this does:** - **Secret key generation**: Creates a cryptographically secure private key - **Keypair creation**: Converts the secret key to a usable keypair - **User client**: Initializes a user client for nilDB operations - **Delegation token**: Grants the user permission to create data - **ACL setup**: Defines access control - builder can execute queries but not read/write - **Data storage**: Stores the user's private key in the nilDB collection ### Encrypt and Upload File Now let's encrypt a file and upload it to Arweave: ```typescript const wallet = await createWallet(); console.log(`💼 Arweave wallet created: ${wallet?.address}`); // Retrieve the user private key from nilDB const retrievedUserKey = await user.readData({ collection: collectionId, document: dataId, }); // Encrypt file contents const fileData = fs.readFileSync("test/demo.txt"); const encrypted = encryptContent(fileData, retrievedUserKey.data.private_key); const upload = await uploadFile(encrypted, wallet?.wallet); console.log(`✅ File uploaded to Arweave with txId: ${upload.id}`); // Download the file and decrypt it const downloadFileName = `./test/encrypted_demo_${Date.now()}.txt`; console.log(`🕒 Downloading file from Arweave`); await downloadFile(`https://turbo-gateway.com/${upload.id}`, downloadFileName); const decrypted = decryptContent(encrypted, retrievedUserKey.data.private_key, `./test/decrypted_demo_${Date.now()}.txt`); ``` **What this does:** - **Arweave wallet**: Creates a new wallet for uploading files - **Key retrieval**: Gets the user's private key from nilDB - **File encryption**: Encrypts the file using the user's private key - **Arweave upload**: Uploads the encrypted file to permanent storage - **File download**: Downloads the file from Arweave - **Decryption**: Decrypts the file using the same private key ### Complete the Application Finish the main function with error handling: ```typescript } catch (error) { console.log(error); console.error("⚠️ Error setting up builder:", JSON.stringify(error, null, 2)); } } main().catch(console.error); ``` **What this does:** - **Error handling**: Catches and logs any errors that occur during execution - **Application startup**: Runs the main function when the script is executed ## Create Test File Before running the demo, create a test file to encrypt: ```bash mkdir test echo "Hello, this is a test file for Nillion + Arweave encryption demo!" > test/demo.txt ``` ## Understanding the Complete Workflow The main application performs the following steps: 1. **Initialize Builder**: Sets up the Nillion Builder client with your private key 2. **Create/Verify Collection**: Creates a new collection or uses an existing one for storing private keys 3. **Generate User**: Creates a new user with a private key stored securely in nilDB 4. **Create Wallet**: Generates an Arweave wallet for uploading data 5. **Encrypt Data**: Encrypts the test file using the user's private key 6. **Upload to Arweave**: Uploads the encrypted data to Arweave using Turbo 7. **Download & Decrypt**: Downloads the encrypted file and decrypts it using the stored private key **Key Security Features:** - Private keys are stored securely in nilDB with proper access control - Data is encrypted using AES-256-GCM with the user's private key - Only authorized users can access the decryption keys - Encrypted data is permanently stored on Arweave ## Running the Complete Demo Add a start script to your `package.json`: ```json { "scripts": { "start": "ts-node src/index.ts", "build": "tsc", "dev": "ts-node --watch src/index.ts" } } ``` Then run the complete workflow: ```bash # Run the complete workflow npm start ``` This will execute the entire process: 1. Initialize the Builder and create a collection 2. Generate a user and store their private key in nilDB 3. Encrypt a sample file using the private key 4. Upload the encrypted file to Arweave 5. Download the encrypted file from Arweave 6. Decrypt the file using the stored private key ## nilCC To securely compute the logic, we could take advantage of [nilCC](https://docs.nillion.com/build/compute/overview), Nillion's Confidential Computing product, that allows you to run application logic inside a TEE. This way, all the interaction could happen inside the confidential environment with no risk of sensitive information leakage. nilCC workloads can be easily triggered via its REST API: ```bash curl --location '{endpoint}/api/v1/workloads/create' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --header 'x-api-key: xxx' \ --data '{ "name": "private-stamement-workload", "artifactsVersion": "0.1.2", "dockerCompose": "services:\n private-statements:\n image: my/workload-logic:v0.2\n environment:\n - DELEGATION_TOKENS=${DELEGATION_TOKENS}\n - COLLECTION_ID=${COLLECTION_ID}\n - DOCUMENT_ID=${DOCUMENT_ID}\n - NODE_ENV=production\n build: .\n ports:\n - \"8080:8080\"", "envVars": { "DOCUMENT_ID": "b05917e6-996c-4e90-a49f-b62fa891da1b", "COLLECTION_ID": "ce9b1d1c-8006-4053-a0c8-f46ad711fc26", "DELEGATION_TOKENS": "W3sidXJsIjoiaHR0cHM6Ly9uaWxkYi1zdGctbjEubmlsbGlvbi5uZXR3b3JrIiwidG9rZW4iOiJleUpoYkdjaU9pSkZVekkxTmtzaWZRLmV5SnBjM01pT2lKa2FXUTZibWxzT2pBeU5qTmlZMkpsTjJVeU5UZGhNamhrTmpjMk5EWTVNemc0WlRnd05EWmxNelEwTW1JeU5UVm1Zakk0TWpZME9EbGhOalE0TW1ZMk9ESmhNRFpsWWpreU1TSXNJbUYxWkNJNkltUnBaRHB1YVd3Nk1ESmxNemcwTm1NME5UVmtZbU5sWldZNVpXWm1PR0U0TkRFeU4yTXpZbVV4WWprM01UbGhZekExTkRFMVpXWmlaamN5Tnprd1pqTXhabUU1Wmpnd01qZGhJaXdpYzNWaUlqb2laR2xrT201cGJEb3dNall6WW1OaVpUZGxNalUzWVRJNFpEWTNOalEyT1RNNE9HVTRNRFEyWlRNME5ESmlNalUxWm1JeU9ESTJORGc1WVRZME9ESm1Oamd5WVRBMlpXSTVNakVpTENKbGVIQWlPakUzTlRrME56Y3hPRGtzSW1OdFpDSTZJaTl1YVd3dlpHSXZkWE5sY25NdmNtVmhaQ0lzSW1GeVozTWlPbnQ5TENKdWIyNWpaU0k2SWpRMVltUTFPVGxsTmpSbU4yVXdaVEZoWWpnNFpqVXdPVEV5TURRME5UWTFJbjAuQ05lMXlISFlUT3cyZW5lYnh4Nm56VWJaQnpJTzdKVmhYZTBMQVo4aTI3TmJzR0JHMHdBbVVGV2VKbG9oc0FxNHBXREVacGQyamtQaXNGMFZCakNHM0EiLCJwdWJsaWNLZXkiOiIwMmUzODQ2YzQ1NWRiY2VlZjllZmY4YTg0MTI3YzNiZTFiOTcxOWFjMDU0MTVlZmJmNzI3OTBmMzFmYTlmODAyN2EifSx7InVybCI6Imh0dHBzOi8vbmlsZGItc3RnLW4yLm5pbGxpb24ubmV0d29yayIsInRva2VuIjoiZXlKaGJHY2lPaUpGVXpJMU5rc2lmUS5leUpwYzNNaU9pSmthV1E2Ym1sc09qQXlOak5pWTJKbE4yVXlOVGRoTWpoa05qYzJORFk1TXpnNFpUZ3dORFpsTXpRME1tSXlOVFZtWWpJNE1qWTBPRGxoTmpRNE1tWTJPREpoTURabFlqa3lNU0lzSW1GMVpDSTZJbVJwWkRwdWFXdzZNREkxTnpreVpUazJZVFk0WXpCaU4yVm1OemM1TkRrMk1ETXlOMlJqTlRjd056QTBZelprWkRVMk5XTm1NbU5oWTJZeU1EWmlaR00zTW1RMk1USXpaamt3SWl3aWMzVmlJam9pWkdsa09tNXBiRG93TWpZelltTmlaVGRsTWpVM1lUSTRaRFkzTmpRMk9UTTRPR1U0TURRMlpUTTBOREppTWpVMVptSXlPREkyTkRnNVlUWTBPREptTmpneVlUQTJaV0k1TWpFaUxDSmxlSEFpT2pFM05UazBOemN4T0Rrc0ltTnRaQ0k2SWk5dWFXd3ZaR0l2ZFhObGNuTXZjbVZoWkNJc0ltRnlaM01pT250OUxDSnViMjVqWlNJNklqVmhNMlV4WW1aaU9ETm1OVGN5WkdVeE5tRTBOV0U1TlRsaFpETXpPREZrSW4wLjE5Nk9uVWZYT3ZnNDVnbHNUaThkZ09OWkc2R3E2NXMxVlBFa01La1huVEZCUlFmamV1R1JzTkVHUnJYci1obmp1Z1BIeWlpWmJiT0JzMU9ndkYzS053IiwicHVibGljS2V5IjoiMDI1NzkyZTk2YTY4YzBiN2VmNzc5NDk2MDMyN2RjNTcwNzA0YzZkZDU2NWNmMmNhY2YyMDZiZGM3MmQ2MTIzZjkwIn0seyJ1cmwiOiJodHRwczovL25pbGRiLXN0Zy1uMy5uaWxsaW9uLm5ldHdvcmsiLCJ0b2tlbiI6ImV5SmhiR2NpT2lKRlV6STFOa3NpZlEuZXlKcGMzTWlPaUprYVdRNmJtbHNPakF5TmpOaVkySmxOMlV5TlRkaE1qaGtOamMyTkRZNU16ZzRaVGd3TkRabE16UTBNbUl5TlRWbVlqSTRNalkwT0RsaE5qUTRNbVkyT0RKaE1EWmxZamt5TVNJc0ltRjFaQ0k2SW1ScFpEcHVhV3c2TURNd05EQXdNVFU1TW1NelpESmhOR0ZtTkdaa01EUTVaamMxWVRVMk1qTmxNVEE1TXpsaU16ZGpNemhqWXpZMFl6STJORGd3TVdFMU5UWTNZalE1TTJGaUlpd2ljM1ZpSWpvaVpHbGtPbTVwYkRvd01qWXpZbU5pWlRkbE1qVTNZVEk0WkRZM05qUTJPVE00T0dVNE1EUTJaVE0wTkRKaU1qVTFabUl5T0RJMk5EZzVZVFkwT0RKbU5qZ3lZVEEyWldJNU1qRWlMQ0psZUhBaU9qRTNOVGswTnpjeE9Ea3NJbU50WkNJNklpOXVhV3d2WkdJdmRYTmxjbk12Y21WaFpDSXNJbUZ5WjNNaU9udDlMQ0p1YjI1alpTSTZJak5rTXpKaVlXSmhaV1ZsTkRjNU5qY3dNbUUyWW1aa016RTVNV013TURaa0luMC5qYTlzQWpJRUFJb09FM2ZmcnIwV190aXlLNWZWWVhQbVdNWjIxbHlhb0d3eUZoMzQycnFhNldUSHhkT3dVSHVXdVhRR0FCOEZTQnhRWks4NXFLbGQtUSIsInB1YmxpY0tleSI6IjAzMDQwMDE1OTJjM2QyYTRhZjRmZDA0OWY3NWE1NjIzZTEwOTM5YjM3YzM4Y2M2NGMyNjQ4MDFhNTU2N2I0OTNhYiJ9XQ==" }, "publicContainerName": "my-workload", "publicContainerPort": 8080, "memory": 1024, "cpus": 1, "disk": 10, "gpus": 0, "workloadId": "88384328-3038-4a8e-8d45-bbebf6d748d4", "creditRate": 1, "status": "scheduled", "accountId": "some-account" }' ``` or the [nilCC Workload Manager](https://nilcc.nillion.com/). Full instructions can be found [here](https://docs.nillion.com/build/compute/quickstart) ## Additional Resources For reference and additional examples, you can also check out the official demo repository: **[Nillion Arweave Demo](https://github.com/elmurci/nillion-arweave-demo)** This repository contains the complete working implementation that this guide is based on. ## Next Steps } > Clone and run the complete working example from the official repository. } > Learn advanced uploading techniques for large encrypted datasets and batch processing. } > Deep dive into Nillion's nilDB for secure key storage and management. } > Learn more about Nillion's native access control and delegation tokens. ## Need Help? If you're interested in implementing encrypted data storage with Nillion, join our [Discord community](https://discord.com/invite/HGG52EtTc2) or explore the [Nillion developer resources](https://docs.nillion.com) for detailed implementation guides. # Deploy a Permanent dApp (/build/guides/hosting-decentralised-apps/deploy-permanent-dapp) Deploy a permanent web application to Arweave with a human-readable ArNS name — using only your Solana wallet. Your app will be served by the network of decentralized gateways with zero ongoing hosting costs. ## What You'll Build By the end of this guide, your app will be: - **Permanently stored** on Arweave (can never be deleted or modified) - **Accessible** at `https://yourname.ar.io` (and every other ar.io gateway) - **Owned by you** as a Metaplex Core NFT (the ArNS name token) ## Prerequisites - **Node.js 18+** - **A Solana wallet** with SOL and ARIO tokens - **A built web app** (any framework that outputs a static folder — React, Next.js, Vue, Svelte, etc.) ```bash npm install @ar.io/sdk @ardrive/turbo-sdk @solana/kit bs58 ``` Requires `@ar.io/sdk` version 3.23+ for Solana support. ## Step-by-Step ### Build Your App Generate a static build of your web application: ```bash # next.config.js must have: output: 'export' npm run build # Output: ./out/ ``` ```bash npm run build # Output: ./dist/ ``` ```bash # Build your app to a static folder npm run build # Use whatever output directory your framework creates ``` ### Set Up Your Signer Create a shared setup file that both Turbo (for uploads) and the ar.io SDK (for naming) can use: ```typescript // setup.ts // Load your Solana keypair const keypairBytes = new Uint8Array( JSON.parse(fs.readFileSync('./solana-keypair.json', 'utf-8')), ); // For ar.io SDK (ArNS names, records) // For Turbo (file uploads to Arweave) // Turbo uses its own signer format — pass the secret key as base58 privateKey: bs58.encode(keypairBytes.slice(0, 32)), token: 'solana', }); ``` **Security:** Never commit your keypair file to version control. Use a dedicated deployment wallet with only the SOL and ARIO needed for the operation. ### Upload to Arweave via Turbo Upload your build folder. Turbo bundles all files into an Arweave manifest — a single transaction ID that maps to all your app's files: ```typescript // deploy.ts const ARNS_NAME = 'my-cool-app'; // the ArNS name you want const BUILD_DIR = './dist'; // your build output folder // Step 1: Upload the build folder console.log('Uploading to Arweave...'); const uploadResult = await turbo.uploadFolder({ folderPath: BUILD_DIR, dataItemOpts: { tags: [ { name: 'App-Name', value: ARNS_NAME }, { name: 'App-Version', value: '1.0.0' }, ], }, }); const manifestTxId = uploadResult.manifestResponse.id; console.log(`Uploaded! Manifest TX: ${manifestTxId}`); console.log(`Direct access: https://turbo-gateway.com/${manifestTxId}`); ``` Files under 100KB are free to upload via Turbo. Larger uploads are paid with SOL from your wallet — no pre-funding needed when using just-in-time payments. ### Register an ArNS Name (If You Don't Have One) Skip this step if you already own the ArNS name. ```typescript // Check if the name is available try { const existing = await ario.getArNSRecord({ name: ARNS_NAME }); console.log(`Name "${ARNS_NAME}" is already registered`); } catch { // Name is available — register it console.log(`Registering "${ARNS_NAME}"...`); // Check the cost first const cost = await ario.getTokenCost({ intent: 'Buy-Name', name: ARNS_NAME, type: 'lease', years: 1, }); console.log(`Cost: ${cost / 1_000_000} ARIO`); // Buy it (this mints an ANT as a Metaplex Core NFT) await ario.buyRecord({ name: ARNS_NAME, type: 'lease', years: 1, }); console.log(`Registered "${ARNS_NAME}"!`); } ``` ### Point Your Name to Your App Set the ANT's root (@) record to your uploaded manifest: ```typescript // Get the ANT mint address from the ArNS record const record = await ario.getArNSRecord({ name: ARNS_NAME }); // Initialize the ANT const ant = ANT.init({ signer, processId: record.processId }); // Set the root record to your manifest console.log('Setting ArNS record...'); await ant.setRecord({ undername: '@', transactionId: manifestTxId, ttlSeconds: 3600, }); console.log('Done! Your app is live at:'); console.log(` https://${ARNS_NAME}.ar.io`); console.log(` https://${ARNS_NAME}.turbo-gateway.com`); ``` ### Verify Wait a minute for gateways to pick up the new record, then verify: ```bash curl -I https://my-cool-app.ar.io # Should return 200 OK with your app's index.html ``` Your app is now permanently hosted and accessible through every ar.io gateway in the network. ## Updating Your App Since Arweave data is immutable, "updating" means uploading a new version and updating your ArNS record to point to it: ```typescript // Upload new version const newUpload = await turbo.uploadFolder({ folderPath: './dist', dataItemOpts: { tags: [ { name: 'App-Name', value: 'my-cool-app' }, { name: 'App-Version', value: '2.0.0' }, ], }, }); // Update the record const ant = ANT.init({ signer, processId: record.processId }); await ant.setRecord({ undername: '@', transactionId: newUpload.manifestResponse.id, ttlSeconds: 3600, }); // Old version is still on Arweave forever — instant rollback if needed ``` ## Using Undernames for Staging You can use undernames to deploy staging environments alongside production: ```typescript // Deploy staging version await ant.setRecord({ undername: 'staging', transactionId: stagingManifestTxId, ttlSeconds: 300, // short TTL for faster updates }); // Access at: https://staging_my-cool-app.ar.io // When ready, promote to production await ant.setRecord({ undername: '@', transactionId: stagingManifestTxId, ttlSeconds: 3600, }); ``` ## Full Script Here's the complete deployment script you can adapt: ```typescript const ARNS_NAME = process.argv[2] || 'my-app'; const BUILD_DIR = process.argv[3] || './dist'; async function deploy() { // Load Solana keypair const keypairBytes = new Uint8Array( JSON.parse(fs.readFileSync('./solana-keypair.json', 'utf-8')), ); // Init SDKs const signer = await createKeyPairSignerFromBytes(keypairBytes); const ario = ARIO.mainnet({ signer }); const turbo = TurboFactory.authenticated({ privateKey: bs58.encode(keypairBytes.slice(0, 32)), token: 'solana', }); // Upload console.log(`Uploading ${BUILD_DIR}...`); const upload = await turbo.uploadFolder({ folderPath: BUILD_DIR, dataItemOpts: { tags: [{ name: 'App-Name', value: ARNS_NAME }], }, }); const txId = upload.manifestResponse.id; console.log(`Uploaded: https://turbo-gateway.com/${txId}`); // Get ANT and update record const arnsRecord = await ario.getArNSRecord({ name: ARNS_NAME }); const ant = ANT.init({ signer, processId: arnsRecord.processId }); console.log('Updating ArNS record...'); await ant.setRecord({ undername: '@', transactionId: txId, ttlSeconds: 3600, }); console.log(`\nLive at: https://${ARNS_NAME}.ar.io`); } deploy().catch(console.error); ``` ```bash # Usage npx tsx deploy.ts my-cool-app ./dist ``` ## Next Steps } /> } /> } /> # Deploying with ARIO Deploy (/build/guides/hosting-decentralised-apps/deploying-with-ario-deploy) import { Terminal, Rocket, Wallet, Zap, GitBranch, BookOpen, } from 'lucide-react'; [ARIO Deploy](https://github.com/ar-io/ar-io-deploy) is the recommended CLI tool for hosting decentralised applications on ar.io. It streamlines the entire deployment process by uploading your build folder to Arweave, creating Arweave manifests, and automatically updating your ArNS records in a single command. Built on the Turbo SDK, ario-deploy offers flexible payment options including pre-funded Turbo Credits or on-demand topups using ARIO or Base-ETH tokens. It works seamlessly with both Arweave and EVM wallets, making it easy to integrate permanent hosting into your existing development workflow. Check out the [series introduction](/build/guides/hosting-decentralised-apps) to learn about permanent hosting and ArNS domains. ## Prerequisites Before starting, ensure you have: - **Node.js 18+** - Download from [nodejs.org](https://nodejs.org/) - **Upload Wallet** - An Arweave JWK or EVM private key to pay for the upload - **Solana Wallet** - A base58 Solana secret key that controls your ArNS name (for ArNS record updates) - **ArNS Name** - Register one at [arns.ar.io](https://arns.ar.io) - **Command Line Familiarity** - Basic terminal/shell knowledge ARIO Deploy uses two separate keys: - **`DEPLOY_KEY`** — pays for the Arweave upload (Arweave, Ethereum, Polygon, or Solana wallet) - **`ARNS_KEY`** — a Solana key that controls the ArNS name and signs the record update These can be different wallets. The upload key handles payment; the ArNS key handles name ownership. If you only need to upload without updating an ArNS record, you can use the `ario-deploy upload` command with just a `DEPLOY_KEY`. ## Project Setup Let's create a new web application and configure it for deployment. ARIO Deploy works with any framework that generates a static build folder. Initialize a new Next.js application: ```bash title="Terminal" npx create-next-app@latest my-permaweb-app cd my-permaweb-app ``` When prompted, select your preferences. For permanent hosting, **enable static export** by updating `next.config.js`: ```javascript title="next.config.js" /** @type {import('next').NextConfig} */ const nextConfig = { output: 'export', images: { unoptimized: true, }, trailingSlash: true, } module.exports = nextConfig ``` - `output: 'export'` generates a static site in the `out` folder - `images.unoptimized: true` prevents server-side image optimization - `trailingSlash: true` ensures URLs work correctly on static hosting Add ario-deploy as a development dependency: ```bash title="Terminal" npm install --save-dev @ar.io/deploy ``` Update your `package.json` to include deployment commands: ```json title="package.json" { "scripts": { "dev": "next dev", "build": "next build", "deploy": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out" } } ``` Replace `your-arns-name` with your actual ArNS domain name. Note the `--deploy-folder out` flag is required since Next.js exports to `./out` instead of the default `./dist`. Initialize a new Vite application with React: ```bash title="Terminal" npm create vite@latest my-permaweb-app -- --template react cd my-permaweb-app npm install ``` Update `vite.config.js` to use relative paths for Arweave: ```javascript title="vite.config.js" plugins: [react()], base: './', // Use relative paths for Arweave }) ``` The `base: './'` setting ensures all asset paths are relative, which is required for proper loading on Arweave gateways. Add ario-deploy as a development dependency: ```bash title="Terminal" npm install --save-dev @ar.io/deploy ``` Update your `package.json` to include deployment commands: ```json title="package.json" { "scripts": { "dev": "vite", "build": "vite build", "deploy": "vite build && ario-deploy deploy --arns-name your-arns-name" } } ``` Replace `your-arns-name` with your actual ArNS domain name. ## Deploying from the Command Line For this walkthrough, we'll deploy directly from the command line using inline credentials. For production apps, we recommend using GitHub Actions with secrets (covered later in this guide). Always use a dedicated wallet for deployments to minimize security risks. Never commit wallet files or keys to version control. ARIO Deploy uses [Turbo](https://console.ar.io) to upload files to Arweave. Before deploying, ensure your wallet has sufficient credits. Visit the [Console app](https://console.ar.io/topup) and connect your deployment wallet to view your current balance. A typical static site (5-10 MB) costs approximately 0.1-0.5 ARIO. Credits are applied instantly and remain in your wallet for future deployments. The `npm run deploy` command we configured earlier will build your app and deploy it to Arweave. You need both the upload key (`DEPLOY_KEY`) and the ArNS authority key (`ARNS_KEY`): For Arweave upload wallets, base64 encode your JWK file. The `ARNS_KEY` is always a base58 Solana secret key: ```bash title="Terminal" DEPLOY_KEY=$(base64 -i wallet.json) ARNS_KEY="your-base58-solana-key" npm run deploy ``` - `DEPLOY_KEY` — your Arweave JWK wallet (base64-encoded), pays for the upload - `ARNS_KEY` — your Solana secret key (base58), signs the ArNS record update For EVM wallets (Ethereum, Polygon, Base), use your raw private key for the upload. The `ARNS_KEY` is always a base58 Solana secret key: ```bash title="Terminal" DEPLOY_KEY="0x1234567890abcdef..." ARNS_KEY="your-base58-solana-key" npm run deploy ``` - `DEPLOY_KEY` — your EVM private key, pays for the upload - `ARNS_KEY` — your Solana secret key (base58), signs the ArNS record update For EVM wallets, ensure your deploy script in `package.json` includes the `--sig-type` flag (e.g., `--sig-type ethereum`). When running from a terminal, ario-deploy will interactively prompt for any missing keys or configuration. You can omit `ARNS_KEY` from the command and the CLI will prompt you for it. ARIO Deploy will: 1. **Upload files** to Arweave via Turbo 2. **Create a manifest** with SPA fallback detection 3. **Update ArNS records** to point to the new transaction Expected output: ``` Starting deployment... ✔ ARIO initialized ✔ ArNS record fetched for your-arns-name ✔ Signer created (arweave) ✔ Turbo initialized ✔ Turbo credits check passed ✔ Folder uploaded: abc123def456ghi789jkl012mno345pqr678stu901v ✔ ANT record updated Deployment Successful! ┌─────────────┬───────────────────────────────────────────────┐ │ Tx ID │ abc123def456ghi789jkl012mno345pqr678stu901v │ │ ArNS Name │ your-arns-name │ │ Undername │ @ │ │ ArNS URL │ https://your-arns-name.ar.io │ └─────────────┴───────────────────────────────────────────────┘ ``` ArNS updates typically propagate across the gateway network within 60 seconds (the default TTL). You may need to hard refresh your browser to see changes immediately. ## On-Demand Payment Instead of pre-funding Turbo Credits, you can pay for deployments on-demand. ARIO Deploy will automatically convert tokens to credits as needed. Update your `package.json` to include an on-demand deployment script: For Arweave wallets using ARIO tokens: ```json title="package.json" { "scripts": { "dev": "next dev", "build": "next build", "deploy": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out", "deploy:on-demand": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --on-demand ario --max-token-amount 2.0" } } ``` - `--on-demand ario` enables ARIO payment mode - `--max-token-amount 2.0` sets maximum ARIO to spend (prevents unexpected costs) For EVM wallets using Base-ETH: ```json title="package.json" { "scripts": { "dev": "next dev", "build": "next build", "deploy": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --sig-type ethereum", "deploy:on-demand": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --sig-type ethereum --on-demand base-eth --max-token-amount 0.01" } } ``` - `--sig-type ethereum` required for EVM wallets - `--on-demand base-eth` enables Base Network payment - Wallet must be funded with ETH on Base Network Base-ETH on-demand payment only works with Ethereum signer types. Your wallet must have ETH on the Base Network, not Ethereum mainnet. For Arweave wallets using ARIO tokens: ```json title="package.json" { "scripts": { "dev": "vite", "build": "vite build", "deploy": "vite build && ario-deploy deploy --arns-name your-arns-name", "deploy:on-demand": "vite build && ario-deploy deploy --arns-name your-arns-name --on-demand ario --max-token-amount 2.0" } } ``` - `--on-demand ario` enables ARIO payment mode - `--max-token-amount 2.0` sets maximum ARIO to spend (prevents unexpected costs) For EVM wallets using Base-ETH: ```json title="package.json" { "scripts": { "dev": "vite", "build": "vite build", "deploy": "vite build && ario-deploy deploy --arns-name your-arns-name --sig-type ethereum", "deploy:on-demand": "vite build && ario-deploy deploy --arns-name your-arns-name --sig-type ethereum --on-demand base-eth --max-token-amount 0.01" } } ``` - `--sig-type ethereum` required for EVM wallets - `--on-demand base-eth` enables Base Network payment - Wallet must be funded with ETH on Base Network Base-ETH on-demand payment only works with Ethereum signer types. Your wallet must have ETH on the Base Network, not Ethereum mainnet. See [Base documentation](https://docs.base.org/) for getting testnet or mainnet ETH. Run the on-demand deployment command with both keys: ```bash title="Terminal" DEPLOY_KEY=$(base64 -i wallet.json) ARNS_KEY="your-base58-solana-key" npm run deploy:on-demand ``` The tool will automatically convert ARIO to Turbo Credits as needed for the deployment. ```bash title="Terminal" DEPLOY_KEY="0x1234567890abcdef..." ARNS_KEY="your-base58-solana-key" npm run deploy:on-demand ``` The tool will automatically convert Base-ETH to Turbo Credits as needed for the deployment. The on-demand approach is ideal for: - **Frequent deployments** where pre-funding isn't convenient - **CI/CD pipelines** that need reliable automated deployments - **Multi-team projects** where different wallets handle different apps ## Automating with GitHub Actions The simplest way to automate deployments is with the official [`ar-io/ar-io-deploy`](https://github.com/ar-io/ar-io-deploy) GitHub Action. It handles node setup, dedup caching, and PR preview comments automatically. In your GitHub repository, navigate to **Settings** → **Secrets and variables** → **Actions** and add: | Secret | Description | |--------|-------------| | `DEPLOY_KEY` | Upload wallet key. For Arweave: base64-encoded JWK. For EVM: raw private key with `0x` prefix. | | `ARNS_KEY` | Base58-encoded Solana secret key that controls the ArNS name. | | `ARNS_NAME` | Your ArNS domain name (e.g., `myapp`). | Create `.github/workflows/deploy.yml`: ```yaml title=".github/workflows/deploy.yml" name: Deploy to Arweave on: push: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - name: Install and build run: | npm ci npm run build - name: Deploy to Arweave uses: ar-io/ar-io-deploy@v1 with: deploy-key: ${{ secrets.DEPLOY_KEY }} arns-key: ${{ secrets.ARNS_KEY }} arns-name: ${{ secrets.ARNS_NAME }} ``` The action automatically detects and uploads the `./dist` folder. For Next.js projects, add `deploy-folder: ./out`. Create a separate workflow for PR preview deployments at `.github/workflows/pr-preview.yml`: ```yaml title=".github/workflows/pr-preview.yml" name: PR Preview on: pull_request: types: [opened, synchronize, reopened, closed] jobs: preview: runs-on: ubuntu-latest if: github.event.pull_request.head.repo.full_name == github.repository steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - name: Install and build run: | npm ci npm run build - name: Deploy preview uses: ar-io/ar-io-deploy@v1 with: deploy-key: ${{ secrets.DEPLOY_KEY }} arns-key: ${{ secrets.ARNS_KEY }} arns-name: ${{ secrets.ARNS_NAME }} preview: 'true' github-token: ${{ secrets.GITHUB_TOKEN }} ``` When `preview` is enabled, the action: - Auto-generates an undername from the PR number (e.g., `myrepo-pr-42`) - Posts a comment on the PR with the preview URL - Cleans up the undername when the PR is closed Make a commit and push to your main branch: ```bash title="Terminal" git add . git commit -m "Set up automated deployments" git push origin main ``` Check the **Actions** tab in your GitHub repository to monitor the deployment progress. Each push triggers a new deployment. For high-traffic repositories, consider adding `paths` filters to deploy only when specific files change, or use manual workflow triggers. ## Summary You now know how to deploy permanent web applications using ario-deploy: - **Static site setup** for Next.js and React + Vite with proper configuration - **Two-key model** with separate upload (`DEPLOY_KEY`) and ArNS authority (`ARNS_KEY`) keys - **Flexible wallet options** supporting Arweave, EVM, and Solana wallets - **Payment methods** including pre-funded Turbo Credits and on-demand topups with ARIO or Base-ETH - **Command-line deployment** with inline wallet credentials for quick deployments - **GitHub Actions automation** with the official `ar-io/ar-io-deploy` action for production deploys and PR previews For more details, see the [ARIO Deploy GitHub repository](https://github.com/ar-io/ar-io-deploy). In the next guide, you'll learn how to use undernames to manage multiple environments and versions of your application. # One Click Deployments with Arlink (/build/guides/hosting-decentralised-apps/deploying-with-arlink) ![Arlink login page showing GitHub, Wander, and MetaMask authentication options](/content/arlink-homepage.png) ## Introduction In the previous guides in this series, we've used a CLI tool called `ario-deploy` to configure and host decentralised apps on ar.io. In this guide we'll be using [Arlink](https://arlink.ar.io), a visual, web-based platform for hosting decentralised apps on ar.io without needing command-line tools. It offers automated builds, GitHub integration, and seamless ArNS management through an intuitive interface. Check out the [series introduction](/build/guides/hosting-decentralised-apps) to learn how ar.io enables permanent, decentralised hosting with 100+ independent gateways. ## What You'll Learn - **Deploying with Arlink** - Using the visual web interface for deployments - **GitHub integration** - Setting up automated deployments from your repository - **ArNS setup** - Connecting your ArNS name or using free Arlink undernames - **Build monitoring** - Tracking deployment progress with live logs - **When to use Arlink** - Understanding limitations and choosing the right tool ## What is Arlink? Arlink is a web-based deployment platform that simplifies hosting applications on Arweave. It provides a visual interface for developers who prefer graphical tools over command-line interfaces, while maintaining the same permanence and decentralization benefits. **Key Features:** - **Visual Interface** - No command-line knowledge required - **Automated Builds** - Auto-detects build settings and handles the entire build process - **GitHub Integration** - Deploy directly from your repositories with continuous deployment - **Real-Time Monitoring** - Live build logs and progress tracking - **ArNS Management** - Connect existing ArNS names or use free Arlink undernames All applications that can be deployed with `ario-deploy` CLI can also be deployed with Arlink. Choose the tool that fits your workflow: - **Arlink**: Visual workflows, quick deployments, smaller projects (under 10MB) - **ario-deploy**: CI/CD pipelines, large applications, custom automation See [Limitations & Considerations](#limitations--considerations) below to help decide. ## Prerequisites Before deploying with Arlink, you'll need: - **Arweave Wallet** - Create one at [Wander](https://www.wander.app/) and add AR tokens or [Turbo credits](https://console.ar.io/topup) - **GitHub Repository** - Your application code in a GitHub repository with build scripts - **Static Build Output** - Application must build to static files (HTML, CSS, JS) - **Optional: ArNS Name** - Purchase at [arns.ar.io](https://arns.ar.io) or use free Arlink undernames Test your build locally (`npm run build`) before deploying to ensure it produces static output. ## Deployment Methods Arlink offers two main deployment approaches: - **GitHub Deploy** - Connect your repository for automated builds with continuous deployment - **Template Hub** - Start with pre-built templates or add your own at [arlink.ar.io/templates](https://arlink.ar.io/templates) This guide focuses on GitHub deployment, which is the most common approach for custom applications. ## Deploying from GitHub The deployment process consists of four main phases. For detailed step-by-step instructions, see the [Arlink Quickstart Guide](https://arlink.gitbook.io/arlink-docs/getting-started/quickstart). ### 1. Connect & Authorize Navigate to the [Arlink Dashboard](https://arlink.ar.io/) and connect your Arweave wallet (Wander). Then authorize GitHub access to enable repository connections. ![Arlink login page showing GitHub, Wander, and MetaMask authentication options](/content/arlink-login.png) Arlink only requests read access to your repositories and webhook permissions for continuous deployment. ### 2. Configure Deployment Select your GitHub repository and branch. Arlink will automatically detect: - **Package Manager** - npm, yarn, or pnpm - **Framework** - React, Next.js, Vue, Astro, etc. - **Build Command** - Usually `npm run build` - **Output Directory** - `dist/`, `build/`, `out/`, etc. ![Arlink repository selection interface showing GitHub repositories with import buttons](/content/arlink-repo-select.png) Review the auto-detected settings and adjust if needed. Ensure your output directory matches your framework: | Framework | Output Directory | Notes | |-----------|-----------------|-------| | Vite/React | `dist/` | Default configuration | | Next.js | `out/` | Requires `output: 'export'` in config | | Astro | `dist/` | Static by default | | Create React App | `build/` | Default configuration | ![Arlink deploy configuration options](/content/arlink-deploy-config.png) ### 3. Choose Domain Select how your application will be accessible: **Free Arlink Undername:** - Format: `yourname_arlink.ar.io` - No ArNS name purchase required - Available immediately **Existing ArNS Name:** - Use your purchased ArNS name (e.g., `myapp.ar.io`) - Optionally add undernames (e.g., `staging_myapp.ar.io`) - Arlink automatically updates ArNS records {/* Screenshot: Domain selection interface showing Arlink undername and existing ArNS options */} ### 4. Deploy & Monitor Click **Deploy** to start the build process. Arlink will clone your repository, install dependencies, build your application, and upload to Arweave. ![Arlink deployment build monitoring](/content/arlink-deployment-process.png) **Build Timeline:** - Small apps (~1MB): 2-3 minutes - Medium apps (1-5MB): 3-5 minutes - Large apps (5-10MB): 5-10 minutes Arlink enforces a **10MB max build output** and **10-minute build timeout**. For larger applications, use `ario-deploy` CLI instead. Once complete, your application is permanently deployed and accessible via: - Your chosen domain (e.g., `myapp_arlink.ar.io`) - Any ar.io gateway (e.g., `myapp_arlink.turbo-gateway.com`) - Direct transaction ID {/* Screenshot: Successful deployment screen showing URL, transaction ID, and deployment stats */} ## ArNS Integration Arlink offers two domain options for your deployments: ### Free Arlink Undernames Arlink provides free subdomains under the `arlink` ArNS name: - **Format**: `yourname_arlink.ar.io` - **Cost**: Free (no ArNS purchase required) - **Availability**: Instant, accessible via all ar.io gateways - **Limitation**: Must be unique across all Arlink deployments ### Existing ArNS Names Connect your owned ArNS names for custom domains: - Select your ArNS name from the dashboard dropdown - Optionally add undernames for versioning (e.g., `staging_myapp`, `v1_myapp`) - Arlink automatically updates ArNS records on deployment {/* Screenshot: ArNS name selector showing owned names and undername input field */} Undernames use underscore separators: `staging_myapp` not `staging.myapp`. See [Using Undernames for Versioning](/build/guides/hosting-decentralised-apps/using-undernames-for-versioning) for versioning strategies. **Deployment Management:** The Arlink dashboard lets you view deployment history, manage undernames, and rollback to previous deployments by updating which transaction ID your ArNS name points to. ![Arlink deployment build monitoring](/content/arlink-history.png) ## Limitations & Considerations Understanding Arlink's limitations helps you choose the right deployment tool for your project. ### Size and Time Constraints | Constraint | Limit | Impact | |------------|-------|--------| | Max Build Output | 10 MB | Applications larger than 10MB cannot be deployed | | Build Timeout | 10 minutes | Complex builds exceeding 10 minutes will fail | | Deployment Cost | Subsidized (beta) | Pricing may change after beta period | The 10MB limit applies to your **build output**, not your source code. Check your build size with: ```bash npm run build du -sh dist/ # or build/, out/, etc. ``` If your build exceeds 10MB, use `ario-deploy` CLI instead. ### Comparison: Arlink vs ario-deploy CLI Choose the right tool for your use case: } title="Use Arlink When..." description="You prefer visual interfaces over command-line tools, your build output is under 10MB, you want automated GitHub deployments, you need quick one-off deployments, you want to use free Arlink undernames, or your build completes in under 10 minutes." /> } title="Use CLI When..." description="Your build output exceeds 10MB, you need custom deployment scripts, you want CI/CD pipeline integration, you need Ethereum wallet deployment, you require Base-ETH payment options, or you want full control over deployment process." /> ### Additional Limitations **Build Environment:** - Standard Node.js environment only - No custom build tools or dependencies - Limited environment variable support - No Docker or custom runtimes **Deployment Features:** - No support for Ethereum wallet signatures - No custom payment methods (Base-ETH, etc.) - Limited automation beyond GitHub integration - No programmatic API access **ArNS Management:** - Cannot create new ArNS names through Arlink - Must purchase ArNS names separately at [arns.ar.io](https://arns.ar.io) - Limited undername configuration options If you outgrow Arlink's capabilities, all your existing deployments can be managed with `ario-deploy` CLI. See the [other guides in this series](/build/guides/hosting-decentralised-apps) for CLI deployment instructions. ## Continuous Deployment Arlink automatically sets up continuous deployment when you authorize GitHub access. ### How It Works Arlink adds webhooks to your repository to detect push events. When you push to your configured branch, Arlink automatically triggers a new build and deployment. ```bash git add . git commit -m "Update homepage content" git push origin main # Triggers automatic deployment ``` {/* Screenshot: Webhook configuration settings in Arlink dashboard */} ### Branch-Based Deployments Configure multiple branches to deploy to different undernames: | Branch | Undername | Purpose | |--------|-----------|---------| | `main` | `myapp` (root) | Production | | `develop` | `staging_myapp` | Staging | | `feature/*` | `dev_myapp` | Development | Monitor all deployments in the Arlink dashboard, which shows build status, commit hashes, build logs, and transaction IDs. For more details on continuous deployment setup, see the [Arlink Documentation](https://arlink.gitbook.io/arlink-docs). ## Summary You now know how to deploy applications using Arlink's visual interface: - **Visual web interface** for deploying without command-line tools - **GitHub integration** with automated builds and continuous deployment - **Domain options** including free Arlink undernames or existing ArNS names - **Build monitoring** with real-time logs and progress tracking - **Understanding limitations** to choose between Arlink and CLI tools for your project needs Arlink provides a quick and accessible way to deploy smaller applications. For larger builds, advanced CI/CD, or custom deployments, consider using the `ario-deploy` CLI covered in earlier guides. In our final guide we'll explore deploying using the ArDrive web UI. # Deploying with ArDrive (/build/guides/hosting-decentralised-apps/hosting-with-ardrive) ## Introduction Create **permanent dApps** using the ArDrive web interface. This guide shows you how to deploy your dApp or website to the permaweb using ArDrive's user-friendly interface. Check out the [series introduction](/build/guides/hosting-decentralised-apps) to learn how ar.io enables permanent, decentralised hosting with 100+ independent gateways. ## What You'll Learn - How to deploy dApps using ArDrive web - Creating manifests for proper file routing - Assigning friendly ArNS names - Updating your dApp with new versions ## Prerequisites **For simple apps and websites:** - Your dApp files ready for deployment - ArDrive account (free to create) **For advanced applications:** - dApp prepared with hash routing and relative file paths - Static files built (for frameworks like React) - Learn more about [preparing your dApp for deployment](https://docs.ardrive.io/docs/misc/deploy/) ## Step-by-Step Deployment Go to the [ArDrive web app](https://app.ardrive.io/#/sign-in) and log in using your preferred method. If you don't have an account, follow the instructions to create one. Navigate to the drive where you want your project hosted. If you need a new drive: - Click the big red "New" button at the top left - Create a new drive - **Important:** Set the drive to **public** for others to access your dApp With your drive selected: - Click the big red "New" button again - Select "Upload Folder" - Navigate to your project's root directory (or built directory if required) - Select the entire directory to maintain your project's file structure Review the upload and associated cost. If everything looks correct, click "Confirm". **Cost Note:** Uploading to Arweave isn't free, but costs are usually quite small compared to the benefits of permanent hosting. While ArDrive displays files as a traditional file structure, they don't actually exist that way on Arweave. The manifest acts as a map to all your dApp files: - Navigate into your newly created folder by double-clicking it - Click the big red "New" button again - Select "New Manifest" in the "Advanced" section - Name the manifest and save it inside the folder you just created Once the manifest is created: - Click on it to expand its details - Go to the "Details" tab - Find the "Data TX ID" on the bottom right - Copy this unique identifier for your dApp Your dApp is now live on the permaweb forever! - Append the Data TX ID to a gateway URL: `https://turbo-gateway.com/YOUR-TX-ID` - It may take a few minutes for files to propagate through the network - Once propagated, your dApp is accessible to anyone, anywhere, at any time Make your dApp easier to access with an ArNS name: - If you own an ArNS name, you'll be prompted during manifest creation - If not, purchase one from [arns.ar.io](https://arns.ar.io) - You can also assign an ArNS name later by clicking the three dots next to any file and selecting "Assign ArNS name" ## Updating Your dApp Files uploaded to Arweave are **permanent and immutable** - they cannot be changed. However, the [Arweave File System (ArFS)](/build/advanced/arfs) protocol lets you "replace" them with new versions while keeping old ones accessible. ### How Updates Work **To update your dApp:** 1. **Make your changes** and build the static directory 2. **Upload the entire folder again** to the same location 3. **Follow the same steps** as the original upload 4. **Create a new manifest** with the same name as the old one 5. **The new manifest generates a new TX ID** for the updated dApp **Important Notes:** - The old version remains accessible to anyone with the correct TX ID - Old files won't display in ArDrive unless you view file history - Each version gets its own unique transaction ID ## Benefits of ArDrive Web Deployment - **User-friendly interface** - No command line required - **Automatic manifest creation** - Handles file routing for you - **Integrated ArNS support** - Easy domain name assignment - **Version management** - Built-in file history and updates - **Cost transparency** - See upload costs before confirming ## Next Steps } /> } /> } /> # Hosting Decentralised Apps on ar.io (/build/guides/hosting-decentralised-apps) import { BookOpen, Rocket, GitBranch, Globe, Zap, Upload, } from "lucide-react"; The goal of decentralised apps (dapps) has always been full stack. However, until now, that has not been possible due to single points of failure for hosting frontends. Ar.io solves this via: - **Permanent Storage**: Pay once and store your site forever on [Arweave](https://arweave.org/). - **Smart Domains**: a sovereign owned domain-space via [Ar.io Name System (ArNS)](/learn/arns). - **Distributed Gateway Network**: access your site as a subdomain of active [gateways](/learn/gateways) For example, this docs portal is a decentralised app: - [docs.ar.io](https://docs.ar.io) - [docs.turbo-gateway.com](https://docs.turbo-gateway.com) Same app, different gateway. You can repeat this by loading the `docs` subdomain for any of the active gateways in the network. ## What You'll Learn In this guide series, you'll learn how to deploy and manage permanent applications on ar.io. We'll cover the following: } /> } /> } /> } /> } /> Each guide will build on the last and by the end of series you'll be equipped to deploy and host any decentralised app. ## Next Steps } /> } /> } /> } /> # Migrating Your App to the New SDKs (/build/guides/hosting-decentralised-apps/migrating-your-app-to-new-sdks) ## Overview This guide will help developers migrate applications that interact with ar.io protocol features to the current SDKs and Solana-based workflows. For now, use this page as a placeholder while final SDK examples and migration details are reviewed. ## What This Guide Will Cover - Updating to the latest `@ar.io/sdk` version - Replacing older protocol interaction patterns with current Solana signer flows - Updating ArNS name registration and record-management code - Reviewing ANT handling for Metaplex Core NFT ownership and control - Separating Arweave data-upload wallets from Solana protocol wallets where needed - Verifying application behavior against devnet or staging examples ## Current References These pages contain related material that should inform the final migration guide: - [ar.io SDK](/sdks/ar-io-sdk) - [Registering ArNS Names Programmatically](/build/guides/working-with-arns/register-arns-programmatically) - [Setting ArNS Records Programmatically](/build/guides/working-with-arns/set-arns-records-programmatically) - [Deploy a Permanent dApp](/build/guides/hosting-decentralised-apps/deploy-permanent-dapp) ## Review Status This page needs technical review before launch. Final content should be checked against the latest `@ar.io/sdk` release, generated SDK docs, Solana signer examples, and current ArNS/ANT behavior. # Undernames for Environments and Versioning (/build/guides/hosting-decentralised-apps/using-undernames-for-versioning) In the [previous guide](/build/guides/hosting-decentralised-apps/deploying-with-ario-deploy), you deployed your application to your base ArNS name. Now you'll learn how to manage multiple versions and environments using **undernames** - subdomains under your ArNS name. ## What You'll Learn - **Strategic versioning patterns** for managing multiple environments - **Package.json workflows** for dev → staging → production - **GitHub Actions automation** for environment-based deployments - **Instant rollbacks** using the ArNS app interface - **Best practices** for TTL configuration and testing ## Understanding Undernames Undernames let you create multiple versions under one ArNS name - like subdomains in traditional DNS, but permanent and decentralized. **Structure:** ``` your-arns-name (your ArNS name) ├─ @ (base) → your-arns-name.ar.io ├─ dev → dev_your-arns-name.ar.io ├─ staging → staging_your-arns-name.ar.io └─ v2 → v2_your-arns-name.ar.io ``` **Key benefits:** - **Unlimited undernames** with one ArNS registration - **Each undername** points to a different deployment (transaction ID) - **All versions** remain permanently accessible - **Instant rollbacks** by updating pointers in the ArNS app --- ## Common Versioning Patterns ### Pattern 1: Environment-Based Versioning Best for: Most production applications This pattern creates separate environments for your development workflow: ``` @ (production) → Current production release staging → Pre-release testing environment dev → Active development builds ``` **Add environment scripts to your project:** In your `my-permaweb-app` project from the previous guide, update `package.json` to add environment-specific deployment commands: ```json title="package.json" { "scripts": { "dev": "next dev", "build": "next build", "deploy": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out", "deploy:dev": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --undername dev --ttl-seconds 60", "deploy:staging": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --undername staging --ttl-seconds 60", "deploy:prod": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --ttl-seconds 3600" } } ``` Replace `your-arns-name` with your actual ArNS name. ```json title="package.json" { "scripts": { "dev": "vite", "build": "vite build", "deploy": "vite build && ario-deploy deploy --arns-name your-arns-name", "deploy:dev": "vite build && ario-deploy deploy --arns-name your-arns-name --undername dev --ttl-seconds 60", "deploy:staging": "vite build && ario-deploy deploy --arns-name your-arns-name --undername staging --ttl-seconds 60", "deploy:prod": "vite build && ario-deploy deploy --arns-name your-arns-name --ttl-seconds 60" } } ``` Replace `your-arns-name` with your actual ArNS name. --- ### Pattern 2: Version Archives Best for: Applications needing long-term version history This pattern extends Pattern 1 by archiving each production release to a version-specific undername for permanent access: ``` @ (production) → v2.0 (current release) staging → v2.1 (next release testing) dev → Latest development v2-0-0 → Version 2.0.0 (archived) v2-1-0 → Version 2.1.0 (archived) ``` **Add archive script:** Create a script that automatically archives based on your `package.json` version: ```javascript title="scripts/deploy-archive.js" const { execSync } = require('child_process'); const packageJson = require('../package.json'); // Get version from package.json (e.g., "2.1.0") const version = packageJson.version.replace(/\./g, '-'); // "2-1-0" console.log(`Archiving version ${version}...`); // Deploy to version-specific undername (build already done by npm script) execSync( `ario-deploy deploy --arns-name your-arns-name --deploy-folder out --undername v${version} --ttl-seconds 3600`, { stdio: 'inherit' } ); console.log(`✓ Archived at: https://v${version}_your-arns-name.ar.io`); ``` Then add to `package.json`: ```json title="package.json" { "scripts": { "deploy:dev": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --undername dev --ttl-seconds 60", "deploy:staging": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --undername staging --ttl-seconds 60", "deploy:prod": "next build && ario-deploy deploy --arns-name your-arns-name --deploy-folder out --ttl-seconds 3600", "deploy:archive": "next build && node scripts/deploy-archive.js" } } ``` ```javascript title="scripts/deploy-archive.js" const { execSync } = require('child_process'); const packageJson = require('../package.json'); // Get version from package.json (e.g., "2.1.0") const version = packageJson.version.replace(/\./g, '-'); // "2-1-0" console.log(`Archiving version ${version}...`); // Deploy to version-specific undername (build already done by npm script) execSync( `ario-deploy deploy --arns-name your-arns-name --undername v${version} --ttl-seconds 3600`, { stdio: 'inherit' } ); console.log(`✓ Archived at: https://v${version}_your-arns-name.ar.io`); ``` Then add to `package.json`: ```json title="package.json" { "scripts": { "deploy:dev": "vite build && ario-deploy deploy --arns-name your-arns-name --undername dev --ttl-seconds 60", "deploy:staging": "vite build && ario-deploy deploy --arns-name your-arns-name --undername staging --ttl-seconds 60", "deploy:prod": "vite build && ario-deploy deploy --arns-name your-arns-name --ttl-seconds 3600", "deploy:archive": "vite build && node scripts/deploy-archive.js" } } ``` Now whenever you run `deploy:archive` a version-specific immutable deployment of your app will be created and hosted permanently. For example, version 2.1.0 would be accessible forever at `https://v2-1-0_your-arns-name.ar.io`. --- ### Pattern 3: Component Architecture Best for: Monorepos with multiple deployable parts This pattern deploys different parts of your application to separate undernames: ``` @ (marketing) → Marketing website app → Main application docs → Documentation site admin → Admin panel ``` **Example package.json:** ```json title="package.json" { "scripts": { "deploy:marketing": "ario-deploy deploy --arns-name your-arns-name --deploy-folder ./marketing/dist --ttl-seconds 3600", "deploy:app": "ario-deploy deploy --arns-name your-arns-name --undername app --deploy-folder ./app/dist --ttl-seconds 60", "deploy:docs": "ario-deploy deploy --arns-name your-arns-name --undername docs --deploy-folder ./docs/dist --ttl-seconds 1800", "deploy:all": "npm run deploy:marketing && npm run deploy:app && npm run deploy:docs" } } ``` Each component can be deployed independently or all at once with `npm run deploy:all`. --- ## Automating Environment Deployments Building on the [GitHub Actions workflow](/build/guides/hosting-decentralised-apps/deploying-with-ario-deploy#automating-with-github-actions) from the previous guide, let's create environment-specific workflows that deploy based on branch activity. Deploy automatically when pushing to the `develop` branch: ```yaml title=".github/workflows/deploy-dev.yml" name: Deploy to Dev on: push: branches: [develop] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '18' cache: 'npm' - name: Install dependencies run: npm ci - name: Deploy to Dev run: npm run deploy:dev env: DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }} ``` **Usage:** Push to `develop` branch → Auto-deploys to `dev_your-arns-name.ar.io` Deploy automatically when pushing to the `staging` branch: ```yaml title=".github/workflows/deploy-staging.yml" name: Deploy to Staging on: push: branches: [staging] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '18' cache: 'npm' - name: Install dependencies run: npm ci - name: Deploy to Staging run: npm run deploy:staging env: DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }} ``` **Usage:** Push to `staging` branch → Auto-deploys to `staging_your-arns-name.ar.io` Deploy to production and create version archive on push to `main`: ```yaml title=".github/workflows/deploy-production.yml" name: Deploy to Production on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest environment: production # Configure in GitHub Settings steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '18' cache: 'npm' - name: Install dependencies run: npm ci - name: Deploy to Production run: npm run deploy:prod env: DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }} - name: Archive Version run: npm run deploy:archive env: DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }} ``` **Usage:** Push to `main` branch → Auto-deploys to `your-arns-name.ar.io` and archives to `v{version}_your-arns-name.ar.io` Every production deployment automatically creates a permanent archive based on your `package.json` version. For example, version 2.1.0 is archived at `v2-1-0_your-arns-name.ar.io`. **Typical git workflow:** ```bash title="Terminal" # Feature development git checkout develop git add . git commit -m "Add new feature" git push origin develop # Auto-deploys to dev # Promote to staging git checkout staging git merge develop git push origin staging # Auto-deploys to staging # Promote to production git checkout main git merge staging git push origin main # Auto-deploys to production ``` --- ## Instant Rollbacks with ArNS App One of the key benefits of undernames is instant rollbacks. Since all versions are permanently stored on Arweave, you can instantly switch between them using the [ArNS app](https://arns.ar.io). **To rollback to a previous version:** Navigate to [arns.ar.io](https://arns.ar.io) and find your ArNS name. You'll see all undernames and their transaction IDs. Find the undername with the version you want to rollback to (e.g., `v2-0-0` or `staging`) and copy its transaction ID. Click on the `@` (base) record, paste the transaction ID, and save. Production now serves that version - instant rollback! You're just updating a pointer to existing data. The old version is already on Arweave, permanent and accessible. No re-upload needed! For more details on managing undernames, see [Managing ArNS via UI](/build/guides/working-with-arns/manage-arns-ui). --- ## Summary You now know how to manage multiple environments using ArNS undernames: - **Environment patterns** for dev → staging → production workflows - **Version archives** for permanent access to all releases - **GitHub Actions automation** for branch-based deployments - **Instant rollbacks** through the ArNS app interface In our next guides we'll explore other tools you can use to host websites on ar.io without needing to configure `ario-deploy`. # Guides (/build/guides) Explore real-world applications and use cases for **Arweave** and **ar.io** infrastructure. These examples show what's possible with permanent data storage and decentralized web services. ## What You Can Build **Arweave and ar.io enable:** - **Decentralized websites** - Host permanent, censorship-resistant web content - **ArNS domains** - Create and manage decentralized domain names - **Data marketplaces** - Trade and sell digital assets and data - **Permanent applications** - Deploy apps that can't be taken down - **And much more** - The permanent web is only limited by your imagination ## Getting Started **Store encrypted data** with blind computing access control **Key topics:** - Encrypted data storage on Arweave - Nillion blind computing integration - Privacy-preserving data processing - Decentralized access control **Build permanent websites and apps** that can't be censored or taken down **Key topics:** - Step-by-step deployment guides - Multiple deployment methods (CLI, web UI) - ArNS domain integration - Production versioning strategies **Create and manage** decentralized domain names **Key topics:** - Primary name registration - Domain management - Integration with applications **Version and organize** your permanent website content **Key topics:** - Undername management - Website versioning - Component organization **Trade and sell** ArNS tokens and digital assets **Key topics:** - Ar.io Name Token (ANT) trading - Marketplace dynamics - Asset ownership **Deploy dApps easily** using the ArDrive web interface **Key topics:** - ArDrive web deployment - Manifest creation - ArNS name assignment - Version management **Build a decentralized NFT minting app** with Arweave and Crossmint **Key topics:** - Permanent NFT storage on Arweave - Crossmint API integration - Payment processing - Decentralized deployment **Distribute software applications** using Arweave manifests and ArNS routing **Key topics:** - Multi-platform application distribution - CI/CD integration with GitHub Actions - Manifest-based routing patterns - Cost optimization with compression **Move large IPFS pin sets** to permanent Arweave storage **Key topics:** - Custom Turbo-based migration scripts - CID-to-transaction mapping and provenance tags - Batch migration at scale (100k+ CIDs) - NFT metadata rewrite patterns ## Why Use Arweave? **Permanent storage** - Data stored on Arweave is permanent and cannot be deleted **Decentralized** - No single point of failure or control **Cost-effective** - Pay once, store forever **Censorship-resistant** - Content cannot be taken down by authorities ## Next Steps } /> } /> } /> # Migrate from IPFS to Arweave (/build/guides/migrate-from-ipfs-to-arweave) If you have thousands, or hundreds of thousands, of IPFS pins and want **permanent, pay-once storage**, Arweave via ar.io is a strong fit. Unlike pinning services that require ongoing fees, data uploaded to Arweave is stored permanently with a single upfront payment. This guide walks through a **custom programmatic migration** using the [Turbo SDK](/sdks/turbo-sdk). It applies to any file set pinned on IPFS, with NFT collections as a concrete example. ## Why Use a Custom Migration? IPFS migrations are usually project-specific. Your CIDs may represent raw files, directories, NFT assets, metadata JSON, application bundles, or a mix of all of them. You may also need custom tagging, retry behavior, validation rules, metadata rewrites, or contract-specific URL formats. For that reason, a custom script is usually the most flexible approach. With modern LLM-assisted development, generating and adapting a migration script for your exact CID inventory, metadata schema, and validation requirements is often faster than forcing a generic tool to fit your project. Useful docs to combine for a custom migration: - [Advanced Uploading with Turbo](/build/upload/advanced-uploading-with-turbo): authentication, uploads, and payment - [Tagging](/build/upload/tagging): metadata and discoverability - [Manifests](/build/upload/manifests): organizing collections under path-based routing - [Receipts](/build/upload/receipts): audit trail for uploads - [Storing NFTs on ar.io](/build/guides/storing-nfts): NFT asset and metadata upload patterns - ArDrive CLI also supports IPFS CID tagging for some workflows, but most large migrations benefit from a custom Turbo SDK script so you can control batching, retries, metadata rewrites, and validation. This guide connects those pieces into a migration workflow. ## When to Use a Custom Migration | Scenario | Recommended approach | | --- | --- | | A few dozen files | Manual upload via [Turbo SDK](/sdks/turbo-sdk) or [console.ar.io](https://console.ar.io) | | Hundreds to millions of CIDs | Custom migration script (this guide) | | NFT collection with linked metadata | Custom script + metadata rewrite (see [NFT example](#nft-collection-example) below) | | Ongoing ingestion pipeline | Build a reusable worker that fetches from IPFS and uploads to Turbo | For full control over tagging, provenance, and reference rewriting, a custom script is usually the most flexible option. ## Migration Architecture At a high level, every migration follows the same pipeline: ```mermaid flowchart LR inventory["Inventory CIDs"] --> fetch["Fetch from IPFS gateway"] fetch --> upload["Upload via Turbo"] upload --> map["Persist CID to txId mapping"] map --> rewrite["Rewrite references"] rewrite --> index["Publish migration index"] index --> validate["Validate on ar.io"] ``` ### Inventory your CIDs Export your pin list from your pinning provider (Pinata, NFT.Storage, a self-hosted node, etc.) into a simple format: ```json [ { "cid": "bafybeig...", "path": "images/0.png", "contentType": "image/png" }, { "cid": "bafkreif...", "path": "metadata/0.json", "contentType": "application/json" } ] ``` Or use a newline-delimited file of CIDs if you have no path metadata: ``` bafybeig... bafkreif... ``` Deduplicate CIDs before starting. For file CIDs, the same CID should resolve to the same content bytes, so you only need to upload each unique file CID once. ### Fetch bytes from IPFS Retrieve each CID through a reliable IPFS gateway or your own node: ```javascript const IPFS_GATEWAY = 'https://ipfs.io/ipfs'; async function fetchFromIpfs(cid) { const response = await fetch(`${IPFS_GATEWAY}/${cid}`, { redirect: 'follow', }); if (!response.ok) { throw new Error(`Failed to fetch ${cid}: ${response.status}`); } const buffer = Buffer.from(await response.arrayBuffer()); const contentType = response.headers.get('content-type') ?? 'application/octet-stream'; return { buffer, contentType }; } ``` **Gateway reliability:** Public gateways can rate-limit or go offline. For large migrations, use your own IPFS node or a dedicated gateway from your pinning provider. Retry failed fetches with exponential backoff. ### Upload to Arweave via Turbo Authenticate with the Turbo SDK and upload each file. Tag every upload with provenance metadata so you can trace it back to its IPFS origin. ```javascript const jwk = JSON.parse(fs.readFileSync('./wallet.json', 'utf-8')); const turbo = TurboFactory.authenticated({ privateKey: jwk, token: 'arweave', }); async function uploadToArweave(cid, buffer, contentType, projectTag) { const result = await turbo.upload({ data: buffer, dataItemOpts: { tags: [ { name: 'Content-Type', value: contentType }, { name: 'App-Name', value: projectTag }, { name: 'Source-Protocol', value: 'ipfs' }, { name: 'Source-CID', value: cid }, ], }, }); return result.id; } ``` See [Tagging best practices](#tagging-for-provenance) below for recommended tags. ### Persist the CID-to-txId mapping Write results to an append-only state file after every upload. This is your migration ledger and lets you resume after failures. ```json { "cid": "bafybeig...", "txId": "Xj9k2Lm8Pq3Rn5Tv7Wz1Yb4Dc6Fg8Hj0Kl2Mn4Pq6Rs8", "contentType": "image/png", "bytes": 48291, "status": "success", "uploadedAt": "2026-05-27T12:00:00Z" } ``` Store this mapping durably, on disk at minimum, and ideally backed up. You will need it to rewrite references and to audit the migration. ### Rewrite references Update any documents, metadata, or application configs that point at `ipfs://` URIs to use Arweave references instead. For broad compatibility today, use gateway URLs in fields that external platforms must fetch immediately, and keep `ar://` references where your application, contract, or metadata consumers support them. The mapping file from the previous step drives this rewrite pass. ### Publish a migration index After the main upload pass, create a JSON index that links every original CID to its Arweave transaction ID. This gives your team and downstream users a durable lookup table for audits, support, and future migrations. ### Validate Fetch each uploaded transaction from an ar.io gateway or via [Wayfinder](/build/access/wayfinder) and compare it with what you fetched from IPFS. Retain [Turbo receipts](/build/upload/receipts) for audit purposes. ## Running Migrations at Scale For collections in the **tens or hundreds of thousands**, treat the migration as a long-running batch job. ### Cost estimation Before starting, estimate total upload cost: 1. Sum the byte size of all unique CIDs in your inventory. 2. Use the [pricing calculator](https://console.ar.io/calculator) or `turbo.getFiatEstimateForBytes()` from the SDK. 3. Purchase sufficient [Turbo Credits](/build/upload/turbo-credits) before beginning. Uploads under **100 KiB are free** and do not require a prior top-up. For large migrations this is negligible, but worth knowing for small metadata files. ### Concurrency and rate limits - Start with **low concurrency** (3–5 parallel uploads) and increase gradually while monitoring for errors. - IPFS gateways and Turbo both have rate limits. Separate fetch concurrency from upload concurrency. - Use exponential backoff on transient failures (HTTP 429, 502, network timeouts). ### Checkpointing and resume - Skip CIDs that already appear in your state file with `status: "success"`. - Write state **after each successful upload**, not in batches. If the process crashes, you lose at most one item. - Log failures separately so you can retry them in a second pass. ### Handling edge cases | Issue | What to do | | --- | --- | | Duplicate CIDs | Upload once, reuse the same txId in your mapping | | Missing/unavailable CID | Log as failed, retry later; do not block the entire run | | Unknown content type | Default to `application/octet-stream`; inspect bytes if needed | | Very large files | Stream via `turbo.uploadFile` with `fileStreamFactory` instead of buffering entirely in memory | | DagPB / directory CIDs | Resolve to individual file CIDs first; directory CIDs are not uploadable as a single blob | ## Tagging for Provenance Every migrated upload should include tags that make the data discoverable and traceable: | Tag | Purpose | | --- | --- | | `Content-Type` | Required: tells gateways how to serve the data | | `App-Name` | Identifies your project (e.g. `MyCollection-Migration-v1`) | | `Source-Protocol` | Set to `ipfs` to mark migrated content | | `Source-CID` | The original IPFS CID, useful for provenance and GraphQL queries | | `Migration-Date` | ISO timestamp of when the upload occurred | | `Collection-Name` | Optional: groups uploads from the same project | **Why tag the original CID?** Storing the source CID on each Arweave transaction lets you query your migrated data via [GraphQL](/build/access/find-data), correlate uploads back to IPFS origins, and support provenance checks without relying on an external mapping file alone. **IPFS-specific tags:** Some ArDrive CLI workflows support adding an `IPFS-Add` tag to public uploads, which may be useful where bridge-aware infrastructure recognizes that tag. For large custom migrations, the Turbo SDK pattern above is usually a better fit because you can choose your own tag names, preserve richer migration state, and adapt the script to your data model. See the full [Tagging guide](/build/upload/tagging) for tag size limits and best practices. ## Publish a Migration Index Your append-only state file is operational state. Once migration is complete, publish a clean migration index that others can use without reading your job logs. ```json { "type": "ipfs-to-arweave-migration", "version": "1.0.0", "project": "MyProject", "createdAt": "2026-05-27T12:00:00Z", "items": [ { "cid": "bafybeig...", "txId": "Xj9k2Lm8Pq3Rn5Tv7Wz1Yb4Dc6Fg8Hj0Kl2Mn4Pq6Rs8", "gatewayUrl": "https://arweave.net/Xj9k2Lm8Pq3Rn5Tv7Wz1Yb4Dc6Fg8Hj0Kl2Mn4Pq6Rs8", "arUri": "ar://Xj9k2Lm8Pq3Rn5Tv7Wz1Yb4Dc6Fg8Hj0Kl2Mn4Pq6Rs8", "contentType": "image/png", "bytes": 48291 } ] } ``` Upload the index with Turbo and tag it so it can be discovered later: ```javascript const migrationIndex = JSON.parse(fs.readFileSync('./migration-index.json', 'utf-8')); const indexUpload = await turbo.upload({ data: Buffer.from(JSON.stringify(migrationIndex)), dataItemOpts: { tags: [ { name: 'Content-Type', value: 'application/json' }, { name: 'App-Name', value: PROJECT_TAG }, { name: 'Data-Type', value: 'IPFS-Migration-Index' }, { name: 'Source-Protocol', value: 'ipfs' }, ], }, }); console.log(`Migration index: https://arweave.net/${indexUpload.id}`); console.log(`Migration index: ar://${indexUpload.id}`); ``` ## Complete Script Skeleton This skeleton ties the pipeline together. Extend it with your own retry logic, logging, and concurrency controls. **Memory usage:** This example buffers each fetched CID in memory before uploading. That keeps the script compact, but it is best for small or medium-sized files. For large media files, stream to disk first and upload with `turbo.uploadFile` so concurrent workers do not hold many large buffers in memory. ```javascript const IPFS_GATEWAY = process.env.IPFS_GATEWAY ?? 'https://ipfs.io/ipfs'; const STATE_FILE = './migration-state.jsonl'; const CONCURRENCY = 5; const PROJECT_TAG = 'MyProject-Migration-v1'; const jwk = JSON.parse(fs.readFileSync('./wallet.json', 'utf-8')); const turbo = TurboFactory.authenticated({ privateKey: jwk, token: 'arweave' }); const inventory = JSON.parse(fs.readFileSync('./cids.json', 'utf-8')); const completed = new Set( fs.existsSync(STATE_FILE) ? fs.readFileSync(STATE_FILE, 'utf-8') .trim() .split('\n') .filter(Boolean) .map((line) => JSON.parse(line)) .filter((r) => r.status === 'success') .map((r) => r.cid) : [], ); function appendState(record) { fs.appendFileSync(STATE_FILE, JSON.stringify(record) + '\n'); } async function fetchFromIpfs(cid) { const response = await fetch(`${IPFS_GATEWAY}/${cid}`, { redirect: 'follow' }); if (!response.ok) throw new Error(`Fetch failed: ${response.status}`); const buffer = Buffer.from(await response.arrayBuffer()); const contentType = response.headers.get('content-type') ?? 'application/octet-stream'; return { buffer, contentType }; } async function migrateOne({ cid, contentType: declaredType }) { if (completed.has(cid)) { console.log(`Skipping ${cid} (already migrated)`); return; } try { const { buffer, contentType } = await fetchFromIpfs(cid); const result = await turbo.upload({ data: buffer, dataItemOpts: { tags: [ { name: 'Content-Type', value: declaredType ?? contentType }, { name: 'App-Name', value: PROJECT_TAG }, { name: 'Source-Protocol', value: 'ipfs' }, { name: 'Source-CID', value: cid }, { name: 'Migration-Date', value: new Date().toISOString() }, ], }, }); appendState({ cid, txId: result.id, contentType: declaredType ?? contentType, bytes: buffer.length, status: 'success', uploadedAt: new Date().toISOString(), }); console.log(`Migrated ${cid} → ar://${result.id}`); } catch (error) { appendState({ cid, status: 'failed', error: error.message, failedAt: new Date().toISOString(), }); console.error(`Failed ${cid}:`, error.message); } } async function runPool(items, fn, concurrency) { const queue = [...items]; const workers = Array.from({ length: concurrency }, async () => { while (queue.length > 0) { const item = queue.shift(); if (item) await fn(item); } }); await Promise.all(workers); } await runPool(inventory, migrateOne, CONCURRENCY); console.log('Migration complete. Review migration-state.jsonl for results.'); ``` ## NFT Collection Example NFT collections are a common migration target because metadata JSON files typically reference images via `ipfs://` URIs. The workflow has an extra rewrite step. ### Separate assets from metadata Split your inventory into two groups: - **Assets**: images, animations, and other media files - **Metadata**: JSON files containing `name`, `description`, `image`, `attributes`, etc. Upload assets first so you have transaction IDs to reference in metadata. ### Upload assets and build the mapping Run the migration script on all asset CIDs. Your state file now maps each image CID to an Arweave transaction ID. ### Rewrite metadata references For each metadata JSON, replace `ipfs://` URIs with Arweave references using your mapping. Because `ar://` is not yet universally supported by NFT marketplaces and wallets, the most robust current approach is to use gateway URLs for widely consumed fields like `image`, while also keeping `ar://` values in additional fields for applications that support them. ```javascript const GATEWAY_URL = 'https://arweave.net'; function getArweaveReferences(uri, cidToTxId) { if (!uri.startsWith('ipfs://')) return uri; const cid = uri.replace('ipfs://', '').split('/')[0]; const txId = cidToTxId[cid]; if (!txId) { console.warn(`No mapping for CID: ${cid}`); return uri; } return { gatewayUrl: `${GATEWAY_URL}/${txId}`, arUri: `ar://${txId}`, }; } function rewriteMetadata(metadata, cidToTxId) { const rewritten = { ...metadata }; for (const field of ['image', 'animation_url', 'external_url']) { if (rewritten[field]) { const references = getArweaveReferences(rewritten[field], cidToTxId); if (typeof references === 'string') continue; rewritten[field] = references.gatewayUrl; rewritten[`${field}_ar`] = references.arUri; } } return rewritten; } ``` **Use both where practical.** Gateway URLs have the broadest compatibility today. `ar://` URIs are more future-proof and can resolve through [Wayfinder](/build/access/wayfinder), but adoption is still growing. Keeping both gives downstream consumers a stable HTTP URL now and a protocol-native reference for clients that support it. ### Upload rewritten metadata Upload each rewritten metadata JSON via Turbo. Tag with the same provenance tags plus the token identifier if applicable. For large collections (100+ tokens), consider uploading metadata as a [manifest](/build/upload/manifests) so your smart contract can use a single base URI. Use an HTTP gateway base URI when you need maximum marketplace compatibility, and keep the `ar://` manifest URI documented for clients that support it: ```solidity string private constant MANIFEST_ID = "your-manifest-transaction-id"; string private constant GATEWAY = "https://arweave.net/"; function tokenURI(uint256 tokenId) public view returns (string memory) { return string(abi.encodePacked(GATEWAY, MANIFEST_ID, "/", tokenId.toString(), ".json")); } ``` See [Storing NFTs on ar.io](/build/guides/storing-nfts) for the full manifest workflow. ### Update onchain references If NFTs are already minted with `ipfs://` token URIs, updating onchain metadata requires a contract-specific approach (owner update functions, redeployment, or a new base URI if your contract supports it). Plan this step before migrating. ## Validation After migration, verify a sample of uploads and ideally all failed items from your retry pass. A size check is a useful first pass; for stronger validation, compare hashes or the full buffers: ```javascript async function validateUpload(txId, expectedBuffer) { const response = await fetch(`https://turbo-gateway.com/${txId}`); if (!response.ok) throw new Error(`Gateway fetch failed: ${response.status}`); const actual = Buffer.from(await response.arrayBuffer()); if (actual.length !== expectedBuffer.length) { throw new Error(`Size mismatch: expected ${expectedBuffer.length}, got ${actual.length}`); } if (!actual.equals(expectedBuffer)) { throw new Error(`Content mismatch for ${txId}`); } console.log(`Validated ar://${txId} (${actual.length} bytes)`); } ``` For production workloads, [Wayfinder](/build/access/wayfinder) can add cryptographic verification for clients that support it. ## Next Steps } /> } /> } /> } /> } /> } /> # Storing NFTs on ar.io (/build/guides/storing-nfts) Storing NFT assets on ar.io ensures they're permanent, immutable and accessible, forever. We achieve this by providing: - **Permanent Storage**: your assets are stored forever on Arweave. - **One-Time Payment**: Pay once upfront for permanent storage. - **Decentralised Gateways**: a network of 100s of incentivized gateways provide access. - **Routing Protocol**: ar://wayfinder ensures an active gateway is selected for all requests. Once your NFT assets are stored on ar.io, no one can delete, edit or "rug" them. That means Creators can be confident their art will live on forever and Collectors can be confident their investment won't disappear. ## How to Store NFT assets on ar.io First, it's important to understand ar.io doesn't let you mint NFTs. What it does provide is permanent storage and access for the media files (images, video, animations) and metadata (artist, attributes, traits) JSON which is used in NFTs. To mint an NFT you'd typically store your assets and metadata via ar.io and then use another service to "mint" the NFTs onchain. This guide is targeted at developers. If you're not a developer we strongly recommend uploading to ar.io via [ArDrive](https://ardrive.io), check out this amazing video guide from [Hashlips](https://x.com/Hashlipsnft): ## Uploading Your First NFT Let's walk through uploading a complete NFT with its metadata using Turbo, ar.io's fast upload service. This guide assumes you're in a node.js environment and using an Arweave wallet, we support uploads in the browser and many other wallet types (EVM, SVM) and payment tokens (ETH, USDC, POL etc.). See [using turbo in the browser](/build/guides/using-turbo-in-a-browser) and [advanced uploading with with turbo](/build/upload/advanced-uploading-with-turbo) for more info. ### Prerequisites Before starting, ensure you have: - **Node.js** (v18 or higher) - **Arweave Wallet (JWK file)** - We recommend [Wander](https://www.wander.app/) if you need a wallet. - **Turbo Credits** - Purchase credits to pay for uploads. See [Turbo Credits guide](/build/upload/turbo-credits) - **Your NFT assets** - The image file(s) you want to upload - **Basic JavaScript knowledge** - Familiarity with Node.js and async/await Let's get started: ### Install and Authenticate Install the Turbo SDK and authenticate with your Arweave wallet: ```bash npm install @ardrive/turbo-sdk ``` Then authenticate and upload your NFT image: ```javascript try { // Authenticate with your Arweave wallet const jwk = JSON.parse(fs.readFileSync('./wallet.json', 'utf-8')); const turbo = TurboFactory.authenticated({ privateKey: jwk, token: 'arweave' }); // Upload the NFT image const imageUpload = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream('./my-nft.png'), fileSizeFactory: () => fs.statSync('./my-nft.png').size, dataItemOpts: { tags: [ { name: 'Content-Type', value: 'image/png' } ] } }); // Get the ar:// URL const imageUrl = `ar://${imageUpload.id}`; console.log('Image uploaded:', imageUrl); // Output: ar://Xj9k2Lm8Pq3Rn5Tv7Wz... } catch (error) { console.error('Upload failed:', error.message); // Check wallet balance, file exists, or network connection } ``` ### Create and Upload Metadata Now create your NFT metadata following the OpenSea/ERC-721 standard, using the `ar://` URL for the image. #### Required Fields ```json { "name": "NFT Name", "description": "Description of your NFT", "image": "ar://image-transaction-id" } ``` #### Full Example with Optional Fields ```json { "name": "Cosmic Explorer #42", "description": "A unique space explorer with permanent storage on Arweave", "image": "ar://Xj9k2Lm8Pq3Rn5Tv7Wz1Yb4Dc6Fg8Hj0Kl2Mn4Pq6Rs8", "animation_url": "ar://Zb1Cd3Ef5Gh7Ij9Kl1Mn3Op5Qr7St9Uv1Wx3Yz", "external_url": "ar://my-collection-website", "attributes": [ { "trait_type": "Class", "value": "Explorer" }, { "trait_type": "Power Level", "value": 9001, "display_type": "number" } ] } ``` **Critical**: All URL fields fetching data from ar.io (`image`, `animation_url`, `external_url`) should use the `ar://` protocol, not hardcoded gateway URLs. This is what makes your NFT future-proof. #### Upload Metadata with Turbo ```javascript // Create metadata with ar:// reference const metadata = { name: 'My Awesome NFT', description: 'Stored permanently on Arweave', image: imageUrl, // ar://transaction-id attributes: [ { trait_type: 'Background', value: 'Cosmic Blue' }, { trait_type: 'Rarity', value: 'Epic' } ] }; // Upload metadata const metadataUpload = await turbo.uploadFile({ fileStreamFactory: () => Buffer.from(JSON.stringify(metadata)), fileSizeFactory: () => Buffer.from(JSON.stringify(metadata)).length, dataItemOpts: { tags: [ { name: 'Content-Type', value: 'application/json' } ] } }); const metadataUrl = `ar://${metadataUpload.id}`; console.log('Metadata uploaded:', metadataUrl); console.log('Use this URL when minting your NFT!'); ``` ### Use in Your Minting Contract Use the metadata URL in your NFT contract's `tokenURI` field: ```solidity function tokenURI(uint256 tokenId) public view returns (string memory) { return "ar://your-metadata-transaction-id"; } ``` For a single NFT, all tokens can share the same metadata. For collections with unique metadata per token, see the manifest section below. **Note:** Uploads are typically available within seconds, but may take a few minutes to propagate across all ar.io gateways. ### Summary This example has shown you how to store your NFT assets and metadata on Arweave via ar.io and then add this to the tokenURI field of your NFT's smart contract. Next we'll explore how to store NFT images and metadata for entire collections. ## Organizing Collections with Manifests For NFT collections, manifests provide an efficient way to organize multiple files under a single transaction ID. ### What Are Manifests? A manifest acts like a folder on Arweave, mapping paths to transaction IDs. For NFT collections, manifests organize metadata access: - `ar://manifest-id/0.json` → Token 0 metadata - `ar://manifest-id/1.json` → Token 1 metadata - `ar://manifest-id/42.json` → Token 42 metadata Each metadata file contains fully qualified `ar://` URIs pointing to permanent image transaction IDs. This approach gives you: - **Organized metadata**: One manifest ID for all collection metadata - **Universal compatibility**: Images use standard `ar://txid` format that works everywhere - **Clean contract code**: Simple `baseURI + tokenId + .json` pattern ### When to Use Manifests - **Large collections** (100+ NFTs): Organize metadata files under a single manifest ID - **Clean contract code**: Use simple `manifest-id/tokenId.json` pattern instead of mapping each token to individual metadata TxIDs - **Future flexibility**: Update by pointing to new manifest without changing image references - **Simplified management**: Track one manifest ID for metadata access **How this workflow balances cost and compatibility:** Images are uploaded once with individual TxIDs (universally compatible), while metadata is organized in a manifest (easy contract integration). For single NFTs or very small collections (under 10), uploading metadata files individually may be simpler. ### Creating a Collection with Manifests ### Organize Your Files Create a folder structure with images and metadata templates: ``` my-collection/ ├── images/ │ ├── 0.png │ ├── 1.png │ ├── 2.png │ └── ... ├── metadata/ │ ├── 0.json │ ├── 1.json │ ├── 2.json │ └── ... ``` Start with metadata templates (image field will be populated in Step 3): ```json { "name": "My NFT #0", "description": "Part of my permanent NFT collection", "image": "", "attributes": [ { "trait_type": "Background", "value": "Blue" } ] } ``` ### Upload Images and Collect Transaction IDs Upload all images individually and collect their permanent transaction IDs: ```javascript // Authenticate const jwk = JSON.parse(fs.readFileSync('./wallet.json', 'utf-8')); const turbo = TurboFactory.authenticated({ privateKey: jwk, token: 'arweave' }); // Upload all images from images folder const imagesDir = './my-collection/images'; const imageFiles = fs.readdirSync(imagesDir).sort(); const imageTxIds = {}; for (const filename of imageFiles) { const filePath = path.join(imagesDir, filename); const upload = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream(filePath), fileSizeFactory: () => fs.statSync(filePath).size, dataItemOpts: { tags: [ { name: 'Content-Type', value: 'image/png' } ] } }); imageTxIds[filename] = upload.id; console.log(`Uploaded ${filename}: ar://${upload.id}`); } console.log('All images uploaded:', imageTxIds); ``` ### Update Metadata with Image Transaction IDs Update each metadata file to reference its permanent image transaction ID: ```javascript // Update metadata files with fully qualified image URIs const metadataDir = './my-collection/metadata'; const metadataFiles = fs.readdirSync(metadataDir).sort(); for (const filename of metadataFiles) { const filePath = path.join(metadataDir, filename); const metadata = JSON.parse(fs.readFileSync(filePath, 'utf-8')); // Map metadata to image: "0.json" -> "0.png" const imageFilename = filename.replace('.json', '.png'); const imageTxId = imageTxIds[imageFilename]; if (!imageTxId) { console.warn(`No image found for ${filename}`); continue; } // Set fully qualified ar:// URI metadata.image = `ar://${imageTxId}`; fs.writeFileSync(filePath, JSON.stringify(metadata, null, 2)); console.log(`Updated ${filename} with image: ${metadata.image}`); } ``` ### Upload Metadata Folder as Manifest Upload only the metadata folder to create a manifest for organized access: ```javascript // Upload metadata folder to create manifest const result = await turbo.uploadFolder({ folderPath: './my-collection/metadata' }); const manifestId = result.manifestId; console.log('Metadata manifest ID:', manifestId); console.log('Token 0 metadata: ar://' + manifestId + '/0.json'); console.log('Token 1 metadata: ar://' + manifestId + '/1.json'); ``` Each metadata file now contains fully qualified `ar://` image references that work with any NFT platform. ### Use in Your Minting Contract Use the metadata manifest ID for clean, organized token URIs: ```solidity string private constant MANIFEST_ID = "your-manifest-transaction-id"; function tokenURI(uint256 tokenId) public view returns (string memory) { return string(abi.encodePacked("ar://", MANIFEST_ID, "/", tokenId.toString(), ".json")); } ``` **How it resolves:** 1. Contract returns: `ar://manifest-id/42.json` 2. Platform fetches metadata containing: `"image": "ar://individual-image-txid"` 3. Platform fetches image from its permanent transaction ID ### Summary By following this workflow, you've created a permanent, organized NFT collection where: 1. **Images stored with permanent transaction IDs** that work universally 2. **Metadata contains fully qualified ar:// image URIs** compatible with all NFT platforms 3. **Metadata organized under one manifest ID** for clean contract integration 4. **Your smart contract uses one manifest ID** to access all token metadata Your NFTs are now stored permanently with guaranteed access, following industry standards, and future-proofed against gateway dependencies. ## Next Steps } /> } /> } /> # Using Turbo SDK with Vanilla HTML (/build/guides/using-turbo-in-a-browser/html) # Using Turbo SDK with Vanilla HTML **Firefox Compatibility**: Some compatibility issues have been reported with the Turbo SDK in Firefox browsers. At this time the below framework examples may not behave as expected in Firefox. ## Overview This guide demonstrates how to integrate the `@ardrive/turbo-sdk` directly into vanilla HTML pages using CDN imports. No build tools, bundlers, or polyfills are required - just modern ES modules support in browsers. **Note**: Vanilla HTML implementation is the simplest way to get started with the Turbo SDK. It's perfect for prototyping, simple applications, or when you want to avoid build complexity. ## Prerequisites - Modern browser with ES modules support (Chrome 61+, Firefox 60+, Safari 10.1+, Edge 16+) - Basic understanding of HTML, CSS, and JavaScript - HTTPS hosting for production (required for browser wallet integrations) Create a basic HTML file with Turbo SDK integration: ```html Turbo SDK Example body { font-family: Arial, sans-serif; max-width: 800px; margin: 0 auto; padding: 20px; } .section { margin: 20px 0; padding: 20px; border: 1px solid #ddd; border-radius: 8px; } .loading { color: #666; font-style: italic; } .error { color: red; } .success { color: green; } button { background: #007cba; color: white; border: none; padding: 10px 20px; border-radius: 4px; cursor: pointer; margin: 5px; } button:hover { background: #005a87; } button:disabled { background: #ccc; cursor: not-allowed; } Turbo SDK - Vanilla HTML Demo Current Rates Loading rates... Upload File Upload File // Initialize Turbo client const turbo = TurboFactory.unauthenticated(); // Fetch and display rates async function loadRates() { try { const rates = await turbo.getFiatRates(); const ratesDiv = document.getElementById("rates"); const ratesText = Object.entries(rates.fiat) .map( ([currency, rate]) => `${currency.toUpperCase()}: $${rate} per GiB` ) .join(""); ratesDiv.innerHTML = ratesText; } catch (error) { document.getElementById( "rates" ).innerHTML = `Error loading rates: ${error.message}`; } } // Handle file upload document .getElementById("uploadForm") .addEventListener("submit", async (e) => { e.preventDefault(); const fileInput = document.getElementById("fileInput"); const uploadBtn = document.getElementById("uploadBtn"); const statusDiv = document.getElementById("uploadStatus"); if (!fileInput.files.length) { statusDiv.innerHTML = 'Please select a file'; return; } const file = fileInput.files[0]; uploadBtn.disabled = true; statusDiv.innerHTML = 'Preparing upload...'; try { // Show upload cost first const costs = await turbo.getUploadCosts({ bytes: [file.size] }); const cost = costs[0]; statusDiv.innerHTML = ` Upload cost: ${cost.winc} winc File size: ${file.size.toLocaleString()} bytes Note: This example cannot complete uploads without wallet authentication. See wallet integration examples below for full upload functionality. `; } catch (error) { statusDiv.innerHTML = `Error: ${error.message}`; } finally { uploadBtn.disabled = false; } }); // Load rates on page load loadRates(); ``` Select the appropriate CDN import method for your needs: **Use esm.sh for best compatibility**: The `unpkg.com` CDN has known issues with ES module exports for complex packages like Turbo SDK. **Latest Version (Recommended for Development)** ```javascript ``` **Specific Version (Recommended for Production)** ```javascript ``` **Alternative CDN Providers** ```javascript // jsDelivr // SkyPack // unpkg.com (not recommended - has ES module issues) ``` Connect your browser wallet to enable file uploads: **Never expose private keys in browser applications!** Always use browser wallet integrations. **Uploading with Wander** **Deprecation Notice**: The signature API used by ArConnect wallets is deprecated and will be removed. Visit [Wander wallet documentation](https://docs.wander.app/api/signature) for alternatives. Complete HTML page with Wander wallet integration: ```html Turbo SDK - Wander Wallet body { font-family: Arial, sans-serif; max-width: 600px; margin: 0 auto; padding: 20px; background: #f5f5f5; } .container { background: white; padding: 30px; border-radius: 10px; box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1); } .wallet-section { border: 2px solid #e0e0e0; border-radius: 8px; padding: 20px; margin: 20px 0; } .connected { border-color: #4caf50; background-color: #f9fff9; } button { background: #000; color: white; border: none; padding: 12px 24px; border-radius: 6px; cursor: pointer; font-size: 16px; margin: 5px; } button:hover { background: #333; } button:disabled { background: #ccc; cursor: not-allowed; } .status { margin: 10px 0; padding: 10px; border-radius: 4px; } .success { background: #d4edda; color: #155724; border: 1px solid #c3e6cb; } .error { background: #f8d7da; color: #721c24; border: 1px solid #f5c6cb; } .info { background: #d1ecf1; color: #0c5460; border: 1px solid #bee5eb; } ⚡ Turbo SDK + Wander Wallet Wander Wallet Connection Connect your Wander wallet to upload files to Arweave using your AR balance. Connect Wander Wallet 📁 File Upload Upload to Arweave import { TurboFactory, ArconnectSigner, } from "https://esm.sh/@ardrive/turbo-sdk"; let connectedAddress = null; let turboClient = null; // Connect to Wander wallet async function connectWanderWallet() { const statusDiv = document.getElementById("walletStatus"); const connectBtn = document.getElementById("connectBtn"); try { if (!window.arweaveWallet) { statusDiv.innerHTML = ` Wander wallet is not installed! Install Wander Wallet `; return; } connectBtn.disabled = true; statusDiv.innerHTML = 'Connecting to Wander wallet...'; // Required permissions for Turbo SDK const permissions = [ "ACCESS_ADDRESS", "ACCESS_PUBLIC_KEY", "SIGN_TRANSACTION", "SIGNATURE", ]; // Connect to wallet await window.arweaveWallet.connect(permissions); // Get wallet address connectedAddress = await window.arweaveWallet.getActiveAddress(); // Create authenticated Turbo client const signer = new ArconnectSigner(window.arweaveWallet); turboClient = TurboFactory.authenticated({ signer }); // Update UI document.getElementById("walletSection").classList.add("connected"); statusDiv.innerHTML = ` ✅ Connected to Wander Wallet Address: ${connectedAddress.slice( 0, 8 )}...${connectedAddress.slice(-8)} `; connectBtn.style.display = "none"; document.getElementById("uploadSection").style.display = "block"; } catch (error) { console.error("Wander wallet connection failed:", error); statusDiv.innerHTML = `Connection failed: ${error.message}`; } finally { connectBtn.disabled = false; } } // Upload file function async function uploadFile() { const fileInput = document.getElementById("fileInput"); const statusDiv = document.getElementById("uploadStatus"); if (!fileInput.files.length) { statusDiv.innerHTML = 'Please select a file first'; return; } if (!turboClient) { statusDiv.innerHTML = 'Please connect Wander wallet first'; return; } const file = fileInput.files[0]; let uploadStartTime = Date.now(); statusDiv.innerHTML = 'Preparing upload...'; try { // Get upload cost first const costs = await turboClient.getUploadCosts({ bytes: [file.size], }); const cost = costs[0]; statusDiv.innerHTML = ` Upload cost: ${cost.winc} winc Starting upload... `; // Upload with comprehensive progress tracking const result = await turboClient.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream", }, { name: "App-Name", value: "Turbo-HTML-Wander-Demo" }, { name: "File-Name", value: file.name }, { name: "Upload-Timestamp", value: new Date().toISOString() }, ], }, events: { onProgress: ({ totalBytes, processedBytes, step }) => { const percent = Math.round((processedBytes / totalBytes) * 100); const elapsed = Math.round( (Date.now() - uploadStartTime) / 1000 ); statusDiv.innerHTML = ` ${step}: ${percent}% Progress: ${processedBytes.toLocaleString()} / ${totalBytes.toLocaleString()} bytes Elapsed: ${elapsed}s `; }, onError: ({ error, step }) => { console.error(`Error during ${step}:`, error); statusDiv.innerHTML = `Error during ${step}: ${error.message}`; }, }, }); const totalTime = Math.round((Date.now() - uploadStartTime) / 1000); // Use original file size for display (result object doesn't contain size info) const displayBytes = file.size; statusDiv.innerHTML = ` 🎉 Upload Successful! Transaction ID: ${ result.id } File Size: ${displayBytes.toLocaleString()} bytes Upload Time: ${totalTime}s Timestamp: ${new Date( result.timestamp ).toLocaleString()} View File: turbo-gateway.com/${result.id} Explorer: ViewBlock `; } catch (error) { console.error("Upload failed:", error); statusDiv.innerHTML = `Upload failed: ${error.message}`; } } // Make functions available globally for onclick handlers window.connectWanderWallet = connectWanderWallet; window.uploadFile = uploadFile; ``` Browser wallet integrations require HTTPS in production: ```html ``` Configure CSP headers to allow CDN imports: ```html ``` Implement comprehensive error handling: ```javascript // Network error handling async function robustApiCall(apiFunction, retries = 3) { for (let i = 0; i setTimeout(resolve, 1000 * Math.pow(2, i)) ); } } } // Usage example const rates = await robustApiCall(() => turbo.getFiatRates()); ``` Optimize for production environments: ```html // Production code here ``` ## Best Practices ### 1. User Experience - **Loading States**: Always show loading indicators during API calls - **Error Recovery**: Provide clear error messages with recovery options - **Progress Tracking**: Show upload progress for large files - **Wallet Detection**: Guide users to install wallets if missing ### 2. Security - **Never expose private keys** in browser applications - **Validate user inputs** before API calls - **Use HTTPS** for all production deployments - **Implement CSP headers** to prevent XSS attacks ### 3. Performance - **Cache API responses** where appropriate (rates, costs) - **Use specific CDN versions** in production - **Implement retry logic** for network failures - **Optimize file handling** for large uploads ### 4. Development - **Use development endpoints** during testing - **Test wallet integrations** across different browsers - **Validate upload functionality** with small files first - **Monitor API rate limits** and implement backoff ## Troubleshooting Common Issues ### CDN Import Errors If you encounter errors like: - `The requested module does not provide an - Module resolution failures **Solution**: Use `esm.sh` instead of `unpkg.com`: ```javascript // ❌ Problematic // ✅ Working ``` ### Function Scope Issues If onclick handlers throw `ReferenceError: function is not defined`: **Solution**: Use explicit global assignment: ```javascript // ❌ Problematic window.myFunction = async function() { ... } // ✅ Working async function myFunction() { ... } window.myFunction = myFunction; ``` ### Upload Result Properties If upload results have undefined properties: **Solution**: Use original file size for display: ```javascript // ❌ Problematic - these properties don't exist in result object const totalBytes = result.totalBytes || result.dataSizeBytes; // ✅ Correct - use original file size const displayBytes = originalFile.size; // Available result properties: id, timestamp, winc, version, // deadlineHeight, dataCaches, fastFinalityIndexes, public, signature, owner ``` ### Upload Cost Properties If cost calculations fail: **Solution**: Use correct cost object structure: ```javascript // ❌ Problematic - adjustedBytes doesn't exist in cost objects const cost = costs[0]; console.log(`Adjusted: ${cost.adjustedBytes.toLocaleString()}`); // ✅ Correct - use available properties const cost = costs[0]; console.log(`Cost: ${cost.winc} winc`); console.log(`File size: ${originalFile.size.toLocaleString()} bytes`); // Available cost properties: winc (string), adjustments (array) ``` ## Testing Your Implementation ### 1. Basic Functionality Test ```javascript // Test CDN import console.log("Testing Turbo SDK import..."); const turbo = TurboFactory.unauthenticated(); console.log("✅ SDK imported successfully"); // Test rate fetching const rates = await turbo.getFiatRates(); console.log("✅ Rates fetched:", rates); ``` ### 2. Wallet Integration Test - Connect to MetaMask/Wander wallet - Verify address display - Test small file upload ( { // Actual structure: { processedBytes: 4326, // number - bytes processed so far totalBytes: 8652, // number - total bytes to process step: "signing" // string - current step: "signing" or "upload" } } } ``` ## Additional Resources - [Turbo SDK Documentation](https://docs.ardrive.io) - [Browser Wallet Security Guide](https://docs.wander.app) - [Arweave Developer Documentation](https://docs.arweave.org) - [CDN Import Best Practices](https://esm.sh) --- For more advanced implementations, see the [Next.js](./nextjs.mdx) and [Vite](./vite.mdx) framework guides, or explore the [Turbo SDK examples](https://github.com/ardriveio/turbo-sdk) repository. # Using Turbo in a Browser (/build/guides/using-turbo-in-a-browser) Integrate the **Turbo SDK** directly into your web applications for fast, reliable data uploads to Arweave. Choose the approach that best fits your development workflow and framework preferences. ## What You Can Build **With Turbo SDK in browsers, you can:** - **Upload files directly** from web applications to Arweave - **Pay with different tokens** (AR, Ethereum, and more) - **Integrate with popular wallets** (MetaMask, Wander, ArConnect) - **Build permanent web apps** that store data on Arweave - **Create data marketplaces** and decentralized applications ## Getting Started **Start with the simplest approach** - no build tools required **Key topics:** - CDN imports for instant setup - Wallet integration examples - Production deployment considerations - Error handling and troubleshooting **Full-stack React applications** with server-side rendering **Key topics:** - Webpack polyfill configuration - Client-side component setup - TypeScript integration - Production optimization **Fast development** with modern build tools **Key topics:** - Vite plugin configuration - React and TypeScript setup - Hot module replacement - Bundle optimization ## Why Use Turbo SDK? - **Fast uploads** - Upload data to Arweave in seconds, not minutes - **Multiple payment options** - Pay with AR, Ethereum, or other supported tokens - **Wallet integration** - Seamlessly connect with popular browser wallets - **Reliable infrastructure** - Built on Arweave's permanent storage network - **Developer-friendly** - Simple APIs with comprehensive documentation ## Next Steps } /> } /> } /> # Using Turbo SDK with Next.js (/build/guides/using-turbo-in-a-browser/nextjs) # Using Turbo SDK with Next.js **Firefox Compatibility**: Some compatibility issues have been reported with the Turbo SDK in Firefox browsers. At this time the below framework examples may not behave as expected in Firefox. ## Overview This guide demonstrates how to configure the `@ardrive/turbo-sdk` in a Next.js application with proper polyfills for client-side usage. Next.js uses webpack under the hood, which requires specific configuration to handle Node.js modules that the Turbo SDK depends on. **Polyfills**: Polyfills are required when using the Turbo SDK in Next.js applications. The SDK relies on Node.js modules like `crypto`, `buffer`, `process`, and `stream` that are not available in the browser by default. ## Prerequisites - Next.js 13+ (with App Router recommended) - Next.js 15 users: See updated configuration examples throughout this guide - Node.js 18+ - Basic familiarity with Next.js configuration and webpack Install the main Turbo SDK package: ```bash npm install @ardrive/turbo-sdk ``` Add required polyfill packages for browser compatibility: ```bash npm install --save-dev crypto-browserify stream-browserify process buffer ``` **Wallet Integration Dependencies**: The Turbo SDK includes `@dha-team/arbundles` as a peer dependency, which provides the necessary signers for browser wallet integration (like `InjectedEthereumSigner` and `ArconnectSigner`). You can import these directly without additional installation. **Next.js 15+**: Use TypeScript config format (`next.config.ts`) with ESM exports. For earlier versions, use `next.config.js` with `module.exports`. Create or update your `next.config.ts` file to include the necessary polyfills: ```typescript const nextConfig: NextConfig = { webpack: (config, { isServer, webpack }) => { // Only configure polyfills for client-side bundles if (!isServer) { config.resolve.fallback = { ...config.resolve.fallback, crypto: require.resolve("crypto-browserify"), stream: require.resolve("stream-browserify"), buffer: require.resolve("buffer"), process: require.resolve("process/browser"), fs: false, net: false, tls: false, }; // Provide global process and Buffer config.plugins.push( new webpack.ProvidePlugin({ process: "process/browser", Buffer: ["buffer", "Buffer"], }) ); // CRITICAL: Handle node: protocol imports (e.g., node:stream, node:crypto) // The Turbo SDK uses modern Node.js imports that webpack doesn't understand config.plugins.push( new webpack.NormalModuleReplacementPlugin( /^node:/, (resource: any) => { const module = resource.request.replace(/^node:/, ""); const fallbackMap: Record = { crypto: "crypto-browserify", stream: "stream-browserify", buffer: "buffer", process: "process/browser", }; resource.request = fallbackMap[module] || module; } ) ); } return config; }, }; ``` **Why `NormalModuleReplacementPlugin` is required**: The Turbo SDK uses `node:stream`, `node:crypto`, etc. imports. Without this plugin, you'll get `UnhandledSchemeError: Reading from "node:stream"` build errors. **Required for SDK Compatibility**: The Turbo SDK requires ES2017 features. Set your TypeScript `target` to `ES2017` or higher. Update your `tsconfig.json`: ```json { "compilerOptions": { "target": "ES2017", "lib": ["es2017", "dom", "dom.iterable", "esnext"], "module": "esnext", "moduleResolution": "bundler", "jsx": "preserve", "paths": { "@/*": ["./src/*"] } // ... other options } } ``` **TypeScript Wallet Types** Create a `types/wallet.d.ts` file to properly type wallet objects: ```typescript // types/wallet.d.ts interface Window { ethereum?: { request: (args: { method: string; params?: any[] }) => Promise; on?: (event: string, handler: (...args: any[]) => void) => void; removeListener?: (event: string, handler: (...args: any[]) => void) => void; isMetaMask?: boolean; }; arweaveWallet?: { connect: (permissions: string[]) => Promise; disconnect: () => Promise; getActiveAddress: () => Promise; getPermissions: () => Promise; sign: (transaction: any) => Promise; getPublicKey: () => Promise; }; } ``` Select between **client-side wallet** or **server-side signing** integration based on your use case: ### When to Use Client-Side Wallets (Steps 6-7) Use MetaMask or Wander wallet integration when: - Users control their own wallets and pay for their uploads - Building a decentralized app (dApp) where users own their data - Users need to sign transactions with their private keys - You want users to manage their own Turbo credits ### When to Use Server-Side Signing (Step 8) Use server-side signing when: - **Your application pays for uploads** (most common for Next.js apps) - Background jobs or automated uploads - API-driven uploads without user wallets - You need centralized control over signing - Building traditional web apps with server-side authentication **Most Next.js apps use server-side signing**. If you're building a typical web application where your backend handles uploads, skip to Step 8. **Never expose private keys in browser applications!** Client-side patterns use browser wallets (no private keys in code). Server-side patterns keep private keys secure on the server. Create a React component for MetaMask wallet integration: For MetaMask integration, you'll need to use `InjectedEthereumSigner` from `@dha-team/arbundles`, which is available as a peer dependency through the Turbo SDK. ```tsx "use client"; const [connected, setConnected] = useState(false); const [address, setAddress] = useState(""); const [uploading, setUploading] = useState(false); const [uploadResult, setUploadResult] = useState(null); const connectMetaMask = useCallback(async () => { try { if (!window.ethereum) { alert("MetaMask is not installed!"); return; } // Request account access await window.ethereum.request({ method: "eth_requestAccounts", }); // Get the current account const accounts = await window.ethereum.request({ method: "eth_accounts", }); if (accounts.length > 0) { setAddress(accounts[0]); setConnected(true); // Log current chain for debugging const chainId = await window.ethereum.request({ method: "eth_chainId", }); console.log("Connected to chain:", chainId); } } catch (error) { console.error("Failed to connect to MetaMask:", error); } }, []); const uploadWithMetaMask = async (event) => { const file = event.target.files?.[0]; if (!file || !connected) return; setUploading(true); try { // Create a provider wrapper for InjectedEthereumSigner const providerWrapper = { getSigner: () => ({ signMessage: async (message: string | Uint8Array) => { const accounts = await window.ethereum!.request({ method: "eth_accounts", }); if (accounts.length === 0) { throw new Error("No accounts available"); } // Convert message to hex if it's Uint8Array const messageToSign = typeof message === "string" ? message : "0x" + Array.from(message) .map((b) => b.toString(16).padStart(2, "0")) .join(""); return await window.ethereum!.request({ method: "personal_sign", params: [messageToSign, accounts[0]], }); }, }), }; // Create the signer using InjectedEthereumSigner const signer = new InjectedEthereumSigner(providerWrapper); const turbo = TurboFactory.authenticated({ signer, token: "ethereum", // Important: specify token type for Ethereum }); // Upload file with progress tracking const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type }, { name: "App-Name", value: "My-Next-App" }, { name: "Funded-By", value: "Ethereum" }, ], }, events: { onProgress: ({ totalBytes, processedBytes, step }) => { console.log( `${step}: ${Math.round((processedBytes / totalBytes) * 100)}%` ); }, onError: ({ error, step }) => { console.error(`Error during ${step}:`, error); console.error("Error details:", JSON.stringify(error, null, 2)); }, }, }); setUploadResult(result); } catch (error) { console.error("Upload failed:", error); console.error("Error details:", JSON.stringify(error, null, 2)); alert(`Upload failed: ${error.message}`); } finally { setUploading(false); } }; return ( MetaMask Upload {!connected ? ( Connect MetaMask ) : ( ✅ Connected: {address.slice(0, 6)}...{address.slice(-4)} Select File to Upload: {uploading && ( 🔄 Uploading... Please confirm transaction in MetaMask )} {uploadResult && ( ✅ Upload Successful! Transaction ID: {uploadResult.id} Data Size: {uploadResult.totalBytes} bytes )} )} ); } ``` Create a React component for Wander wallet integration: ```tsx "use client"; const [connected, setConnected] = useState(false); const [address, setAddress] = useState(""); const [uploading, setUploading] = useState(false); const [uploadResult, setUploadResult] = useState(null); const connectWanderWallet = useCallback(async () => { try { if (!window.arweaveWallet) { alert("Wander wallet is not installed!"); return; } // Required permissions for Turbo SDK const permissions = [ "ACCESS_ADDRESS", "ACCESS_PUBLIC_KEY", "SIGN_TRANSACTION", "SIGNATURE", ]; // Connect to wallet await window.arweaveWallet.connect(permissions); // Get wallet address const walletAddress = await window.arweaveWallet.getActiveAddress(); setAddress(walletAddress); setConnected(true); } catch (error) { console.error("Failed to connect to Wander wallet:", error); } }, []); const uploadWithWanderWallet = async (event) => { const file = event.target.files?.[0]; if (!file || !connected) return; setUploading(true); try { // Create ArConnect signer using Wander wallet const signer = new ArconnectSigner(window.arweaveWallet); const turbo = TurboFactory.authenticated({ signer }); // Note: No need to specify token for Arweave as it's the default // Upload file with progress tracking const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type }, { name: "App-Name", value: "My-Next-App" }, { name: "Funded-By", value: "Arweave" }, ], }, events: { onProgress: ({ totalBytes, processedBytes, step }) => { console.log( `${step}: ${Math.round((processedBytes / totalBytes) * 100)}%` ); }, onError: ({ error, step }) => { console.error(`Error during ${step}:`, error); }, }, }); setUploadResult(result); } catch (error) { console.error("Upload failed:", error); alert(`Upload failed: ${error.message}`); } finally { setUploading(false); } }; return ( Wander Wallet Upload {!connected ? ( Connect Wander Wallet ) : ( ✅ Connected: {address.slice(0, 6)}...{address.slice(-4)} Select File to Upload: {uploading && ( 🔄 Uploading... Please confirm transaction in Wander wallet )} {uploadResult && ( ✅ Upload Successful! Transaction ID: {uploadResult.id} Data Size: {uploadResult.totalBytes} bytes )} )} ); } ``` **Recommended for most Next.js applications**. This pattern keeps your private keys secure on the server while allowing uploads from the client. ### Architecture - **Client**: Uses `walletAdapter` with Turbo SDK, delegates signing to API route - **Server**: API route signs data with `ethers.js` Wallet, returns signature - **Security**: Private keys stay server-side, never exposed to browser ### Installation Install ethers.js (if not already installed): ```bash npm install ethers ``` ### Environment Setup #### Getting a Private Key You'll need an Ethereum-compatible private key for server-side signing: **For Testing/Development:** - Generate a new wallet with any Ethereum wallet (MetaMask, Rainbow, etc.) - Export the private key (usually in wallet settings under "Show private keys") - Copy the private key (starts with `0x`) **For Production:** - Use a dedicated wallet for your application - Fund it with the appropriate token (USDC on Base, ETH, etc.) - Never use a personal wallet with significant funds **Quick Test Key Generation (Node.js):** ```bash node -e "console.log(require('crypto').randomBytes(32).toString('hex'))" # Add 0x prefix: 0x[generated_hex] ``` Create a `.env.local` file in your project root: ```bash # .env.local (NEVER commit this file to git!) ETHEREUM_PRIVATE_KEY=0xYourPrivateKeyHere ``` **Security**: Add `.env.local` to your `.gitignore` file. Never commit private keys to version control. **Funding Your Wallet**: After setting up your private key, you'll need to fund the corresponding wallet address with Turbo credits. Visit [https://turbo.ardrive.io](https://turbo.ardrive.io) to add credits to your wallet. ### Token Selection Choose your token type based on the blockchain you're using: - `"base-usdc"` - USDC on Base Network (recommended for low fees) - `"ethereum"` - ETH on Ethereum Mainnet - `"polygon-usdc"` - USDC on Polygon Network - `"matic"` or `"pol"` - MATIC/POL on Polygon Network - `"solana"` - SOL on Solana Your private key must match the blockchain you select. For example, if using `"base-usdc"`, use an Ethereum-compatible private key. ### Server API Route Create `app/api/sign/route.ts`: ```typescript import "server-only"; // Validate that private key exists in environment variables if (!process.env.ETHEREUM_PRIVATE_KEY) { throw new Error( "ETHEREUM_PRIVATE_KEY is not set. Please add it to your .env.local file." ); } const wallet = new Wallet(process.env.ETHEREUM_PRIVATE_KEY); async function signData(data: string | Uint8Array): Promise { // ethers.js handles both UTF-8 strings and binary data return await wallet.signMessage(data); } function getPublicKey(): string { // Return uncompressed public key (keep '04' prefix, remove only '0x') return wallet.signingKey.publicKey.slice(2); } return NextResponse.json({ publicKey: getPublicKey() }); } const body = await req.json() as { signatureData: string; isHex?: boolean; }; if (!body?.signatureData) { return NextResponse.json( { error: "Missing signatureData" }, { status: 400 } ); } // Handle both string messages (UTF-8) and binary data (hex-encoded) const dataToSign = body.isHex ? Buffer.from(body.signatureData, "hex") : body.signatureData; const signature = await signData(dataToSign); return NextResponse.json({ signature }); } ``` **Next.js 15 Requirement**: Route handlers can ONLY ### Client Component Create `app/upload/component.tsx`: ```typescript "use client"; // Buffer is available globally from webpack polyfills configured in next.config.ts // No import needed - it's provided by the ProvidePlugin // API endpoint for server-side signing (corresponds to app/api/sign/route.ts) const signEndpoint = "/api/sign"; const [uploading, setUploading] = useState(false); // Create turbo instance with useMemo to avoid recreating on every render // This prevents SSR issues and unnecessary re-initialization const turbo = useMemo(() => { return TurboFactory.authenticated({ token: "base-usdc", // Match the token type from your environment walletAdapter: { getSigner: () => { return { signMessage: async (_message: string | Uint8Array) => { // Determine if message is binary (Uint8Array) or string const isHex = typeof _message !== "string"; const message = isHex ? Buffer.from(_message).toString("hex") : _message; const res = await fetch(signEndpoint, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ signatureData: message, isHex }), }); if (!res.ok) { throw new Error(`Signing failed: ${res.statusText}`); } const { signature } = await res.json(); return signature; }, sendTransaction: async () => { throw new Error("Not implemented"); }, provider: { // Fetch public key from server getPublicKey: async () => { const res = await fetch(signEndpoint); const { publicKey } = await res.json(); return publicKey; }, getSigner: () => ({ // Return address (same as public key for this implementation) getAddress: async () => { const res = await fetch(signEndpoint); const { publicKey } = await res.json(); return publicKey; }, // Sign typed data (used for certain Turbo operations) _signTypedData: async ( _domain: never, _types: never, message: { address: string; "Transaction hash": Uint8Array; } ) => { const convertedMsg = Buffer.from( message["Transaction hash"] ).toString("hex"); const res = await fetch(signEndpoint, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ signatureData: convertedMsg, isHex: true }), }); if (!res.ok) { throw new Error(`Signing failed: ${res.statusText}`); } const { signature } = await res.json(); return signature; }, }), _ready: () => {}, }, } as unknown as EthereumWalletSigner; }, }, }); }, []); // Upload string data const handleUploadText = async () => { setUploading(true); try { const result = await turbo.upload({ data: "Hello, ArDrive Turbo!", }); console.log("Upload successful! ID:", result.id); alert(`Upload successful! ID: ${result.id}`); } catch (error) { console.error("Upload failed:", error); alert(`Upload failed: ${(error as Error).message}`); } finally { setUploading(false); } }; // Upload file const handleFileUpload = async (e: React.ChangeEvent) => { const file = e.target.files?.[0]; if (!file) return; setUploading(true); try { const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type }, { name: "File-Name", value: file.name }, ], }, }); console.log("File upload successful! ID:", result.id); alert(`File uploaded! ID: ${result.id}`); } catch (error) { console.error("Upload failed:", error); alert(`Upload failed: ${(error as Error).message}`); } finally { setUploading(false); } }; return ( {uploading ? "Uploading..." : "Upload Text"} ); } ``` **Styling**: This is a basic example. Add your own styles, labels, and accessibility features as needed for your application. ### Key Implementation Details 1. **`useMemo` for Turbo Instance**: Creating the turbo instance inside `useMemo` prevents it from being recreated on every render and avoids SSR issues. The turbo client is initialized once when the component mounts. 2. **Provider Implementation**: The `provider` object is required for Ethereum wallets and includes: - `getPublicKey()` - Fetches public key from `GET /api/sign` - `getSigner().getAddress()` - Returns the wallet address (same as public key) - `getSigner()._signTypedData()` - Signs EIP-712 typed data for certain Turbo operations - `_ready()` - Lifecycle method (can be empty) 3. **Buffer Global**: `Buffer` is available globally from the webpack polyfills you configured in Step 3. No import needed - the `ProvidePlugin` makes it available everywhere. 4. **`isHex` Flag**: The SDK passes both UTF-8 strings (e.g., during key setup) and binary data (file content). Use this flag to differentiate handling on the server. 5. **File Uploads**: Use `uploadFile()` with `fileStreamFactory` and `fileSizeFactory` for files. Use `upload()` with `data` for strings or Blobs. 6. **Public Key Format**: Server returns uncompressed public key with `04` prefix byte. Use `.slice(2)` to remove only `0x`, keeping `04`. 7. **Signature Format**: `ethers.js` produces EIP-191 signatures compatible with the SDK's verification. 8. **Token Matching**: Ensure the `token` in your client matches your server's blockchain (e.g., both use `"base-usdc"`). 9. **Security**: Private keys stay in `.env.local` on the server, never exposed to the browser. ### Complete Working Example To use this component, add it to a page: ```typescript // app/page.tsx return ( Upload to ArDrive Turbo ); } ``` Now you can: - Click "Upload Text" to upload a string - Use the file input to upload any file - Both use server-side signing for security ## Common Issues and Solutions ### Build Errors If you encounter build errors related to missing modules: 1. **"UnhandledSchemeError: Reading from 'node:stream'"** ⚠️ **CRITICAL** - The Turbo SDK uses `node:` protocol imports that webpack doesn't understand - **Solution**: Add `NormalModuleReplacementPlugin` to your webpack config (see Step 3) - This is the most common build error in Next.js 15 2. **"Module not found: Can't resolve 'fs'"** - Ensure `fs: false` is set in your webpack fallback configuration 3. **"process is not defined"** - Make sure you have the `ProvidePlugin` configuration for process 4. **"Buffer is not defined"** - Verify the Buffer polyfill is properly configured in `ProvidePlugin` ### Next.js 15 Specific Issues 1. **"Type does not satisfy constraint" in route handlers** - Next.js 15 requires route files to ONLY - **Solution**: Remove `export` from helper functions ```typescript // ❌ INCORRECT // ✅ CORRECT async function helper() { ... } // Not exported ``` 2. **TypeScript compilation errors** - **Solution**: Set `"target": "ES2017"` in `tsconfig.json` - The SDK requires ES2017 features ### Runtime Errors 1. **"crypto.getRandomValues is not a function"** - This usually indicates the crypto polyfill isn't working. Double-check your webpack configuration. 2. **"TypeError: e.startsWith is not a function"** - This indicates incorrect signer usage. For MetaMask integration, use `InjectedEthereumSigner` from `@dha-team/arbundles`, not `EthereumSigner`. - `EthereumSigner` expects a private key string, while `InjectedEthereumSigner` expects a provider wrapper. 3. **"No accounts available" during wallet operations** - Ensure the wallet is properly connected before attempting operations - Add validation to check account availability after connection 4. **Message signing failures with wallets** - For `InjectedEthereumSigner`, ensure your provider wrapper correctly implements the `getSigner()` method - Handle both string and Uint8Array message types in your `signMessage` implementation - Use MetaMask's `personal_sign` method with proper parameter formatting 5. **Server-side rendering issues** - Always use `'use client'` directive for components that use the Turbo SDK - Consider dynamic imports with `ssr: false` for complex cases: ```tsx const TurboUploader = dynamic(() => import("./TurboUploader"), { ssr: false, }); ``` ### Server-Side Signing Issues 1. **"Invalid Data Item" errors (400 Bad Request)** - Usually caused by incorrect signature format or public key issues - **Common causes**: - Public key missing `04` prefix byte - Incorrect handling of string vs binary data - Wrong signature encoding - Token mismatch between client and server 2. **Turbo Instance SSR/Re-render Issues** ```typescript // ❌ INCORRECT - Creates instance at module level const turbo = TurboFactory.authenticated({ ... }); // Uses turbo - will cause SSR issues } // ✅ CORRECT - Create inside component with useMemo const turbo = useMemo(() => TurboFactory.authenticated({ ... }) , []); } ``` - Creating at module level runs on server during SSR - Creates new instance on every render without memoization - **Solution**: Always use `useMemo` hook inside component 3. **Public Key Format Issues** ```typescript // ❌ INCORRECT - Removes both '0x' AND '04' wallet.signingKey.publicKey.slice(4) // ✅ CORRECT - Removes only '0x', keeps '04' prefix wallet.signingKey.publicKey.slice(2) ``` - The uncompressed Ethereum public key format is `04 + x_coord + y_coord` - The SDK expects this full format 4. **String vs Binary Data Handling** - The SDK passes BOTH types to `signMessage`: - **UTF-8 strings** (e.g., "sign this message to connect to Bundlr.Network") during setup - **Binary data** (Uint8Array) for actual file content - **Solution**: Use `isHex` flag to differentiate: ```typescript const isHex = typeof message !== "string"; const messageStr = isHex ? Buffer.from(message).toString("hex") : message; // Send to server with isHex flag fetch("/api/sign", { body: JSON.stringify({ signatureData: messageStr, isHex }) }); ``` 5. **"Buffer is not defined"** - `Buffer` should be available globally from webpack polyfills - **Solution**: Verify `ProvidePlugin` is configured correctly in `next.config.ts` (Step 3) - No import needed - it's provided globally 6. **Using TypedEthereumSigner instead of ethers.js Wallet** - `TypedEthereumSigner` from `@dha-team/arbundles` produces incompatible signature format - **Solution**: Use `ethers.js` Wallet for server-side signing - Ethers produces proper EIP-191 signatures with recovery parameter 7. **Token Mismatch Errors** - Client and server must use the same token type - **Solution**: Ensure `token: "base-usdc"` matches between: - Client `TurboFactory.authenticated()` - Server environment (private key's blockchain) - Your Turbo account funding ### Wallet Integration Issues 1. **Incorrect Signer Import** ```tsx // ❌ INCORRECT - For Node environments // ✅ CORRECT - For browser wallets ``` 2. **Provider Interface Mismatch** ```tsx // ❌ INCORRECT - window.ethereum doesn't have getSigner() const signer = new InjectedEthereumSigner(window.ethereum); // ✅ CORRECT - Use a provider wrapper const providerWrapper = { getSigner: () => ({ signMessage: async (message: string | Uint8Array) => { // Implementation here }, }), }; const signer = new InjectedEthereumSigner(providerWrapper); ``` 3. **Missing Dependencies** If you encounter import errors for `@dha-team/arbundles`, note that it's available as a peer dependency through `@ardrive/turbo-sdk`. You may need to ensure it's properly resolved in your build process. ## Best Practices 1. **Use Client Components**: Always mark components using the Turbo SDK with `'use client'` 2. **Error Handling**: Implement proper error handling for network requests and wallet interactions 3. **Environment Variables**: Store sensitive configuration in environment variables: ```javascript // next.config.js const nextConfig = { env: { TURBO_UPLOAD_URL: process.env.TURBO_UPLOAD_URL, TURBO_PAYMENT_URL: process.env.TURBO_PAYMENT_URL, }, // ... webpack config }; ``` 4. **Bundle Size**: Consider code splitting for large applications to reduce bundle size 5. **Wallet Security**: - **Never expose private keys** in client-side code - Always use browser wallet integrations (MetaMask, Wander, etc.) - Request only necessary permissions from wallets - Validate wallet connections before use - Handle wallet disconnection gracefully ## Production Deployment Checklist For production deployments: 1. **Verify polyfills work correctly** in your build environment 2. **Test wallet connections** with various providers (Wander, MetaMask, etc.) 3. **Monitor bundle sizes** to ensure polyfills don't significantly increase your app size 4. **Use environment-specific configurations** for different Turbo endpoints 5. **Implement proper error boundaries** for wallet connection failures 6. **Add loading states** for wallet operations to improve UX 7. **Test across different browsers** to ensure wallet compatibility ## Implementation Verification To verify your MetaMask integration is working correctly: 1. **Check Console Logs**: After connecting to MetaMask, you should see: ``` Connected to chain: 0x1 (or appropriate chain ID) ``` 2. **Test Balance Retrieval**: Add this to verify your authenticated client works: ```tsx // After creating authenticated turbo client const balance = await turbo.getBalance(); console.log("Current balance:", balance); ``` 3. **Verify Signer Setup**: Your implementation should: - Use `InjectedEthereumSigner` from `@dha-team/arbundles` - Include a proper provider wrapper with `getSigner()` method - Handle both string and Uint8Array message types - Use MetaMask's `personal_sign` method 4. **Common Success Indicators**: - No `TypeError: e.startsWith is not a function` errors - Successful wallet connection and address display - Ability to fetch balance without errors - Upload operations work with proper MetaMask transaction prompts ## Additional Resources - [Turbo SDK Documentation](https://docs.ardrive.io) - [Web Usage Examples](https://docs.ardrive.io) - [Next.js Webpack Configuration](https://nextjs.org/docs/pages/api-reference/next-config-js/webpack) - [ArDrive Examples Repository](https://github.com/ardriveio/turbo-sdk) --- For more examples and advanced usage patterns, refer to the [Turbo SDK examples directory](https://github.com/ardriveio/turbo-sdk) or the main [SDK documentation](https://docs.ardrive.io). # Using Turbo SDK with Vite (/build/guides/using-turbo-in-a-browser/vite) # Using Turbo SDK with Vite **Firefox Compatibility**: Some compatibility issues have been reported with the Turbo SDK in Firefox browsers. At this time the below framework examples may not behave as expected in Firefox. ## Overview This guide demonstrates how to configure the `@ardrive/turbo-sdk` in a Vite application with proper polyfills for client-side usage. Vite provides excellent support for modern JavaScript features and can be easily configured to work with the Turbo SDK through plugins. **Polyfills**: Vite simplifies polyfill management compared to other bundlers. The `vite-plugin-node-polyfills` plugin handles most of the complexity automatically. ## Prerequisites - Vite 5+ - Node.js 18+ - React 18+ (or your preferred framework) - Basic familiarity with Vite configuration Install the main Turbo SDK package: ```bash npm install @ardrive/turbo-sdk ``` Add the Vite node polyfills plugin for browser compatibility: ```bash npm install --save-dev vite-plugin-node-polyfills ``` **Wallet Integration Dependencies**: The Turbo SDK includes `@dha-team/arbundles` as a peer dependency, which provides the necessary signers for browser wallet integration (like `InjectedEthereumSigner` and `ArconnectSigner`). You can import these directly without additional installation. Add React and TypeScript dependencies (if using React): ```bash npm install react react-dom npm install --save-dev @vitejs/plugin-react @types/react @types/react-dom ``` Create or update your `vite.config.js` file: ```javascript base: "/", plugins: [ react(), nodePolyfills({ // Enable specific polyfills for Turbo SDK requirements include: ["crypto", "stream", "buffer", "process"], globals: { Buffer: true, global: true, process: true, }, }), ], define: { // Define globals for browser compatibility global: "globalThis", }, }); ``` If you're using TypeScript, update your `tsconfig.json`: ```json { "compilerOptions": { "target": "ESNext", "lib": ["DOM", "DOM.Iterable", "ESNext"], "allowJs": true, "skipLibCheck": true, "esModuleInterop": true, "allowSyntheticDefaultImports": true, "strict": true, "forceConsistentCasingInFileNames": true, "module": "ESNext", "moduleResolution": "Bundler", "isolatedModules": true, "jsx": "react-jsx", "paths": { "buffer/": ["./node_modules/vite-plugin-node-polyfills/shims/buffer"] } }, "include": ["src"] } ``` **TypeScript Wallet Types** Create a `types/wallet.d.ts` file to properly type wallet objects: ```typescript // types/wallet.d.ts interface Window { ethereum?: { request: (args: { method: string; params?: any[] }) => Promise; on?: (event: string, handler: (...args: any[]) => void) => void; removeListener?: (event: string, handler: (...args: any[]) => void) => void; isMetaMask?: boolean; }; arweaveWallet?: { connect: (permissions: string[]) => Promise; disconnect: () => Promise; getActiveAddress: () => Promise; getPermissions: () => Promise; sign: (transaction: any) => Promise; getPublicKey: () => Promise; }; } ``` Select between MetaMask or Wander wallet integration: **Never expose private keys in browser applications!** Always use browser wallet integrations for security. Create a React component for MetaMask wallet integration: ```tsx const [connected, setConnected] = useState(false); const [address, setAddress] = useState(""); const [uploading, setUploading] = useState(false); const [uploadResult, setUploadResult] = useState(null); const connectMetaMask = useCallback(async () => { try { if (!window.ethereum) { alert("MetaMask is not installed!"); return; } // Request account access await window.ethereum.request({ method: "eth_requestAccounts", }); // Get the current account const accounts = await window.ethereum.request({ method: "eth_accounts", }); if (accounts.length > 0) { setAddress(accounts[0]); setConnected(true); // Log current chain for debugging const chainId = await window.ethereum.request({ method: "eth_chainId", }); console.log("Connected to chain:", chainId); } } catch (error) { console.error("Failed to connect to MetaMask:", error); } }, []); const uploadWithMetaMask = async (event) => { const file = event.target.files?.[0]; if (!file || !connected) return; setUploading(true); try { // Create a provider wrapper for InjectedEthereumSigner const providerWrapper = { getSigner: () => ({ signMessage: async (message: string | Uint8Array) => { const accounts = await window.ethereum!.request({ method: "eth_accounts", }); if (accounts.length === 0) { throw new Error("No accounts available"); } // Convert message to hex if it's Uint8Array const messageToSign = typeof message === "string" ? message : "0x" + Array.from(message) .map((b) => b.toString(16).padStart(2, "0")) .join(""); return await window.ethereum!.request({ method: "personal_sign", params: [messageToSign, accounts[0]], }); }, }), }; // Create the signer using InjectedEthereumSigner const signer = new InjectedEthereumSigner(providerWrapper); const turbo = TurboFactory.authenticated({ signer, token: "ethereum", // Important: specify token type for Ethereum }); // Upload file with progress tracking const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type }, { name: "App-Name", value: "My-Vite-App" }, { name: "Funded-By", value: "Ethereum" }, ], }, events: { onProgress: ({ totalBytes, processedBytes, step }) => { console.log( `${step}: ${Math.round((processedBytes / totalBytes) * 100)}%` ); }, onError: ({ error, step }) => { console.error(`Error during ${step}:`, error); console.error("Error details:", JSON.stringify(error, null, 2)); }, }, }); setUploadResult(result); } catch (error) { console.error("Upload failed:", error); console.error("Error details:", JSON.stringify(error, null, 2)); alert(`Upload failed: ${error.message}`); } finally { setUploading(false); } }; return ( MetaMask Upload {!connected ? ( Connect MetaMask ) : ( ✅ Connected: {address.slice(0, 6)}...{address.slice(-4)} Select File to Upload: {uploading && ( 🔄 Uploading... Please confirm transaction in MetaMask )} {uploadResult && ( ✅ Upload Successful! Transaction ID: {uploadResult.id} Data Size: {uploadResult.totalBytes} bytes )} )} ); } ``` Create a React component for Wander wallet integration: ```tsx const [connected, setConnected] = useState(false); const [address, setAddress] = useState(""); const [uploading, setUploading] = useState(false); const [uploadResult, setUploadResult] = useState(null); const connectWanderWallet = useCallback(async () => { try { if (!window.arweaveWallet) { alert("Wander wallet is not installed!"); return; } // Required permissions for Turbo SDK const permissions = [ "ACCESS_ADDRESS", "ACCESS_PUBLIC_KEY", "SIGN_TRANSACTION", "SIGNATURE", ]; // Connect to wallet await window.arweaveWallet.connect(permissions); // Get wallet address const walletAddress = await window.arweaveWallet.getActiveAddress(); setAddress(walletAddress); setConnected(true); } catch (error) { console.error("Failed to connect to Wander wallet:", error); } }, []); const uploadWithWanderWallet = async (event) => { const file = event.target.files?.[0]; if (!file || !connected) return; setUploading(true); try { // Create ArConnect signer using Wander wallet const signer = new ArconnectSigner(window.arweaveWallet); const turbo = TurboFactory.authenticated({ signer }); // Note: No need to specify token for Arweave as it's the default // Upload file with progress tracking const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type }, { name: "App-Name", value: "My-Vite-App" }, { name: "Funded-By", value: "Arweave" }, ], }, events: { onProgress: ({ totalBytes, processedBytes, step }) => { console.log( `${step}: ${Math.round((processedBytes / totalBytes) * 100)}%` ); }, onError: ({ error, step }) => { console.error(`Error during ${step}:`, error); }, }, }); setUploadResult(result); } catch (error) { console.error("Upload failed:", error); alert(`Upload failed: ${error.message}`); } finally { setUploading(false); } }; return ( Wander Wallet Upload {!connected ? ( Connect Wander Wallet ) : ( ✅ Connected: {address.slice(0, 6)}...{address.slice(-4)} Select File to Upload: {uploading && ( 🔄 Uploading... Please confirm transaction in Wander wallet )} {uploadResult && ( ✅ Upload Successful! Transaction ID: {uploadResult.id} Data Size: {uploadResult.totalBytes} bytes )} )} ); } ``` Here's a complete `package.json` example for a Vite + React + Turbo SDK project: ```json { "name": "vite-turbo-app", "version": "0.1.0", "private": true, "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview", "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0" }, "dependencies": { "@ardrive/turbo-sdk": "^1.20.0", "react": "^18.3.1", "react-dom": "^18.3.1" }, "devDependencies": { "@types/react": "^18.3.1", "@types/react-dom": "^18.3.0", "@vitejs/plugin-react": "^4.2.1", "typescript": "^5.3.3", "vite": "^5.2.14", "vite-plugin-node-polyfills": "^0.17.0" } } ``` ## Common Issues and Solutions ### Build Errors 1. **"global is not defined"** - Ensure you have `global: 'globalThis'` in your Vite config's `define` section 2. **Buffer polyfill issues** - Make sure `vite-plugin-node-polyfills` is properly configured with Buffer globals - Add the buffer path mapping in your `tsconfig.json` 3. **Module resolution errors** - Use `moduleResolution: "Bundler"` in TypeScript configuration - Ensure you're importing from `@ardrive/turbo-sdk/web` for browser usage ### Runtime Errors 1. **"process is not defined"** - Enable process globals in the node polyfills plugin configuration 2. **Wallet integration errors** - For MetaMask, use `InjectedEthereumSigner` from `@dha-team/arbundles` - For Wander wallet, use `ArconnectSigner` from the Turbo SDK - Always check wallet availability before attempting connection ### Development Experience 1. **Hot reload issues with wallet connections** - Wallet state may not persist across hot reloads - Consider using localStorage to persist connection state 2. **Console warnings about dependencies** - Some peer dependency warnings are normal for wallet libraries - Focus on runtime functionality rather than dependency warnings ## Best Practices 1. **Development vs Production** - Use debug logs during development: `TurboFactory.setLogLevel('debug')` - Remove debug logs in production builds 2. **Error Handling** - Always wrap wallet operations in try-catch blocks - Provide meaningful error messages to users - Log detailed error information for debugging 3. **Performance** - Initialize Turbo clients once and reuse them - Consider lazy loading wallet integration components - Use loading states for better user experience 4. **Security** - Never expose private keys in browser applications - Always validate wallet connections before operations - Use secure wallet connection methods in production ## Production Deployment Checklist For production builds: 1. **Build optimization** - Vite automatically optimizes builds with tree shaking - Polyfills are only included when needed 2. **Testing** - Test wallet connections across different browsers - Verify polyfills work in production builds - Test with actual wallet extensions 3. **Monitoring** - Monitor bundle sizes to ensure polyfills don't bloat your app - Set up error tracking for wallet connection failures ## Implementation Verification To verify your Vite setup is working correctly: 1. **Check Development Server**: Start your dev server and verify no polyfill errors 2. **Test Wallet Connections**: Ensure both MetaMask and Wander wallet integrations work 3. **Build Verification**: Run `npm run build` and check for any build errors 4. **Bundle Analysis**: Use `vite-bundle-analyzer` to inspect your bundle size ## Additional Resources - [Vite Documentation](https://vitejs.dev/) - [vite-plugin-node-polyfills](https://www.npmjs.com/package/vite-plugin-node-polyfills) - [Turbo SDK Documentation](https://docs.ardrive.io) - [Web Usage Examples](https://docs.ardrive.io) - [ArDrive Examples Repository](https://github.com/ardriveio/turbo-sdk) --- For more examples and advanced usage patterns, refer to the [Turbo SDK examples directory](https://github.com/ardriveio/turbo-sdk) or the main [SDK documentation](https://docs.ardrive.io). # Working With Primary Names (/build/guides/working-with-arns/arns-primary-names) Create **web3 identity** using ArNS names. Primary names allow you to use human-readable names as your identity in the Arweave ecosystem, making it easy for others to find and interact with you. ## What Are Primary Names? **Primary names** are ArNS names used as identity that: - **Resolve to wallet addresses** - Link human-readable names to wallet addresses - **Provide web3 identity** - Give users friendly names for their Arweave identity - **Are bidirectional** - Can resolve from name to address or address to name - **Require ownership** - Only the owner of an ArNS name can set it as their primary name - **Enable secure verification** - Ownership requirement ensures identity authenticity - **Work across gateways** - Accessible from any ar.io gateway - **Are unique** - A name cannot be the Primary Name for more than one wallet ## How It Works ### 1. Identity Registration **Register a primary name:** - Choose a unique name (e.g., `jonniesparkles`) - Pay the Primary Name fee - Link the name to your wallet address - Use as your web3 identity The Primary Name fee is equivalent to a single undername purchase on a 51-character name of the same purchase type, adjusted by the current Demand Factor. ### 2. Bidirectional Resolution **Name to address resolution:** - `jonniesparkles` → `OU48aJtcq3KjsEqSUWDVpynh1xP2Y1VI-bwiSukAktU` - Others can find your wallet using your name - Use in dApps and applications **Address to name resolution:** - `OU48aJtcq3KjsEqSUWDVpynh1xP2Y1VI-bwiSukAktU` → `jonniesparkles` - Find the name associated with any wallet - Verify identity in transactions ### 3. Application Integration **Use in supported apps:** - **Send tokens to "jonniesparkles"** instead of copying long wallet addresses - **Display friendly names** as usernames when connecting wallets - **Apps resolve names** to wallet addresses using the ar.io SDK - **Seamless user experience** with human-readable identifiers ## Basic Integration ### Using the ar.io SDK **Get a primary name by address:** ```javascript const ario = new ARIO(); // Get the primary name for a wallet address const nameData = await ario.getPrimaryName({ address: "OU48aJtcq3KjsEqSUWDVpynh1xP2Y1VI-bwiSukAktU", }); console.log(nameData.name); // e.g., "jonniesparkles" ``` **Get primary name data:** ```javascript const ario = new ARIO(); // Get primary name data for a name const nameData = await ario.getPrimaryName({ name: "jonniesparkles", }); console.log(nameData.owner); // e.g., "OU48aJtcq3KjsEqSUWDVpynh1xP2Y1VI-bwiSukAktU" console.log(nameData.name); // e.g., "jonniesparkles" ``` ## How Apps Use Primary Names **Token transfers:** - Send tokens to "jonniesparkles" instead of copying `OU48aJtcq3KjsEqSUWDVpynh1xP2Y1VI-bwiSukAktU` - Apps automatically resolve the name to the wallet address - Much more user-friendly than long wallet addresses **User interfaces:** - Display "jonniesparkles" as username when wallet is connected - Show friendly names in transaction histories - Make interactions more personal and memorable **Developer integration:** - Use the [ar.io SDK](/sdks/ar-io-sdk/primary-names#getprimaryname) to resolve names - Support primary names in your dApp - Enhance user experience with human-readable identifiers - **Trust identity ownership** - Only name owners can set primary names, ensuring secure verification **Ownership controls:** - A wallet can have one Primary Name at a time - A name can only be the Primary Name for one wallet - The base name's ANT owner can remove any Primary Name set on one of its undernames ## Benefits - **Web3 identity** - Use human-readable names as your identity - **Easy discovery** - Others can find you by name instead of wallet address - **Bidirectional resolution** - Resolve name to address or address to name - **Secure verification** - Only name owners can set primary names, preventing impersonation - **Network uniqueness** - A name can identify only one wallet as its Primary Name - **Permanent ownership** - Own your identity forever - **App integration** - Works in any app that supports primary names ## Ready to Learn More? } > Learn about using undernames for website versioning. } > Use the ar.io SDK to programmatically manage ArNS records. } > Learn how to deploy and host permanent apps on Arweave. # Working with ArNS (/build/guides/working-with-arns) The **Ar.io Name System (ArNS)** provides human-readable names for Arweave content, making it easy to access and manage your permanent websites and applications. Learn how to register, manage, and use ArNS names effectively. ## What is ArNS? ArNS is a decentralized naming system that allows you to: - **Register human-readable names** like `myapp` instead of using long transaction IDs - **Point names to content** stored on Arweave for permanent access - **Create subdomains** (undernames) for organizing different versions and components - **Establish web3 identity** with primary names linked to wallet addresses - **Transfer ownership** of names through the ANT (Ar.io Name Token) system — ANTs are Metaplex Core NFTs tradeable on Solana marketplaces ## Getting Started } title="Purchase an ArNS Name" description="Learn how to register and purchase ArNS names using the arns.ar.io interface" href="/build/guides/working-with-arns/purchase-arns-ui" /> } title="Manage ArNS Names" description="Update records, transfer ownership, and configure your ArNS names" href="/build/guides/working-with-arns/manage-arns-ui" /> } title="Primary Names" description="Set up web3 identity using ArNS names that resolve to wallet addresses" href="/build/guides/working-with-arns/arns-primary-names" /> ## Advanced Usage } title="Register ArNS Names Programmatically" description="Use the ar.io SDK to programmatically register and purchase ArNS names" href="/build/guides/working-with-arns/register-arns-programmatically" /> } title="Set Records Programmatically" description="Use the ar.io SDK to programmatically set and manage ArNS records" href="/build/guides/working-with-arns/set-arns-records-programmatically" /> } title="Undername Versioning" description="Use undernames to manage different versions and components of your permanent website" href="/build/guides/hosting-decentralised-apps/using-undernames-for-versioning" /> ## Key Concepts ### ArNS Names Human-readable identifiers that point to Arweave content, making it easy to access your permanent websites and applications. ### Undernames Subdomains under your main ArNS name that allow you to organize different versions, components, and content types. ### Primary Names ArNS names used as web3 identity that resolve to wallet addresses, enabling human-readable usernames across applications. ### ANT (Ar.io Name Token) A Metaplex Core NFT on Solana that represents ownership and control of an ArNS name. Each ANT stores DNS-like records pointing to Arweave data. ANTs are managed by the `ario-ant` Solana program and are tradeable on NFT marketplaces like Tensor and Magic Eden. ## Common Use Cases - **Permanent websites** - Host decentralized websites with memorable names - **Application versioning** - Maintain multiple versions of your app using undernames - **Web3 identity** - Use primary names as your identity across the ecosystem - **Content organization** - Structure different types of content using undernames - **API endpoints** - Create dedicated subdomains for different services ## Next Steps # Managing ArNS names using arns.ar.io (/build/guides/working-with-arns/manage-arns-ui) ## arns.ar.io The simplest way to register or manage an ArNS name is by using the user interface maintained by the ar.io team at [arns.ar.io](https://arns.ar.io). This site can also be accessed as an ArNS name itself on any ar.io gateway via ar://arns. Visit [arns.ar.io](https://arns.ar.io) in your browser and connect your wallet using the "Connect" button at the top right of the screen. Use the "Manage Assets" button at the top right of the screen to view and manage your registered ArNS names. The "Manage Assets" page will display a list of your registered ArNS names. You can click on an ArNS name to view the details of the name, or click the "gear" icon to the right of the name to view the settings for the name. The "Settings" page will display the settings for the selected ArNS name. You can transfer the name to a different wallet, add or remove controllers, set info about the name like the description, keywords, and logo, extend the lease duration, and more from this page. The most common setting you will want to change is the data that the name is pointing to. You can change this by using the "Target ID" field. Once you have copied in a new Arweave transaction ID to the "Target ID" field, you can click the "Save" button to save and deploy the changes. Any updates to the ArNS name will require a signature from the wallet that owns the name, or an authorized controller. ## Next Steps } > Use the ar.io SDK to programmatically register ArNS names. } > Use the ar.io SDK to programmatically manage ArNS records. } > Set up web3 identity with primary names. # Purchasing an ArNS name using arns.ar.io (/build/guides/working-with-arns/purchase-arns-ui) ## arns.ar.io The simplest way to register or manage an ArNS name is by using the user interface maintained by the ar.io team at [arns.ar.io](https://arns.ar.io). This site can also be accessed as an ArNS name itself on any ar.io gateway via ar://arns. Visit [arns.ar.io](https://arns.ar.io) in your browser and connect your wallet using the "Connect" button at the top right of the screen. Search for an ArNS name in the search bar in the center of the screen to see if your chosen name is available. If it is, click the "Register" button to register it. On the registration page, you will be able to configure the lease duration for registering the name, or choose to permabuy the name. The displayed price for the registration will automatically recalculate based on the selected purchase type and lease duration. Once you are satisfied with the configuration, click the "Next" button to proceed to the payment page. On the payment page, you will be able to review the details of the registration and the price you will be charged. You will also be able to select the payment method between a credit card, ARIO tokens, or Turbo Credits. If you do not have enough ARIO tokens or Turbo Credis for your chosen registration configuration, the payment button will display "Insufficient Balance" and you will not be able to proceed using that payment method. Once you are satisfied with the configuration, click the "Pay Now" button to complete the registration and celebrate your new ArNS name! Use the "Manage Assets" button at the top right of the screen to view and manage your registered ArNS names. ## Next Steps } > Learn how to update records and configure your ArNS names. } > Use the ar.io SDK to programmatically register ArNS names. } > Set up web3 identity with primary names. # Registering ArNS Names Programmatically (/build/guides/working-with-arns/register-arns-programmatically) Use the **ar.io SDK** to programmatically register and purchase ArNS names. This approach allows you to automate the registration process, check availability, calculate costs, and complete purchases programmatically. ## Prerequisites **Install the required packages:** ```bash npm install @ar.io/sdk ``` **Required:** - A Solana wallet with ARIO tokens and SOL for transaction fees ## Basic Setup **Initialize the SDK with your Solana wallet:** ```javascript const { ARIO } = require("@ar.io/sdk"); // Initialize ARIO with a Solana signer for transactions // See the SDK Configuration page for how to create a signer from @solana/kit const ario = ARIO.mainnet({ signer }); ``` ## Complete Registration Process **Check Name Availability** First, verify that the ArNS name you want to register is available: ```javascript async function checkNameAvailability(name) { try { const record = await ario.getArNSRecord({ name }); if (record) { console.log(`❌ Name '${name}' is already registered`); console.log("Record details:", record); return false; } } catch (error) { if (error.message.includes("not found") || error.message.includes("404")) { console.log(`✅ Name '${name}' is available`); return true; } throw error; } } // Usage const isAvailable = await checkNameAvailability("myapp"); ``` **Calculate Registration Cost** Get the cost for registering the name before proceeding: ```javascript async function getRegistrationCost(name, type = "lease", years = 1) { try { const cost = await ario.getTokenCost({ intent: "Buy-Name", name: name, type: type, years: years, }); // Convert to ARIO for readability const arioCost = new mARIOToken(cost).toARIO(); console.log(`Cost to register '${name}': ${arioCost} ARIO`); return { cost: arioCost, rawCost: cost, type: type, years: years, }; } catch (error) { console.error("Error calculating cost:", error); throw error; } } // Usage examples const leaseCost = await getRegistrationCost("myapp", "lease", 1); const permabuyCost = await getRegistrationCost("myapp", "permabuy"); ``` **Register the ArNS Name** Complete the registration process with full error handling: ```javascript async function registerArNSName(name, options = {}) { const { type = "lease", years = 1, processId = null, referrer = null, tags = [], } = options; try { console.log(`Starting registration process for '${name}'...`); // Step 1: Check availability console.log("1. Checking name availability..."); const isAvailable = await checkNameAvailability(name); if (!isAvailable) { throw new Error(`Name '${name}' is not available`); } // Step 2: Calculate cost console.log("2. Calculating registration cost..."); const costInfo = await getRegistrationCost(name, type, years); console.log(`Cost: ${costInfo.cost} ARIO`); // Step 3: Register the name console.log("3. Registering ArNS name..."); const record = await ario.buyRecord( { name: name, type: type, years: years, processId: processId, referrer: referrer, }, { tags: [{ name: "App-Name", value: "ArNS-Registration" }, ...tags], onSigningProgress: (step, event) => { console.log(`Signing progress: ${step}`); if (step === "spawning-ant") { console.log("Spawning ANT:", event); } if (step === "registering-ant") { console.log("Registering ANT:", event); } if (step === "verifying-state") { console.log("Verifying state:", event); } if (step === "buying-name") { console.log("Buying name:", event); } }, } ); console.log(`✅ Successfully registered '${name}'`); console.log("Record details:", record); return { success: true, record: record, cost: costInfo.cost, name: name, }; } catch (error) { console.error(`❌ Failed to register '${name}':`, error.message); return { success: false, error: error.message, name: name, }; } } // Usage const result = await registerArNSName("myapp", { type: "lease", years: 1, referrer: "my-app.com", }); ``` ## Advanced Registration Features ### Cost Details and Funding **Get detailed cost breakdown:** ```javascript async function getDetailedCost(name, type = "lease", years = 1) { try { const costDetails = await ario.getCostDetails({ intent: "Buy-Name", fromAddress: "your-wallet-address", fundFrom: "stakes", // or "balance" or "any" name: name, type: type, years: years, }); console.log("Cost details:", costDetails); return costDetails; } catch (error) { console.error("Error getting cost details:", error); throw error; } } ``` ### Batch Registration **Register multiple names:** ```javascript async function registerMultipleNames(names, options = {}) { const results = []; for (const name of names) { console.log(`\n--- Registering ${name} ---`); const result = await registerArNSName(name, options); results.push(result); // Add delay between registrations to avoid rate limiting await new Promise((resolve) => setTimeout(resolve, 2000)); } const successful = results.filter((r) => r.success); const failed = results.filter((r) => !r.success); console.log(`\n📊 Registration Summary:`); console.log(`✅ Successful: ${successful.length}`); console.log(`❌ Failed: ${failed.length}`); return results; } // Usage const names = ["myapp", "myapi", "mydocs"]; const results = await registerMultipleNames(names, { type: "lease", years: 1, }); ``` ### Registration with Validation **Enhanced registration with pre-checks:** ```javascript async function registerWithValidation(name, options = {}) { const { type = "lease", years = 1, maxCost = null, // Maximum cost in ARIO validateName = true, } = options; try { // Validate name format if (validateName) { if (!/^[a-z0-9-]+$/.test(name)) { throw new Error( "Name must contain only lowercase letters, numbers, and hyphens" ); } if (name.length 63) { throw new Error("Name must be between 3 and 63 characters"); } } // Check availability const isAvailable = await checkNameAvailability(name); if (!isAvailable) { throw new Error(`Name '${name}' is not available`); } // Get cost const costInfo = await getRegistrationCost(name, type, years); // Check cost limit if (maxCost && costInfo.cost > maxCost) { throw new Error( `Cost ${costInfo.cost} ARIO exceeds maximum ${maxCost} ARIO` ); } // Proceed with registration return await registerArNSName(name, options); } catch (error) { console.error(`Validation failed for '${name}':`, error.message); return { success: false, error: error.message, name: name, }; } } // Usage with validation const result = await registerWithValidation("myapp", { type: "lease", years: 1, maxCost: 1000, // Maximum 1000 ARIO validateName: true, }); ``` ## Registration Types ### Lease Registration **Register with a time-limited lease:** ```javascript // 1-year lease const leaseResult = await registerArNSName("myapp", { type: "lease", years: 1, }); // 5-year lease const longLeaseResult = await registerArNSName("myapp", { type: "lease", years: 5, }); ``` ### Permanent Registration **Register with permanent ownership:** ```javascript const permabuyResult = await registerArNSName("myapp", { type: "permabuy", }); ``` ## Error Handling **Comprehensive error handling:** ```javascript async function safeRegister(name, options = {}) { try { return await registerArNSName(name, options); } catch (error) { if (error.message.includes("insufficient funds")) { console.error("❌ Insufficient funds for registration"); } else if (error.message.includes("already registered")) { console.error("❌ Name is already registered"); } else if (error.message.includes("invalid name")) { console.error("❌ Invalid name format"); } else if (error.message.includes("network")) { console.error("❌ Network error, please try again"); } else { console.error(`❌ Unexpected error: ${error.message}`); } return { success: false, error: error.message, name: name, }; } } ``` ## Ready to Register? } > Learn how to set and manage ArNS records after registration. } > Set up web3 identity with your registered ArNS names. } > Use undernames to organize different versions of your content. # Setting ArNS Records Programmatically (/build/guides/working-with-arns/set-arns-records-programmatically) Use the **ar.io SDK** to programmatically set and manage ArNS records. This approach gives you full control over your ArNS operations and allows for automation and integration into your applications. ## Prerequisites **Install the required packages:** ```bash npm install @ar.io/sdk ``` **Required:** - A Solana wallet with ARIO tokens and SOL for transaction fees ## Basic Setup **Initialize the SDK:** ```javascript const { ARIO, ANT } = require("@ar.io/sdk"); // Initialize ARIO for mainnet const ario = ARIO.mainnet(); // See the Configuration page for how to create a signer from @solana/kit // const signer = await createKeyPairSignerFromBytes(keypairBytes); ``` ## Setting Base Name Records **Set the main domain record:** ```javascript async function setBaseNameRecord(arnsName, transactionId) { try { // Get the ArNS record for your domain const pid = await ario.getArNSRecord({ name: arnsName }); if (!pid) { throw new Error(`ArNS name '${arnsName}' not found`); } // Initialize ANT contract const ant = ANT.init({ processId: pid.processId, signer, }); // Set the base name record const response = await ant.setBaseNameRecord({ transactionId: transactionId, ttlSeconds: 900, // 15 minutes TTL }); console.log("Base name record updated:", response); return response; } catch (error) { console.error("Error setting base name record:", error); throw error; } } // Usage await setBaseNameRecord("myapp", "abc123...def789"); ``` ## Setting Undername Records **Set subdomain records:** ```javascript async function setUndernameRecord(arnsName, undername, transactionId) { try { // Get the ArNS record for your domain const pid = await ario.getArNSRecord({ name: arnsName }); if (!pid) { throw new Error(`ArNS name '${arnsName}' not found`); } // Initialize ANT contract const ant = ANT.init({ processId: pid.processId, signer, }); // Set the undername record const response = await ant.setUndernameRecord({ undername: undername, transactionId: transactionId, ttlSeconds: 900, // 15 minutes TTL }); console.log(`Undername '${undername}' record updated:`, response); return response; } catch (error) { console.error("Error setting undername record:", error); throw error; } } // Usage examples await setUndernameRecord("myapp", "v1", "abc123...def789"); await setUndernameRecord("myapp", "api", "def456...ghi012"); await setUndernameRecord("myapp", "docs", "ghi789...jkl345"); ``` ## Complete Deployment Example **Upload and set records in one function:** ```javascript const { TurboFactory } = require("@ardrive/turbo-sdk"); async function deployAndSetRecord(arnsName, undername, distFolderPath) { try { // Initialize Turbo for file uploads const turbo = TurboFactory.authenticated({ privateKey: jwk }); // Upload your website folder const { manifestResponse } = await turbo.uploadFolder({ folderPath: distFolderPath, dataItemOpts: { tags: [{ name: "App-Name", value: "ar.io docs deploy" }], }, manifestOptions: { fallbackFile: "index.html", }, }); console.log(`Manifest uploaded: ${manifestResponse.id}`); // Set the appropriate record if (undername === "@") { await setBaseNameRecord(arnsName, manifestResponse.id); } else { await setUndernameRecord(arnsName, undername, manifestResponse.id); } console.log(`Deployment complete!`); console.log( `Access at: https://${arnsName}.turbo-gateway.com${undername === "@" ? "" : `/${undername}`}` ); } catch (error) { console.error("Deployment failed:", error); throw error; } } // Usage await deployAndSetRecord("myapp", "@", "./build"); await deployAndSetRecord("myapp", "v1", "./build-v1"); await deployAndSetRecord("myapp", "api", "./api-docs"); ``` ## Managing Multiple Records **Set up a complete website structure:** ```javascript async function setupWebsiteStructure(arnsName) { const records = [ { undername: "@", folder: "./build", description: "Main site" }, { undername: "v1", folder: "./build-v1", description: "Version 1" }, { undername: "v2", folder: "./build-v2", description: "Version 2" }, { undername: "api", folder: "./api-docs", description: "API documentation", }, { undername: "docs", folder: "./user-docs", description: "User documentation", }, ]; for (const record of records) { try { console.log(`Deploying ${record.description}...`); await deployAndSetRecord(arnsName, record.undername, record.folder); console.log(`✅ ${record.undername} deployed successfully`); } catch (error) { console.error(`❌ Failed to deploy ${record.undername}:`, error.message); } } } // Usage await setupWebsiteStructure("myapp"); ``` ## Advanced Configuration **Custom TTL and error handling:** ```javascript async function setRecordWithOptions( arnsName, undername, transactionId, options = {} ) { const { ttlSeconds = 900, retries = 3, retryDelay = 1000 } = options; for (let attempt = 1; attempt setTimeout(resolve, retryDelay)); } } } // Usage with custom options await setRecordWithOptions("myapp", "v1", "abc123...def789", { ttlSeconds: 3600, // 1 hour retries: 5, retryDelay: 2000, }); ``` ## Reading Records **Get current record information:** ```javascript async function getRecordInfo(arnsName, undername = "@") { try { const pid = await ario.getArNSRecord({ name: arnsName }); if (!pid) { throw new Error(`ArNS name '${arnsName}' not found`); } const ant = ANT.init({ processId: pid.processId, signer }); if (undername === "@") { return await ant.getBaseNameRecord(); } else { return await ant.getUndernameRecord({ undername }); } } catch (error) { console.error("Error getting record info:", error); throw error; } } // Usage const mainRecord = await getRecordInfo("myapp", "@"); const v1Record = await getRecordInfo("myapp", "v1"); console.log("Main record:", mainRecord); console.log("V1 record:", v1Record); ``` ## Error Handling Best Practices **Comprehensive error handling:** ```javascript async function safeSetRecord(arnsName, undername, transactionId) { try { // Validate inputs if (!arnsName || !undername || !transactionId) { throw new Error("Missing required parameters"); } // Check if ArNS name exists const pid = await ario.getArNSRecord({ name: arnsName }); if (!pid) { throw new Error( `ArNS name '${arnsName}' not found or not owned by this wallet` ); } // Set the record const ant = ANT.init({ processId: pid.processId, signer }); let response; if (undername === "@") { response = await ant.setBaseNameRecord({ transactionId, ttlSeconds: 900, }); } else { response = await ant.setUndernameRecord({ undername, transactionId, ttlSeconds: 900, }); } console.log(`✅ Successfully set ${undername} record for ${arnsName}`); return response; } catch (error) { if (error.message.includes("not found")) { console.error(`❌ ArNS name '${arnsName}' not found`); } else if (error.message.includes("insufficient funds")) { console.error("❌ Insufficient funds for transaction"); } else if (error.message.includes("signature")) { console.error("❌ Invalid wallet signature"); } else { console.error(`❌ Unexpected error: ${error.message}`); } throw error; } } ``` ## Ready to Get Started? } > Learn about using undernames for website versioning. } > Set up web3 identity with primary names. } > Explore the full ar.io SDK documentation. # Get Started (/build) Welcome to ar.io's developer documentation. This section will guide you through everything you need to know for building on top of ar.io from uploading and accessing your data, to operating and extending your own infrastructure to support your unique use-case. If you're unfamiliar with Arweave's permanent storage and ar.io we recommend reading this [introduction section](/learn) first. ## Get Started Building with ar.io } title="Uploading Data" description="Learn how to permanently store files, websites, and application data on Arweave" href="/build/upload" /> } title="Accessing Data" description="Query, retrieve, and interact with data stored on the permanent web" href="/build/access" /> } title="Running Your Own Gateway" description="Deploy and operate ar.io gateway infrastructure to support the network" href="/build/run-a-gateway" /> } title="Testnet Sandbox" description="Try the full stack — upload, pay, buy ArNS names, and serve data — free on Solana devnet, no mainnet" href="/build/testnet" /> } title="AI Agents & LLMs" description="Machine-readable docs — llms.txt, SKILL.md, and per-SDK llm.txt — for building with AI coding agents" href="/build/agents" /> ## Developer Resources } title="Turbo SDK Reference" description="Fast upload service SDK with payment processing and instant confirmation" href="/sdks/turbo-sdk" /> } title="Wayfinder SDK Reference" description="Decentralized data access SDK with built-in verification and routing" href="/sdks/wayfinder" /> } title="ar.io SDK Reference" description="Complete SDK documentation for interacting with ar.io protocols" href="/sdks/ar-io-sdk" /> } title="Gateway API Reference" description="Direct API access for custom implementations and integrations" href="/apis" /> ## Use Cases & Guides } title="Hosting Unstoppable Apps" description="Build and deploy permanent, censorship-resistant applications with comprehensive guides" href="/build/guides/hosting-decentralised-apps" /> } title="ArNS Primary Names" description="Set up primary names for user-friendly wallet addresses" href="/build/guides/working-with-arns/arns-primary-names" /> } title="ArNS Undernames & Versioning" description="Manage subdomains and versioning for your ArNS names" href="/build/guides/hosting-decentralised-apps/using-undernames-for-versioning" /> } title="ArNS Marketplace" description="Explore the ArNS marketplace for name trading and management" href="/build/guides/arns-marketplace" /> } title="Verifiable AI" description="Anchor MLflow lifecycle proofs and verify them through ar.io gateways" href="/build/verifiable-ai" /> ## Get Help } title="Browse Examples" description="Explore code examples and sample applications" href="https://github.com/ar-io" /> } title="Join the Community" description="Connect with other developers on Discord" href="https://discord.com/invite/HGG52EtTc2" /> Ready to build on the permanent web? Choose your path above and start creating applications that last forever. # Run a Gateway (/build/run-a-gateway) Join the decentralized network that powers permanent data access. Run your own **ar.io Gateway** to support the permaweb infrastructure and earn rewards. ## Gateway Options Choose the deployment approach that fits your needs - from local testing to production infrastructure. Production Gateway Earn Rewards} description={ Join ar.io and earn ARIO tokens Earn ARIO token rewards Serve Wayfinder traffic Cache and serve Arweave data } href="/build/run-a-gateway/quick-start#production-setup-with-custom-domain" icon={} /> Perfect for development and testing Quick Docker setup Test gateway features No commitment required } href="/build/run-a-gateway/quick-start" icon={} /> Optimize for specific use cases • Data filtering options • Performance tuning • Advanced features } href="/build/run-a-gateway/manage/filters" icon={} /> ## Why Run a Gateway? **Economic Benefits** - Earn ARIO tokens through network participation - Set custom pricing for premium services - Build sustainable infrastructure business **Technical Advantages** - Full control over data access and caching - Custom configuration for your applications - Direct integration with your services **Network Impact** - Support decentralized web infrastructure - Increase network reliability and redundancy - Enable censorship-resistant data access ## Quick Start in 30 Seconds Get a gateway running locally with a single command: ```bash # Prerequisites: Docker installed on your system docker run -p 4000:4000 ghcr.io/ar-io/ar-io-core:latest ``` Test your gateway: ```bash # Fetch a transaction curl localhost:4000/4jBV3ofWh41KhuTs2pFvj-KBZWUkbrbCYlJH0vLA6LM ``` Your gateway is now serving Arweave data! This local setup is perfect for: - Testing gateway functionality - Developing applications - Understanding gateway operations ## Learn Before You Build Understanding gateways helps you make informed infrastructure decisions. } /> } /> } /> ## Ready to Deploy? Whether you're exploring gateway capabilities or ready to join the network, we have resources to help: # Join the Network (/build/run-a-gateway/join-the-network) Take control of the permanent web by running your own **ar.io Gateway**. Join the decentralized network that powers the permaweb and earn rewards for providing infrastructure services. ## Prerequisites ### Running Gateway Required You must have a fully functional ar.io Gateway running with a custom domain and SSL certificates. **Don't have a gateway yet?** Follow our [Production Setup Guide](/build/run-a-gateway/quick-start#production-setup-with-custom-domain) to get your gateway running with proper DNS configuration. **Requirements:** - Gateway accessible via your custom domain (e.g., `https://yourdomain.com`) - SSL certificates properly configured - ArNS subdomain resolution working - Gateway responding to test requests ### Minimum Stake Requirement To join the network as a gateway operator, you need **20,000 ARIO tokens** as the minimum stake requirement. **Need to acquire ARIO tokens?** Visit our [Get the Token guide](/learn/token/get-the-token) to learn about all available methods including exchanges, DEXs, and network participation. **Acquisition Options:** - Purchase on centralized exchanges like Gate.io - Trade on Solana DEXs (Jupiter, Raydium) - Use a Solana wallet (Phantom, Solflare, Backpack) to manage tokens - Earn through network participation and community programs You will also need SOL in your wallet for Solana transaction fees when joining the network. ## Join the Network Choose your preferred method to register your gateway: ### Visit the Network Portal Go to [gateways.ar.io](https://gateways.ar.io/#/gateways) to access the ar.io's Network Portal. The portal shows all active gateways on the network and provides the interface to register your own gateway. ### Connect Your Wallet Click the "Start your own gateway" button to begin the registration process. You'll be prompted to connect your wallet. Choose your preferred Solana wallet (Phantom, Solflare, or Backpack) to connect. Use the same wallet address that you configured in your gateway's `AR_IO_WALLET` environment variable. This wallet will be the owner of your gateway registration. ### Fill Out Gateway Information Complete the gateway registration form with your gateway details: **Required Fields:** - **Label**: A display name for your gateway (e.g., "My New Gateway") - **Address**: Your gateway's domain with port (e.g., `https://fastandfurious.io:443`) - **Observer Wallet**: The public address of your observer wallet - **Properties ID**: Transaction ID of your gateway properties - **Stake (ARIO)**: Minimum stake required (20,000 ARIO) - **Delegated Staking**: Enable to allow others to delegate stake to your gateway - **Minimum Delegated Stake**: Set minimum delegation amount (e.g., 100 ARIO) - **Reward Share Ratio**: Percentage of rewards shared with delegators (e.g., 50%) - **Note**: Additional information about your gateway (e.g., "ar.io rules!") ### Confirm Registration Review all information carefully and click "Confirm" to submit your gateway registration to the network. **What happens next:** - Your gateway will be added to the Gateway Address Registry - Observers will start observing your gateway at the next Epoch (day) - You will begin to receive rewards based on your gateway performance - You can monitor your gateway's performance in the portal **Confirm your gateway registration:** Your gateway should now be viewable at `gateways.ar.io/#/` with the wallet address you used to join. This dashboard shows your gateway's information, stats, and performance metrics including join date, uptime, operator stake, and delegated stake details. ### Install the ar.io CLI First, install the ar.io CLI tool if you haven't already: ```npm npm install -g @ar.io/sdk ``` ### Run the Join Network Command Use the `ar.io join-network` command with your gateway configuration. The operator key can be supplied either as a JSON keypair file (`--wallet-file`) or as a base58-encoded 64-byte secret key string (`--private-key`). The latter is the format Phantom and similar browser wallets export, so most operators can paste it directly. ```bash ar.io join-network \ --wallet-file ./path/to/solana-keypair.json \ -t solana \ --mainnet \ --rpc-url https://api.mainnet-beta.solana.com \ --core-program-id 73YoECm6NKXpVRoe5f1Q9BcP5DJGPFUjnFy6AxBE5Nvh \ --gar-program-id 89fNiiwgpFSPHKuqfNUkgYTYjtAJAhyqHjXmgXeppGpf \ --arns-program-id 2yCUx5edFvUrkibYaUa2ZXWyx9kuJkS8CwyzsgHPWdZZ \ --ant-program-id 2MWexMHfMhGJwMHv9Qm9YAVCqjUFUJwDJAysW4oCUGk5 \ --operator-stake 20000 \ --auto-stake \ --allow-delegated-staking \ --min-delegated-stake 100000000 \ --delegate-reward-share-ratio 10 \ --label "My Gateway" \ --note "Production ar.io gateway" \ --observer-address 7xKXtR2qpZm8FjvKNsG3kL9p5yMnYhVdEbxQ4oWc2Rn \ --fqdn my-gateway.example.com \ --port 443 \ --protocol https ``` '`. The SDK decodes the 64-byte secret, derives the public key, and signs in-memory — no JSON file required. The public key derived from your `--wallet-file` / `--private-key` becomes your `AR_IO_WALLET` (operator). The `--observer-address` must be a **unique** Solana address; no two gateways can share an observer. If you don't have a separate observer key, set `--observer-address` to your operator address. The gateway accepts this single-key setup. **Parameter explanations:** - `--operator-stake 20000` — operator stake in whole ARIO units (CLI converts to mARIO internally). Minimum is 20,000 ARIO on mainnet. - `--auto-stake` — flag-only (no value). When set, operator rewards auto-restake to your gateway. - `--allow-delegated-staking` — flag-only. Permits other wallets to delegate stake to you. - `--min-delegated-stake 100000000` — minimum delegation in mARIO (100 ARIO × 1,000,000). Unlike `--operator-stake`, this argument is in mARIO directly. - `--delegate-reward-share-ratio 10` — percentage of rewards shared with delegators (0-100). - `--observer-address` — Solana address of the observer key (often the same as the operator address). - `--fqdn` — your gateway's public domain name. - `-t solana` — required to switch the CLI into Solana mode (default is Arweave). - `--mainnet` — selects Solana mainnet behavior. The explicit `--core-program-id`, `--gar-program-id`, `--arns-program-id`, `--ant-program-id`, and `--rpc-url` flags pin the command to the canonical mainnet deployment. ### Verify Registration Verify the on-chain registration via the CLI: ```bash ar.io get-gateway \ --mainnet \ -t solana \ --rpc-url https://api.mainnet-beta.solana.com \ --core-program-id 73YoECm6NKXpVRoe5f1Q9BcP5DJGPFUjnFy6AxBE5Nvh \ --gar-program-id 89fNiiwgpFSPHKuqfNUkgYTYjtAJAhyqHjXmgXeppGpf \ --arns-program-id 2yCUx5edFvUrkibYaUa2ZXWyx9kuJkS8CwyzsgHPWdZZ \ --ant-program-id 2MWexMHfMhGJwMHv9Qm9YAVCqjUFUJwDJAysW4oCUGk5 \ --address ``` The result should show `"status": "joined"` and the settings (FQDN, stake, allow-delegated, etc.) you passed at join time. Once your gateway is running and pointed at the same network, cross-check from the gateway side too: ```bash curl https:///ar-io/info | jq '{wallet, programIds}' ``` - `wallet` should match the operator pubkey you joined with. - `programIds.{core,gar,arns,ant}` should match the network you joined (mainnet program IDs, or whichever staging/devnet set you're using). If those don't line up, your gateway is querying a different network than the one you joined. That's the usual reason a freshly-joined gateway doesn't show up as observing. You can also verify in the portal: `gateways.ar.io/#/`. ## What Happens After Registration After joining the network: - Your gateway will be monitored by the Observer system - You'll earn rewards for providing reliable service - You can monitor your gateway's performance and earnings in the portal - You may be selected as an Observer to help monitor other gateways ## Next Steps Your gateway is now part of ar.io! Here are some next steps to maximize your participation: } /> } /> } /> } /> # CDB64 Root Transaction Index (/build/run-a-gateway/manage/cdb64) ## Overview When your gateway receives a request for a data item (content inside an ANS-104 bundle), it needs to find the root Arweave transaction containing that data. The CDB64 index provides O(1) lookups for this mapping, enabling instant resolution of historical data items. **Default Behavior**: As of Release 67, CDB64 is enabled by default with no configuration required. The gateway ships with a pre-built index covering approximately 964 million data items. ## How It Works The gateway checks multiple sources when resolving a data item ID to its root transaction. The order is controlled by `ROOT_TX_LOOKUP_ORDER`: 1. **db** - Your local SQLite database (fastest, but requires locally parsing ANS-104 bundles to index discovered items) 2. **gateways** - HEAD requests to other AR.IO gateways 3. **cdb** - CDB64 file-based index (O(1) lookup from local files or cached remote data) 4. **graphql** - GraphQL queries to trusted gateways The default configuration tries each source in order until a match is found: ```bash ROOT_TX_LOOKUP_ORDER=db,gateways,cdb,graphql ``` ## Default Coverage The shipped CDB64 index covers: - Non-AO data items (excludes `Bundler-App-Name: AO`) - Non-Redstone data items - Data items with content types - Block heights 0 through 1,820,000 This means most historical ArDrive, Akord, and similar application data can be resolved via the CDB64 index. The default shipped index stores partition data on Arweave, so network requests are made to fetch CDB data (with intelligent byte-range caching). For zero network latency, you can download the CDB files locally. ## Configuration Options ### Disabling CDB64 If you want to disable CDB64 lookups (not recommended), remove `cdb` from the lookup order: ```bash ROOT_TX_LOOKUP_ORDER=db,gateways,graphql ``` ### Using Custom Index Sources You can configure custom CDB64 index sources to supplement or replace the default index: ```bash CDB64_ROOT_TX_INDEX_SOURCES=/path/to/custom-index.cdb ``` ```bash # Directory containing multiple .cdb files or a partitioned index CDB64_ROOT_TX_INDEX_SOURCES=/path/to/index-directory/ ``` ```bash CDB64_ROOT_TX_INDEX_SOURCES=https://cdn.example.com/index.cdb ``` ```bash # 43-character base64url transaction ID CDB64_ROOT_TX_INDEX_SOURCES=ABC123def456xyz789ABC123def456xyz789ABC12 ``` ```bash # Sources are tried in order until a match is found CDB64_ROOT_TX_INDEX_SOURCES=/local/index.cdb,https://cdn.example.com/index/,TxId123... ``` ### Remote Index Configuration When using HTTP or Arweave-stored indexes, you can tune the caching and request behavior: ```bash # Caching settings CDB64_REMOTE_CACHE_MAX_REGIONS=100 # Max cached byte-range regions per source CDB64_REMOTE_CACHE_TTL_MS=300000 # Cache TTL (5 minutes) # Request settings CDB64_REMOTE_REQUEST_TIMEOUT_MS=30000 # Request timeout CDB64_REMOTE_MAX_CONCURRENT_REQUESTS=4 # Max concurrent HTTP requests # Retrieval order for fetching CDB files from Arweave CDB64_REMOTE_RETRIEVAL_ORDER=gateways,chunks ``` ### File Watching For local CDB64 directories, the gateway automatically watches for new or removed `.cdb` files: ```bash # Enable/disable automatic reloading (default: true) CDB64_ROOT_TX_INDEX_WATCH=true ``` When enabled, you can add new index files to the directory without restarting your gateway. ## Partitioned Indexes Large CDB64 indexes can be split across up to 256 partition files for better manageability. Records are partitioned by the first byte of the binary data item ID, represented as a hex prefix (00-ff). A partitioned index consists of: - `manifest.json` - Describes all partitions and their locations - `00.cdb` through `ff.cdb` - Partition files (only populated prefixes exist) Partitions can be stored in different locations (local files, HTTP, Arweave), allowing flexible deployment strategies. ```bash # Point to directory containing manifest.json CDB64_ROOT_TX_INDEX_SOURCES=/path/to/partitioned-index/ ``` ```bash # HTTP URL to manifest CDB64_ROOT_TX_INDEX_SOURCES=https://cdn.example.com/index/manifest.json ``` ```bash # Append :manifest to transaction ID CDB64_ROOT_TX_INDEX_SOURCES=ABC123def456xyz789ABC123def456xyz789ABC12:manifest ``` ## Generating Custom Indexes If you need to create CDB64 indexes for specific data sets, the gateway includes CLI tools: ```bash # Generate from CSV file ./tools/generate-cdb64-root-tx-index --input data.csv --output index.cdb # Generate partitioned index (creates manifest.json automatically) ./tools/generate-cdb64-root-tx-index --input data.csv --partitioned --output-dir ./index/ # Export from local SQLite database ./tools/export-sqlite-to-cdb64 --output index.cdb # Verify index completeness ./tools/verify-cdb64 --index index.cdb --gateway https://turbo-gateway.com ``` The `--partitioned` flag automatically shards records by ID prefix and generates the `manifest.json` with local file locations. For high-throughput generation, a Rust-backed tool is also available: ```bash ./tools/generate-cdb64-root-tx-index-rs --input data.csv --output index.cdb ``` ## Uploading Indexes to Arweave You can upload partitioned CDB64 indexes to Arweave for permanent, decentralized storage: ```bash ./tools/upload-cdb64-to-arweave \ --input-dir ./partitioned-index/ \ --wallet ./wallet.json \ --concurrency 5 ``` This tool: 1. Uploads each partition file to Arweave via Turbo 2. Resolves the bundle IDs and byte offsets for each partition 3. Updates the manifest with `arweave-bundle-item` locations The resulting manifest can be shared with other gateway operators or uploaded to Arweave for decentralized index distribution. ## Performance Considerations - **O(1) lookups** - Each lookup requires only 2-3 file reads regardless of index size - **Byte-range caching** - The 4KB header is cached permanently; other regions use LRU caching - **Lazy loading** - Partitioned indexes only open accessed partitions, reducing memory usage - **Circuit breakers** - If CDB64 lookups fail repeatedly, the gateway automatically falls back to other sources ## Troubleshooting ### CDB64 lookups not working 1. Verify `cdb` is in your `ROOT_TX_LOOKUP_ORDER` 2. Check that index files exist and are readable 3. Review gateway logs for CDB64-related errors ### Slow remote index performance 1. Increase `CDB64_REMOTE_CACHE_MAX_REGIONS` for frequently accessed indexes 2. Consider downloading the index locally for best performance 3. Check network connectivity to remote sources ### Missing data items in index The default shipped index excludes AO and Redstone data. For these, you'll need to: - Generate a custom index covering the desired data - Rely on other lookup sources (db, gateways, graphql) For the complete list of CDB64 environment variables, see [Environment Variables Reference](/build/run-a-gateway/manage/environment-variables#cdb64-root-transaction-index). # Content Moderation (/build/run-a-gateway/manage/content-moderation) ## Overview Arweave is a network designed for permanent storage of data. It is a practical impossibility for data to be wholly removed from the network once it has been uploaded. Ar.io has adopted Arweave's voluntary content moderation model, whereby every participant of the network has the autonomy to decide which content they want to (or can legally) store, serve, and see. Each gateway operating on the network has the right and ability to blocklist any content, ArNS name, or address that is deemed in violation of its content policies or is non-compliant with local regulations. Gateway operators may set content to be blocked by their gateway by submitting a `PUT` request to their gateway defining the content to be blocked. This requires that the `ADMIN_API_KEY` environmental variable to be set in order to authenticate the moderation request. The simplest method for submitting moderation requests to a gateway is to use `curl` in a terminal. ## Quick Start ### Set Up Admin API Key Configure your admin API key in your `.env` file: ```bash # Set a secure admin API key ADMIN_API_KEY=your_secure_admin_key_here ``` Choose a strong, unique admin API key. This key provides administrative access to your gateway and should be kept secure. ### Test API Access Verify your admin API key is working: ```bash # Test admin endpoint access curl -H "Authorization: Bearer your_secure_admin_key_here" \ http://localhost:3000/ar-io/admin/debug ``` ### Block Your First Content Block a specific transaction ID: ```bash curl -X 'PUT' \ 'http://localhost:3000/ar-io/admin/block-data' \ -H 'accept: */*' \ -H 'Authorization: Bearer your_secure_admin_key_here' \ -H 'Content-Type: application/json' \ -d '{ "id": "3lyxgbgEvqNSvJrTX2J7CfRychUD5KClFhhVLyTPNCQ", "notes": "Content violates our policies", "source": "Manual Review" }' ``` ## Authentication Moderation requests must contain the gateway's `ADMIN_API_KEY` in the request Header, as `Authorization: Bearer`. For example, if a gateway's `ADMIN_API_KEY` is set to `secret`, any request must contain `Authorization: Bearer secret` in the Header. ## Block Data Specific data items can be blocked by a gateway operator by submitting a `PUT` request containing a json object with three keys: - **id**: The Arweave transaction Id of the data item to be blocked. - **notes**: Any note the gateway operator wants to leave him/herself as to the reason the content is blocked. - **source**: A note as to where the content was identified as requiring moderation. i.e. a public block list. Requests to block data must be submitted to the gateway's `/ar-io/admin/block-data` endpoint. ```bash {{ title: 'curl' }} curl -X 'PUT' \ 'http://localhost:3000/ar-io/admin/block-data' \ -H 'accept: */*' \ -H 'Authorization: Bearer secret' \ -H 'Content-Type: application/json' \ -d '{ "id": "3lyxgbgEvqNSvJrTX2J7CfRychUD5KClFhhVLyTPNCQ", "notes": "This content is offensive", "source": "Public Block list" }' ``` ### Unblock Data At this time, blocked data items can only be unblocked by manually deleting the corresponding row from the `data/sqlite/moderation.db` database. The Arweave transaction Id of the blocked data item is stored in the database as raw bytes, which sqlite3 accepts as a BLOB (Binary Large OBject), and so cannot be accessed easily using the original transaction Id, which is a base64url. Sqlite3 is able to interact with a hexadecimal representation of the BLOB, by using a BLOB literal. To do so, wrap a hexadecimal representation of the Arweave transaction Id in single quotes, and prepend an `X` i.e. `X'de5cb181b804bea352bc9ad35f627b09f472721503e4a0a51618552f24cf3424'`. Where possible, consider using the `notes` or `source` values to identify rows for deletion rather than the `id`. ```bash {{ title: 'id' }} sqlite3 data/sqlite/moderation.db "DELETE FROM blocked_ids WHERE id=X'de5cb181b804bea352bc9ad35f627b09f472721503e4a0a51618552f24cf3424';" # Note that the id in this command is a BLOB literal using the hexadecimal representation of the Arweave transaction Id, not the transaction Id in its normal base64url format ``` ```bash {{ title: 'source' }} sqlite3 data/sqlite/moderation.db "DELETE FROM blocked_ids WHERE block_source_id = (SELECT id FROM block_sources WHERE name='Public Block List');" # This command uses a subquery to look up the id in block_sources where name='Public Block List' # This command will unblock ALL data items marked with this source value ``` ## Block ArNS Name ArNS names can be blocked so that a gateway will refuse to serve their associated content even if the name holder updates the Arweave transaction Id that the name points at. This is done via an authenticated `PUT` request to the endpoint `/ar-io/admin/block-name` containing a json object with three keys: - **name**: The ArNS name to be blocked. - **notes**: Any note the gateway operator wants to leave him/herself as to the reason the content is blocked. - **source**: A note as to where the content was identified as requiring moderation. i.e. a public block list. ```bash {{ title: 'curl'}} curl -X 'PUT' \ 'http://localhost:3000/ar-io/admin/block-name' \ -H 'accept: */*' \ -H 'Authorization: Bearer secret' \ -H 'Content-Type: application/json' \ -d '{ "name": "i-bought-a-potato", "notes": "Potatoes are offensive", "source": "Public Block list" }' ``` For moderation purposes, each [undername](/learn/arns) of an ArNS name is treated as a separate name and must be moderated separately. ### Unblock ArNS Name Gateway operators can unblock ArNS names that were previously blocked. This is done via an authenticated `PUT` request to the endpoint `/ar-io/admin/unblock-name` containing a json object with a single key: - **name**: The ArNS name to be unblocked ```bash {{title: 'curl'}} curl -X 'PUT' \ 'http://localhost:3000/ar-io/admin/unblock-name' \ -H 'accept: */*' \ -H 'Authorization: Bearer secret' \ -H 'Content-Type: application/json' \ -d '{ "name": "i-bought-a-potato", }' ``` # Environment Variables (/build/run-a-gateway/manage/environment-variables) **Default Values**: Most environment variables have sensible defaults. Only set variables when you need to override the default behavior. ## Core ar.io Node The main ar.io Gateway service that handles data retrieval, indexing, and serving. ### Server Configuration | Variable | Type | Default | Description | | --------------------- | ------- | ----------------- | ---------------------------------------- | | `PORT` | number | `4000` | HTTP server port | | `NODE_ENV` | string | `production` | Node.js environment | | `LOG_LEVEL` | string | `info` | Logging level (error, warn, info, debug) | | `LOG_FORMAT` | string | `simple` | Log format (simple, json) | | `LOG_FILTER` | string | `{"always":true}` | Log filtering configuration | | `LOG_ALL_STACKTRACES` | boolean | `false` | Include full stack traces in logs | | `INSTANCE_ID` | string | - | Unique instance identifier | ### Authentication & Security | Variable | Type | Default | Description | | -------------------- | ------ | --------- | ------------------------------------------------------- | | `ADMIN_API_KEY` | String | Generated | API key for admin endpoints (auto-generated if not set) | | `ADMIN_API_KEY_FILE` | String | - | Path to file containing admin API key | ### HTTP Message Signatures (RFC 9421) Configure [response signing](/build/run-a-gateway/manage/verification-headers#http-message-signatures-rfc-9421) to provide cryptographic proof that your gateway produced a given response. If `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` is set, the observer's Ed25519 Solana key signs responses directly. The key's Solana address is already in the on-chain Gateway Registry, so verifiers can confirm the signer with a single GAR lookup. If neither is set, the gateway auto-generates a standalone Ed25519 key at `HTTPSIG_KEY_FILE`; responses are still signed but can't be tied back to the registry. Setting both at once is rejected at startup as ambiguous. | Variable | Type | Default | Description | | ----------------------- | ------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `HTTPSIG_ENABLED` | boolean | `false` | Enable RFC 9421 response signing | | `HTTPSIG_KEY_FILE` | string | `data/keys/httpsig.pem` | Path to standalone Ed25519 private key PEM. Auto-generated on first startup if missing. Ignored when `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` is set | | `HTTPSIG_BIND_REQUEST` | boolean | `true` | Include request method and path in signature (prevents replay) | | `OBSERVER_KEYPAIR_PATH` | string | - | Path to a 64-byte Solana keypair JSON file (e.g. `solana-keygen new` output). When set, used as the HTTPSIG signing key | | `OBSERVER_PRIVATE_KEY` | string | - | Alternative to `OBSERVER_KEYPAIR_PATH`: base58-encoded 64-byte Solana secret key (the format Phantom and other browser wallets export) | ### Network Configuration | Variable | Type | Default | Description | | ------------------------------------- | ------ | ---------------------------- | ------------------------------------ | | `TRUSTED_NODE_URL` | string | `https://turbo-gateway.com` | Trusted Arweave node URL | | `TRUSTED_GATEWAY_URL` | string | `https://turbo-gateway.com` | Primary trusted gateway URL | | `TRUSTED_GATEWAYS_URLS` | JSON | `{"https://turbo-gateway.com": 1}` | Weighted trusted gateway URLs | | `TRUSTED_GATEWAYS_REQUEST_TIMEOUT_MS` | number | `10000` | Request timeout for trusted gateways | | `ARWEAVE_NODE_IGNORE_URLS` | string | - | Comma-separated URLs to ignore | ### Chunk Management | Variable | Type | Default | Description | | ---------------------------------------- | ------ | --------------------------- | ----------------------------------- | | `CHUNK_POST_URLS` | string | `https://turbo-gateway.com/chunk` | URLs for posting chunks | | `CHUNK_POST_CONCURRENCY_LIMIT` | number | `2` | Max concurrent chunk posts | | `CHUNK_POST_MIN_SUCCESS_COUNT` | number | `3` | Min successful chunk posts required | | `CHUNK_POST_RESPONSE_TIMEOUT_MS` | number | - | Chunk POST response timeout | | `CHUNK_POST_ABORT_TIMEOUT_MS` | number | - | Chunk POST abort timeout | | `SECONDARY_CHUNK_POST_URLS` | string | - | Secondary chunk POST URLs | | `SECONDARY_CHUNK_POST_CONCURRENCY_LIMIT` | number | `2` | Secondary chunk POST concurrency | | `SECONDARY_CHUNK_POST_MIN_SUCCESS_COUNT` | number | `1` | Secondary chunk POST success count | ### Data Sources | Variable | Type | Default | Description | | ---------------------------- | ------ | ------------------------------------ | -------------------------------------------- | | `ON_DEMAND_RETRIEVAL_ORDER` | string | `s3,trusted-gateways,chunks,tx-data` | On-demand data retrieval priority | | `BACKGROUND_RETRIEVAL_ORDER` | string | `chunks,s3,trusted-gateways,tx-data` | Background data retrieval priority | | `CHUNK_DATA_SOURCE_TYPE` | string | `fs` | Chunk data source type (fs, legacy-s3) | | `CHUNK_METADATA_SOURCE_TYPE` | string | `fs` | Chunk metadata source type (fs, legacy-psql) | ### CDB64 Root Transaction Index The CDB64 index provides O(1) constant-time lookups for resolving data item IDs to their root Arweave transactions. As of Release 67, a pre-built index is enabled by default covering ~964 million records. | Variable | Type | Default | Description | | ------------------------------------- | ------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `ROOT_TX_LOOKUP_ORDER` | string | `db,gateways,cdb,graphql` | Comma-separated list of root TX lookup sources. Options: db, cdb, gateways, turbo, graphql | | `CDB64_ROOT_TX_INDEX_SOURCES` | string | shipped manifest | Comma-separated list of CDB64 sources: local paths, directories, HTTP URLs, Arweave TX IDs, or bundle data items | | `CDB64_ROOT_TX_INDEX_WATCH` | boolean | `true` | Enable file watching for local CDB64 directories. New files auto-load without restart | | `CDB64_REMOTE_RETRIEVAL_ORDER` | string | `gateways,chunks` | Data sources for fetching remote CDB64 files. Options: gateways, chunks, tx-data | | `CDB64_REMOTE_CACHE_MAX_REGIONS` | number | `100` | Maximum byte-range regions to cache per remote source | | `CDB64_REMOTE_CACHE_TTL_MS` | number | `300000` | TTL for cached byte-range regions (5 minutes) | | `CDB64_REMOTE_REQUEST_TIMEOUT_MS` | number | `30000` | Request timeout for remote CDB64 sources | | `CDB64_REMOTE_MAX_CONCURRENT_REQUESTS`| number | `4` | Maximum concurrent HTTP requests across all remote CDB64 sources | | `CDB64_REMOTE_SEMAPHORE_TIMEOUT_MS` | number | `5000` | Maximum wait time for a request slot before failing | For detailed configuration and usage, see [CDB64 Root TX Index](/build/run-a-gateway/manage/cdb64). ### Indexing & Synchronization | Variable | Type | Default | Description | | -------------------------------- | ------- | ---------- | ---------------------------------- | | `START_WRITERS` | boolean | `true` | Enable indexing processes | | `START_HEIGHT` | number | `0` | Starting block height for indexing | | `STOP_HEIGHT` | number | `Infinity` | Stopping block height for indexing | | `SKIP_CACHE` | boolean | `false` | Bypass header cache | | `SIMULATED_REQUEST_FAILURE_RATE` | number | `0` | Rate of simulated request failures | ### ANS-104 Bundle Processing | Variable | Type | Default | Description | | ------------------------- | ------- | ----------------- | ------------------------------------- | | `ANS104_UNBUNDLE_FILTER` | JSON | `{"never": true}` | Filter for bundles to unbundle | | `ANS104_INDEX_FILTER` | JSON | `{"never": true}` | Filter for data items to index | | `ANS104_UNBUNDLE_WORKERS` | number | `1` | Number of unbundling workers | | `ANS104_DOWNLOAD_WORKERS` | number | `5` | Number of download workers | | `FILTER_CHANGE_REPROCESS` | boolean | `false` | Reprocess old bundles with new filter | | `BACKFILL_BUNDLE_RECORDS` | boolean | `false` | Backfill bundle records | ### Data Management | Variable | Type | Default | Description | | --------------------------------------- | ------- | -------- | ---------------------------------- | | `WRITE_ANS104_DATA_ITEM_DB_SIGNATURES` | boolean | `false` | Write data item signatures to DB | | `WRITE_TRANSACTION_DB_SIGNATURES` | boolean | `false` | Write transaction signatures to DB | | `ENABLE_DATA_DB_WAL_CLEANUP` | boolean | `false` | Enable data DB WAL cleanup | | `MAX_DATA_ITEM_QUEUE_SIZE` | number | `100000` | Max data items in queue | | `BUNDLE_DATA_IMPORTER_QUEUE_SIZE` | number | `1000` | Max bundles in import queue | | `VERIFICATION_DATA_IMPORTER_QUEUE_SIZE` | number | `1000` | Max verification items in queue | | `DATA_ITEM_FLUSH_COUNT_THRESHOLD` | number | `1000` | Data items threshold for flushing | | `MAX_FLUSH_INTERVAL_SECONDS` | number | `600` | Max interval between flushes | ### File System Cleanup | Variable | Type | Default | Description | | ------------------------------------------ | ------ | ---------- | ------------------------------------ | | `FS_CLEANUP_WORKER_BATCH_SIZE` | number | `2000` | Files per cleanup batch | | `FS_CLEANUP_WORKER_BATCH_PAUSE_DURATION` | number | `5000` | Pause between cleanup batches (ms) | | `FS_CLEANUP_WORKER_RESTART_PAUSE_DURATION` | number | `14400000` | Pause before restarting cleanup (ms) | ### Background Verification | Variable | Type | Default | Description | | ------------------------------------------------ | ------- | ------- | ----------------------------------- | | `ENABLE_BACKGROUND_DATA_VERIFICATION` | boolean | `false` | Enable background data verification | | `BACKGROUND_DATA_VERIFICATION_INTERVAL_SECONDS` | number | `600` | Verification interval | | `BACKGROUND_DATA_VERIFICATION_WORKER_COUNT` | number | `1` | Number of verification workers | | `BACKGROUND_DATA_VERIFICATION_STREAM_TIMEOUT_MS` | number | `30000` | Stream timeout for verification | ### Bundle Repair | Variable | Type | Default | Description | | -------------------------------------------------- | ------ | ------- | ----------------------------- | | `BUNDLE_REPAIR_RETRY_INTERVAL_SECONDS` | number | `300` | Bundle repair retry interval | | `BUNDLE_REPAIR_UPDATE_TIMESTAMPS_INTERVAL_SECONDS` | number | `300` | Timestamp update interval | | `BUNDLE_REPAIR_BACKFILL_INTERVAL_SECONDS` | number | `900` | Backfill interval | | `BUNDLE_REPAIR_FILTER_REPROCESS_INTERVAL_SECONDS` | number | `300` | Filter reprocess interval | | `BUNDLE_REPAIR_RETRY_BATCH_SIZE` | number | `5000` | Batch size for repair retries | ### ArNS Configuration | Variable | Type | Default | Description | | ------------------------------------ | ------ | ------------------- | ------------------------------------------------- | | `ARNS_ROOT_HOST` | string | - | Root hostname for ArNS | | `SANDBOX_PROTOCOL` | string | - | Protocol for sandboxing redirects (http or https) | | `AR_IO_SDK_LOG_LEVEL` | string | `none` | ar.io SDK log level | | `ARNS_CACHE_TYPE` | string | `node` | ArNS cache type | | `ARNS_CACHE_TTL_SECONDS` | number | `86400` | ArNS cache TTL | | `ARNS_CACHE_MAX_KEYS` | number | `10000` | Max ArNS cache keys | | `ARNS_RESOLVER_PRIORITY_ORDER` | string | `gateway,on-demand` | ArNS resolver priority | | `ARNS_COMPOSITE_RESOLVER_TIMEOUT_MS` | number | `3000` | Composite resolver timeout | | `ARNS_NAMES_CACHE_TTL_SECONDS` | number | `3600` | Names cache TTL | | `ARNS_MAX_CONCURRENT_RESOLUTIONS` | number | `1` | Max concurrent resolutions | ### ar.io The gateway talks to four Solana programs that together implement the ar.io protocol (see [protocol architecture](/learn/protocol-architecture)). Each program ID is configured independently, so the same image can run against mainnet, staging-devnet, or a local devnet. Canonical mainnet IDs are listed in the [Token](/learn/token#mainnet-addresses) docs. To confirm which set a running gateway is using, `GET /ar-io/info` returns the resolved `programIds` object. | Variable | Type | Default | Description | | -------------------------- | ------ | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------- | | `AR_IO_WALLET` | string | - | Operator Solana public key (base58). Display label surfaced on `/ar-io/info` | | `SOLANA_RPC_URL` | string | `https://api.mainnet-beta.solana.com`| Solana JSON-RPC endpoint. Public defaults throttle hard — use a premium provider (QuickNode, Helius, Triton) in production | | `SOLANA_KEYPAIR_PATH` | string | - | Path to the operator's 64-byte Solana keypair JSON file. Signs `join_network`, `update_gateway_settings`, and cranker instructions. Inside the container the path must start with `/app/wallets/` | | `SOLANA_PRIVATE_KEY` | string | - | Alternative to `SOLANA_KEYPAIR_PATH`: base58-encoded 64-byte secret (Phantom | `ENABLE_EPOCH_CRANKING` | boolean | unset (= off) | When `true`, the observer runs permissionless epoch instructions (`close_observation`, `tick_epoch`, etc.). "When unset, observer skips cranking." Set `false` to make the off-state explicit | | `ARIO_CORE_PROGRAM_ID` | string | - | `ario-core` program ID (token, staking, epoch state) | | `ARIO_GAR_PROGRAM_ID` | string | - | `ario-gar` program ID (Gateway Registry; joins, observations, distributions) | | `ARIO_ARNS_PROGRAM_ID` | string | - | `ario-arns` program ID (ArNS name registry) | | `ARIO_ANT_PROGRAM_ID` | string | - | `ario-ant` program ID (ANT records — Metaplex Core NFTs that route names to data) | | `AR_IO_NODE_RELEASE` | string | `33` | ar.io node release version | | `APEX_TX_ID` | string | - | Apex transaction ID | | `APEX_ARNS_NAME` | string | - | Apex ArNS name | | `ARNS_NOT_FOUND_TX_ID` | string | - | Not found transaction ID | | `ARNS_NOT_FOUND_ARNS_NAME` | string | `unregistered_arns` | Not found ArNS name | ### Apex Domain | Variable | Type | Default | Description | | ---------------- | ------ | ------- | ------------------- | | `APEX_TX_ID` | string | - | Apex transaction ID | | `APEX_ARNS_NAME` | string | - | Apex ArNS name | ### Caching | Variable | Type | Default | Description | | ----------------------------------------- | ------- | ------------------------ | --------------------------------------- | | `CHAIN_CACHE_TYPE` | string | `lmdb` | Chain cache type (lmdb, fs, redis) | | `REDIS_CACHE_URL` | string | `redis://localhost:6379` | Redis cache URL | | `REDIS_USE_TLS` | boolean | `false` | Use TLS for Redis | | `REDIS_CACHE_TTL_SECONDS` | number | `28800` | Redis cache TTL | | `ENABLE_FS_HEADER_CACHE_CLEANUP` | boolean | `false` | Enable FS header cache cleanup | | `CONTIGUOUS_DATA_CACHE_CLEANUP_THRESHOLD` | string | - | Contiguous data cache cleanup threshold | ### Webhooks | Variable | Type | Default | Description | | ------------------------ | ------ | ----------------- | -------------------------------------- | | `WEBHOOK_TARGET_SERVERS` | string | - | Comma-separated webhook target servers | | `WEBHOOK_INDEX_FILTER` | JSON | `{"never": true}` | Webhook index filter | | `WEBHOOK_BLOCK_FILTER` | JSON | `{"never": true}` | Webhook block filter | ### Mempool Watcher | Variable | Type | Default | Description | | ----------------------------- | ------- | ------- | ------------------------ | | `ENABLE_MEMPOOL_WATCHER` | boolean | `false` | Enable mempool watcher | | `MEMPOOL_POLLING_INTERVAL_MS` | number | `30000` | Mempool polling interval | ### AWS S3 | Variable | Type | Default | Description | | ------------------------------- | ------ | ------- | ----------------------------- | | `AWS_ACCESS_KEY_ID` | string | - | AWS access key ID | | `AWS_SECRET_ACCESS_KEY` | string | - | AWS secret access key | | `AWS_SESSION_TOKEN` | string | - | AWS session token | | `AWS_REGION` | string | - | AWS region | | `AWS_ENDPOINT` | string | - | AWS endpoint | | `AWS_S3_CONTIGUOUS_DATA_BUCKET` | string | - | S3 bucket for contiguous data | | `AWS_S3_CONTIGUOUS_DATA_PREFIX` | string | - | S3 prefix for contiguous data | ### ClickHouse | Variable | Type | Default | Description | | --------------------- | ------ | ------- | ------------------- | | `CLICKHOUSE_URL` | string | - | ClickHouse URL | | `CLICKHOUSE_USER` | string | - | ClickHouse username | | `CLICKHOUSE_PASSWORD` | string | - | ClickHouse password | ### PostgreSQL (Legacy) | Variable | Type | Default | Description | | ------------------------------------- | ------- | ------- | ----------------------------------- | | `LEGACY_PSQL_CONNECTION_STRING` | string | - | PostgreSQL connection string | | `LEGACY_PSQL_PASSWORD_FILE` | string | - | Path to PostgreSQL password file | | `LEGACY_PSQL_SSL_REJECT_UNAUTHORIZED` | boolean | `true` | Reject unauthorized SSL connections | ### Solana | Variable | Type | Default | Description | | ----------------- | ------ | ------------------------------------------ | ----------------------------- | | `SOLANA_RPC_URL` | string | `https://api.mainnet-beta.solana.com` | Solana RPC endpoint | The default public Solana RPC is rate-limited and may block `getProgramAccounts` queries needed for full registry enumeration. For production gateways, use a dedicated RPC provider such as [Helius](https://helius.dev), [Triton](https://triton.one), or [QuickNode](https://quicknode.com). ### Circuit Breaker | Variable | Type | Default | Description | | ----------------------------------------------------------------- | ------ | --------- | -------------------------- | | `ARIO_PROCESS_DEFAULT_CIRCUIT_BREAKER_TIMEOUT_MS` | number | `60000` | Circuit breaker timeout | | `ARIO_PROCESS_DEFAULT_CIRCUIT_BREAKER_ERROR_THRESHOLD_PERCENTAGE` | number | `30` | Error threshold percentage | | `ARIO_PROCESS_DEFAULT_CIRCUIT_BREAKER_ROLLING_COUNT_TIMEOUT_MS` | number | `600000` | Rolling count timeout | | `ARIO_PROCESS_DEFAULT_CIRCUIT_BREAKER_RESET_TIMEOUT_MS` | number | `1200000` | Reset timeout | ### Performance Tuning | Variable | Type | Default | Description | | ----------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | | `NODE_JS_MAX_OLD_SPACE_SIZE` | string | - | Node.js max old space size | | `WEIGHTED_PEERS_TEMPERATURE_DELTA` | number | `2` | Weighted peers temperature delta | | `GATEWAY_PEERS_WEIGHTS_CACHE_DURATION_MS` | number | `5000` | Gateway peers weights cache duration | | `GATEWAY_PEERS_REQUEST_WINDOW_COUNT` | number | `20` | Gateway peers request window count | | `TAG_SELECTIVITY` | JSON | `{"Parent-Folder-Id": 20, "Message": 20, "Drive-Id": 10, "Process": 10, "Recipient": 10, "App-Name": -10, "Content-Type": -10, "Data-Protocol": -10}` | Tag selectivity configuration | ### Data Paths | Variable | Type | Default | Description | | ---------------------- | ------ | ------------------- | ----------------------- | | `CHUNKS_DATA_PATH` | string | `./data/chunks` | Path to chunks data | | `CONTIGUOUS_DATA_PATH` | string | `./data/contiguous` | Path to contiguous data | | `HEADERS_DATA_PATH` | string | `./data/headers` | Path to headers data | | `SQLITE_DATA_PATH` | string | `./data/sqlite` | Path to SQLite data | | `DUCKDB_DATA_PATH` | string | `./data/duckdb` | Path to DuckDB data | | `TEMP_DATA_PATH` | string | `./data/tmp` | Path to temporary data | | `LMDB_DATA_PATH` | string | `./data/lmdb` | Path to LMDB data | | `PARQUET_DATA_PATH` | string | `./data/parquet` | Path to Parquet data | ### Rate Limiter | Variable | Type | Default | Description | | ----------------------------------------- | ------- | --------- | ----------------------------------- | | `ENABLE_RATE_LIMITER` | boolean | `false` | Enable rate limiting system | | `RATE_LIMITER_TYPE` | string | `memory` | Rate limiter type (memory or redis) | | `RATE_LIMITER_REDIS_ENDPOINT` | string | - | Redis endpoint for rate limiter | | `RATE_LIMITER_IP_TOKENS_PER_BUCKET` | number | `100000` | IP bucket token capacity | | `RATE_LIMITER_IP_REFILL_PER_SEC` | number | `20` | IP bucket refill rate per second | | `RATE_LIMITER_RESOURCE_TOKENS_PER_BUCKET` | number | `1000000` | Resource bucket token capacity | | `RATE_LIMITER_RESOURCE_REFILL_PER_SEC` | number | `100` | Resource bucket refill rate | | `RATE_LIMITER_IPS_AND_CIDRS_ALLOWLIST` | string | - | Comma-separated IP/CIDR allowlist | | `RATE_LIMITER_ARNS_ALLOWLIST` | string | - | Comma-separated ArNS allowlist | ### x402 Payment Protocol | Variable | Type | Default | Description | | -------------------------------------- | ------- | -------------- | ------------------------------------------ | | `ENABLE_X_402_USDC_DATA_EGRESS` | boolean | `false` | Enable x402 USDC payments | | `X_402_USDC_NETWORK` | string | - | Base network (base-sepolia or base) | | `X_402_USDC_WALLET_ADDRESS` | string | - | USDC wallet address for receiving payments | | `X_402_USDC_FACILITATOR_URL` | string | - | Payment facilitator URL | | `X_402_USDC_PER_BYTE_PRICE` | number | `0.0000000001` | Price per byte in USDC | | `X_402_USDC_DATA_EGRESS_MIN_PRICE` | number | `0.001` | Minimum payment amount | | `X_402_USDC_DATA_EGRESS_MAX_PRICE` | number | `1.00` | Maximum payment amount | | `X_402_RATE_LIMIT_CAPACITY_MULTIPLIER` | number | `10` | Paid token capacity multiplier | | `X_402_APP_NAME` | string | - | Application name for paywall | | `X_402_APP_LOGO` | string | - | Application logo URL for paywall | | `X_402_CDP_CLIENT_KEY` | string | - | **PUBLIC**: CDP client key for Onramp (optional testnet, required mainnet) | | `CDP_API_KEY_ID` | string | - | **SECRET**: CDP API key ID for Onramp (optional testnet, required mainnet) | | `CDP_API_KEY_SECRET` | string | - | **SECRET**: CDP API secret for Onramp (use CDP_API_KEY_SECRET_FILE instead) | | `CDP_API_KEY_SECRET_FILE` | string | - | **SECRET**: Path to CDP secret file for Onramp (takes precedence, recommended) | | `CHUNK_GET_BASE64_SIZE_BYTES` | number | `368640` | Fixed size for chunk pricing | ## Observer Service ### Basic Configuration | Variable | Type | Default | Description | | -------------------- | ------ | ------- | -------------------------- | | `PORT` | number | `5050` | Observer service port | | `LOG_LEVEL` | string | - | Observer log level | | `OBSERVER_WALLET` | string | - | Observer wallet | | `IO_PROCESS_ID` | string | - | ar.io Solana program address | | `AR_IO_NODE_RELEASE` | string | `33` | ar.io node release version | ### Observer Operation | Variable | Type | Default | Description | | ------------------------------------- | ------- | ------- | ------------------------------------------ | | `SUBMIT_CONTRACT_INTERACTIONS` | boolean | `true` | Submit observations to Solana programs | | `NUM_ARNS_NAMES_TO_OBSERVE_PER_GROUP` | number | `8` | Number of ArNS names per observation group | | `REPORT_GENERATION_INTERVAL_MS` | string | - | Report generation interval | | `REPORT_DATA_SINK` | string | - | Report data sink | | `TURBO_UPLOAD_SERVICE_URL` | string | - | Turbo upload service URL | | `RUN_OBSERVER` | boolean | `true` | Run observer service | | `MIN_RELEASE_NUMBER` | number | `0` | Minimum release number | ### Report Configuration | Variable | Type | Default | Description | | ------------------------------- | ------ | ------- | -------------------------- | | `REPORT_GENERATION_INTERVAL_MS` | string | - | Report generation interval | | `REPORT_DATA_SINK` | string | - | Report data sink | ### Gateway Assessment | Variable | Type | Default | Description | | ------------------------------------- | ------ | ------- | ------------------------------------------ | | `NUM_ARNS_NAMES_TO_OBSERVE_PER_GROUP` | number | `8` | Number of ArNS names per observation group | ### ArNS Names | Variable | Type | Default | Description | | ------------------------------------- | ------ | ------- | ------------------------------------------ | | `NUM_ARNS_NAMES_TO_OBSERVE_PER_GROUP` | number | `8` | Number of ArNS names per observation group | ### On-Chain Observations | Variable | Type | Default | Description | | ------------------------------ | ------- | ------- | ------------------------------------- | | `SUBMIT_CONTRACT_INTERACTIONS` | boolean | `true` | Submit observations to Solana programs. Pre-flight no-ops unless your pubkey is in `epoch.prescribed_observers` — harmless to leave at default before `join_network` | ### Upload Wallet Identities The observer uploads report bundles to Turbo. The upload signer is resolved from the first matching env in the [precedence chain](/build/run-a-gateway/manage/solana-migration#upload-signing-precedence). Setting envs from more than one chain group at once is rejected at startup. If your observer logs warn that `TurboReportSink` is not configured, explicitly set a Solana upload signer. Most operators can use the same base58 secret for both `OBSERVER_PRIVATE_KEY` and `SOLANA_UPLOAD_PRIVATE_KEY`. | Variable | Type | Default | Description | | --------------------------------- | ------ | ------- | --------------------------------------------------------------------------- | | `ARWEAVE_UPLOAD_KEY_FILE` | string | - | Path to an Arweave JWK file. Highest priority for upload signing | | `ARWEAVE_UPLOAD_JWK` | string | - | Inline Arweave JWK JSON. Lower priority than the file form | | `ETHEREUM_UPLOAD_PRIVATE_KEY_FILE`| string | - | Path to a 32-byte hex private key (with or without `0x` prefix) | | `ETHEREUM_UPLOAD_PRIVATE_KEY` | string | - | Inline hex private key. Lower priority than the file form | | `SOLANA_UPLOAD_KEYPAIR_PATH` | string | - | Path to a separate Solana keypair JSON for uploads. Ignored when any `ARWEAVE_UPLOAD_*` or `ETHEREUM_UPLOAD_*` is set | | `SOLANA_UPLOAD_PRIVATE_KEY` | string | - | Alternative to above: base58 secret. Mutually exclusive with the file form | When none of the above are set, uploads fall back to the observer key, then the operator key. For production observers, prefer setting `SOLANA_UPLOAD_KEYPAIR_PATH` or `SOLANA_UPLOAD_PRIVATE_KEY` explicitly so report uploads do not depend on fallback behavior. ### Offset Observation | Variable | Type | Default | Description | | ------------------------------- | ------ | ------- | -------------------------- | | `REPORT_GENERATION_INTERVAL_MS` | string | - | Report generation interval | ### Data Paths | Variable | Type | Default | Description | | ------------------- | ------ | ---------------- | ---------------------- | | `TEMP_DATA_PATH` | string | `./data/tmp` | Path to temporary data | | `REPORTS_DATA_PATH` | string | `./data/reports` | Path to reports data | | `WALLETS_PATH` | string | `./wallets` | Path to wallets | ## Envoy Proxy ### Basic Configuration | Variable | Type | Default | Description | | --------------------- | ------ | ------------- | --------------- | | `LOG_LEVEL` | string | `info` | Envoy log level | | `TVAL_AR_IO_HOST` | string | `core` | ar.io host | | `TVAL_AR_IO_PORT` | number | `4000` | ar.io port | | `TVAL_OBSERVER_HOST` | string | `observer` | Observer host | | `TVAL_OBSERVER_PORT` | number | `5050` | Observer port | | `TVAL_GATEWAY_HOST` | string | `turbo-gateway.com` | Gateway host | | `TVAL_GRAPHQL_HOST` | string | `core` | GraphQL host | | `TVAL_GRAPHQL_PORT` | number | `4000` | GraphQL port | | `TVAL_ARNS_ROOT_HOST` | string | - | ArNS root host | ## Redis Cache ### Basic Configuration | Variable | Type | Default | Description | | ------------------- | ------ | --------------------------- | ----------------- | | `REDIS_IMAGE_TAG` | string | `7` | Redis image tag | | `REDIS_MAX_MEMORY` | string | `256mb` | Redis max memory | | `EXTRA_REDIS_FLAGS` | string | `--save "" --appendonly no` | Extra Redis flags | ### Data Paths | Variable | Type | Default | Description | | ----------------- | ------ | -------------- | ------------------ | | `REDIS_DATA_PATH` | string | `./data/redis` | Path to Redis data | ## ClickHouse ### Basic Configuration | Variable | Type | Default | Description | | ---------------------- | ------ | ------- | -------------------- | | `CLICKHOUSE_IMAGE_TAG` | string | `25.4` | ClickHouse image tag | | `CLICKHOUSE_USER` | string | - | ClickHouse username | | `CLICKHOUSE_PASSWORD` | string | - | ClickHouse password | ### Data Paths | Variable | Type | Default | Description | | ---------------------- | ------ | ------------------- | ----------------------- | | `CLICKHOUSE_DATA_PATH` | string | `./data/clickhouse` | Path to ClickHouse data | | `CLICKHOUSE_LOGS_PATH` | string | `./logs/clickhouse` | Path to ClickHouse logs | ### ClickHouse Auto-Import | Variable | Type | Default | Description | | ------------------------------------------ | ------ | ------------------------------------------ | ------------------------------------------- | | `CLICKHOUSE_AUTO_IMPORT_IMAGE_TAG` | string | `79792e1b549f64edad3e338769949fd9bffa62db` | ClickHouse auto-import image tag | | `CLICKHOUSE_DEBUG` | string | - | ClickHouse debug flag | | `AR_IO_HOST` | string | `core` | ar.io host | | `AR_IO_PORT` | number | `4000` | ar.io port | | `ADMIN_API_KEY` | string | - | Admin API key | | `PARQUET_DATA_PATH` | string | `./data/parquet` | Path to Parquet data | | `CLICKHOUSE_HOST` | string | `clickhouse` | ClickHouse host | | `CLICKHOUSE_PORT` | string | - | ClickHouse port (defaults to 9000) | | `CLICKHOUSE_USER` | string | - | ClickHouse username (defaults to 'default') | | `CLICKHOUSE_PASSWORD` | string | - | ClickHouse password (required) | | `CLICKHOUSE_AUTO_IMPORT_SLEEP_INTERVAL` | string | - | Auto-import sleep interval | | `CLICKHOUSE_AUTO_IMPORT_HEIGHT_INTERVAL` | string | - | Auto-import height interval | | `CLICKHOUSE_AUTO_IMPORT_MAX_ROWS_PER_FILE` | string | - | Max rows per file for auto-import | ## Litestream Backup ### S3 Configuration | Variable | Type | Default | Description | | ------------------------------------------ | ------ | ------------------------------------------ | ----------------------------------- | | `LITESTREAM_IMAGE_TAG` | string | `be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8` | Litestream image tag | | `AR_IO_SQLITE_BACKUP_S3_BUCKET_NAME` | string | - | S3 bucket name for SQLite backups | | `AR_IO_SQLITE_BACKUP_S3_BUCKET_REGION` | string | - | S3 bucket region for SQLite backups | | `AR_IO_SQLITE_BACKUP_S3_BUCKET_ACCESS_KEY` | string | - | S3 access key for SQLite backups | | `AR_IO_SQLITE_BACKUP_S3_BUCKET_SECRET_KEY` | string | - | S3 secret key for SQLite backups | | `AR_IO_SQLITE_BACKUP_S3_BUCKET_PREFIX` | string | - | S3 prefix for SQLite backups | ### Data Paths | Variable | Type | Default | Description | | ------------------ | ------ | --------------- | ------------------- | | `SQLITE_DATA_PATH` | string | `./data/sqlite` | Path to SQLite data | ## Autoheal Service ### Configuration | Variable | Type | Default | Description | | ------------------------------- | ------- | ---------- | ------------------------------- | | `AUTOHEAL_CONTAINER_LABEL` | string | `autoheal` | Container label for autoheal | | `AUTOHEAL_ONLY_MONITOR_RUNNING` | boolean | `false` | Only monitor running containers | | `RUN_AUTOHEAL` | boolean | `false` | Enable autoheal service | ## OpenTelemetry Tracing ### Basic Configuration | Variable | Type | Default | Description | | --------------------------------- | ------ | ------------ | ---------------------------------- | | `OTEL_SERVICE_NAME` | string | `ar-io-node` | OpenTelemetry service name | | `OTEL_EXPORTER_OTLP_ENDPOINT` | string | - | OTLP exporter endpoint | | `OTEL_EXPORTER_OTLP_HEADERS` | string | - | OTLP exporter headers | | `OTEL_EXPORTER_OTLP_HEADERS_FILE` | string | - | Path to OTLP exporter headers file | ### Performance Tuning | Variable | Type | Default | Description | | ------------------------------------------------ | ------ | ------- | ----------------------------------- | | `OTEL_BATCH_LOG_PROCESSOR_SCHEDULED_DELAY_MS` | number | `5000` | Batch log processor scheduled delay | | `OTEL_BATCH_LOG_PROCESSOR_MAX_EXPORT_BATCH_SIZE` | number | `512` | Max | `OTEL_TRACING_SAMPLING_RATE_DENOMINATOR` | number | `1000` | Tracing sampling rate denominator | ## Image Tags ### Service Images | Variable | Type | Default | Description | | ---------------------------------- | ------ | ------------------------------------------ | -------------------------------- | | `ENVOY_IMAGE_TAG` | string | `4789af164fcd3029a65a1d6739f2d9026567206e` | Envoy image tag | | `CORE_IMAGE_TAG` | string | `3a793c6ee06f5e1df56920fc70184b213ceb8c6e` | Core image tag | | `OBSERVER_IMAGE_TAG` | string | `e5f6ae36fd6eea04be5ebba2624f8ecc08db4ea0` | Observer image tag | | `LITESTREAM_IMAGE_TAG` | string | `be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8` | Litestream image tag | | `CLICKHOUSE_AUTO_IMPORT_IMAGE_TAG` | string | `79792e1b549f64edad3e338769949fd9bffa62db` | ClickHouse auto-import image tag | ## Additional Paths ### Data Directories | Variable | Type | Default | Description | | ---------------------- | ------ | ------------------- | ----------------------- | | `CHUNKS_DATA_PATH` | string | `./data/chunks` | Path to chunks data | | `CONTIGUOUS_DATA_PATH` | string | `./data/contiguous` | Path to contiguous data | | `HEADERS_DATA_PATH` | string | `./data/headers` | Path to headers data | | `SQLITE_DATA_PATH` | string | `./data/sqlite` | Path to SQLite data | | `DUCKDB_DATA_PATH` | string | `./data/duckdb` | Path to DuckDB data | | `TEMP_DATA_PATH` | string | `./data/tmp` | Path to temporary data | | `LMDB_DATA_PATH` | string | `./data/lmdb` | Path to LMDB data | | `PARQUET_DATA_PATH` | string | `./data/parquet` | Path to Parquet data | | `REDIS_DATA_PATH` | string | `./data/redis` | Path to Redis data | | `CLICKHOUSE_DATA_PATH` | string | `./data/clickhouse` | Path to ClickHouse data | | `CLICKHOUSE_LOGS_PATH` | string | `./logs/clickhouse` | Path to ClickHouse logs | | `REPORTS_DATA_PATH` | string | `./data/reports` | Path to reports data | | `WALLETS_PATH` | string | `./wallets` | Path to wallets | ## Usage Notes - All environment variables are optional unless otherwise specified - Default values are shown in the "Default" column - Boolean values should be set to `true` or `false` - JSON values should be valid JSON strings - Path values should be absolute or relative to the project root - Some variables are only used in specific deployment scenarios (e.g., ClickHouse, Litestream) - Image tags can be updated to use different versions of the services - Data paths can be customized based on your storage requirements ## Configuration Examples ### Basic Gateway Setup ```bash # Core configuration NODE_ENV=production LOG_LEVEL=info PORT=4000 ADMIN_API_KEY=your-admin-key-here # Network configuration TRUSTED_NODE_URL=https://turbo-gateway.com TRUSTED_GATEWAY_URL=https://turbo-gateway.com # Data paths CHUNKS_DATA_PATH=/data/chunks CONTIGUOUS_DATA_PATH=/data/contiguous SQLITE_DATA_PATH=/data/sqlite ``` ### Advanced Gateway with ClickHouse ```bash # Core configuration NODE_ENV=production LOG_LEVEL=info PORT=4000 ADMIN_API_KEY=your-admin-key-here # ClickHouse configuration CLICKHOUSE_URL=http://clickhouse:8123 CLICKHOUSE_USER=default CLICKHOUSE_PASSWORD=your-password # Bundle processing ANS104_UNBUNDLE_FILTER={"and": [{"equals": {"App-Name": "MyApp-v1.0"}}]} ANS104_INDEX_FILTER={"and": [{"equals": {"App-Name": "MyApp-v1.0"}}]} ANS104_UNBUNDLE_WORKERS=2 ANS104_DOWNLOAD_WORKERS=5 ``` ### Gateway with Redis Caching ```bash # Core configuration NODE_ENV=production LOG_LEVEL=info PORT=4000 ADMIN_API_KEY=your-admin-key-here # Redis configuration CHAIN_CACHE_TYPE=redis REDIS_CACHE_URL=redis://redis:6379 REDIS_USE_TLS=false REDIS_CACHE_TTL_SECONDS=28800 # ArNS configuration ARNS_ROOT_HOST=your-domain.com ARNS_CACHE_TYPE=redis ``` This comprehensive reference should help you configure your ar.io Gateway with the appropriate environment variables for your specific use case. # Gateway Filters (/build/run-a-gateway/manage/filters) Configure your ar.io Gateway to efficiently process and index only the data you need. This comprehensive guide covers advanced filtering techniques, performance optimization, and real-world use cases. ## Overview The ar.io Gateway uses a flexible JSON-based filtering system to control data processing and indexing. The system provides precise control over which bundles are processed and which data items are indexed for querying. ## Understanding the Filtering System The ar.io Gateway uses two primary filters to control data processing: 1. **ANS104_UNBUNDLE_FILTER** - Controls which bundles are processed and unbundled 2. **ANS104_INDEX_FILTER** - Controls which data items from unbundled bundles are indexed for querying By default, gateways process no bundles and index no data items. You must explicitly configure filters to start processing data. ## Core Environment Variables ### Configure Data Management Optimize data storage and processing: ```bash # Number of new data items before flushing to stable storage DATA_ITEM_FLUSH_COUNT_THRESHOLD=1000 # Maximum time between flushes (in seconds) MAX_FLUSH_INTERVAL_SECONDS=600 # Maximum number of data items to queue for indexing MAX_DATA_ITEM_QUEUE_SIZE=100000 # Enable background verification ENABLE_BACKGROUND_DATA_VERIFICATION=true ``` ### Set Up GraphQL Configuration Choose between local-only or proxied queries: ```bash # For new gateways - proxy to turbo-gateway.com for complete index GRAPHQL_HOST=turbo-gateway.com GRAPHQL_PORT=443 # For local-only queries (uncomment to use) # GRAPHQL_HOST= ``` ## Filter Construction While the filters below are displayed on multiple lines for readability, they must be stored in the `.env` file as a single line for proper processing. ### Basic Filters The simplest filters you can use are `"always"` and `"never"` filters. The `"never"` filter is the default behavior and will match nothing, while the `"always"` filter matches everything. ```json {{title: "Never Match"}} { "never": true //default behavior } ``` ```json {{title: "Always Match"}} { "always": true } ``` ### Tag Filters Tag filters allow you to match items based on their tags in three different ways. You can match exact tag values, check for the presence of a tag regardless of its value, or match tags whose values start with specific text. All tag values are automatically base64url-decoded before matching. ```json {{title: "Exact Match"}} { "tags": [ { "name": "Content-Type", "value": "image/jpeg" } ] } ``` ```json {{title: "Match Tag Name Only"}} { "tags": [ { "name": "App-Name" } ] } ``` ```json {{title: "Starts With Match"}} { "tags": [ { "name": "Protocol", "valueStartsWith": "AO" } ] } ``` ### Attribute Filters Attribute filtering allows you to match items based on their metadata properties. The system automatically handles owner public key to address conversion, making it easy to filter by owner address. You can combine multiple attributes in a single filter: ```json {{title: "Basic Attributes"}} { "attributes": { "owner_address": "xyz123...", "data_size": 1000 } } ``` ### Nested Bundle Filter The `isNestedBundle` filter is a specialized filter that checks whether a data item is part of a nested bundle structure. It's particularly useful when you need to identify or process data items in bundles that are contained within other bundles. ```json {{title: "Basic Nested Bundle"}} { "isNestedBundle": true } ``` **Note**: When processing nested bundles, be sure to include filters that match the nested bundles in both `ANS104_UNBUNDLE_FILTER` and `ANS104_INDEX_FILTER`. The bundle data items (nested bundles) need to be indexed to be matched by the unbundle filter. ### Complex Filters Using Logical Operators For more complex scenarios, the system provides logical operators (AND, OR, NOT) that can be combined to create sophisticated filtering patterns. These operators can be nested to any depth: ```json {{title: "AND Operation"}} { "and": [ { "tags": [ { "name": "App-Name", "value": "ArDrive-App" } ] }, { "tags": [ { "name": "Content-Type", "valueStartsWith": "image/" } ] } ] } ``` ```json {{title: "OR Operation"}} { "or": [ { "tags": [ { "name": "App-Name", "value": "ArDrive-App" } ] }, { "attributes": { "data_size": 1000 } } ] } ``` ```json {{title: "NOT Operation"}} { "not": { "tags": [ { "name": "Content-Type", "value": "application/json" } ] } } ``` ## Filter Configuration Strategies ### Process Everything ```json { "always": true } ``` ### Process Nothing (Default) ```json { "never": true } ``` ### Process Specific App Data ```json { "tags": [ { "name": "App-Name", "valueStartsWith": "MyApp" } ] } ``` ### Single Application ```json { "tags": [ { "name": "App-Name", "value": "MyApp-v1.0" } ] } ``` ### Multiple Applications ```json { "or": [ { "tags": [ { "name": "App-Name", "value": "MyApp-v1.0" } ] }, { "tags": [ { "name": "App-Name", "value": "AnotherApp-v2.1" } ] } ] } ``` ### Application with Version Range ```json { "tags": [ { "name": "App-Name", "valueStartsWith": "MyApp" } ] } ``` ### Content Type Filtering ```json { "tags": [ { "name": "Content-Type", "valueStartsWith": "image/" } ] } ``` ### Specific File Types ```json { "or": [ { "tags": [ { "name": "Content-Type", "value": "application/json" } ] }, { "tags": [ { "name": "Content-Type", "value": "text/plain" } ] } ] } ``` ### File Size Filtering ```json { "attributes": { "data_size": 1000000 } } ``` ### Single Owner ```json { "attributes": { "owner_address": "YOUR_WALLET_ADDRESS" } } ``` ### Multiple Owners ```json { "or": [ { "attributes": { "owner_address": "WALLET_ADDRESS_1" } }, { "attributes": { "owner_address": "WALLET_ADDRESS_2" } } ] } ``` ### Exclude Specific Owners ```json { "not": { "attributes": { "owner_address": "UNWANTED_WALLET_ADDRESS" } } } ``` ### Complex Multi-Condition Filter ```json { "and": [ { "tags": [ { "name": "App-Name", "valueStartsWith": "MyApp" } ] }, { "attributes": { "owner_address": "YOUR_WALLET_ADDRESS" } }, { "not": { "tags": [ { "name": "Content-Type", "value": "application/octet-stream" } ] } } ] } ``` ### Exclude Common Bundlers ```json { "and": [ { "not": { "or": [ { "tags": [ { "name": "Bundler-App-Name", "value": "Warp" } ] }, { "tags": [ { "name": "Bundler-App-Name", "value": "Redstone" } ] }, { "attributes": { "owner_address": "-OXcT1sVRSA5eGwt2k6Yuz8-3e3g9WJi5uSE99CWqsBs" } } ] } }, { "tags": [ { "name": "App-Name", "valueStartsWith": "MyApp" } ] } ] } ``` ## Real-World Use Cases ### Personal Data Gateway Perfect for individuals who want to process only their own data: **Unbundle Filter:** ```json { "and": [ { "not": { "or": [ { "tags": [ { "name": "Bundler-App-Name", "value": "Warp" } ] }, { "tags": [ { "name": "Bundler-App-Name", "value": "Redstone" } ] } ] } }, { "tags": [ { "name": "App-Name", "valueStartsWith": "MyApp" } ] } ] } ``` **Index Filter:** ```json { "attributes": { "owner_address": "YOUR_WALLET_ADDRESS" } } ``` ### Application-Specific Service Ideal for building services around specific applications: **Unbundle Filter:** ```json { "tags": [ { "name": "App-Name", "valueStartsWith": "MyApp" } ] } ``` **Index Filter:** ```json { "or": [ { "tags": [ { "name": "ArFS", "value": "0.10" } ] }, { "tags": [ { "name": "ArFS", "value": "0.11" } ] }, { "tags": [ { "name": "ArFS", "value": "0.12" } ] } ] } ``` ### Content-Type Focused Gateway For gateways specializing in specific content types: **Unbundle Filter:** ```json { "tags": [ { "name": "Content-Type", "valueStartsWith": "image/" } ] } ``` **Index Filter:** ```json { "and": [ { "tags": [ { "name": "Content-Type", "valueStartsWith": "image/" } ] }, { "attributes": { "data_size": 100000 } } ] } ``` ## Performance Optimization ### Worker Configuration ### Understanding Default Worker Settings The gateway uses sensible defaults that work well for most users: ```bash # Default values (no need to set unless customizing) # ANS104_UNBUNDLE_WORKERS=1 (default: 0, or 1 if filters are set) # ANS104_DOWNLOAD_WORKERS=5 (default: 5) # Only adjust if you have specific hardware requirements # or want to optimize for your system's capabilities ``` **When to Adjust Workers:** Only modify worker counts if you have high-performance hardware and want to maximize throughput, or if you're experiencing resource constraints and need to reduce load. ### Optimize Data Flushing Balance between memory usage and database performance: ```bash # For high-memory systems, increase threshold DATA_ITEM_FLUSH_COUNT_THRESHOLD=2000 # For low-memory systems, decrease threshold DATA_ITEM_FLUSH_COUNT_THRESHOLD=500 # Adjust flush interval based on data volume MAX_FLUSH_INTERVAL_SECONDS=300 ``` ### Enable Background Processing ```bash # Enable background verification ENABLE_BACKGROUND_DATA_VERIFICATION=true # Enable WAL cleanup for better performance ENABLE_DATA_DB_WAL_CLEANUP=true ``` ## Webhook Filters There are also two filters available that are used to trigger webhooks. When a transaction is processed that matches one of the webhook filters, the gateway will send a webhook to the specified `WEBHOOK_TARGET_SERVERS` urls containing the transaction data. ```bash WEBHOOK_INDEX_FILTER="" WEBHOOK_BLOCK_FILTER="" ``` The `WEBHOOK_INDEX_FILTER` is used to trigger a webhook when a transaction is indexed. The `WEBHOOK_BLOCK_FILTER` is used to trigger a webhook when a block is processed. ## Important Notes - All tag names and values are base64url-decoded before matching - Owner addresses are automatically converted from owner public keys - Empty or undefined filters default to "never match" - Tag matching requires all specified tags to match - Attribute matching requires all specified attributes to match - The filter system supports nested logical operations to any depth, allowing for very precise control over what data gets processed ## Best Practices ### Filter Design 1. **Start Simple** - Begin with basic filters and gradually add complexity 2. **Test Thoroughly** - Use `FILTER_CHANGE_REPROCESS=true` when changing filters 3. **Monitor Performance** - Watch system resources during processing 4. **Document Changes** - Keep track of filter modifications and their effects ### Maintenance 1. **Regular Monitoring** - Check gateway logs for errors and warnings 2. **Resource Cleanup** - Periodically clean up old data and logs 3. **Filter Optimization** - Refine filters based on actual data patterns 4. **Backup Configuration** - Keep copies of working filter configurations ### Troubleshooting If your gateway stops processing data after changing filters, check: - Filter syntax is valid JSON - Required environment variables are set - Gateway has been restarted after changes - System has sufficient resources ## Next Steps Now that you understand gateway filtering, continue building your infrastructure: } title="Set Up Monitoring" description="Deploy Grafana to visualize your gateway's performance metrics" href="/build/extensions/grafana" /> } title="Add ClickHouse" description="Improve query performance with ClickHouse and Parquet integration" href="/build/extensions/clickhouse" /> } title="Deploy Bundler" description="Accept data uploads directly through your gateway" href="/build/extensions/bundler" /> # Importing SQLite Database Snapshots (/build/run-a-gateway/manage/index-snapshots) ## Overview One of the challenges of running an ar.io Gateway is the initial synchronization time as your gateway builds its local index of the Arweave network. This process can take days or even weeks, depending on your hardware and the amount of data you want to index. To accelerate this process, you can import a pre-synchronized SQLite database snapshot that contains transaction and data item records already indexed. This guide will walk you through the process of importing a database snapshot into your ar.io Gateway. The below instructions are designed to be used in a linux environment. Windows and MacOS users must modify the instructions to use the appropriate package manager/ command syntax for their platform. Unless otherwise specified, all commands should be run from the root directory of the gateway. ## Quick Start ### Download Database Snapshot Download the latest database snapshot using BitTorrent: ```bash transmission-cli "magnet:?xt=urn:btih:62ca6e05248e6df59fac9e38252e9c71951294ed&dn=2025-04-23-sqlite.tar.gz&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=http%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Fp4p.arenabg.com%3A1337%2Fannounce&tr=https%3A%2F%2Ftracker.bt4g.com%3A443%2Fannounce" ``` This downloads a 42.8GB snapshot current to April 23, 2025. ### Extract the Snapshot Extract the downloaded tarball: ```bash tar -xzf 2025-04-23-sqlite.tar.gz ``` This creates a directory with the extracted database files. ### Import the Snapshot Replace your existing database with the snapshot: ```bash # Stop the gateway docker compose down # Backup existing database (optional) mkdir sqlite-backup mv data/sqlite/* sqlite-backup/ # Remove old database rm data/sqlite/* # Import new snapshot mv 2025-04-23-sqlite/* data/sqlite/ # Start the gateway docker compose up -d ``` ## Detailed Instructions ### Obtaining a Database Snapshot SQLite database snapshots are very large and not easy to incrementally update. For these reasons, ar.io is distributing them using BitTorrent. ### Install Torrent Client Install a BitTorrent client. We recommend [transmission-cli](https://github.com/transmission/transmission): ```bash # Ubuntu/Debian sudo apt-get install transmission-cli # CentOS/RHEL sudo yum install transmission-cli # macOS brew install transmission-cli ``` ### Download Snapshot Download the latest snapshot using the magnet link: ```bash transmission-cli "magnet:?xt=urn:btih:62ca6e05248e6df59fac9e38252e9c71951294ed&dn=2025-04-23-sqlite.tar.gz&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=http%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Fp4p.arenabg.com%3A1337%2Fannounce&tr=https%3A%2F%2Ftracker.bt4g.com%3A443%2Fannounce" ``` This will download a snapshot, current to April 23, 2025, of an unbundled data set that includes all data items uploaded via an ArDrive product, including Turbo. The file will be named `2025-04-23-sqlite.tar.gz` and be approximately 42.8GB in size. ### Consider Seeding While continuing to seed the torrent after download is not required, it is highly recommended to help ensure the continued availability of the snapshot for others, as well as the integrity of the data. Seeding this file should not cause any issues with your internet service provider. ### Extracting the Database Snapshot Once the file has downloaded, you can extract it using the following command. ### Verify Download Check that the file downloaded completely: ```bash ls -lh 2025-04-23-sqlite.tar.gz # Should show approximately 42.8GB ``` ### Extract the Archive Extract the tarball: ```bash tar -xzf 2025-04-23-sqlite.tar.gz ``` Be sure to replace the filename with the actual filename of the snapshot you are using, if not using the example above. ### Verify Extraction Check that the extraction was successful: ```bash ls -la 2025-04-23-sqlite/ # Should show SQLite database files ``` This will extract the file into a directory matching the filename, minus the `.tar.gz` extension. ### Importing the Database Snapshot Once you have an extracted database snapshot, you can import it into your ar.io gateway by replacing the existing SQLite database files. Importing a database snapshot will delete your existing database and replace it with the snapshot you are importing. ### Stop the Gateway Stop your ar.io gateway: ```bash docker compose down ``` ### Backup Existing Database (Optional) Backup your existing SQLite database files: ```bash mkdir sqlite-backup mv data/sqlite/* sqlite-backup/ ``` ### Remove Old Database Delete the existing SQLite database files: ```bash rm data/sqlite/* ``` ### Import New Snapshot Move the snapshot files into the `data/sqlite` directory: ```bash mv 2025-04-23-sqlite/* data/sqlite/ ``` Be sure to replace `2025-04-23-sqlite` with the actual directory name of the extracted snapshot you are using. ### Start the Gateway Start your ar.io gateway: ```bash docker compose up -d ``` ### Verifying the Import The simplest way to verify the import is to check the gateway logs to see what block number is being imported. ### Check Gateway Logs View the gateway logs to see the current block height: ```bash docker compose logs -f gateway ``` Look for messages indicating the current block being processed. ### Verify Block Height The 2025-04-23 snapshot was taken at block `1645229`, so the gateway will start importing blocks after this height if the snapshot was imported successfully. You should see logs showing blocks being processed starting from block 1645230 or higher. ### Use Grafana (Optional) You can also use the [Grafana Extension](/build/extensions/grafana) to view the last block imported in a more human readable format. # Manage your Gateway (/build/run-a-gateway/manage) import { Database, RotateCcw, TrendingUp, Zap, Globe, Shield, Settings, Wrench, CreditCard, Search, Bot, } from "lucide-react"; Master the advanced features and configurations of your ar.io Gateway. These comprehensive guides cover everything from performance optimization to content moderation, helping you run a professional-grade gateway infrastructure. ## Gateway Management }> Learn how to import pre-synchronized database snapshots to quickly bootstrap your gateway and reduce initial sync time from weeks to hours. }> Step-by-step guide to safely upgrade your ar.io Gateway to the latest version without losing data or progress. ## Monitoring & Analytics } > Deploy and configure Grafana for comprehensive gateway monitoring, performance analytics, and operational insights. ## Performance Optimization }> Configure advanced filters to efficiently process and index only the data you need, optimizing performance and resource usage. }> Configure the CDB64 index for O(1) data item lookups. Enabled by default with ~964 million records for instant historical data resolution. } > Customize your gateway's root domain to serve custom content, project information, or documentation instead of default network info. }> Configure x402 payment protocol to monetize data egress with frictionless cryptocurrency payments and premium access tiers. ## Content Management } > Implement content moderation policies using blocklisting and filtering to control what content your gateway serves. ## Configuration Reference } > Comprehensive reference for all ar.io Gateway environment variables organized by service component. ## AI-Assisted Operations } > The `ar-io-node` repo ships a Claude Code skill at `.claude/skills/ar-io-gateway-operator/` with a one-screen `scripts/health-check` snapshot and an operational runbook covering the ANS-104 pipeline, ClickHouse, ArNS resolution, observer/cranker, and common pitfalls. Claude Code auto-loads it when run from the repo root. Readable as plain Markdown if you don't use Claude Code. ## Support & Troubleshooting } > Comprehensive troubleshooting guide and FAQ for common gateway issues, failed epoch guidance, and frequently asked questions. # Advanced NGINX Caching (/build/run-a-gateway/manage/nginx-caching) The [quick-start guide](/build/run-a-gateway/quick-start) covers basic NGINX reverse proxy setup for SSL termination and header forwarding. This guide covers adding a caching layer in front of your ar.io node for high-traffic gateways, based on production configurations running at scale. NGINX caching is optional. The ar.io node has its own internal caching and serves data correctly without it. Add NGINX caching when you need to reduce load on the node process or serve high volumes of traffic for popular content. ## Prerequisites - A running ar.io gateway with NGINX already configured as a reverse proxy (see the [quick-start guide](/build/run-a-gateway/quick-start)) - Root or sudo access on the host running NGINX If you are running the default Docker Compose setup, NGINX runs on the host and proxies to **envoy** on port 3000, which in turn proxies to the ar.io node core on port 4000. The `proxy_pass http://127.0.0.1:3000` directives in this guide target envoy, which is the correct entry point. The cache directories described below are on the host filesystem. If you already have an NGINX configuration (e.g., `/etc/nginx/sites-available/default` from the quick-start), you will be adding cache directives to it. The `http`-block directives (cache zones, maps) go outside your existing `server` block, and the location blocks replace or extend the ones in your existing `server` block. ## Why Cache at NGINX Arweave data is immutable - once a transaction is confirmed, its content never changes. This makes it an ideal candidate for aggressive edge caching: - **Reduce node load** - Serve repeated requests for the same content directly from NGINX's disk cache without hitting the node process. - **Thundering herd protection** - When many clients request the same uncached content simultaneously, NGINX ensures only one request reaches the node while others wait for the cached result. - **Stale serving during failures** - If the node is temporarily unavailable, NGINX can serve stale cached content rather than returning errors. - **Faster response times** - Cached responses skip the node entirely, reducing latency. **Rate limiter interaction**: Once NGINX caches a significant portion of traffic, fewer requests reach the ar.io node's rate limiter. This effectively increases per-IP limits for cached content. This is generally beneficial but means abusive clients can hammer cached endpoints without triggering rate limits. Consider this when tuning rate limit values. ## http Block Configuration All `proxy_cache_path` and `map` directives must be placed in the `http` block of your NGINX config, **outside** the `server` block. On Debian/Ubuntu, files in `/etc/nginx/sites-enabled/` are included inside the `http` block via `nginx.conf`, so you can place these directives at the top of your site config file (before the `server` block). ### Create Cache Directories Create the directories and set ownership before reloading NGINX. NGINX will fail to start if these directories don't exist. ```bash sudo mkdir -p /var/lib/nginx/cache/api sudo mkdir -p /var/lib/nginx/cache/block sudo mkdir -p /var/lib/nginx/cache/tx-and-chunk sudo mkdir -p /var/lib/nginx/cache/data sudo mkdir -p /var/lib/nginx/cache/arns sudo chown -R www-data:www-data /var/lib/nginx/cache ``` ```bash sudo mkdir -p /var/lib/nginx/cache/api sudo mkdir -p /var/lib/nginx/cache/block sudo mkdir -p /var/lib/nginx/cache/tx-and-chunk sudo mkdir -p /var/lib/nginx/cache/data sudo mkdir -p /var/lib/nginx/cache/arns sudo chown -R nginx:nginx /var/lib/nginx/cache ``` If your cache directories are on a different filesystem than NGINX's temp path, cached file writes will use slow cross-filesystem copies instead of fast renames. For best performance, keep cache directories on the same filesystem as NGINX's default temp path. Also avoid placing the cache on the same disk as your ar.io node's SQLite/ClickHouse databases if I/O is constrained. ### Tiered Cache Zones Different types of gateway content have different caching characteristics. A tiered approach uses separate cache zones sized and configured for each content type: ```nginx # --- Tiered cache zones (http block, outside server) --- # High churn - small short-TTL API responses proxy_cache_path /var/lib/nginx/cache/api levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=10m use_temp_path=off; # Block metadata - highly cacheable, rarely changes proxy_cache_path /var/lib/nginx/cache/block levels=1:2 keys_zone=block_cache:10m max_size=10g inactive=365d use_temp_path=off; # TX + chunk metadata - tens of millions of items proxy_cache_path /var/lib/nginx/cache/tx-and-chunk levels=1:2 keys_zone=tx_and_chunk_cache:30m max_size=30g inactive=90d use_temp_path=off manager_threshold=500ms loader_files=1000 loader_threshold=500ms; # Data - immutable Arweave content, bulk of disk usage # Set max_size based on your available disk space proxy_cache_path /var/lib/nginx/cache/data levels=1:2 keys_zone=data_cache:200m max_size=500g inactive=90d use_temp_path=off manager_threshold=500ms loader_files=1000 loader_threshold=500ms; # ArNS - web app content served via ArNS subdomains proxy_cache_path /var/lib/nginx/cache/arns levels=1:2 keys_zone=arns_cache:10m max_size=50g inactive=30d use_temp_path=off; ``` Size the `data_cache` zone based on your actual available disk space. The data cache will be the largest zone by far. Leave headroom for the node's own data storage, databases, and OS needs. The `keys_zone` memory size determines how many entries can be tracked — 1MB holds approximately 8,000 keys. | Parameter | Purpose | |-----------|---------| | `levels=1:2` | Two-level directory structure for cache files. Prevents any single directory from containing too many files. | | `keys_zone=name:size` | Shared memory zone for cache keys. Size based on expected number of cached items. | | `max_size` | Maximum disk space for this cache zone. NGINX evicts least-recently-used entries when exceeded. | | `inactive` | Remove entries not accessed within this period, even if not expired. | | `use_temp_path=off` | Write cache files directly to the cache directory (avoids cross-filesystem copies). | | `manager_threshold` | Maximum time the cache manager spends per cleanup iteration. Prevents disk I/O spikes on large caches. | | `loader_files` / `loader_threshold` | Controls how NGINX loads cache metadata on startup. Important for large caches to avoid slow restarts. | ### Map Directives All `map` directives must also be in the `http` block. These must be defined **before** the `server` block because the server references the variables they create. ```nginx # --- Map directives (http block, outside server) --- # WebSocket upgrade support (used by /graphql and catch-all) map $http_upgrade $connection_upgrade { default upgrade; '' close; } # Route ArNS subdomains to arns_cache, everything else to api_cache. # Excludes sandbox subdomains (52-char base32) which are not ArNS. map $host $catch_all_cache { "~^[a-z2-7]{52}\." api_cache; "~^[^.]+\..+\..+" arns_cache; default api_cache; } # Never cache 429 responses — prevent rate-limit errors from # overwriting valid cache entries. map $upstream_status $no_cache_429 { 429 1; default 0; } # Never cache responses with Cache-Control: no-store — prevents # caching 402 Payment Required and other uncacheable responses. map $upstream_http_cache_control $no_cache_no_store { "~no-store" 1; default 0; } ``` ## Cache Key Design For Arweave content, query strings are not meaningful - the content is identified by its transaction ID in the URL path. The cache key excludes query strings to prevent cache fragmentation: ```nginx proxy_cache_key "$scheme://$host$uri"; ``` The `$host` component is important because ArNS subdomains serve different content at the same path. ## Server Block Configuration The following directives go inside your `server` block. If you have an existing server block from the quick-start guide, merge these directives into it. ### Shared Defaults These directives apply as defaults across all locations. Individual locations override them as needed: ```nginx server { # ... your existing SSL, server_name, listen directives ... # Default cache zone (overridden per-location) proxy_cache api_cache; proxy_cache_key "$scheme://$host$uri"; # Thundering herd protection: first request fetches from origin, # others wait up to lock_timeout, then go to origin if lock_age exceeded proxy_cache_lock on; proxy_cache_lock_age 60s; proxy_cache_lock_timeout 60s; # Serve stale entries during errors, timeouts, or background updates proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; proxy_cache_revalidate on; proxy_cache_background_update on; # Never cache 429 or no-store responses proxy_no_cache $no_cache_429 $no_cache_no_store; proxy_cache_bypass $no_cache_429 $no_cache_no_store; # Error TTLs shared across all locations proxy_cache_valid 400 60s; proxy_cache_valid 403 10s; proxy_cache_valid 451 30d; proxy_cache_valid 500 502 503 504 10s; # Prevent cache fragmentation from Vary headers. # Trade-off: this drops ALL Vary values, not just Vary: Origin. # For Arweave gateways this is safe because the node does not use # Vary: Accept-Encoding for content negotiation. proxy_ignore_headers Vary; # Proxy buffer settings proxy_buffering on; proxy_buffer_size 32k; proxy_buffers 64 32k; proxy_max_temp_file_size 8192m; # Default timeouts proxy_read_timeout 120s; proxy_send_timeout 30s; # ... location blocks follow ... } ``` ### Route-Specific Caching #### Immutable Data (30-day TTL) Transaction data and raw content are immutable on Arweave. Cache aggressively with extended timeouts for large file downloads: ```nginx # Raw data endpoint location ^~ /raw/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_read_timeout 600s; proxy_send_timeout 600s; # Lock holder gets 10 minutes for large file downloads; # waiters give up after 5s and go to origin directly proxy_cache_lock_age 600s; proxy_cache_lock_timeout 5s; proxy_cache data_cache; # Disable background refresh — avoids expensive multi-GB origin fetches proxy_cache_background_update off; proxy_ignore_headers Set-Cookie Vary; proxy_hide_header Set-Cookie; proxy_cache_valid 200 30d; proxy_cache_valid 404 60s; } # Transaction data (43-char base64url IDs) location ~ "^/[a-zA-Z0-9_-]{43}(/|$)" { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_read_timeout 600s; proxy_send_timeout 600s; proxy_cache_lock_age 600s; proxy_cache_lock_timeout 5s; proxy_cache data_cache; proxy_cache_background_update off; proxy_ignore_headers Set-Cookie Vary; proxy_hide_header Set-Cookie; proxy_cache_valid 200 30d; proxy_cache_valid 404 60s; } ``` Note that `/raw/TX_ID` and `/TX_ID` serve the same content but are cached as separate entries because the cache key includes the URI path. For most gateways this duplication is acceptable. If disk space is tight, you can normalize the cache key with a `map` — see the [Troubleshooting](#troubleshooting) section. #### Chunks (24-hour TTL) Arweave chunks are immutable but accessed less frequently than full transactions: ```nginx location ^~ /chunk/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache tx_and_chunk_cache; proxy_cache_background_update off; proxy_ignore_headers Cache-Control Expires Set-Cookie Vary; proxy_hide_header Set-Cookie; proxy_cache_valid 200 24h; proxy_cache_valid 404 30s; } ``` #### Volatile Metadata (Short TTLs) API endpoints and metadata change frequently and need short TTLs: ```nginx # Health, info, metrics, tx_anchor (2-minute TTL) location ~ ^/(health|info|metrics|tx_anchor)$ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache_valid 200 120s; proxy_cache_valid 404 30s; } # Height and time (20-second TTL - changes every block) location ~ ^/(height|time)$ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache_valid 200 20s; proxy_cache_valid 404 30s; } # Peers, current_block (30-second TTL) location ~ ^/(peers|current_block)$ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache_valid 200 30s; proxy_cache_valid 404 15s; } # Wallet, price, unconfirmed TX lookups (30-second TTL) location ~ ^/(wallet|price|unconfirmed_tx)/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache_valid 200 30s; proxy_cache_valid 404 15s; } # AR.IO API (30-second TTL) location ^~ /ar-io/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache_valid 200 30s; proxy_cache_valid 404 5s; } # Block metadata location ^~ /block/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache block_cache; proxy_cache_valid 404 30s; } # TX metadata location /tx/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache tx_and_chunk_cache; proxy_cache_valid 404 30s; } # TX status (30-second TTL) location ~ ^/tx/[A-Za-z0-9_-]+/status$ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache tx_and_chunk_cache; proxy_cache_valid 200 30s; proxy_cache_valid 404 30s; } # Pending transactions (20-second TTL) location = /tx/pending { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache tx_and_chunk_cache; proxy_cache_valid 200 20s; proxy_cache_valid 404 30s; } # Current block (2-minute TTL) location = /block/current { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; add_header X-Cache-Status $upstream_cache_status always; proxy_cache_valid 200 120s; proxy_cache_valid 404 30s; } ``` #### Endpoints to Never Cache Some endpoints must never be cached because they handle writes, WebSocket connections, or state-changing operations. These locations use `proxy_cache off` and explicitly set `Cache-Control: no-store` (this is a response header injected by NGINX, distinct from the upstream `no-store` bypass in the map directives): ```nginx # GraphQL - supports WebSocket upgrades and POST mutations location = /graphql { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_cache off; proxy_hide_header Cache-Control; add_header Cache-Control "no-store" always; } # Transaction submission (POST endpoint) location = /tx { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_cache off; proxy_hide_header Cache-Control; add_header Cache-Control "no-store" always; } # Chunk upload (POST endpoint) location = /chunk { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_cache off; proxy_hide_header Cache-Control; add_header Cache-Control "no-store" always; } ``` #### Catch-All with ArNS Routing The catch-all location handles remaining requests including ArNS subdomain content: ```nginx location / { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; add_header X-Cache-Status $upstream_cache_status always; proxy_cache $catch_all_cache; proxy_cache_valid 200 300s; proxy_cache_valid 404 30s; } ``` ## TTL Summary | Route | Cache Zone | 200 TTL | 404 TTL | Notes | |-------|-----------|---------|---------|-------| | `/raw/`, `/[TX_ID]` | data_cache | 30d | 60s | Immutable content, extended timeouts | | `/chunk/` | tx_and_chunk_cache | 24h | 30s | Immutable chunks | | `/block/` | block_cache | origin | 30s | Origin-controlled TTL | | `/block/current` | api_cache | 120s | 30s | Changes every block | | `/tx/` | tx_and_chunk_cache | origin | 30s | Origin-controlled TTL | | `/tx/[ID]/status` | tx_and_chunk_cache | 30s | 30s | Status can change | | `/tx/pending` | tx_and_chunk_cache | 20s | 30s | Changes frequently | | `/ar-io/` | api_cache | 30s | 5s | Short-lived API data | | `/health`, `/info` | api_cache | 120s | 30s | Metadata endpoints | | `/height`, `/time` | api_cache | 20s | 30s | Changes every block | | `/peers`, `/current_block` | api_cache | 30s | 15s | Network state | | `/wallet/`, `/price/` | api_cache | 30s | 15s | Volatile data | | `/` (catch-all) | dynamic | 300s | 30s | ArNS or API cache | | `/graphql`, `/tx` POST, `/chunk` POST | none | - | - | Never cached | ## Content Moderation & Cache Purging Caching immutable content for 30 days creates a compliance risk: if the ar.io node blocks a transaction via [content moderation](/build/run-a-gateway/manage/content-moderation), NGINX may continue serving the cached copy for the remainder of the TTL. This is a compliance concern for production gateways. If you use content moderation filters, you need a way to force-refresh cached entries after blocking. ### Cache Bypass Header Add a `map` that checks a secret bypass header value, and use it with `proxy_cache_bypass`. When triggered, NGINX skips the cache and fetches fresh from the ar.io node — which now returns the blocked response — and overwrites the stale cache entry. In the `http` block (with your other maps), add: ```nginx # Cache bypass for content moderation purging. # Only bypass when the header matches your secret — prevents abuse. map $http_x_cache_bypass $purge_allowed { "your-secret-here" 1; default 0; } ``` In each data-serving location (`/raw/`, `/[TX_ID]`, catch-all), add: ```nginx proxy_cache_bypass $purge_allowed; ``` Then force-refresh a blocked transaction: ```bash # Replace "your-secret-here" and domain with your actual values curl -s -o /dev/null -H "X-Cache-Bypass: your-secret-here" \ https://your-domain.example/TX_ID_HERE curl -s -o /dev/null -H "X-Cache-Bypass: your-secret-here" \ https://your-domain.example/raw/TX_ID_HERE ``` The node returns its blocked/404 response, NGINX caches that instead, and subsequent requests get the blocked response. If you run a content scanning sidecar, fire these bypass requests automatically after each block event to complete the purge without any extra infrastructure. ## Monitoring Cache Performance Add the `X-Cache-Status` header to expose cache behavior on every response: ```nginx add_header X-Cache-Status $upstream_cache_status always; ``` Check cache performance: ```bash curl -s -D - -o /dev/null https://your-gateway.example/TX_ID 2>&1 | grep -i x-cache ``` | Status | Meaning | |--------|---------| | `HIT` | Served from cache | | `MISS` | Fetched from origin, now cached | | `EXPIRED` | Cache entry expired, fetched fresh from origin | | `UPDATING` | Stale entry served while background update runs | | `STALE` | Stale entry served due to origin error | | `BYPASS` | Cache was bypassed (e.g., 429 or no-store response) | | (empty) | Response status not covered by any `proxy_cache_valid` directive for this location | ## Validate and Reload Always test your configuration before reloading NGINX: ```bash sudo nginx -t && sudo systemctl reload nginx ``` If `nginx -t` reports errors, fix them before reloading. A bad reload with `systemctl reload` is safe (NGINX keeps the old config running), but `systemctl restart` with a broken config will take NGINX offline. ## Additional Configuration ### Load Balancer Real IP If your gateway is behind a load balancer, configure NGINX to trust the `X-Forwarded-For` header from the load balancer's IP: ```nginx # Replace with your load balancer's actual IP set_real_ip_from 10.0.0.1/32; real_ip_header X-Forwarded-For; real_ip_recursive on; ``` Without this, all requests appear to come from the load balancer's IP. ## Troubleshooting ### Duplicate Cache Entries for /raw/ and / `/raw/TX_ID` and `/TX_ID` serve the same bytes but are cached as separate entries because the cache key includes the full URI path. If disk space is a concern, normalize the key with a map in the `http` block: ```nginx map $uri $normalized_cache_uri { "~^/raw/(.+)$" "/$1"; default $uri; } ``` Then use `proxy_cache_key "$scheme://$host$normalized_cache_uri";` in both the `/raw/` and TX data locations. This causes both endpoints to share one cache entry. ### Cache Directories Not Writable If NGINX logs show write errors, check ownership matches the NGINX worker user (`www-data` on Debian/Ubuntu, `nginx` on RHEL/CentOS): ```bash ls -la /var/lib/nginx/cache/ ``` ### Shared Memory Exhaustion If `keys_zone` is too small for the number of cached items, NGINX silently evicts entries. Monitor with `stub_status` and increase `keys_zone` size if your hit rate drops unexpectedly despite having disk space available. ## Related } /> } /> } /> } /> # Setting Apex Domain Content (/build/run-a-gateway/manage/setting-apex-domain) Configure your ar.io Gateway to serve custom content from the apex domain instead of the default Arweave network information. This allows you to customize your gateway's root domain with useful information, project details, or any content you wish to share. ## Overview Prior to gateway Release 28, the apex domain of a gateway would only display information about the Arweave network. Release 28 introduced two new environment variables that allow a gateway to serve custom content from the apex domain: - `APEX_TX_ID`: Set to serve content from a specific transaction ID - `APEX_ARNS_NAME`: Set to serve content from an ArNS name These variables enable gateway operators to customize their gateway's apex domain with useful information, details about the operator or associated projects, or any other content they wish to share. ## Quick Start ### Choose Your Content Source Decide how you want to serve your content: **Option 1: Direct Transaction ID** - Upload your content to Arweave - Use the transaction ID directly **Option 2: ArNS Name (Recommended)** - Upload your content to Arweave - Assign your content's transaction ID to an ArNS name - Use the ArNS name for easier management ### Upload Your Content Upload your dApp, website, or other content to Arweave using your preferred method: - **ArDrive** - For simple file uploads - **Turbo** - For application bundles - **Direct upload** - For advanced users ### Configure Environment Variable Add one of these variables to your `.env` file: ```bash # Option 1: Direct transaction ID APEX_TX_ID=your-transaction-id # Option 2: ArNS name (recommended) APEX_ARNS_NAME=your-arns-name ``` You cannot set both variables simultaneously. Providing both variables will result in an error. ### Restart Your Gateway Restart your gateway to apply the changes: ```bash docker compose down docker compose up -d ``` ### Verify Configuration Visit your gateway's apex domain to confirm the custom content is being served correctly. ## Configuration Methods ### Using Direct Transaction ID ### Upload Content Upload your content to Arweave and note the transaction ID: ```bash # Example: Upload using ArDrive CLI ardrive upload-file --file-path ./my-website.html # Note the returned transaction ID # Example: abc123...def789 ``` ### Set Environment Variable Add the transaction ID to your `.env` file: ```bash APEX_TX_ID=abc123...def789 ``` ### Restart Gateway Restart your gateway to apply the configuration: ```bash docker compose down docker compose up -d ``` ### Update Content To update your content: 1. Upload new content to Arweave 2. Update `APEX_TX_ID` with the new transaction ID 3. Restart your gateway **Advantages:** - Direct control over content - No additional ArNS setup required - Simple for one-time content **Disadvantages:** - Requires gateway restart for updates - Less flexible for content management ### Using ArNS Name (Recommended) ### Upload Content Upload your content to Arweave: ```bash # Upload your website or dApp ardrive upload-file --file-path ./my-dapp.html # Note the transaction ID: xyz789...abc123 ``` ### Register ArNS Name Register an ArNS name pointing to your content: 1. Visit [ArNS App](https://arns.ar.io) 2. Connect your wallet 3. Choose your desired name (e.g., `my-gateway-content`) 4. Set the transaction ID: `xyz789...abc123` 5. Pay the registration fee ### Configure Environment Variable Add the ArNS name to your `.env` file: ```bash APEX_ARNS_NAME=my-gateway-content ``` ### Restart Gateway Restart your gateway to apply the configuration: ```bash docker compose down docker compose up -d ``` ### Update Content To update your content: 1. Upload new content to Arweave 2. Update the ArNS name to point to the new transaction ID 3. **No gateway restart required!** **Advantages:** - No restart required for content updates - Easy content management - Professional domain naming - Can be updated independently **Disadvantages:** - Requires ArNS setup - Additional cost for ArNS registration ### Advanced Setup Options ### Custom Content Types Configure different types of content: **Static Website:** ```bash # Upload HTML/CSS/JS files APEX_ARNS_NAME=my-gateway-website ``` **Single Page Application:** ```bash # Upload SPA bundle APEX_ARNS_NAME=my-dapp ``` **Documentation Site:** ```bash # Upload documentation APEX_ARNS_NAME=my-gateway-docs ``` ### Content Management Workflow Implement a content management workflow: 1. **Development** - Test content locally 2. **Upload** - Deploy to Arweave 3. **Register** - Create/update ArNS name 4. **Verify** - Check content on gateway 5. **Monitor** - Track performance and usage ## Use Cases and Examples ### Display Gateway Service Information Perfect for showcasing your gateway service: **Content Ideas:** - Gateway operator information - Service capabilities and features - Contact information - Status and uptime statistics - Network participation details **Example Structure:** ```html My ar.io Gateway ar.io Gateway Service Reliable gateway infrastructure for the permanent web High availability Fast response times Global CDN Contact: operator@example.com ``` ### Showcase Associated Projects Highlight your projects and services: **Content Ideas:** - Project portfolio - Service offerings - Recent updates and news - Links to other projects - Integration examples **Example Structure:** ```html My Projects - ar.io Gateway My Projects Project Alpha Description of project and its features Visit Project Project Beta Another project description Visit Project ``` ### Host Documentation Provide comprehensive documentation: **Content Ideas:** - Gateway setup guides - API documentation - Integration tutorials - Troubleshooting guides - FAQ sections **Example Structure:** ```html Gateway Documentation .nav { float: left; width: 200px; } .content { margin-left: 220px; } Navigation Setup Guide API Reference Troubleshooting Gateway Documentation Setup Guide Step-by-step setup instructions... ``` ### Real-World Examples Several gateway operators have implemented this feature: **arweave.tech** - Serves a custom landing page with gateway service information - Professional presentation of capabilities **arlink.xyz** - Serves the permaDapp for the Arlink project - Demonstrates integration with existing projects **frostor.xyz / love4src.com** - Serves information about the Memetic Block Software Guild - Showcases community and project information **vilenarios.com** - Serves personalized portfolio/link tree information - Personal branding and contact information These examples demonstrate the flexibility of the apex domain feature and how different operators use it to create unique, personalized experiences for their users. ## Troubleshooting ### Fix Configuration Problems ### Check Environment Variables Verify your `.env` file configuration: ```bash # Check if variables are set correctly grep -E "APEX_(TX_ID|ARNS_NAME)" .env # Should show only one of: # APEX_TX_ID=your-transaction-id # APEX_ARNS_NAME=your-arns-name ``` Ensure you have only ONE of the APEX variables set, not both. ### Verify Gateway Restart Ensure your gateway has been restarted after configuration changes: ```bash # Check if gateway is running docker compose ps # Restart if needed docker compose down docker compose up -d ``` ### Check Gateway Logs Review logs for any error messages: ```bash docker compose logs ar-io-core | grep -i apex ``` ### Resolve Content Issues ### Verify Content Accessibility Test if your content is accessible: ```bash # Test transaction ID directly curl -I https://turbo-gateway.com/your-transaction-id # Test ArNS name resolution curl -I https://your-arns-name.ar.io ``` ### Check Content Format Ensure your content is properly formatted: - **HTML content** should have proper DOCTYPE - **Text content** should be UTF-8 encoded - **Binary content** should have appropriate Content-Type headers ### Test Content Rendering Verify content renders correctly in different browsers: 1. Test in Chrome, Firefox, Safari 2. Check mobile responsiveness 3. Verify all links work correctly 4. Test with different screen sizes ### Fix ArNS Problems ### Verify ArNS Resolution Check if your ArNS name resolves correctly: ```bash # Test ArNS resolution nslookup your-arns-name.ar.io # Check if it points to the correct transaction curl -s https://your-arns-name.ar.io | head -10 ``` ### Update ArNS Record If ArNS name points to wrong content: 1. Go to [ArNS App](https://arns.ar.io) 2. Find your ArNS name 3. Update the transaction ID 4. Wait for propagation (usually immediate) ### Check ArNS Status Verify ArNS name is active and not expired: 1. Visit the ArNS app 2. Check your name's status 3. Ensure it's not expired 4. Verify payment is up to date ## Best Practices ### Content Design ### Optimize for Performance - Keep file sizes reasonable - Use efficient HTML/CSS - Optimize images and assets - Minimize external dependencies ### Ensure Accessibility - Use semantic HTML - Include alt text for images - Ensure good color contrast - Test with screen readers ### Mobile Responsiveness - Design for mobile-first - Use responsive CSS - Test on various devices - Ensure touch-friendly interfaces ### Content Management ### Version Control - Keep content in version control - Document changes and updates - Test changes before deployment - Maintain backup copies ### Regular Updates - Keep information current - Update contact details - Refresh project information - Monitor for broken links ### Backup Strategy - Backup content regularly - Keep multiple copies - Document restoration procedures - Test backup recovery ## Next Steps **Ready to customize your gateway?** Start with the Quick Start section above, then explore the different configuration methods and use cases to find what works best for your needs. ### Additional Resources - **ArNS Documentation** - Learn more about ArNS names and management - **Content Upload Guides** - Best practices for uploading content to Arweave - **Gateway Configuration** - Advanced gateway configuration options - **Community Examples** - See how other operators use this feature ### Getting Help If you encounter issues: 1. Check the troubleshooting section above 2. Verify your configuration is correct 3. Test content accessibility independently 4. Consult the [ar.io Discord](https://discord.com/invite/HGG52EtTc2) for community support # Solana Migration for Operators (/build/run-a-gateway/manage/solana-migration) ## Overview The ar.io network has migrated protocol execution from AO to Solana. If you're an existing gateway operator, this guide covers everything you need to change. **Your gateway will continue serving Arweave data uninterrupted** — the changes affect how your node interacts with the protocol layer (staking, observations, rewards). Complete these steps before the cutover date to ensure uninterrupted reward eligibility. Gateways that fail 30 consecutive epochs will be [pruned](/learn/oip/pruning) with 100% of minimum stake slashed. ## What Changed | Before (AO) | After (Solana) | |-------------|----------------| | Arweave wallet (RSA JWK) | Solana keypair (Ed25519) | | AO Compute Unit for state reads | Solana RPC for state reads | | `AO_CU_URL`, `AO_MU_URL`, etc. | `SOLANA_RPC_URL` | | Observations submitted via AO messages | Observations submitted as Solana transactions | | Observer wallet pays in AR/Turbo credits | Observer wallet pays in SOL | | Rewards distributed atomically | Rewards distributed via [cranker pipeline](/learn/oip/epoch-pipeline) | | Observer address not unique | Observer address must be unique per gateway | | Minimum stake = 10,000 ARIO | Minimum stake = 20,000 ARIO | ## What Didn't Change - Your gateway still serves Arweave data (caching, indexing, ArNS resolution) - Docker-based deployment workflow is the same - Epoch duration remains 24 hours - Staking, delegation, and reward mechanics are functionally equivalent ## Step-by-Step Migration ### Map Your Address Before the cutover, register your Solana wallet address using the ar.io claim app. This maps your existing Arweave address to your new Solana address so your stake, delegation, and gateway registration carry over automatically. ### Generate a Solana Keypair Create a new Solana keypair for your gateway and observer: ```bash # Install Solana CLI tools sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)" # Generate gateway operator keypair solana-keygen new --outfile operator-keypair.json # Generate observer keypair (must be unique — not shared with any other gateway) solana-keygen new --outfile observer-keypair.json ``` Your observer address must be **unique across all gateways**. If another gateway is already using the same observer address, your registration will fail. This is enforced onchain by the ObserverLookup PDA. ### Fund Your Wallets with SOL Your observer wallet needs SOL for submitting observation transactions. Typical costs are less than 0.01 SOL per transaction. ```bash # Check your observer address solana-keygen pubkey observer-keypair.json # Send SOL to it from an exchange or existing wallet ``` We recommend keeping at least **0.5 SOL** in the observer wallet to cover several months of observation submissions. ### Update Environment Variables Edit your `.env` file with the new Solana configuration: ```bash # Operator + observer addresses (Solana pubkeys, base58) AR_IO_WALLET= OBSERVER_WALLET= # Solana RPC endpoint (see Callout below — use a premium provider in production) SOLANA_RPC_URL=https://your-rpc-provider.com # Keypair file paths inside the container (mounted from ./wallets/) SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json OBSERVER_KEYPAIR_PATH=/app/wallets/observer-keypair.json # The four ar.io Solana programs (canonical mainnet) ARIO_CORE_PROGRAM_ID=73YoECm6NKXpVRoe5f1Q9BcP5DJGPFUjnFy6AxBE5Nvh ARIO_GAR_PROGRAM_ID=89fNiiwgpFSPHKuqfNUkgYTYjtAJAhyqHjXmgXeppGpf ARIO_ARNS_PROGRAM_ID=2yCUx5edFvUrkibYaUa2ZXWyx9kuJkS8CwyzsgHPWdZZ ARIO_ANT_PROGRAM_ID=2MWexMHfMhGJwMHv9Qm9YAVCqjUFUJwDJAysW4oCUGk5 ``` **Remove the old AO variables** (they are no longer used): ```bash # DELETE THESE from your .env: # AO_CU_URL=... # NETWORK_AO_CU_URL=... # ANT_AO_CU_URL=... # AO_MU_URL=... # AO_GATEWAY_URL=... # AO_GRAPHQL_URL=... # HTTPSIG_UPLOAD_ATTESTATION=... # WALLETS_PATH=... ``` The default public Solana RPC (`api.mainnet-beta.solana.com`) is rate-limited. Under any real load your observer and cranker will start dropping transactions, and the resulting timeouts cause most observation cycles to miss their submission window. Use a dedicated RPC provider in production: [Helius](https://helius.dev), [Triton](https://triton.one), or [QuickNode](https://quicknode.com). ` (or `SOLANA_PRIVATE_KEY=`) instead of the `*_KEYPAIR_PATH` env. When using an inline observer key, also set `SOLANA_UPLOAD_PRIVATE_KEY=` so observer report uploads to Turbo use the same signer. Setting both forms for the same role is rejected at startup. ### Place Keypair Files Copy your operator and observer Solana keypair JSON files into the `wallets/` directory in your ar-io-node checkout. Docker Compose mounts this directory into `/app/wallets/` inside the container, which is where the `*_KEYPAIR_PATH` env vars point. ```bash cp operator-keypair.json wallets/operator-keypair.json cp observer-keypair.json wallets/observer-keypair.json chmod 600 wallets/*.json ``` Env vars must use the **in-container** path (`/app/wallets/...`), not the host path (`./wallets/...`). Docker Compose bind-mounts `${WALLETS_PATH:-./wallets}` to `/app/wallets` inside the container. | Wrong (host path) | Right (container path) | |---|---| | `SOLANA_KEYPAIR_PATH=./wallets/operator-keypair.json` | `SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json` | Skip this step entirely if you set `OBSERVER_PRIVATE_KEY` / `SOLANA_PRIVATE_KEY` env vars (base58 strings) instead. ### Update ar-io-node Pull the latest Solana-compatible release and recreate the containers: ```bash cd ar-io-node git pull docker compose pull docker compose up -d --force-recreate core envoy observer ``` Avoid `docker compose down -v` here — the `-v` flag deletes named volumes, which on most setups wipes the sqlite index and chunk data the gateway has spent days/weeks building. Use `docker compose up -d --force-recreate ` instead, which restarts containers in place without touching volumes. Check your release version at `https:///ar-io/info`. ### Optional: Enable Epoch Cranking Your observer can optionally help drive the epoch reward pipeline. This is completely permissionless and costs minimal SOL (~0.000155 SOL per epoch): ```bash # Add to .env ENABLE_EPOCH_CRANKING=true ``` See the [epoch pipeline](/learn/oip/epoch-pipeline) docs for details. Running the cranker creates redundancy so the network is never dependent on a single bot. ### Verify After restarting, verify your gateway is operating correctly: 1. **Release + program IDs (cross-check config is what you intended):** ```bash curl -s https:///ar-io/info | jq '{release, wallet, programIds}' ``` `programIds.core/gar/arns/ant` should match the network you're targeting. If they don't, the gateway is still pointed at the old program set, which is usually why a migrated gateway looks inert. 2. **Gateway registration is live on the new network:** ```bash ar.io get-gateway -t solana \ --rpc-url https://api.mainnet-beta.solana.com \ --core-program-id 73YoECm6NKXpVRoe5f1Q9BcP5DJGPFUjnFy6AxBE5Nvh \ --gar-program-id 89fNiiwgpFSPHKuqfNUkgYTYjtAJAhyqHjXmgXeppGpf \ --arns-program-id 2yCUx5edFvUrkibYaUa2ZXWyx9kuJkS8CwyzsgHPWdZZ \ --ant-program-id 2MWexMHfMhGJwMHv9Qm9YAVCqjUFUJwDJAysW4oCUGk5 \ --address ``` Should return `"status": "joined"` with the FQDN/stake/settings you set at join time. 3. **ArNS resolution works end-to-end:** ```bash curl -I -H "Host: ardrive." https:/// ``` Expect HTTP 200 (or a manifest redirect). 404 here means the cache hasn't hydrated yet — give it a minute and re-check. 4. **Observer reports are flowing:** `https:///ar-io/observer/reports/current` 5. **Gateway listed in the portal:** [gateways.ar.io](https://gateways.ar.io) 6. **No accumulating failed epochs** in the portal's gateway view. ## Wallet Roles and Configuration Patterns The gateway uses up to four distinct wallet roles. Understanding these helps you pick the right configuration for your setup. | Role | What it signs | Env vars | Fallback | |---|---|---|---| | **Operator** (+ cranker) | `join_network`, `update_gateway_settings`, permissionless cranker instructions | `SOLANA_KEYPAIR_PATH` or `SOLANA_PRIVATE_KEY` | — (required) | | **Observer** | `save_observations` transactions | `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` | Falls back to operator key | | **Upload** | Observer report bundles sent to Turbo | See [upload precedence](#upload-signing-precedence) below | Falls back to observer → operator Solana key, but explicit upload env is recommended | | **HTTPSIG signer** | RFC 9421 response headers | Uses observer Solana key when set | Auto-generated standalone Ed25519 key | Setting both the file-path and inline forms for the same role (e.g. `SOLANA_KEYPAIR_PATH` **and** `SOLANA_PRIVATE_KEY`) is rejected at startup as ambiguous. Pick one. ### Supported Configurations These are the five supported wallet setups. **Pattern 1 is the recommended default** — one key does everything. Pattern 2 is the most common migration path for operators who already have an Arweave JWK. | # | Operator | Observer | Upload | Required envs | |---|---|---|---|---| | **1** | Solana | = operator | = operator (Solana) | `SOLANA_KEYPAIR_PATH` | | **2** | Solana | = operator | Arweave JWK | `SOLANA_KEYPAIR_PATH` + `ARWEAVE_UPLOAD_KEY_FILE` | | **3** | Solana A | Solana B | Solana C | `SOLANA_KEYPAIR_PATH` + `OBSERVER_KEYPAIR_PATH` + `SOLANA_UPLOAD_KEYPAIR_PATH` | | **4** | Solana A | Solana B | Arweave JWK | `SOLANA_KEYPAIR_PATH` + `OBSERVER_KEYPAIR_PATH` + `ARWEAVE_UPLOAD_KEY_FILE` | | **5** | Solana A | Solana B | Ethereum | `SOLANA_KEYPAIR_PATH` + `OBSERVER_KEYPAIR_PATH` + `ETHEREUM_UPLOAD_PRIVATE_KEY_FILE` | #### Pattern 1 — Single Solana keypair (recommended) ```bash # One key for operator + observer + uploads SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json SOLANA_UPLOAD_KEYPAIR_PATH=/app/wallets/operator-keypair.json SOLANA_RPC_URL= AR_IO_WALLET= OBSERVER_WALLET= ENABLE_EPOCH_CRANKING=false # flip to true when ready ``` If you use a base58 private key instead of a keypair file, explicitly set the upload key too: ```bash OBSERVER_PRIVATE_KEY= SOLANA_UPLOAD_PRIVATE_KEY= ``` #### Pattern 2 — Keep existing Arweave JWK for uploads The most common path for operators migrating from a pre-Solana setup. Your existing Arweave JWK continues signing report bundles while the Solana keypair handles protocol interactions. ```bash SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json ARWEAVE_UPLOAD_KEY_FILE=/app/wallets/.json SOLANA_RPC_URL= AR_IO_WALLET= OBSERVER_WALLET= ENABLE_EPOCH_CRANKING=false ``` ### Upload Signing Precedence The gateway picks the first matching upload signer from this list: ``` 1. ARWEAVE_UPLOAD_KEY_FILE (file) → ArweaveSigner 2. ARWEAVE_UPLOAD_JWK (inline) → ArweaveSigner 3. ETHEREUM_UPLOAD_PRIVATE_KEY_FILE (file) → EthereumSigner 4. ETHEREUM_UPLOAD_PRIVATE_KEY (inline) → EthereumSigner 5. SOLANA_UPLOAD_KEYPAIR_PATH (explicit file) → SolanaSigner 6. SOLANA_UPLOAD_PRIVATE_KEY (explicit inline) → SolanaSigner 7. Fallback: OBSERVER_KEYPAIR_PATH ?? SOLANA_KEYPAIR_PATH → SolanaSigner ``` Setting upload envs from more than one chain at once (e.g. `ARWEAVE_UPLOAD_KEY_FILE` **plus** `ETHEREUM_UPLOAD_PRIVATE_KEY`) raises a startup error listing every conflicting env. Pick exactly one upload chain. If observer logs warn that `TurboReportSink` is not configured, the observer does not have an upload signer for report data. Add `SOLANA_UPLOAD_PRIVATE_KEY=` or `SOLANA_UPLOAD_KEYPAIR_PATH=`, then restart the observer and check the next epoch logs. ### Key Formats Solana keypairs come in two common formats. Both encode the same 64-byte secret (`seed(32) || pubkey(32)`): | Format | Example | Source | |---|---|---| | **JSON array** (Solana CLI standard) | `[12,34,56,...]` — 64 uint8 integers | `solana-keygen new --outfile keypair.json` | | **base58 secret** | 87–88 character base58 string | Phantom " Use the JSON file with `*_KEYPAIR_PATH` env vars, or the base58 string with `*_PRIVATE_KEY` env vars — never both for the same role. ## Troubleshooting ### "Observer is restart-looping with 'Epoch 0 PDA not found'" If the observer logs repeatedly: ``` error: Continuous observer start() rejected — attempting auto-restart ...error: "Epoch 0 PDA not found at — has prescribe_epoch run yet?" ``` …the network you're targeting hasn't had its first epoch initialized. The observer needs entropy from `epoch[N].prescribed_observers` to bootstrap, which doesn't exist until someone calls `create_epoch` (typically a cranker). Verify the epoch state directly: ```bash ar.io get-current-epoch -t solana \ --rpc-url https://api.mainnet-beta.solana.com \ --core-program-id 73YoECm6NKXpVRoe5f1Q9BcP5DJGPFUjnFy6AxBE5Nvh \ --gar-program-id 89fNiiwgpFSPHKuqfNUkgYTYjtAJAhyqHjXmgXeppGpf \ --arns-program-id 2yCUx5edFvUrkibYaUa2ZXWyx9kuJkS8CwyzsgHPWdZZ \ --ant-program-id 2MWexMHfMhGJwMHv9Qm9YAVCqjUFUJwDJAysW4oCUGk5 ``` For staging-devnet or local devnet, replace the RPC URL and program IDs with that environment's values. If this returns `"Epoch 0 not found"`, the network is configured but inactive. This is a network-operations state, not a gateway misconfiguration. Wait for an active cranker (yours or another operator's) to bootstrap epoch 0. ### "Cranker started but never submits any instructions" The cranker silently bails out if `EpochSettings.enabled` is `false`. This is intentional: operators shouldn't pay SOL fees attempting cranker instructions against a paused network. Confirm with `ar.io get-epoch-settings` and check whether epochs are enabled on your target network. If `enabled` is false, no cranker activity is expected. ### "ArNS names return 404 but the on-chain record exists" The gateway hydrates an ArNS names cache at boot, paginating through the on-chain registry. If your SDK pin is significantly older than the deployed `ario-arns` program, the paginated response shape may not match and the cache hydrates incompletely. Symptoms: - Cache hydration logs `Successfully hydrated ArNS names cache` quickly (e.g. ~4s for thousands of records) - Resolver logs report `Base name not found in ArNS names cache` for names that demonstrably exist via the SDK CLI's `get-arns-record --name ` Fix: bump `@ar.io/sdk` in your gateway's image (`package.json`) to the latest `^4.0.0-solana.*` and rebuild. ### Wallet Configuration Startup Errors The gateway validates wallet configuration at startup. Errors are loud and name the offending env: | Error pattern | Cause | Fix | |---|---|---| | `multiple chain groups configured for upload role` | Upload envs from more than one chain are set (e.g. `ARWEAVE_UPLOAD_*` and `ETHEREUM_UPLOAD_*`) | Pick one upload chain and remove the others | | `ambiguous: both ... set for role` | File-path and inline forms for the same role are both set (e.g. `SOLANA_KEYPAIR_PATH` + `SOLANA_PRIVATE_KEY`) | Use one form per role | | `material at SOLANA_KEYPAIR_PATH does not look like a Solana keypair` | An Arweave JWK or other JSON was placed at the Solana keypair path | Check you copied the right file — Solana keypairs are a JSON array of 64 integers, not a JWK object | | `material at ARWEAVE_UPLOAD_KEY_FILE does not look like an Arweave JWK` | A Solana keypair (JSON array) was placed at the Arweave upload slot | Swap the file for your Arweave JWK | | `SOLANA_KEYPAIR_PATH not set in Solana mode` | The operator key is missing entirely | Set `SOLANA_KEYPAIR_PATH` or `SOLANA_PRIVATE_KEY` | | `OBSERVER_KEYPAIR_PATH does not match on-chain Gateway.observer_address` | The observer key doesn't match what was registered at `join_network` | Update the key to match, or call `update_observer_address` on-chain | ## New Risks to Be Aware Of ### Gateway Pruning Gateways that fail **30 consecutive epochs** are automatically pruned from the network. When pruned: - 100% of minimum stake (20,000 ARIO) is slashed to the protocol balance - Excess operator stake enters the standard 30-day withdrawal queue - Delegated stakes enter the standard 30-day withdrawal queue (delegators are not slashed) See [Gateway Pruning](/learn/oip/pruning) for full details. ### SOL Balance Monitoring Keep your observer wallet funded with SOL. If it runs out, your observer cannot submit observations, which leads to failed epochs and eventually pruning. Set up monitoring/alerts for your observer wallet balance. ## FAQ **Do I need to re-register my gateway?** No. If you mapped your address before cutover, your gateway registration, stake, and delegations are migrated automatically. **What happens to my delegators?** Delegations are migrated as-is. Delegators who mapped their addresses will see their stakes in their Solana wallet. Unmapped delegations are held in escrow for claiming. **Can I still use my Arweave wallet for data uploads?** Yes. Arweave wallets are still used for uploading data to Arweave via Turbo. The Solana wallet is only for protocol interactions (staking, observations, ArNS). **How do I check my observer's SOL balance?** ```bash solana balance --url mainnet-beta ``` # Automating SSL Certificate Renewal (/build/run-a-gateway/manage/ssl-certs) Secure your ar.io Gateway with automated SSL certificate renewal using Certbot and DNS challenge validation. This guide covers setup for different DNS providers to automatically renew certificates without manual intervention. ## Overview Using DNS challenge validation with Certbot allows you to: - Automatically renew SSL certificates - Support wildcard certificates - Avoid manual certificate management - Ensure continuous gateway security ## Prerequisites - A running ar.io Gateway - Domain name configured with your DNS provider - Administrative access to your server - API access to your DNS provider ## DNS Provider Setup ### Cloudflare Configuration ### Create Cloudflare API Token Navigate to **Cloudflare → My Profile → API Tokens → Create Token** Configure the token with these permissions: - **Zone → Zone → Read** - **Zone → DNS → Edit** ![Cloudflare API Token Configuration](https://turbo-gateway.com/GMzqNXNCQMSLqyt7SV7FrGOgCuGBeaO5qjRWibFkVBE) ### Install Certbot and Cloudflare Plugin ```bash apt update apt install certbot python3-certbot-dns-cloudflare -y ``` ### Configure API Credentials Create the credentials file: ```bash nano /etc/letsencrypt/cloudflare.ini ``` Add your API token: ```ini dns_cloudflare_api_token = your_api_token_here ``` Secure the file: ```bash chmod 600 /etc/letsencrypt/cloudflare.ini ``` ### Generate SSL Certificate Request the certificate with wildcard support: ```bash certbot certonly --dns-cloudflare \ --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \ -d example.com -d *.example.com ``` **Expected output:** ```bash Successfully received certificate. Certificate is saved at: /etc/letsencrypt/live/example.com/fullchain.pem Key is saved at: /etc/letsencrypt/live/example.com/privkey.pem ``` ### Test Automatic Renewal Perform a dry run to validate the renewal process: ```bash certbot renew --dry-run ``` **Expected output:** ```bash Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/example.com.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Account registered. Simulating renewal of an existing certificate for example.com and *.example.com Waiting 10 seconds for DNS changes to propagate - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Congratulations, all simulated renewals succeeded: /etc/letsencrypt/live/example.com/fullchain.pem (success) ``` ### Verify Automatic Renewal Timer Check that the certbot timer is active: ```bash systemctl list-timers | grep certbot ``` **Expected output:** ```bash Tue 2024-11-05 02:22:10 UTC 3h 21min Mon 2024-11-04 17:16:51 UTC 5h 43min ago certbot.timer certbot.service ``` ### Namecheap Configuration **API Requirements:** Namecheap requires specific conditions to create API keys: - At least 20 domains under your account - Minimum $50 account balance - At least $50 spent within the last 2 years If you don't meet these requirements, contact Namecheap support for a waiver. ### Create Namecheap API Key Navigate to **Namecheap → Profile → Tools → Manage API Access Keys** Create your API credentials and note: - Your username - Your API key ### Install Certbot and Dependencies ```bash apt update apt install certbot python3-pip -y ``` Install the Namecheap DNS plugin: ```bash pip install certbot-dns-namecheap ``` ### Configure API Credentials Create the credentials file: ```bash nano /etc/letsencrypt/namecheap.ini ``` Add your API credentials: ```ini dns_namecheap_username = your_username dns_namecheap_api_key = your_api_key ``` Secure the file: ```bash chmod 600 /etc/letsencrypt/namecheap.ini ``` ### Generate SSL Certificate Request the certificate with wildcard support: ```bash certbot certonly --dns-namecheap \ --dns-namecheap-credentials /etc/letsencrypt/namecheap.ini \ -d example.com -d *.example.com ``` **Expected output:** ```bash Successfully received certificate. Certificate is saved at: /etc/letsencrypt/live/example.com/fullchain.pem Key is saved at: /etc/letsencrypt/live/example.com/privkey.pem ``` ### Test Automatic Renewal Perform a dry run to validate the renewal process: ```bash certbot renew --dry-run ``` **Expected output:** ```bash Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/example.com.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Account registered. Simulating renewal of an existing certificate for example.com and *.example.com Waiting 10 seconds for DNS changes to propagate - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Congratulations, all simulated renewals succeeded: /etc/letsencrypt/live/example.com/fullchain.pem (success) ``` ### Verify Automatic Renewal Timer Check that the certbot timer is active: ```bash systemctl list-timers | grep certbot ``` **Expected output:** ```bash Tue 2024-11-05 02:22:10 UTC 3h 21min Mon 2024-11-04 17:16:51 UTC 5h 43min ago certbot.timer certbot.service ``` ## Post-Installation Steps After successfully setting up automatic SSL renewal: ### Update Gateway Configuration Configure your ar.io Gateway to use the new certificates. Update your gateway's SSL configuration to point to: - **Certificate:** `/etc/letsencrypt/live/your-domain.com/fullchain.pem` - **Private Key:** `/etc/letsencrypt/live/your-domain.com/privkey.pem` ### Reload Web Server (Optional) If you're using nginx or another web server, reload it to apply the new certificates: ```bash systemctl reload nginx ``` ### Monitor Renewal Process Certbot automatically sets up a systemd timer for renewal. Certificates will be renewed when they have 30 days or less remaining. To manually check renewal status: ```bash certbot certificates ``` ## Troubleshooting ### Common Issues - **DNS propagation delays:** Wait 5-10 minutes for DNS changes to propagate - **API rate limits:** Check your DNS provider's API rate limits - **Permission errors:** Ensure credential files have correct permissions (600) ### Logs and Debugging Check certbot logs for detailed error information: ```bash tail -f /var/log/letsencrypt/letsencrypt.log ``` ## Next Steps With SSL certificates automated, consider: - [Setting up monitoring](/build/extensions/grafana) to track certificate expiration - [Configuring gateway filters](/build/run-a-gateway/manage/filters) for optimal performance - [Implementing content moderation](/build/run-a-gateway/manage/content-moderation) policies # Troubleshooting (/build/run-a-gateway/manage/troubleshooting) Welcome to the comprehensive troubleshooting and FAQ resource for ar.io Gateway operators. Use the quick lookup table below for fast answers, or browse the detailed sections for in-depth guidance. ## Quick Lookup Below is a quick summary of what you should check when troubleshooting your gateway. Find more detailed information in the sections below. | Issue | What to Check | | ------------------------------------------------------------ | ------------------------------------------------------------------------- | | My release number is wrong | Pull the latest github updates and make sure you are on the `main` branch | | Gateway appears offline on Viewblock or https://gateways.ar.io | Probably fine, but verify that your gateway is still running. | | '/ar-io/observer/reports/current' just says "report pending" | Normal behavior, wait for the report to complete. | | Observer error "Cannot read properties of undefined" | Normal behavior, Observer is checking for data not implemented yet. | | Observing my gateway shows failures | Check `AR_IO_WALLET` and `ARNS_ROOT_HOST` settings. | | Updated .env settings not reflected on gateway | Rebuild your gateway after editing .env file. | | Out of disk space error | Check for inode exhaustion and delete files if necessary. | | Can't load ArNS names | Check `ARNS_ROOT_HOST` setting in .env file, and DNS records. | | "Your connection is not private" error | Generate or renew SSL certificates. | | 404/Nginx error when accessing domain | Check Nginx settings and restart Nginx if necessary. | | 502 error from Nginx | Check for errors in your gateway. | | Trouble generating SSL certificates | Ensure TXT records have propagated and follow certbot instructions. | ## General Troubleshooting ### My Gateway Seems to be Running but... If your release number when you go to `/ar-io/info` is lower than the current release, you simply need to upgrade your gateway in order to reach the latest release. If your release number includes the suffix "-pre" it means you are running your gateway from the development branch of the github repository, instead of the main branch. The development branch is used for staging work that the engineering team is in the middle of. Because of this, it can be much less stable than the main branch used for production and can cause significant issues. Ensure that you are running the latest release, from the main branch, by running the below commands in your terminal: ```console sudo docker compose down --rmi all git checkout main git pull sudo docker compose up -d ``` If this doesn't resolve the issue, you can also try a more extreme method of clearing out the incorrect docker images: ```console sudo docker compose down sudo docker system prune -a sudo docker compose up -d ``` Viewblock and https://gateways.ar.io use a very simple ping method for determining if a gateway is "up". There are plenty of reasons why this ping may fail while the gateway is running perfectly, so showing as down is not cause for concern. Just verify that your gateway is still running, and wait. Your gateway will show as up again soon. /ar-io/observer/reports/current just says 'report pending'"> This is normal. Your Observer is working to generate a report and that report will be displayed once it is complete. This is not an issue with your observer. The short explanation is that your Observer is looking for tasks assigned to it by the ar.io network contract, but there isnt anything there. You can safely ignore this error message. When observing a gateway, there are two main pass/fail tests. "Ownership" and "ArNS Assessment" - Ownership: This tests to see if the value set in your gateway `AR_IO_WALLET` value (in .env) matches the wallet used to join ar.io. If they don't match, update the value in your .env file and restart your gateway. - ArNS Assessment: This tests to see if a gateway is able to resolve ArNS names correctly. The first thing you should check is if you have the `ARNS_ROOT_HOST` value set in your .env file. If not, set the value and restart your gateway. If this value is set, check to make sure you have current DNS records and SSL certificates for wildcard subdomains on your gateway. Once you edit your .env file, you need to "rebuild" your gateway for the changes to take effect. As of release 3, every time you start your gateway with `docker compose` it is automatically rebuilt. So all you need to do is shut your gateway down and restart it. The most likely cause of this is inode exhaustion. Test this by running the command: ``` df -i ``` If one of the lines in the output says 100%, you have run out of inodes and so your filesystem is not capable of creating new files, even if you have available space. The solution is to delete files from your `data` folder in order to free up inodes. This was a common issue prior to release #3, when Redis caching was introduced to reduce the number of small files created. If you are using an older version of the gateway, consider upgrading to mitigate the risk of inode exhaustion. The first thing you should check if your gateway is not resolving ArNS names is that you have `ARNS_ROOT_HOST` set in your .env file. If not, set it to your domain name used for the gateway. For example, `ARNS_ROOT_HOST=turbo-gateway.com`. Once this value is set, restart your gateway for the changes to take effect. If that doesn't resolve the issue, check your dns records. You need to have a wildcard subdomain ( \*. ) set with your domain registrar so that ArNS names will actually point at your gateway. You can set this record, and generate an SSL certificate for it, in the same way you set the records for your primary domain. This error message means that your SSL certificates have expired. You need to renew your certificates by running the same certbot command you used when you initially started your gateway: ``` sudo certbot certonly --manual --preferred-challenges dns --email -d .com -d '*..com' ``` Certbot SSL certificates expire after 90 days, and you will need to rerun this command to renew every time. If you provide an email address, you will receive an email letting you know when it is time to renew. If you navigate to your domain and see a 404 error from Nginx (the reverse proxy server used in the setup guide) it means that your domain is correctly pointed at the machine running your gateway, but you have not properly configured your Nginx settings (or your gateway is not running). The [Set up Networking](./linux-setup.md#set-up-networking) section of the setup guide has detailed instructions on configuring your Nginx server. If all else fails, try restarting Nginx, that usually clears any issues with the server clinging to old configurations. ``` sudo service nginx restart ``` A 502 error from Nginx means that Nginx is working correctly, but it is receiving an error from your gateway when it tries to forward traffic. When using the manual certbot command provided in the setup guide: ``` sudo certbot certonly --manual --preferred-challenges dns --email -d .com -d '*..com' ``` You need to be sure that you are waiting after creating your TXT records for them to completely propagate. You can check propagation using a tool like [dnschecker.org](https://dnschecker.org). If you continue to have issues, you can check the [official certbot instructions guide](https://certbot.eff.org/instructions). - Visit your gateway in a browser and see if your SSL certs are expired. This is the most common issue causing sudden stops in proper operation. - Try restarting nginx, it sometimes has trouble looking at the new certs without a restart. - Make sure `ARNS_ROOT_HOST` is properly set in your `.env` file. Updating this requires restarting your gateway. - Make sure you have a DNS record set for `*.`. Since ArNS names are served as subdomains, you need to make sure all subdomains are pointed at your gateway. - If your gateway is attempting to resolve the name, but times out, it's most likely a CU issue. - ar.io gateways are very robust, they can handle temporary errors gracefully and not affect normal operation. You should only be concerned if the error is consistent or it is causing your gateway to not function properly. - Observers generate and submit their reports at specific times throughout the epoch. This is to ensure a healthy network throughout the entire epoch, not just at the start. - Your observer wallet must match the observer wallet associated with your gateway in the ar.io contract. You can check this by navigating to your gateway in https://gateways.ar.io. - This happens when a request to a CU fails, and your gateway receives an html failure message instead of the expected JSON response. This will normally clear up on its own after congestion on that CU dies down, but if it is persistent try switching to a different CU. - This is normal. It means you have reached the current Arweave block and need to wait for more before you can index them. - This is normal. If a gateway fails to resolve an arns name within 3 seconds, it will fall back to a trusted gateway (turbo-gateway.com by default) to help resolve the name. - There are many reasons a gateway could fail an epoch. Following these steps is usually enough to identify and correct the issue: - Try to visit your gateway in a browser and see if your SSL certs are bad - Try to resolve an ArNS name on your gateway. If it fails to resolve, check the console and your gateway logs for errors - Look at the observation reports that failed your gateway, they will list the reason for failure ## Troubleshooting Failed Epochs ### Overview The ARIO Network provides several tools to help troubleshoot problems with a gateway. The most powerful among these is the [Observer](/learn/oip). The Observer, which is a component of every gateway joined to the ARIO Network, checks all gateways in the network to ensure that they are functioning properly, and returning the correct data. The Observer then creates a report of the results of these checks, including the reasons why a gateway might have failed the checks. If a gateway fails the checks from more than half of the prescribed observers, the gateway is marked as failed for the epoch, and does not receive any rewards for that epoch. The first step in troubleshooting a failed gateway is always to attempt to resolve data on that gateway in a browser, but if that does not make the issue clear, the Observer report can be used to diagnose the problem. ### Manual Observation Manual observations may be run on a gateway at any time buy using the [Network Portal](https://gateways.ar.io). This allows operators (or anyone with an interest in the gateway's performance) to check the gateway's performance at any time. To run a manual observation: 1. Navigate to the [Network Portal](https://gateways.ar.io) 2. Select the gateway you are interested in from the list of gateways 3. Click on the "Observe" button in the top right corner of the page. ![Diagram](https://turbo-gateway.com/0G52dTWe65abQ6qDGvI99ERAaGU7DHR9srimJXnYRGA) 4. Click on the "Run Observation" button in the bottom right corner of the page. ![Diagram](https://turbo-gateway.com/A_B_58rufQ0Pj4ri0AKuC0DJn61u5ayO5ONWpkMerQw) Two randomly selected ArNS names will be entered automatically in the "ArNS names" field to the left of the "Run Observation" button. These can be changed, or additional ArNS names can be added to the list before running the observation. The Manual observation will run the same checks as the observer, and will display the results on the right side of the page. ![Diagram](https://turbo-gateway.com/vgRXfbx4fa47qGDpjndq128VCHl1wajKaq464KeA0Qg) ### Accessing the Observer Report The simplest way to access an observer report is via the [Network Portal](https://gateways.ar.io), following the steps below: 1. Navigate to the [Network Portal](https://gateways.ar.io) 2. Select the gateway you are interested in from the list of gateways 3. In the Observation window, select the epoch you are interested in. This will display a list of the observers that failed the gateway for that epoch. 4. Click on the "View Report" button to the right any observer on that list. This will display the entire report that observer generated. ![Diagram](https://turbo-gateway.com/ynbxYU_8xLRaU1D6a_LMoUq00roWwsMKgr-xrsDE0Sk) 5. Locate the gateway you are interested in in the report, and click on that row. This will display the report for that gateway. ### Understanding the Observer Report The observer report will display a list of checked ArNS names, and a reason if the gateway failed to return the correct data for that name. There are several reasons why a gateway might fail to return the correct data for an ArNS name. Below is a list of the most common reasons, and how to resolve them. #### Timeout awaiting 'socket', or Timeout awaiting 'connect' ![Diagram](https://turbo-gateway.com/_GupbMa-EW_wiCD201MuOkDQrLT0MXTfxDXhLSDmyh4) ![Diagram](https://turbo-gateway.com/0WkzxdyN-9hJfv0pSiTs0Ozg_wqFvE3-OWlgzYPimtU) This failure means that the observer was unable to connect to the gateway when it tried to check the ArNS name. There are lots of reasons why this might happen, many of them unrelated to the gateway itself. If an observer report has a small number of these failures, among a larger number of successful checks, it is unlikely to be an issue with the gateway. If this failure occurs persistently for a large number, or all ArNS names checked, it likely means that the observer is having trouble connecting to the gateway at all. You can verify this by: - Attempting to connect to the gateway in a browser - Running manual observations on the gateway using the [Network Portal](https://gateways.ar.io) - Using tools like `curl` or `ping` to check the gateway's connectivity If these methods consistently fail to connect to the gateway, it is likely that the gateway is not properly configured or powered on. If this is the case: - Check Docker and the gateway's logs to see if the gateway is on. - Ensure that the SSL certificates are valid for the gateway's domain. - Check DNS records for the gateway's domain, misconfigured or conflicting DNS records can cause connectivity issues. Some gateway operators who run their gateways on their personal home networks have also reported issues with their ISP blocking, throttling, or otherwise delaying traffic to a gateway. If none of the above steps resolve the issue, it may be worth checking with your ISP to see if they are blocking or throttling traffic to the gateway. Using [Grafana](/build/extensions/grafana) can also provide a visual representation of the gateway's ArNS resolution times. If this is consistently high (above 10 seconds), it is likely that the gateway is not properly configured to resolve ArNS names. Ensure that the gateway is operating on the latest Release. #### Cert has expired This failure means that the gateway's SSL certificate has expired. Obtaining a new SSL certificate and updating the gateway's reverse proxy (nginx, etc) configuration to use the new certificate is the only solution to this issue. #### dataHashDigest mismatch ![Diagram](https://turbo-gateway.com/xXe0bHne--0JJv-HRf5HT9R1V1UbzaOh2AxvAdQZhjg) This failure means that the gateway did respond to a resolution request, but the data it returned did not match the data that was expected. This could be due to a number of reasons, including: - Cached data was returned by the gateway that doesnt match the most current data on the network. - The gateway is configured to operate on testnet or devnet. Gateways joined to the ARIO Network MUST operate on mainnet in order to pass observation checks. - The gateway is intentionally returning fraudulent data. A gateway will not return fraudulent data unless that operator intentionally rewrote the gateway's code to do so, and a major purpose of the Observation and Incentive Protocol is to catch and prevent this behavior. A gateway may return mistaken data on occasion, usually due to a cache mismatch between the gateway and the observer's authority (usually arweavae.net). This is a relatively rare occurrence, and should only be considered an issue if it occurs persistently. If most or all of the ArNS names checked are failing for this reason, it is likely that the gateway is not operating on mainnet. #### Response code 502 (Bad Gateway) ![Diagram](https://turbo-gateway.com/NBQsYUKP6IZt_rYg77QXgzwUUPvimFGXCQqtesbW1_I) This failure means that the observer was able to connect to the gateway's network, but the reverse proxy returned a 502 error. This is almost always a reverse proxy issue. Ensure that the gateway's reverse proxy is running, and that it is configured to forward requests to the gateway. Testing the validity of the reverse proxy's configuration file (`sudo nginx -t` on Nginx) may provide more information about the issue, and restarting the reverse proxy (`sudo nginx -s reload`) often resolves the issue if there are no problems with the configuration file. It is also possible that the gateway itself is not running at all. Check Docker and the gateway's logs to see if the gateway is on. #### Response code 503 (Service Unavailable) ![Diagram](https://turbo-gateway.com/7eFKSm-cs81-aJ_H4xkolR2nSlxl5tYWXJdFTei8Dbs) This failure means that the observer was able to connect to the gateway's network, but the reverse proxy was unable to forward the request to the gateway. It differs from the 502 error in that the reverse proxy is likely able to see that the gateway is running, but is unable to communicate with it. This is often a temporary issue, caused by the gateway not being able to handle a heavy load of requests, or the gateway being in the process of restarting. If this failure occurs once or twice in a report, it is likely a temporary issue and should not be considered an issue with the gateway. However, when this failure occurs persistently, particularly for every ArNS name checked on the report, it is likely that the gateway may have crashed. Manually restarting the gateway can likely resolve the issue. #### connect EHOSTUNREACH ![Diagram](https://turbo-gateway.com/O-uG-yGm5bNxjw2ADH_yBjOcGo-ZEiFym8GeFZZNueY) This failure means that the observer was unable to connect to the gateway at all. The connection was either refused, or the gateway was not able to find a target based on the domain name's DNS records. This is almost always an issue with DNS records or local network configuration. Ensure that the gateway domain has correct DNS records, and that the local network is set up to allow connections. Checking logs from the local network's reverse proxy (nginx, etc) may provide more information about the issue. #### getaddrinfo ENOTFOUND ![Diagram](https://turbo-gateway.com/WJDwW0NM29uKC-9puvhXK_n75vgFXLpa6VKFVMhRsLQ) This is another DNS related issue. Likely, the gateway does not have a valid DNS record either for the top level domain or the required wildcard subdomain. Having this failure occur once or twice in a report could mean that the DNS server being used by the observer is having temporary issues and should not be considered an issue with the gateway. However, when this failure occurs persistently, particularly for every ArNS name checked on the report, it is likely that the gateway's DNS records are not set, or are misconfigured. #### Hostname/IP does not match certificate's altnames: Host: \. is not in the cert's altnames: DNS:\ ![Diagram](https://turbo-gateway.com/HfOfpAYm811dWFPNQC7bANEvjGVK4ch3kO7K7qMN9qs) This failure means that the observer's SSL certificate does not match the gateway's domain name. This is almost always an issue with the gateway's SSL certificate. This most likely occurred because the gateway's operator did not update the gateway's SSL certificate when the gateway's domain name was changed. Obtaining a new SSL certificate and updating the gateway's reverse proxy configuration to use the new certificate is the only solution to this issue. #### write EPROTO \:error:\:SSL routines:ssl3_read_bytes:tlsv1 unrecognized name:\:SSL alert number 112 ![Diagram](https://turbo-gateway.com/Hbip_ZmqmFN8-uXijw1aylyYp1YllwgyZTNAcsPCxSg) This failure almost always means that the gateway operator did not properly obtain SSL certificates for the gateway's wildcard subdomain. Obtaining a new SSL certificate and updating the gateway's reverse proxy configuration to use the new certificate is the only solution to this issue. ## FAQ - Gateway protocol rewards are calculated as 0.1% of the protocol balance (0.05% after August 2025) split between all gateways in the network. A change in the protocol balance or the number of gateways in the network between epochs will result in the reward for an individual gateway changing. - The Observer rewards are separate from protocol rewards, and if your gateway is selected as an observer for an epoch, assuming it performs its duties well, it will receive additional rewards The observer selection process uses a weighted random selection method that considers multiple factors beyond just stake: - **Stake Weight (SW)**: Ratio of your total staked ARIO tokens (including delegated stake) to the network minimum - **Tenure Weight (TW)**: How long your gateway has been part of the network (capped at 4 after 2 years) - **Gateway Performance Ratio Weight (GPRW)**: Ratio of epochs where you correctly resolved names vs total participation - **Observer Performance Ratio Weight (OPRW)**: Ratio of epochs where you successfully submitted reports vs total observer periods A composite weight (CW) is calculated as: CW = SW × TW × GPRW × OPRW Up to 50 gateways are chosen as observers per epoch. If there are more than 50 gateways, selection is randomized based on these normalized weights. Even with a high stake, other factors like performance and tenure affect your chances of being selected. - There is a 30 day locking period when withdrawing delegated stake or excess operator stake from your gateway. This locking period can be skipped, for a fee. The fee starts at 50% of the withdrawal amount, and goes down over time. If you selected instant withdrawal, you paid the fee to skip the locking period. The required network-join stake follows the separate network leave process. - The minimum operator stake for gateways (20,000 ARIO) cannot be withdrawn while the gateway is active. It can only be returned through the network leave process, and it is unrecoverable if the gateway is pruned for sustained poor performance. - If possible, leave your original server running while you prepare the new one - Set up the new server following the same steps you used to set up the original server - This includes setting up SSL certificates for the new server - You must use the same gateway wallet when setting up the new server - The observer wallet may be changed at any point, but requires extra steps. It is recommended you use the original observer wallet as well - Once the new server is set up, change your DNS A records to point at the new server - After your DNS records are set and you have verified your gateway is operating correctly, shut down the original server - No changes need to be made in the network contract or on https://gateways.ar.io - Yes - Configure your new domain to point at your gateway, including setting up SSL certificates - Update your NGINX (or other reverse proxy) server to recognize the new domain. This usually requires a restart of NGINX - Update the `ARNS_ROOT_HOST` variable in your `.env` and restart the gateway - Using https://gateways.ar.io, update your gateway settings to change the FQDN in the contract - Your gateway is now using the new domain name for normal operation. ## Getting Help If you encounter any issues during the troubleshooting process, please seek assistance from the [ar.io community](https://discord.com/invite/HGG52EtTc2). **Ready to get back to building?** Once your gateway is running smoothly, check out [Manage your Gateway](/build/run-a-gateway/manage) for guides on optimization, monitoring, and more. # Upgrading your Gateway (/build/run-a-gateway/manage/upgrading-a-gateway) To ensure the optimal performance and security of your ar.io Gateway, it's essential to regularly upgrade to the latest version. Notably, indexed data resides separate from Docker. As a result, neither upgrading the Gateway nor pruning Docker will erase your data or progress. Here's how you can perform the upgrade: ## Prerequisites - Your Gateway should have been cloned using git. If you haven't, follow the [installation instructions](/build/run-a-gateway/join-the-network). ## Checking your Release Number Effective with release 3, you can view the currently implemented release on any gateway by visiting `https:///ar-io/info` in a browser. Be sure to replace `` with the domain of the gateway you are checking. If the release number displayed includes `-pre` it means that your gateway is using the `develop` branch of the github repo for the gateway code. Follow steps in our [troubleshooting guide](/build/run-a-gateway/manage/troubleshooting) to switch over to the more stable `main` branch. Announcements will be made in our [discord server](https://discord.com/invite/HGG52EtTc2) showing each new release. ## Quick Start ### Pull Latest Changes Navigate to your cloned repository directory and execute: ```bash git pull ``` ### Shut Down Docker Stop your gateway: ```bash sudo docker compose down -v ``` ```bash docker compose down -v ``` ### Restart Gateway Start your gateway with the new version: ```bash sudo docker compose up -d ``` ```bash docker compose up -d ``` Effective with Release #3, it is no longer required to include the `--build` flag when starting your gateway. Docker will automatically build using the image specified in the `docker-compose.yaml` file. ## Detailed Upgrade Process ### Full Upgrade Process ### Pull Latest Changes Navigate to your cloned repository directory and execute: ```bash git pull ``` ### Shut Down Docker Stop your gateway: ```bash sudo docker compose down -v ``` ```bash docker compose down -v ``` ### Check for New Environment Variables Read the update release change logs and community announcements to see if the new version includes any new environmental variables that you should set before restarting your gateway. ### Restart the Gateway Start your gateway with the new version: ```bash sudo docker compose up -d ``` ```bash docker compose up -d ``` Effective with Release #3, it is no longer required to include the `--build` flag when starting your gateway. Docker will automatically build using the image specified in the `docker-compose.yaml` file. ### Docker Pruning (Optional) It's a good practice to clean up unused Docker resources after shutting down your gateway. This will erase all inactive docker containers on your machine. If you use docker for anything beyond running a gateway be extremely careful using this command. ### Shut Down Gateway First, stop your gateway: ```bash sudo docker compose down -v ``` ```bash docker compose down -v ``` ### Prune Docker System Clean up unused Docker resources: ```bash sudo docker system prune ``` ```bash docker system prune ``` ### Restart Gateway Start your gateway: ```bash sudo docker compose up -d ``` ```bash docker compose up -d ``` ### Checking for New Environment Variables New gateway releases may introduce new environment variables that you need to configure. ### Review Release Notes Check the release notes and community announcements for any new environment variables: - Review the [GitHub releases](https://github.com/ar-io/ar-io-node/releases) - Check the [ar.io Discord](https://discord.com/invite/HGG52EtTc2) for announcements - Look for changes in the `.env.example` file ### Update Your .env File Add any new environment variables to your `.env` file: ```bash # Example: Add new environment variables NEW_FEATURE_ENABLED=true NEW_CONFIG_VALUE=default_value ``` ### Restart Gateway Restart your gateway to apply the new environment variables: ```bash sudo docker compose up -d ``` ```bash docker compose up -d ``` That's it! Your ar.io Gateway is now upgraded to the latest version. Ensure to test and verify that everything is functioning as expected. If you encounter any issues, reach out to the [ar.io community](https://discord.com/invite/HGG52EtTc2) for assistance. # Verification & Trust Headers (/build/run-a-gateway/manage/verification-headers) Ar.io gateways attach HTTP response headers to every data response that communicate whether the data has been verified against the Arweave base layer, whether it has reached finality, and whether the source is trusted. These headers enable clients to make informed trust decisions without relying solely on the gateway operator's reputation. ## Trust & Verification Headers Every data response from an ar.io gateway includes the following headers: | Header | Values | Description | |--------|--------|-------------| | `X-AR-IO-Verified` | `true` / `false` | `true` only when data is served from local cache **and** has been verified against the Arweave base layer. Data streamed from network sources is marked `false` even if the gateway's database indicates prior verification, because the hash cannot be confirmed in-flight during streaming. | | `X-AR-IO-Stable` | `true` / `false` | `true` when the data exists beyond Arweave's maximum fork depth, indicating finality. Once stable, the data cannot be reorganized out of the chain. | | `X-AR-IO-Trusted` | `true` / `false` | `true` when the data was retrieved from a source the operator has configured as trusted (via `TRUSTED_GATEWAY_URL` or `TRUSTED_GATEWAYS_URLS`). | | `Content-Digest` | `sha-256=:base64:` | SHA-256 hash of the response body in [RFC 9530](https://www.rfc-editor.org/rfc/rfc9530.html) format. Only set on cache hits and HEAD requests, since the hash cannot be computed during network streaming. Enables direct body integrity verification by clients. | | `X-AR-IO-Digest` | base64url string | The raw SHA-256 hash of the content in base64url encoding. Set alongside `Content-Digest`. | | `ETag` | `"hash"` | Content hash for HTTP conditional requests (`If-None-Match`). Enables 304 Not Modified responses for efficient caching. | | `X-AR-IO-Data-Id` | base64url string | The Arweave transaction ID or data item ID of the content being served. | | `X-AR-IO-Hops` | number | The number of inter-gateway hops the request traversed before reaching this gateway. | | `X-Cache` | `HIT` / `MISS` | Whether the data was served from the gateway's local cache or fetched from the network. | ### Why X-AR-IO-Verified Can Be False for Verified Data A common source of confusion: data may be verified in the gateway's database but still return `X-AR-IO-Verified: false`. This happens when the data is streamed from a network source rather than served from local cache. During streaming, the gateway cannot compute a hash of the data in-flight to confirm it matches the verified hash. Only when the data is served from local cache can the gateway guarantee that the bytes being sent match what was previously verified. ``` # Cache hit - verified data served from local storage X-AR-IO-Verified: true X-Cache: HIT Content-Digest: sha-256=:4ROTs2lTPAKbr8Y41WrjHu+2q+7S+m+yTuO7fAUzZI4=: # Network fetch - same data, but hash can't be confirmed in-flight X-AR-IO-Verified: false X-Cache: MISS # Content-Digest is NOT set (can't hash during streaming) ``` ## ANS-104 Bundle Headers For data items served from [ANS-104 bundles](https://specs.ar.io/?tx=fFJEsPOLDJKRfNgaeaRPlfSvcNtHp0Y3JVxD-HHzmwA), gateways include additional headers in two categories. ### Position Headers These headers describe the data item's location within its parent bundle, allowing clients to independently verify inclusion: | Header | Description | |--------|-------------| | `X-AR-IO-Root-Transaction-Id` | The root Arweave transaction containing the bundle | | `X-AR-IO-Data-Item-Offset` | Byte offset of this data item within the bundle | | `X-AR-IO-Data-Item-Size` | Size of the data item in bytes | | `X-AR-IO-Data-Item-Data-Offset` | Byte offset of the data payload within the data item | | `X-AR-IO-Root-Item-Offset` | Offset of the root item in the bundle | | `X-AR-IO-Root-Item-Size` | Size of the root item | ### Metadata Headers ANS-104 fields are exposed at the HTTP layer so clients can read data item tags and owner information directly from response headers without parsing the binary format: | Header | Description | |--------|-------------| | `X-Arweave-Owner` | The raw owner public key of the data item | | `X-Arweave-Owner-Address` | The data item signer's wallet address (derived from the owner key) | | `X-Arweave-Tag-*` | One header per tag, with the tag name as the suffix (e.g., `X-Arweave-Tag-Content-Type`) | | `X-Arweave-Tags-Truncated` | Set when the tag set exceeds a configurable byte budget, indicating partial tag exposure | ## Client-Side Verification Clients can verify gateway responses at multiple levels, from lightweight header checks to full cryptographic verification: ### Body Integrity Compare the `Content-Digest` header (when present) against a locally computed SHA-256 hash of the response body. For Arweave content, the `X-AR-IO-Data-Id` is itself a content hash - clients can walk the Arweave or ANS-104 signature chain from this ID to verify the content. ```javascript // Verify Content-Digest in a browser const response = await fetch('https://gateway.example/raw/TX_ID'); const body = await response.arrayBuffer(); const hash = await crypto.subtle.digest('SHA-256', body); const base64Hash = btoa(String.fromCharCode(...new Uint8Array(hash))); const digestHeader = response.headers.get('Content-Digest'); // Header format: sha-256=:base64hash=: // Extract the hash between the colons const match = digestHeader?.match(/sha-256=:(.+?):/); const verified = match?.[1] === base64Hash; console.log('Integrity check:', verified); ``` ### Cross-Gateway Comparison Query the same data from multiple gateways and compare their response headers. If `X-AR-IO-Verified: true` and `Content-Digest` values match across independent gateways, the data is authentic with high confidence. ### Wayfinder Verification The [Wayfinder](/learn/wayfinder) protocol provides automated client-side routing and verification across the gateway network. It supports configurable strategies including balanced (random), fastest ping, and static gateway selection, with built-in response verification. ## HTTP Message Signatures (RFC 9421) Gateways can opt in to signing response headers using [RFC 9421 HTTP Message Signatures](https://www.rfc-editor.org/rfc/rfc9421.html). When enabled, an Ed25519 sub-key signs all trust-relevant headers at response time, producing a `Signature` and `Signature-Input` header pair. This provides cryptographic proof that a specific gateway operator produced a given response. ### Enabling HTTP Signatures Add the following to your gateway's `.env` file: ```bash # Enable response signing HTTPSIG_ENABLED=true ``` #### Signing Key Selection When `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` is set, the gateway uses the observer's Solana keypair directly as the HTTPSIG signing key. Verifiers derive the Solana address from the public key in the `keyId` and look it up in the on-chain Gateway Registry — no separate attestation is needed. When neither observer env is set, the gateway auto-generates a standalone Ed25519 key at `data/keys/httpsig.pem`. Responses are still signed, but the signer can't be tied back to the on-chain registry. If you use a single Solana key for both operator and observer (Pattern 1 in the [migration guide](/build/run-a-gateway/manage/solana-migration#supported-configurations)), HTTPSIG falls back to the auto-generated key because the gateway only piggybacks on `OBSERVER_KEYPAIR_PATH`/`OBSERVER_PRIVATE_KEY` when they're **explicitly set**. To get on-chain-verifiable HTTPSIG with a single key, explicitly set `OBSERVER_KEYPAIR_PATH` (or `OBSERVER_PRIVATE_KEY`) to the same value as your operator key. ### Configuration Reference | Variable | Default | Description | |----------|---------|-------------| | `HTTPSIG_ENABLED` | `false` | Enable RFC 9421 response signing | | `HTTPSIG_KEY_FILE` | `data/keys/httpsig.pem` | Path to standalone Ed25519 private key (auto-generated if missing). Ignored when `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` is set | | `HTTPSIG_BIND_REQUEST` | `true` | Include request method and path in signature (prevents replay) | | `OBSERVER_KEYPAIR_PATH` | - | Path to a 64-byte Solana keypair JSON. When set, used as the HTTPSIG signing key — verifiable against the on-chain GAR | | `OBSERVER_PRIVATE_KEY` | - | Alternative: base58-encoded 64-byte secret. Mutually exclusive with the file form | ### Verifying It's Working Once enabled, check the `/ar-io/info` endpoint to confirm: ```bash curl -s https://your-gateway.example/ar-io/info | jq '.httpsig' ``` The response includes the public key, key ID, Solana address, and attestation details (if configured). Signed responses will include `Signature` and `Signature-Input` headers on all trust-relevant data responses. ### What Gets Signed The gateway signs all trust-relevant response headers but **not the response body**. Body integrity is achieved through the signed `X-AR-IO-Data-Id` (which is a content hash) and the `Content-Digest` header. Header-only signing preserves streaming performance for large responses. **Signed headers** (when present on the response): - Trust headers: `X-AR-IO-Data-Id`, `X-AR-IO-Verified`, `X-AR-IO-Stable`, `X-AR-IO-Trusted` - Bundle headers: `X-AR-IO-Root-Transaction-Id`, `X-Arweave-Owner-Address`, `X-Arweave-Tags-Truncated` - ArNS headers: `X-ArNS-Name`, `X-ArNS-Resolved-Id`, `X-ArNS-TTL-Seconds`, `X-ArNS-Process-Id` - Chunk headers: `X-Arweave-Chunk-Data-Root`, `X-Arweave-Chunk-Tx-Id`, `X-AR-IO-Chunk-Source-Type` - Content headers: `Content-Type`, `Content-Digest` - All `X-Arweave-Tag-*` headers (dynamically matched) - Request components (when `HTTPSIG_BIND_REQUEST=true`): `@method`, `@path`, `@status` Only responses containing at least one trust-relevant header are signed. Non-data responses (health checks, info endpoint, errors) are not signed. ### Verification Chain Clients can verify signed responses through the following chain: 1. **Signature verification** — The public key is embedded in the `Signature-Input` header's `keyid` parameter and is verifiable via the Web Crypto API in modern browsers. 2. **Identity verification** — When the observer Solana key is used for signing, derive the Solana address from the public key and look it up in the on-chain Gateway Registry (GAR). A match confirms the signer is a registered gateway operator. 3. **Body integrity** — Compare the `Content-Digest` header against a locally computed hash, or walk the Arweave signature chain from the signed `X-AR-IO-Data-Id`. ## Related } /> } /> } /> } /> # x402 Payment Setup (/build/run-a-gateway/manage/x402-setup) Learn how to set up x402 payment protocol on your ar.io Gateway to monetize data egress. For information about what x402 is and how gateways leverage it, see our [x402 Payments guide](/learn/gateways/x402-payments). **Release Requirement**: x402 payments first became available with gateway Release 56. Ensure your gateway is running Release 56 or later to use this feature. ## Prerequisites - Running ar.io Gateway - Administrative access to your server - USDC wallet for receiving payments - API access to x402 payment facilitator **Important**: x402 payments require the rate limiter to be enabled. There is no "payments only" configuration - both systems must work together. ## Quick Start ### Enable Rate Limiter Configure basic rate limiting in your `.env` file: ```bash # Enable rate limiter (required for x402) ENABLE_RATE_LIMITER=true RATE_LIMITER_TYPE=redis RATE_LIMITER_IP_TOKENS_PER_BUCKET=100000 RATE_LIMITER_IP_REFILL_PER_SEC=20 RATE_LIMITER_RESOURCE_TOKENS_PER_BUCKET=1000000 RATE_LIMITER_RESOURCE_REFILL_PER_SEC=100 ``` ### Configure x402 Payments Add x402 configuration to your `.env` file: ```bash # Enable x402 payments ENABLE_X_402_USDC_DATA_EGRESS=true X_402_USDC_NETWORK=base-sepolia # or base for production X_402_USDC_WALLET_ADDRESS=0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb X_402_USDC_FACILITATOR_URL=https://x402.org/facilitator X_402_USDC_PER_BYTE_PRICE=0.0000000001 X_402_USDC_DATA_EGRESS_MIN_PRICE=0.001 X_402_USDC_DATA_EGRESS_MAX_PRICE=1.00 X_402_RATE_LIMIT_CAPACITY_MULTIPLIER=10 ``` ### Restart Gateway Apply the configuration changes: ```bash docker compose down docker compose up -d ``` ### Test Configuration Verify x402 is working by checking gateway logs: ```bash docker compose logs -f core | grep -i "x402\|payment" ``` ## Detailed Configuration ### Testnet Configuration Perfect for development and testing with free testnet USDC: ### Configure Rate Limiter Set up memory-based rate limiting for single-node deployment: ```bash # Rate limiter configuration ENABLE_RATE_LIMITER=true RATE_LIMITER_TYPE=memory RATE_LIMITER_IP_TOKENS_PER_BUCKET=100000 RATE_LIMITER_IP_REFILL_PER_SEC=20 RATE_LIMITER_RESOURCE_TOKENS_PER_BUCKET=1000000 RATE_LIMITER_RESOURCE_REFILL_PER_SEC=100 ``` ### Configure x402 Testnet Set up x402 with Base Sepolia testnet: ```bash # x402 testnet configuration ENABLE_X_402_USDC_DATA_EGRESS=true X_402_USDC_NETWORK=base-sepolia X_402_USDC_WALLET_ADDRESS=0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb X_402_USDC_FACILITATOR_URL=https://x402.org/facilitator X_402_USDC_PER_BYTE_PRICE=0.0000000001 X_402_USDC_DATA_EGRESS_MIN_PRICE=0.001 X_402_USDC_DATA_EGRESS_MAX_PRICE=1.00 X_402_RATE_LIMIT_CAPACITY_MULTIPLIER=10 ``` ### Get Testnet USDC Obtain free testnet USDC from a faucet: 1. Visit [Base Sepolia Faucet](https://www.coinbase.com/faucets/base-ethereum-sepolia-faucet) 2. Connect your wallet 3. Request testnet USDC 4. Use the same wallet address in your configuration ### Test Payment Flow Test the payment system: ```bash # Make multiple requests to trigger rate limiting for i in {1..200}; do curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3000/TX_ID done # Check for 402 Payment Required responses curl -v http://localhost:3000/TX_ID ``` ### Mainnet Configuration Production setup with real USDC payments: ### Configure Redis Rate Limiter Set up distributed rate limiting for multi-node deployment: ```bash # Redis-based rate limiter ENABLE_RATE_LIMITER=true RATE_LIMITER_TYPE=redis RATE_LIMITER_REDIS_ENDPOINT=redis://redis:6379 RATE_LIMITER_IP_TOKENS_PER_BUCKET=100000 RATE_LIMITER_IP_REFILL_PER_SEC=20 RATE_LIMITER_RESOURCE_TOKENS_PER_BUCKET=1000000 RATE_LIMITER_RESOURCE_REFILL_PER_SEC=100 ``` ### Configure x402 Mainnet Set up x402 with Base mainnet: ```bash # x402 mainnet configuration ENABLE_X_402_USDC_DATA_EGRESS=true X_402_USDC_NETWORK=base X_402_USDC_WALLET_ADDRESS=0xYOUR_MAINNET_WALLET X_402_USDC_FACILITATOR_URL=https://facilitator.x402.rs X_402_USDC_PER_BYTE_PRICE=0.0000000001 X_402_USDC_DATA_EGRESS_MIN_PRICE=0.001 X_402_USDC_DATA_EGRESS_MAX_PRICE=1.00 X_402_RATE_LIMIT_CAPACITY_MULTIPLIER=10 ``` ### Set Up CDP Integration (Onramp) Configure Coinbase Developer Platform for browser paywall with easy USDC purchases: **What are CDP keys for?** These keys enable Coinbase Onramp integration in the browser paywall, allowing users to easily purchase USDC without leaving your gateway. They are **optional for testnet** but **required for mainnet** when using Coinbase facilitators. **Security Note**: - `X_402_CDP_CLIENT_KEY` is **PUBLIC** (safe for client-side use) - `CDP_API_KEY_ID` and `CDP_API_KEY_SECRET_FILE` are **SENSITIVE SECRETS** - CDP keys are for Onramp integration, NOT facilitator authentication ```bash # CDP configuration for Coinbase Onramp (browser paywall) CDP_API_KEY_ID=YOUR_API_KEY_ID CDP_API_KEY_SECRET_FILE=/app/secrets/cdp_secret_key X_402_CDP_CLIENT_KEY=YOUR_PUBLIC_CLIENT_KEY ``` Create the secret file: ```bash mkdir -p ./secrets chmod 700 ./secrets echo "YOUR_CDP_SECRET_KEY" > ./secrets/cdp_secret_key chmod 600 ./secrets/cdp_secret_key ``` **Note**: If both `CDP_API_KEY_SECRET_FILE` and `CDP_API_KEY_SECRET` are set, the file version takes precedence. ### Configure Redis Persistence Ensure paid tokens persist across restarts: ```bash # Redis persistence for paid tokens EXTRA_REDIS_FLAGS=--save 300 10 --appendonly yes --appendfsync everysec ``` ### Advanced Setup Options Customize your x402 implementation: ### Configure Paywall Customization Customize the payment interface: ```bash # Paywall customization X_402_APP_NAME=My ar.io Gateway X_402_APP_LOGO=https://example.com/logo.png ``` ### Set Up Chunk Pricing Configure fixed-size pricing for chunk requests: ```bash # Chunk pricing configuration CHUNK_GET_BASE64_SIZE_BYTES=368640 ``` ### Configure Allowlists Exempt specific IPs or ArNS names from rate limiting: ```bash # IP/CIDR allowlist (comma-separated) RATE_LIMITER_IPS_AND_CIDRS_ALLOWLIST=192.168.1.100,10.0.0.50 # ArNS name allowlist (comma-separated) RATE_LIMITER_ARNS_ALLOWLIST=trusted-app,premium-service ``` ### Set Up Monitoring Configure monitoring and metrics: ```bash # Enable detailed logging LOG_LEVEL=debug # Monitor payment metrics # Access metrics at: http://localhost:3000/ar-io/__gateway_metrics ``` ## Environment Variables Reference ### Required Variables | Variable | Description | Example | | ------------------------------- | ---------------------------------------- | ------------------------------ | | `ENABLE_RATE_LIMITER` | Enable rate limiting (required for x402) | `true` | | `ENABLE_X_402_USDC_DATA_EGRESS` | Enable x402 payments | `true` | | `X_402_USDC_NETWORK` | Base network (base-sepolia or base) | `base-sepolia` | | `X_402_USDC_WALLET_ADDRESS` | Your USDC wallet address | `0x742d35Cc...` | | `X_402_USDC_FACILITATOR_URL` | Payment facilitator URL | `https://x402.org/facilitator` | ### Pricing Configuration | Variable | Description | Default | | -------------------------------------- | ---------------------- | -------------- | | `X_402_USDC_PER_BYTE_PRICE` | Price per byte in USDC | `0.0000000001` | | `X_402_USDC_DATA_EGRESS_MIN_PRICE` | Minimum payment amount | `0.001` | | `X_402_USDC_DATA_EGRESS_MAX_PRICE` | Maximum payment amount | `1.00` | | `X_402_RATE_LIMIT_CAPACITY_MULTIPLIER` | Paid token multiplier | `10` | ### Rate Limiter Configuration | Variable | Description | Default | | ----------------------------------------- | ----------------------------------- | --------- | | `RATE_LIMITER_TYPE` | Rate limiter type (memory or redis) | `redis` (docker-compose), `memory` (standalone) | | `RATE_LIMITER_IP_TOKENS_PER_BUCKET` | IP bucket size | `100000` | | `RATE_LIMITER_IP_REFILL_PER_SEC` | IP refill rate | `20` | | `RATE_LIMITER_RESOURCE_TOKENS_PER_BUCKET` | Resource bucket size | `1000000` | | `RATE_LIMITER_RESOURCE_REFILL_PER_SEC` | Resource refill rate | `100` | ## Testing Your Configuration ### Verify Rate Limiting Test that rate limiting is working: ```bash # Make requests to trigger rate limits for i in {1..150}; do curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3000/TX_ID done # Should see 402 responses after rate limit is exceeded ``` ### Test Payment Response Check the 402 payment response: ```bash # Get detailed payment information curl -v http://localhost:3000/TX_ID # Look for X-Payment header in response ``` ### Monitor Gateway Logs Watch for payment-related activity: ```bash # Monitor x402 and payment logs docker compose logs -f core | grep -i "x402\|payment\|rate limit" ``` ### Check Metrics View payment and rate limiting metrics: ```bash # Access gateway metrics curl -s http://localhost:3000/ar-io/__gateway_metrics | grep rate_limit # Check token consumption curl -s http://localhost:3000/ar-io/__gateway_metrics | grep tokens_consumed ``` ## Troubleshooting ### Common Issues **Rate Limiter Not Working**: Ensure `ENABLE_RATE_LIMITER=true` is set and gateway has been restarted. **No 402 Responses**: Check that rate limits are being exceeded and x402 is properly configured. **Payment Verification Fails**: Verify wallet address and facilitator URL are correct. ### Debug Steps 1. **Check Configuration**: ```bash # Verify environment variables docker compose exec core env | grep -E "RATE_LIMITER|X_402" ``` 2. **Monitor Logs**: ```bash # Watch for errors docker compose logs -f core | grep -i error ``` 3. **Test Rate Limits**: ```bash # Check if rate limiting is active curl -s http://localhost:3000/ar-io/__gateway_metrics | grep rate_limit ``` 4. **Verify Network Connectivity**: ```bash # Test facilitator connectivity curl -I https://x402.org/facilitator ``` ## Next Steps Once x402 payments are configured: } > Use Grafana to track payment metrics, rate limiting statistics, and gateway performance analytics. } > Adjust rate limits and pricing based on usage patterns, and configure gateway filters for optimal performance. } > Consider Redis for distributed deployments and review environment variables for advanced configuration options. } > Configure paywall branding, messaging, and customize your gateway's apex domain for a professional appearance. # Installation & Setup (/build/run-a-gateway/quick-start) New to ar.io gateways? Learn more about what they are and how they work at [ar.io Gateways](/learn/gateways). Get your ar.io gateway running in **30 seconds** with Docker. No configuration needed - just run and test. ## Quickstart ```bash # Start ar.io gateway with Docker docker run -p 4000:4000 ghcr.io/ar-io/ar-io-core:latest ``` **Test it's working:** ```bash # Fetch a test transaction curl localhost:4000/4jBV3ofWh41KhuTs2pFvj-KBZWUkbrbCYlJH0vLA6LM # Expected output: test ``` That's it! Your gateway is now serving Arweave data at `localhost:4000`. ## Production Setup with Custom Domain Ready to run a gateway with your own domain name and SSL certificates? Follow these comprehensive steps: ### System Requirements **Minimum requirements:** - 4 core CPU - 4 GB RAM - 500 GB storage (SSD recommended) - Stable 50 Mbps internet connection **Recommended:** - 12 core CPU - 32 GB RAM - 2 TB SSD storage - Stable 1 Gbps internet connection External storage devices should be formatted as ext4. ### Install Required Packages **Quick install all packages:** ```bash sudo apt update -y && sudo apt upgrade -y && sudo apt install -y curl openssh-server git certbot nginx sqlite3 build-essential && sudo systemctl enable ssh && curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash && source ~/.bashrc && sudo ufw allow 22 80 443 && sudo ufw enable ``` **Install Docker:** ```bash # Add Docker's official GPG key sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update # Install Docker sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin ``` **Install Node.js and Yarn:** ```bash nvm install 20.11.1 && nvm use 20.11.1 && npm install -g yarn@1.22.22 ``` ### Install the Node **Clone the repository:** ```bash git clone -b main https://github.com/ar-io/ar-io-node cd ar-io-node ``` **Note:** Your indexing databases will be created in the project directory unless otherwise specified in your .env file, not your Docker environment. So, if you are using an external hard drive, you should install the node directly to that external drive. **Create environment file:** ```bash nano .env ``` **Add configuration:** ```bash GRAPHQL_HOST=turbo-gateway.com GRAPHQL_PORT=443 START_HEIGHT=1000000 RUN_OBSERVER=true ARNS_ROOT_HOST= AR_IO_WALLET= OBSERVER_WALLET= OBSERVER_PRIVATE_KEY= SOLANA_UPLOAD_PRIVATE_KEY= ``` `AR_IO_WALLET` and `OBSERVER_WALLET` must be Solana public keys (base58 Ed25519). The observer address must be unique — no other gateway can use the same observer. **Both wallets need SOL** — the observer pays for `save_observations` instructions, and the operator pays for cranker instructions when `ENABLE_EPOCH_CRANKING=true` (`tally_weights`, `prescribe_epoch`, `distribute_epoch`, `close_epoch`). Budget ~0.5 SOL each to cover several months of activity. **Supply Observer Wallet Keyfile:** Save your Solana keypair JSON file as `.json` in the `wallets` directory. If you use a keypair file instead of inline base58 keys, remove `OBSERVER_PRIVATE_KEY` and `SOLANA_UPLOAD_PRIVATE_KEY` from `.env`, then set both paths explicitly: ```bash OBSERVER_KEYPAIR_PATH=/app/wallets/.json SOLANA_UPLOAD_KEYPAIR_PATH=/app/wallets/.json ``` By default, the Observer will use [Turbo Credits](https://docs.ardrive.io/docs/turbo/credits) to pay for uploading reports to Arweave. This allows reports under 100kb to be uploaded for free, but larger reports will fail if the Observer wallet does not contain Credits. Set `SOLANA_UPLOAD_PRIVATE_KEY` to the same base58 secret as `OBSERVER_PRIVATE_KEY` so Turbo has an explicit upload signer. If logs warn that `TurboReportSink` is not configured, this upload key is usually missing. Including `REPORT_DATA_SINK=arweave` in your `.env` file will configure the Observer to use AR tokens instead of Turbo Credits, without any free limit. **Start the Docker container:** ```bash sudo docker compose up -d ``` ### Set Up Networking **Register a Domain Name:** Choose a domain registrar (e.g., [Namecheap](https://namecheap.com)) to register a domain name. **Point Domain at Your Home Network:** - Get your public IP address: `curl ifconfig.me` - Create A records for your domain and wildcard subdomains (`*.yourdomain.com`) **Set up Port Forwarding:** - Get local IP: `ip addr show | grep -w inet | awk '{print $2}' | awk -F'/' '{print $1}'` - Configure router to forward ports 80 and 443 to your local machine **Create SSL Certificates:** ```bash sudo certbot certonly --manual --preferred-challenges dns -d .com -d '*..com' ``` Previous versions of these instructions advised providing an email address to Certbot. As of June 2025, LetsEncrypt (the certificate authority used by Certbot) no longer supports email notifications. **Important:** Wild card subdomain (*.<your-domain>.com) cannot auto renew without obtaining an API key from your domain registrar. Not all registrars offer this. Certbot certificates expire every 90 days. Be sure to consult with your chosen registrar to see if they offer an API for this purpose, or run the above command again to renew your certificates. ### Configure nginx **Open nginx configuration:** ```bash sudo nano /etc/nginx/sites-available/default ``` **Replace with this configuration:** ```nginx # Force redirects from HTTP to HTTPS server { listen 80; listen [::]:80; server_name .com *..com; location / { return 301 https://$host$request_uri; } } # Forward traffic to your node and provide SSL certificates server { listen 443 ssl; listen [::]:443 ssl; server_name .com *..com; ssl_certificate /etc/letsencrypt/live/.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/.com/privkey.pem; location / { proxy_pass http://localhost:3000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_http_version 1.1; # Forward ar.io headers if present in the request proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; } } ``` **Test and restart nginx:** ```bash sudo nginx -t sudo service nginx restart ``` **Note:** Previous versions of these instructions advised checking a gateway's ability to fetch content using `localhost`. Subsequent security updates prevent this without first unsetting `ARNS_ROOT_HOST` in your `.env`. ### Test Your Gateway **Verify it's working:** ```bash curl https:///3lyxgbgEvqNSvJrTX2J7CfRychUD5KClFhhVLyTPNCQ ``` **Expected output:** ``` 1984 ``` If you see `1984`, your gateway is working perfectly! ### System Requirements **Minimum requirements:** - 4 core CPU - 4 GB RAM - 500 GB storage (SSD recommended) - Stable 50 Mbps internet connection **Recommended:** - 12 core CPU - 32 GB RAM - 2 TB SSD storage - Stable 1 Gbps internet connection External storage devices should be formatted as ext4. ### Install Required Software **Install Docker Desktop:** - Download from [Docker Desktop for Windows](https://www.docker.com/products/docker-desktop/) - Run installer and follow prompts - Select WSL (Windows Subsystem for Linux) during installation - Restart your PC - Update WSL: ```cmd wsl --update wsl --shutdown ``` - Restart Docker Desktop **Install Git:** - Download from [Git for Windows](https://git-scm.com/download/win) - Run installer with default settings ### Clone the Repository **Open Command Prompt:** - Press `Windows Key + R` - Type `cmd` and press `Enter` **Navigate to desired directory:** ```cmd cd Documents ``` **Clone the repository:** ```cmd git clone -b main https://github.com/ar-io/ar-io-node ``` **Note:** Your database will be created in the project directory, not Docker. If using an external hard drive, install directly to that drive. ### Create Environment File **Open a text editor (e.g., Notepad):** - Press `Windows Key` and search for "Notepad" **Create .env file with this content:** ```bash GRAPHQL_HOST=turbo-gateway.com GRAPHQL_PORT=443 START_HEIGHT=0 RUN_OBSERVER=true ARNS_ROOT_HOST= AR_IO_WALLET= OBSERVER_WALLET= OBSERVER_PRIVATE_KEY= SOLANA_UPLOAD_PRIVATE_KEY= ``` **Save as `.env`** (select "All Files" as file type) **Supply Observer Wallet Keyfile:** Save your wallet keyfile as `.json` in the `wallets` directory. If you use a keypair file instead of inline base58 keys, remove `OBSERVER_PRIVATE_KEY` and `SOLANA_UPLOAD_PRIVATE_KEY` from `.env`, then set both paths explicitly: ```bash OBSERVER_KEYPAIR_PATH=/app/wallets/.json SOLANA_UPLOAD_KEYPAIR_PATH=/app/wallets/.json ``` ### Start Docker Containers **Navigate to project directory:** ```cmd cd Documents\ar-io-node ``` **Start the container:** ```cmd docker compose up -d ``` **Explanation of flags:** - `up`: Start the Docker containers - `-d`: Run containers as background processes (detached mode) **Shutdown command:** ```cmd docker compose down ``` ### Set Up Router Port Forwarding **Obtain a Domain Name:** Choose a domain registrar (e.g., [Namecheap](https://namecheap.com)) and purchase a domain name. **Point Domain at Your Home Network:** - Visit https://www.whatsmyip.org/ to get your public IP address - Access your domain registrar's settings - Create A records for your domain and wildcard subdomains (`*.yourdomain.com`) **Get Local IP Address:** ```cmd ipconfig ``` Look for IPv4 Address (format: `192.168.X.X` or `10.X.X.X`) **Set Up Router Port Forwarding:** - Access router settings (usually `192.168.0.1`) - Navigate to port forwarding settings - Forward ports 80 and 443 to your local machine's IP address ### Install and Configure NGINX Docker **Clone NGINX Docker repository:** ```cmd cd Documents git clone -b main https://github.com/bobinstein/dockerized-nginx ``` **Follow the repository instructions** for setting up NGINX Docker. **Important:** When configuring your nginx setup, ensure that your nginx configuration includes the following ar.io headers in the proxy configuration: ```nginx # Forward ar.io headers if present in the request proxy_set_header X-AR-IO-Origin $http_x_ar_io_origin; proxy_set_header X-AR-IO-Origin-Node-Release $http_x_ar_io_origin_node_release; proxy_set_header X-AR-IO-Hops $http_x_ar_io_hops; ``` These headers are essential for proper ar.io network functionality. ### Test Your Gateway **Verify it's working:** Visit `https:///3lyxgbgEvqNSvJrTX2J7CfRychUD5KClFhhVLyTPNCQ` in your browser. **Expected output:** ``` 1984 ``` If you see `1984`, your gateway is working perfectly! ## Useful Docker Commands Monitor and manage your ar.io gateway with these commands: ```bash # View all running services docker ps # Run services in background daemon docker compose up -d # Turn off all services docker compose down # Pull latest images docker compose pull # Follow the logs of the core service docker compose logs core -f -n 10 # Follow the logs of the observer docker compose logs core observer -f -n 10 ``` ## What's Next? Your gateway is running! Now you can: } /> } /> } /> } /> **Need more context?** Learn [What is an ar.io Gateway](/learn/gateways) to understand the full capabilities. # Admin UI (/build/run-wayfinder-router/admin-ui) The admin UI is a built-in web dashboard that runs on a **separate port** (default 3001) from the public router (default 3000). This ensures admin endpoints are never exposed to public traffic. ``` http://localhost:3001 ``` ## Dashboard Pages ### Status Live dashboard showing: - **Uptime** - How long the router has been running - **Operating Mode** - Current mode (proxy/route) - **Verification Status** - Whether verification is enabled and working - **Gateway Health Bar** - Visual indicator of healthy vs unhealthy gateways - **Cache Utilization** - Content cache size and hit rate - **Ping Service Stats** - Background ping status and results ### Gateways Sortable table of all ar.io network gateways with: - **Health** - Current gateway status (healthy/unhealthy/unknown) - **Temperature Score** - Performance score for routing decisions - **Latency** - Average response time - **Success Rate** - Percentage of successful requests - **Traffic Stats** - Requests served and bytes transferred ### Telemetry Time-ranged metrics with - **Time Ranges** - 1 hour, 6 hours, 24 hours, 7 days - **Request Totals** - Total requests, success rate, error breakdown - **Bytes Served** - Total data transferred - **Per-Gateway Performance** - Table with CSV Content moderation management: - **Block/Unblock** - Add or remove ArNS names and transaction IDs - **View Blocklist** - See all currently blocked content - **Enable Moderation** - If not yet configured, shows setup instructions ### Settings View current configuration grouped by category: - Server, Mode, Routing, Verification - Cache, Telemetry, Rate Limiting - HTTP, Shutdown, Admin ## Setup Wizard On first run (when `BASE_DOMAIN=localhost`), the admin UI shows a guided setup wizard that walks you through domain, routing, and verification configuration. See [Quick Start](/build/run-wayfinder-router/quick-start#setup-wizard) for details. ## Security Model | | Public Port (3000) | Admin Port (3001) | |---|---|---| | **Default bind** | `0.0.0.0` (all interfaces) | `127.0.0.1` (localhost only) | | **Admin UI** | Not available (404) | Full access | | **Content serving** | Normal operation | N/A | Key security features: - Admin is **never** exposed on the public port - Default localhost binding means only local access - `ADMIN_PORT` must differ from `PORT` (validated at startup) If you set `ADMIN_HOST=0.0.0.0` to expose the admin UI over the network, you **must** set `ADMIN_TOKEN` to protect it. ## Remote Access To access the admin UI from a remote machine while keeping it secure: ### Option 1: SSH Tunnel (Recommended) ```bash # On your local machine ssh -L 3001:localhost:3001 your-server # Then open in your browser http://localhost:3001 ``` This forwards port 3001 on your local machine to the server's localhost:3001, keeping the admin UI secure. ### Option 2: Token Authentication If you need direct remote access: ```bash # In .env ADMIN_HOST=0.0.0.0 ADMIN_TOKEN=your-secure-random-token ``` Generate a secure token: ```bash openssl rand -base64 32 ``` The UI will prompt for the token when you load it. ## Admin API Endpoints The admin UI is backed by JSON API endpoints: | Endpoint | Method | Description | |----------|--------|-------------| | `/api/status` | GET | Aggregated status data | | `/api/gateways` | GET | Gateway list with health and scores | | `/api/telemetry` | GET | Time-ranged telemetry stats | | `/api/config` | GET | Current configuration (sanitized) | | `/api/moderation` | GET | Moderation status and blocklist | | `/api/config/save` | POST | Save .env file | | `/api/restart` | POST | Validate config and restart router | All endpoints require the `Authorization: Bearer ` header when `ADMIN_TOKEN` is set. ## Restart from Admin UI The admin UI can restart the router after configuration changes: 1. Make changes in the Settings page 2. Click "Save Configuration" 3. Click "Restart Router" to apply changes The restart endpoint validates the new configuration before restarting, preventing configuration errors from taking down the router. # Configuration (/build/run-wayfinder-router/configuration) All configuration is via environment variables. Create a `.env` file or set them directly in your environment. The admin UI setup wizard at `http://localhost:3001` can generate a `.env` file for you. ## Server | Variable | Default | Description | |----------|---------|-------------| | `PORT` | `3000` | Public server port | | `HOST` | `0.0.0.0` | Public server bind address | | `BASE_DOMAIN` | `localhost` | Base domain for ArNS subdomain routing | | `ROOT_HOST_CONTENT` | _(empty)_ | ArNS name or txId to serve at root domain | | `RESTRICT_TO_ROOT_HOST` | `false` | Only serve root domain content (404 for subdomains/txIds) | | `GRAPHQL_PROXY_URL` | _(empty)_ | Upstream GraphQL endpoint for `/graphql` proxy | ### Root Host Configuration `ROOT_HOST_CONTENT` accepts either an ArNS name or a transaction ID (auto-detected by format): ```bash # ArNS name ROOT_HOST_CONTENT=wayfinder # Transaction ID (43-char base64url) ROOT_HOST_CONTENT=bNbA3TEQVL60xlgCcqdz4ZPHFZ711cZ3hmkpGttDt_U ``` When set, the root domain serves this content instead of the info page. The info page moves to `/wayfinder/info`. ## Operating Mode | Variable | Default | Description | |----------|---------|-------------| | `DEFAULT_MODE` | `proxy` | Operating mode: `proxy` or `route` | | `ALLOW_MODE_OVERRIDE` | `true` | Allow `?mode=` query parameter to override | **Modes:** - **proxy** - Fetch, verify, and serve content through the router - **route** - Redirect clients directly to a gateway URL ## Routing Controls where content is fetched from. | Variable | Default | Description | |----------|---------|-------------| | `ROUTING_STRATEGY` | `fastest` | Gateway selection strategy | | `ROUTING_GATEWAY_SOURCE` | `network` | Where to get the gateway list | | `ROUTING_STATIC_GATEWAYS` | _(see below)_ | Comma-separated gateway URLs (when source=`static`) | | `TRUSTED_PEER_GATEWAY` | `https://turbo-gateway.com` | Gateway for peer list (when source=`trusted-peers`) | | `TRUSTED_ARIO_GATEWAYS` | _(empty)_ | Trusted gateways (when source=`trusted-ario`) | ### Routing Strategies | Strategy | Behavior | Best For | |----------|----------|----------| | `fastest` | Concurrent ping, use first responder | Lowest latency | | `random` | Random selection from healthy gateways | Load distribution | | `round-robin` | Sequential rotation through gateways | Predictable distribution | | `temperature` | Weighted selection based on recent performance | Production (recommended) | The `temperature` strategy tracks per-gateway latency and success rate. Better-performing gateways get more traffic, but slower gateways still receive some requests to detect improvements. ### Gateway Sources | Source | Behavior | |--------|----------| | `network` | All ar.io gateways from the onchain registry (recommended) | | `trusted-peers` | Peer list from `TRUSTED_PEER_GATEWAY` | | `trusted-ario` | Specific gateways from `TRUSTED_ARIO_GATEWAYS` | | `static` | Manual list from `ROUTING_STATIC_GATEWAYS` | ## Verification Controls content integrity verification. | Variable | Default | Description | |----------|---------|-------------| | `VERIFICATION_ENABLED` | `true` | Enable content hash verification | | `VERIFICATION_GATEWAY_SOURCE` | `top-staked` | Source for trusted gateways | | `VERIFICATION_GATEWAY_COUNT` | `3` | Number of gateways to query for verification | | `VERIFICATION_STATIC_GATEWAYS` | _(see below)_ | Comma-separated URLs (when source=`static`) | | `VERIFICATION_RETRY_ATTEMPTS` | `3` | Gateways to try before failing | | `ARNS_CONSENSUS_THRESHOLD` | `2` | Minimum gateways that must agree on ArNS resolution | ### Verification Sources | Source | Behavior | |--------|----------| | `top-staked` | Top N gateways by stake (economic security, recommended) | | `static` | Manual list from `VERIFICATION_STATIC_GATEWAYS` | ## Cache | Variable | Default | Description | |----------|---------|-------------| | `CONTENT_CACHE_ENABLED` | `true` | Enable verified content cache | | `CONTENT_CACHE_MAX_SIZE_BYTES` | `53687091200` (50GB) | Maximum total cache size | | `CONTENT_CACHE_MAX_ITEM_SIZE_BYTES` | `2147483648` (2GB) | Maximum single item size | | `CONTENT_CACHE_PATH` | _(empty)_ | Disk path for persistence (empty = in-memory only) | | `ARNS_CACHE_TTL_MS` | `300000` (5min) | ArNS resolution cache TTL | ### Disk-Backed Cache For production, enable disk persistence: ```bash CONTENT_CACHE_PATH=./data/content-cache ``` When set: - LRU holds metadata only (low memory footprint) - Content stored as files on disk (`.bin` + `.meta.json`) - Cache survives restarts - Atomic writes via temp file + rename for crash safety ## Admin UI The admin UI runs on a **separate port** for security. | Variable | Default | Description | |----------|---------|-------------| | `ADMIN_UI_ENABLED` | `true` | Enable admin UI server | | `ADMIN_PORT` | `3001` | Admin server port (must differ from `PORT`) | | `ADMIN_HOST` | `127.0.0.1` | Admin bind address | | `ADMIN_TOKEN` | _(empty)_ | Bearer token for auth (required when not localhost) | | `ADMIN_OPEN_BROWSER` | `true` | Auto-open browser on startup | When `ADMIN_HOST=0.0.0.0`, you **must** set `ADMIN_TOKEN` to protect the admin UI. ## Resilience | Variable | Default | Description | |----------|---------|-------------| | `RETRY_ATTEMPTS` | `3` | Retry attempts for failed requests | | `RETRY_DELAY_MS` | `100` | Delay between retries | | `CIRCUIT_BREAKER_THRESHOLD` | `3` | Failures before marking gateway unavailable | | `CIRCUIT_BREAKER_RESET_MS` | `60000` (1min) | Time before retrying a broken gateway | | `GATEWAY_HEALTH_TTL_MS` | `300000` (5min) | Health status cache TTL | | `STREAM_TIMEOUT_MS` | `120000` (2min) | Per-chunk stream read timeout | ## Telemetry | Variable | Default | Description | |----------|---------|-------------| | `TELEMETRY_ENABLED` | `true` | Enable telemetry collection | | `TELEMETRY_ROUTER_ID` | `router-{timestamp}` | Instance identifier | | `TELEMETRY_DB_PATH` | `./data/telemetry.db` | SQLite database path | | `TELEMETRY_RETENTION_DAYS` | `30` | Data retention period | | `TELEMETRY_SAMPLE_SUCCESS` | `0.1` | Sampling rate for successful requests (0.0-1.0) | | `TELEMETRY_SAMPLE_ERRORS` | `1.0` | Sampling rate for errors | ## Rate Limiting | Variable | Default | Description | |----------|---------|-------------| | `RATE_LIMIT_ENABLED` | `false` | Enable per-IP rate limiting | | `RATE_LIMIT_WINDOW_MS` | `60000` (1min) | Rate limit window | | `RATE_LIMIT_MAX_REQUESTS` | `1000` | Max requests per IP per window | ## Content Moderation | Variable | Default | Description | |----------|---------|-------------| | `MODERATION_ENABLED` | `false` | Enable content moderation | | `MODERATION_BLOCKLIST_PATH` | `./data/blocklist.json` | Blocklist file (auto-created, hot-reloaded) | | `MODERATION_ADMIN_TOKEN` | _(empty)_ | Bearer token for moderation API | ## Arweave HTTP API Proxy Proxy Arweave node API endpoints (`/info`, `/tx/{id}`, `/block/height/{h}`, etc.). | Variable | Default | Description | |----------|---------|-------------| | `ARWEAVE_API_ENABLED` | `false` | Enable Arweave API proxy | | `ARWEAVE_READ_NODES` | _(Arweave tip nodes)_ | Nodes for GET requests | | `ARWEAVE_WRITE_NODES` | _(falls back to read)_ | Nodes for POST requests | | `ARWEAVE_API_CACHE_ENABLED` | `true` | Cache API responses | ## Gateway Ping Service Background latency probing for the `temperature` routing strategy. | Variable | Default | Description | |----------|---------|-------------| | `PING_ENABLED` | `true` | Enable ping service | | `PING_INTERVAL_HOURS` | `4` | How often to refresh ping data | | `PING_GATEWAY_COUNT` | `50` | Gateways to ping each round | | `PING_TIMEOUT_MS` | `5000` | Timeout per ping | | `PING_CONCURRENCY` | `10` | Maximum concurrent pings | ## Shutdown | Variable | Default | Description | |----------|---------|-------------| | `SHUTDOWN_DRAIN_TIMEOUT_MS` | `15000` (15s) | Grace period for in-flight requests | | `SHUTDOWN_TIMEOUT_MS` | `30000` (30s) | Total shutdown timeout | ## Logging | Variable | Default | Description | |----------|---------|-------------| | `LOG_LEVEL` | `info` | Log level: `trace`, `debug`, `info`, `warn`, `error`, `fatal` | ## Example Configuration ```bash # .env - Production example # Server PORT=3000 HOST=0.0.0.0 BASE_DOMAIN=yourdomain.com ROOT_HOST_CONTENT=your-arns-name # Mode DEFAULT_MODE=proxy ALLOW_MODE_OVERRIDE=false # Routing ROUTING_STRATEGY=temperature ROUTING_GATEWAY_SOURCE=network # Verification VERIFICATION_ENABLED=true VERIFICATION_GATEWAY_SOURCE=top-staked VERIFICATION_GATEWAY_COUNT=3 ARNS_CONSENSUS_THRESHOLD=2 # Cache CONTENT_CACHE_ENABLED=true CONTENT_CACHE_PATH=./data/content-cache CONTENT_CACHE_MAX_SIZE_BYTES=53687091200 # Admin ADMIN_UI_ENABLED=true ADMIN_PORT=3001 ADMIN_HOST=127.0.0.1 # Telemetry TELEMETRY_ENABLED=true TELEMETRY_DB_PATH=./data/telemetry.db # Rate Limiting RATE_LIMIT_ENABLED=true RATE_LIMIT_MAX_REQUESTS=1000 ``` # Wayfinder Router (/build/run-wayfinder-router) Wayfinder Router is a lightweight proxy service that provides a **single trusted endpoint** for accessing Arweave data through the decentralized ar.io gateway network. It fetches content from multiple gateways, verifies integrity via cryptographic hash checking, and serves verified data to your users. ``` Your Users | v +------------------+ | Wayfinder Router | |------------------| | - Verification | | - Caching | | - Telemetry | +------------------+ | +--------------+--------------+ | | | v v v +------+ +------+ +------+ | GW1 | | GW2 | | GW3 | +------+ +------+ +------+ \___________+___________/ | ar.io Gateway Network ``` ## Why Use Wayfinder Router? } title="Content Verification" description="Cryptographically verify all content against trusted gateways before serving" /> } title="Single Endpoint" description="Give your users one reliable URL instead of managing gateway failover in client code" /> } title="Built-in Caching" description="LRU cache with optional disk persistence for fast repeat access" /> } title="Admin Dashboard" description="Web UI for configuration, monitoring, and content moderation" /> ## Two Operating Modes | Mode | Behavior | Use Case | |------|----------|----------| | **Proxy** | Fetch, verify, and serve content through the router | Full verification, caching, single domain | | **Route** | Redirect clients directly to a gateway URL | Lower latency, client-side verification | ## When to Use Wayfinder Router **Use Wayfinder Router when you need:** - A single trusted endpoint for your application - Server-side content verification - Centralized caching and rate limiting - Content moderation capabilities - Monitoring and telemetry **Use the SDKs directly when you need:** - Client-side verification in browsers - Custom routing logic in your application - Integration with existing infrastructure ## Architecture Overview The router separates **routing** (where to fetch data) from **verification** (who to trust): - **Routing Gateways** - Where content is fetched from (all ar.io gateways, trusted peers, or static list) - **Verification Gateways** - Who to trust for hash verification (top-staked gateways or static list) This means even if a routing gateway is compromised, tampered content is detected and rejected. When verification fails, the router returns an error to the client rather than serving potentially malicious content. ## Get Started } title="Quick Start" description="Get running in 30 seconds with a standalone binary or Docker" href="/build/run-wayfinder-router/quick-start" /> } title="Configuration" description="Environment variables for routing, verification, caching, and more" href="/build/run-wayfinder-router/configuration" /> } title="Admin UI" description="Web dashboard for monitoring, configuration, and moderation" href="/build/run-wayfinder-router/admin-ui" /> } title="Operations" description="Monitoring, troubleshooting, and production deployment" href="/build/run-wayfinder-router/operations" /> # Operations (/build/run-wayfinder-router/operations) ## Monitoring ### Health and Readiness ```bash # Health check (is the process running?) curl http://localhost:3000/wayfinder/health # Readiness check (is the router ready to serve traffic?) curl http://localhost:3000/wayfinder/ready # API Guard compatible health check curl http://localhost:3000/ar-io/healthcheck ``` Use these endpoints for load balancer health checks and orchestration systems. ### Prometheus Metrics ```bash curl http://localhost:3000/wayfinder/metrics ``` Exposes standard metrics for scraping by Prometheus. Configure your Prometheus scrape targets to point at `/wayfinder/metrics`. ### Gateway Statistics ```bash # Summary statistics curl http://localhost:3000/wayfinder/stats/gateways # List all tracked gateways curl http://localhost:3000/wayfinder/stats/gateways/list # Detailed stats for a specific gateway curl http://localhost:3000/wayfinder/stats/gateways/:gateway # Export telemetry data curl http://localhost:3000/wayfinder/stats/ ### Router Info ```bash curl http://localhost:3000/wayfinder/info ``` Returns current configuration, version, uptime, and operating mode. ### Telemetry Storage Telemetry is stored in SQLite at `TELEMETRY_DB_PATH` (default `./data/telemetry.db`). Configure sampling rates to control storage growth: ```bash TELEMETRY_SAMPLE_SUCCESS=0.1 # Sample 10% of successful requests TELEMETRY_SAMPLE_ERRORS=1.0 # Record all errors TELEMETRY_RETENTION_DAYS=30 # Auto-purge old data ``` ### Log Levels Set log verbosity via `LOG_LEVEL`: ```bash LOG_LEVEL=debug # trace, debug, info, warn, error, fatal ``` ## Content Moderation Block ArNS names or transaction IDs from being served. ### Setup ```bash MODERATION_ENABLED=true MODERATION_ADMIN_TOKEN= ``` Generate a secure token: ```bash openssl rand -base64 32 ``` The blocklist is stored at `MODERATION_BLOCKLIST_PATH` (default `./data/blocklist.json`) and is hot-reloaded on changes. ### API Endpoints All admin endpoints require `Authorization: Bearer ` header. ```bash # Block an ArNS name curl -X POST http://localhost:3000/wayfinder/moderation/block \ -H "Authorization: Bearer YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{"type":"arns","value":"badcontent","reason":"Policy violation"}' # Block a transaction ID curl -X POST http://localhost:3000/wayfinder/moderation/block \ -H "Authorization: Bearer YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{"type":"txid","value":"abc123...","reason":"DMCA takedown"}' ``` ```bash # List all blocked content curl http://localhost:3000/wayfinder/moderation/blocklist \ -H "Authorization: Bearer YOUR_TOKEN" # Check if content is blocked (no auth required) curl http://localhost:3000/wayfinder/moderation/check/arns/somename # Moderation statistics curl http://localhost:3000/wayfinder/moderation/stats \ -H "Authorization: Bearer YOUR_TOKEN" ``` ```bash # Unblock content curl -X DELETE http://localhost:3000/wayfinder/moderation/block/arns/badcontent \ -H "Authorization: Bearer YOUR_TOKEN" # Reload blocklist from disk curl -X POST http://localhost:3000/wayfinder/moderation/reload \ -H "Authorization: Bearer YOUR_TOKEN" ``` ## Cache Management For production, enable disk-backed caching. See [Configuration](/build/run-wayfinder-router/configuration#cache) for all cache settings. ### Clearing Caches **From source:** ```bash bun run clear:telemetry # Clear telemetry database bun run clear:all # Clear all data (telemetry + cache) ``` **Binary or Docker:** Delete the data directory contents directly: ```bash rm -rf ./data/content-cache/* rm ./data/telemetry.db ``` ## Graceful Shutdown The router handles SIGTERM and SIGINT signals with a two-phase shutdown: 1. **Drain phase** - Stop accepting new connections, wait for in-flight requests 2. **Force exit** - If drain exceeds timeout, force shutdown ```bash # Configuration SHUTDOWN_DRAIN_TIMEOUT_MS=15000 # 15s drain period SHUTDOWN_TIMEOUT_MS=30000 # 30s total timeout ``` ```bash # Graceful stop kill -TERM # Docker (sends SIGTERM, waits 10s, then SIGKILL) docker stop wayfinder-router # Docker with custom timeout docker stop -t 30 wayfinder-router ``` ## Troubleshooting ### Port Conflicts If port 3000 or 3001 is already in use: ```bash PORT=3080 ADMIN_PORT=3081 ./wayfinder-router-linux-x64 ``` `ADMIN_PORT` must differ from `PORT` - the router validates this at startup. ### Gateway Health Issues Check gateway status via the admin UI Gateways page or: ```bash curl http://localhost:3000/wayfinder/stats/gateways ``` If all gateways show as unhealthy: - Verify internet connectivity - Check `ROUTING_GATEWAY_SOURCE` - if `static`, ensure URLs are correct - Check circuit breaker settings - Review logs: `LOG_LEVEL=debug` ### ArNS Resolution Failures ArNS names require consensus across multiple verification gateways. If resolution fails: - Check `ARNS_CONSENSUS_THRESHOLD` (default: 2) - Verify the ArNS name exists on the network - Check verification gateway health - Ensure `VERIFICATION_ENABLED=true` ### Subdomain Routing Not Working ArNS subdomains require `BASE_DOMAIN` to match your actual domain: ```bash # For local development BASE_DOMAIN=localhost # For production BASE_DOMAIN=yourdomain.com ``` Requests to `{name}.yourdomain.com` are only recognized as ArNS subdomains if `BASE_DOMAIN=yourdomain.com`. ### Content Verification Failures If content consistently fails verification: - Check `VERIFICATION_GATEWAY_COUNT` - more gateways increases reliability but adds latency - Verify `VERIFICATION_GATEWAY_SOURCE` is correctly configured - Check if the content transaction is still being seeded - Review logs for specific hash mismatch details ### Memory Usage If memory usage is high: - Enable disk-backed cache: `CONTENT_CACHE_PATH=./data/content-cache` - Reduce cache size: `CONTENT_CACHE_MAX_SIZE_BYTES` - Reduce telemetry retention: `TELEMETRY_RETENTION_DAYS` - Increase sampling: `TELEMETRY_SAMPLE_SUCCESS=0.01` ### Slow Response Times If responses are slow: - Check routing strategy - `temperature` adapts to gateway performance - Verify verification gateway health - Check network connectivity to gateways - Review `STREAM_TIMEOUT_MS` setting - Consider reducing `VERIFICATION_GATEWAY_COUNT` if latency is acceptable ## Production Checklist Before deploying to production: - [ ] Set `BASE_DOMAIN` to your actual domain - [ ] Configure `ROOT_HOST_CONTENT` if serving at root - [ ] Enable disk cache: `CONTENT_CACHE_PATH=./data/content-cache` - [ ] Set appropriate cache size limits - [ ] Enable rate limiting: `RATE_LIMIT_ENABLED=true` - [ ] Secure admin UI: `ADMIN_TOKEN` or keep `ADMIN_HOST=127.0.0.1` - [ ] Configure telemetry sampling rates - [ ] Set up monitoring (health checks, Prometheus) - [ ] Configure reverse proxy (nginx/Caddy) with SSL - [ ] Set up log aggregation - [ ] Plan for graceful shutdown in your orchestration # Quick Start (/build/run-wayfinder-router/quick-start) Get your Wayfinder Router running in **30 seconds**. No configuration needed - the admin UI wizard will guide you through setup. ## Quickstart Download the latest binary for your platform from [GitHub Releases](https://github.com/ar-io/wayfinder-router/releases): | Platform | Binary | |----------|--------| | Linux x64 | `wayfinder-router-linux-x64` | | Linux ARM64 | `wayfinder-router-linux-arm64` | | macOS Intel | `wayfinder-router-darwin-x64` | | macOS Apple Silicon | `wayfinder-router-darwin-arm64` | | Windows x64 | `wayfinder-router-windows-x64.exe` | Each release includes `checksums.txt` with SHA256 hashes for verification. ### Download and make executable ```bash # Download (example for Linux x64) curl -LO https://github.com/ar-io/wayfinder-router/releases/latest/download/wayfinder-router-linux-x64 # Make executable chmod +x wayfinder-router-linux-x64 # Verify checksum (optional but recommended) sha256sum wayfinder-router-linux-x64 ``` ### Run ```bash ./wayfinder-router-linux-x64 ``` The admin UI automatically opens at `http://localhost:3001` with a setup wizard. ### Test ```bash # Fetch a test transaction curl http://localhost:3000/4jBV3ofWh41KhuTs2pFvj-KBZWUkbrbCYlJH0vLA6LM # Expected output: test ``` ### Run with Docker ```bash docker run -p 3000:3000 -p 3001:3001 ghcr.io/ar-io/wayfinder-router:latest ``` Or with Docker Compose (uses ports 3020/3021 by default): ```bash git clone https://github.com/ar-io/wayfinder-router cd wayfinder-router docker compose up -d ``` ### Open Admin UI Navigate to `http://localhost:3001` in your browser. The setup wizard will guide you through configuration. ### Test ```bash curl http://localhost:3000/4jBV3ofWh41KhuTs2pFvj-KBZWUkbrbCYlJH0vLA6LM # Expected output: test ``` **Production Docker:** ```bash docker run -p 3000:3000 -p 3001:3001 \ --env-file .env \ -v ./data:/app/data \ ghcr.io/ar-io/wayfinder-router:latest ``` The `./data` volume persists telemetry, content cache, and blocklist data. Requires [Bun](https://bun.sh) >= 1.0.0. ### Clone and install ```bash git clone https://github.com/ar-io/wayfinder-router cd wayfinder-router bun install ``` ### Configure (optional) ```bash cp .env.example .env # Edit .env as needed, or skip and use the setup wizard ``` ### Run ```bash # Development with hot reload bun run dev # Production bun run start ``` The admin UI opens at `http://localhost:3001`. **Build standalone binaries:** ```bash bun run build:binaries # outputs to ./builds/ ``` ## Setup Wizard On first run (when `BASE_DOMAIN=localhost`), the admin UI shows a guided setup wizard: ### Domain Configuration Configure your base domain, port, and optional root host content. For local development, keep the defaults. ### Routing Settings Choose your operating mode and routing strategy: | Setting | Options | Recommended | |---------|---------|-------------| | **Mode** | `proxy` or `route` | `proxy` for full verification | | **Strategy** | `fastest`, `random`, `round-robin`, `temperature` | `temperature` for production | | **Gateway Source** | `network`, `trusted-peers`, `static` | `network` for full decentralization | ### Verification Settings Enable content verification and configure trust settings: | Setting | Default | Description | |---------|---------|-------------| | **Enabled** | `true` | Verify content hashes before serving | | **Trust Source** | `top-staked` | Use top-staked gateways for verification | | **Gateway Count** | `3` | Number of gateways to query | | **Consensus** | `2` | Minimum gateways that must agree | ### Save Configuration The wizard generates a `.env` file. You can copy it to clipboard or save directly to the server. ## Accessing Content Once running, access Arweave content through your router: ```bash # By transaction ID http://localhost:3000/{txId} http://localhost:3000/{txId}/path/to/file # By ArNS name (subdomain) http://{arns-name}.localhost:3000/ http://{arns-name}.localhost:3000/path/to/file ``` ## Serving Content at Root Domain Configure `ROOT_HOST_CONTENT` to serve an ArNS name or transaction ID at your root domain: ```bash # In .env ROOT_HOST_CONTENT=wayfinder # Or a transaction ID ROOT_HOST_CONTENT=bNbA3TEQVL60xlgCcqdz4ZPHFZ711cZ3hmkpGttDt_U ``` With this configuration: - `https://yourdomain.com/` serves the configured content - `https://yourdomain.com/docs` serves the `/docs` path within that content - `https://yourdomain.com/wayfinder/info` still returns router info ## What's Next? } title="Configuration" description="Full reference for all environment variables" href="/build/run-wayfinder-router/configuration" /> } title="Admin UI" description="Monitor gateways, view telemetry, manage moderation" href="/build/run-wayfinder-router/admin-ui" /> } title="Operations" description="Production deployment, monitoring, and troubleshooting" href="/build/run-wayfinder-router/operations" /> # Accessing Data (/build/testnet/accessing-data) The gateway at [**ar-io.dev**](https://ar-io.dev) indexes your upload optimistically — usually within seconds — then serves and queries it. ## Fetching your data - **Exact bytes:** `GET https://ar-io.dev/raw/` returns the data item's exact bytes, unmodified, with no redirect. - **Rendered / path-aware:** `GET https://ar-io.dev/` serves the content and, for **manifests**, resolves paths (e.g. `//index.html`). For browser security the gateway **302-redirects** each item to its own **sandbox subdomain** — `https://.ar-io.dev/` — so each item is isolated on its own origin. Follow redirects (`curl -L`; browsers do it automatically). ## Querying with GraphQL `POST https://ar-io.dev/graphql` — query your data by id, tags, or owner as soon as it's indexed: ```graphql query { transactions(first: 1, owners: [""]) { edges { node { id tags { name value } } } } } ``` ## Data response headers On a data fetch (`/raw/` or the sandbox `/`): | Header | Meaning | |---|---| | `x-ar-io-data-id` | The data item id served | | `x-ar-io-verified` | **`false`** here — optimistic sandbox data, **not** weave-verified | | `x-ar-io-trusted` | `true` — served from the trusted sandbox bundler | | `x-ar-io-stable` | `false` — the tx is not in a stable (mainnet) block (it never will be) | | `x-ar-io-digest` / `content-digest` / `etag` | sha-256 of the bytes (integrity / caching) | | `x-ar-io-hops`, `x-cache` | Gateway hop count; cache HIT/MISS | | `x-arweave-owner-address`, `x-arweave-tag-*` | The data item's owner and ANS-104 tag metadata | Seeing **`x-ar-io-verified: false`** is expected and correct on this sandbox — the data isn't on mainnet Arweave. Don't treat it as an error. ## ArNS resolution headers On an ArNS resolution (`https://.ar-io.dev`): | Header | Meaning | |---|---| | `x-arns-name` / `x-arns-basename` | The resolved name | | `x-arns-resolved-id` | The target data id the name points at | | `x-arns-record` | The record served (`@` = apex, else the undername) | | `x-arns-ttl-seconds` | Resolution TTL (e.g. `3600`) | | `x-arns-ant-id` / `x-arns-ant-program-id` | The backing ANT and the **devnet** ANT program | | `x-arns-undername-limit`, `x-arns-record-index` | Undername capacity and record index | ## Next steps # ArNS Names (/build/testnet/arns) The sandbox runs a **devnet ArNS registry**. You can buy **real names** paid in **staging-ARIO credits** and resolve them at `https://.ar-io.dev`. New to ArNS? Start with the [ArNS overview](/learn/arns). There are two ways to register: the **bundler route** (simplest — pay with your Turbo credits), or **directly with the ar.io SDK** against the devnet programs. ## Buying through the bundler Buying through the bundler debits your [upload credits](/build/testnet/uploading-and-credits#getting-credits). If you don't bring your own ANT, a **custodial ANT** is spawned and linked to you, so the name resolves out of the box. ### Check the price ```http GET https://payment.services.ar-io.dev/v1/arns/price/buy-record/?type=lease&years=1 ``` Use `type=permabuy` for a permanent registration. ### Buy the name ```http POST https://payment.services.ar-io.dev/v1/arns/purchase/buy-record/?type=lease&years=1 ``` A signed request that debits your credits. With provisioning on, a buy **without** your own ANT spawns a custodial one and links it to you. ### Poll until it's recorded ```http GET https://payment.services.ar-io.dev/v1/arns/purchase/:nonce ``` `status: recorded` means the purchase settled on Solana devnet. **Minimum name length: 8 characters — bundler route only.** This is an anti-squat rule on the shared registry; shorter names return `400` (`Name must be at least 8 characters…`). Buying [directly with the SDK](#advanced-buying-directly-with-the-sdk) is not subject to this floor. ## Advanced: buying directly with the SDK You can also buy and manage names **directly** against the devnet programs — paying the name price in **staging ARIO** and **devnet SOL** yourself — instead of going through the bundler proxy. This path is **not** bound by the bundler's 8-character floor (though very short or premium names may still be reserved on-chain). ```ts const ario = ARIO.init({ backend: 'solana', programIds: DEVNET_PROGRAM_IDS, // see Reference rpc, signer, }); const cost = await ario.getTokenCost({ /* name, type, years */ }); await ario.buyRecord({ /* name, type, years */ }); ``` The payment mint is read from on-chain config automatically (the staging ARIO mint). For the devnet program IDs and the ar.io SDK's devnet configuration, see the [Reference](/build/testnet/reference) and the SDK's [advanced networks](/sdks/ar-io-sdk/advanced-networks) and [networks](/sdks/ar-io-sdk/networks) guides. ## Resolving a name Once bought, `https://.ar-io.dev` resolves the record, returning `x-arns-*` response headers. To point a name at your uploaded data, set its record via the manage routes (`/v1/arns/manage/...`) or through your ANT directly. **Names outlive data.** A name keeps resolving because the record lives on the devnet registry — but the **data behind it disappears** after the ~3-day purge. Re-upload and re-point when you need it back. See [Accessing Data](/build/testnet/accessing-data#arns-resolution-headers) for the full set of ArNS resolution headers. ## Next steps # Funds & Faucet (/build/testnet/funds-and-faucet) Everything in the sandbox runs on **testnet value**. Depending on what you're doing you need two or three tokens — all free. | Token | Network | What it's for | Where to get it | |---|---|---|---| | **SOL** | Solana **devnet** | Transaction fees on every Solana action | `solana airdrop 1 --url devnet` or [faucet.solana.com](https://faucet.solana.com) | | **ARIO (staging)** | Solana **devnet** | Buying ArNS names and paying for uploads | The ar.io faucet → [faucet.services.ar-io.dev](https://faucet.services.ar-io.dev) | | **Base Sepolia ETH** | Base **Sepolia** | *(optional)* Funding credits with `base-eth` or the x402 path | Any [Base Sepolia faucet](https://docs.base.org/docs/tools/network-faucets/) | You always need a little **devnet SOL** — even after claiming ARIO — because SOL pays the transaction fee for *using* your tokens (funding credits, buying names, etc.). ## The ARIO faucet The staging-ARIO faucet lives at [**faucet.services.ar-io.dev**](https://faucet.services.ar-io.dev). It's **GitHub-gated** to prevent sybil abuse: ### Sign in with GitHub Your GitHub account must be at least **30 days old**. ### Enter your Solana address The address you want to receive staging ARIO on. ### Claim The faucet dispenses **2,500 ARIO per claim** and pays the transfer fee for you. **Rate limit:** one claim per GitHub account per **8-hour** window. A wallet that **already holds the maximum (2,500 ARIO)** won't receive more — you'll see `Recipient … already has more than the maximum quantity`. Claim to a fresh address, or spend down first. The faucet page also displays the current **Solana devnet program IDs and the ARIO mint**, copy-ready — handy if you're accessing the devnet contracts directly with the [ar.io SDK](/build/testnet/arns#advanced-buying-directly-with-the-sdk). See the [Reference](/build/testnet/reference) for the current values. Trouble claiming, or your GitHub account didn't pass verification? Ask in the [ar.io Discord](https://discord.com/invite/HGG52EtTc2). ## Agents and CI The faucet's GitHub OAuth consent is a browser flow and **can't be completed headlessly**, so an agent or CI job can't claim on its own. The reliable pattern is: ### A human claims once A person signs in with GitHub and claims **2,500 ARIO** to the wallet address your agent will use. ### The agent uses the ARIO From there, everything is a scriptable API call — uploading, funding credits, and buying/resolving ArNS names all work without any browser interaction. Re-claim (again by a human) when the balance runs low. **Faucet agent skill.** The faucet ships an `ario-testnet-faucet` agent skill documenting the claim API and flow. Pair it with the human-claims-once pattern above so your agent has funds without needing to solve the OAuth gate. ## Next steps # Testnet Sandbox (/build/testnet) The **ar.io Testnet Sandbox** is a free, public environment where you can build against the **entire ar.io stack** — uploading, payments, ArNS names, and gateway serving — running on **Solana devnet** with a staging ARIO token. It's the fastest way to wire up an integration end to end **without spending real money or touching mainnet**. It's also an ideal target for **AI coding agents and CI**: the services are plain HTTP APIs, the tokens are free from a faucet, and nothing you do here can incur real cost or post permanent data. See [building with an agent](#building-with-an-agent) below. **Read this first — the three rules of the sandbox:** 1. **Data is ephemeral.** Uploads are served and indexed for a short window (**~3 days**), then purged. Nothing here is permanent — don't store anything you need to keep. 2. **Never on-chain.** Your data is *never* posted to mainnet Arweave. The gateway serves it optimistically, so it returns `x-ar-io-verified: false` and disappears when purged. 3. **Testnet value only.** Fund with testnet tokens or Stripe **test cards**. Real tokens and cards are rejected — and there is no mainnet settlement, so you'd be throwing value away. ## What you get } title="Turbo bundler (upload + payment)" description="Accepts testnet tokens, issues upload credits, and includes a free tier for small items." /> } title="Gateway (ar-io.dev)" description="Indexes, serves, and GraphQL-queries your uploads, and resolves ArNS names on Solana devnet." /> } title="ArNS on devnet" description="Buy real ArNS names on the devnet registry, paid in staging ARIO, and resolve them at .ar-io.dev." /> } title="Faucets" description="Free devnet SOL, staging ARIO, and Base Sepolia ETH for everything you need to test." /> ## Endpoints | Service | URL | Health check | |---|---|---| | Upload API | `https://upload.services.ar-io.dev` | `GET /v1/info` | | Payment API | `https://payment.services.ar-io.dev` | `GET /v1/info` | | Gateway (serve / resolve / GraphQL) | `https://ar-io.dev` | `GET /ar-io/healthcheck` | | ARIO faucet (staging ARIO) | `https://faucet.services.ar-io.dev` | `GET /healthcheck` | **Network:** primarily Solana **devnet** — staging **ARIO** SPL mint `6vTw5CysRXQ4ybbHkDUiisHWVsBeMtUzYvJqs2iqHyaN` (6 decimals). The optional `base-eth` funding path and x402 use **Base Sepolia**. Point your app at `*.services.ar-io.dev` and `ar-io.dev` to exercise the full **ar.io** stack — upload, payment, ArNS, and gateway — together. These are shared sandbox endpoints, so treat them as a testbed rather than a production plane. ## Quick start ### Get a Solana devnet keypair and a little devnet SOL SOL covers transaction fees on every Solana action. Airdrop some from [faucet.solana.com](https://faucet.solana.com) or `solana airdrop 1 --url devnet`. ### Claim staging ARIO from the faucet Sign in with GitHub at [faucet.services.ar-io.dev](https://faucet.services.ar-io.dev) and claim **2,500 ARIO** to your Solana address. See [Funds & Faucet](/build/testnet/funds-and-faucet). ### Upload a small file for free Items up to **105 KiB** are eligible for the free tier. Use the Turbo SDK pointed at the sandbox, or `POST /v1/tx`. See [Uploading & Credits](/build/testnet/uploading-and-credits). ### Need more? Top up credits with your staging ARIO Fund upload credits with `ario`, `solana`, or `base-eth`, or Stripe **test cards**. See [Uploading & Credits](/build/testnet/uploading-and-credits#getting-credits). ### Buy an ArNS name (≥ 8 characters) Paid in ARIO credits, provisioned with a custodial ANT so it resolves out of the box. See [ArNS Names](/build/testnet/arns). ### Resolve it Your name is live at `https://.ar-io.dev`. See [Accessing Data](/build/testnet/accessing-data). ## What you must NOT do - **Don't rely on persistence.** Treat every upload as disposable (≤ 3 days). - **Don't send mainnet tokens or real cards** to the payment service — they're rejected. - **Don't point production apps at these endpoints.** This is a shared, wipeable sandbox with reserved short names and rate limits, not a production plane. - **Don't try to defeat the faucet's GitHub gate or the rate limits** — they keep the shared testbed usable for everyone. ## Building with an agent The sandbox is a natural fit for AI coding assistants and automated tests — free tokens, plain HTTP endpoints, and no way to spend real value or write permanent data. **One human step: the faucet.** The ARIO faucet is **GitHub-gated** to prevent abuse, and the OAuth consent can't be completed headlessly. The standard pattern is: a **human claims once** to the wallet address your agent will use, then the agent simply **uses** the ARIO. After that, uploading, funding, and buying ArNS names are all scriptable API calls. See [Funds & Faucet → Agents and CI](/build/testnet/funds-and-faucet#agents-and-ci). This documentation is also available as machine-readable text (`/llms-full.txt`) and every page has an **Open in AI** action, so you can hand the full sandbox reference to your agent in one step. ## Next steps # Reference (/build/testnet/reference) ## Solana devnet identifiers For direct access with the [ar.io SDK](/build/testnet/arns#advanced-buying-directly-with-the-sdk) (`ARIO.init({ backend: 'solana', programIds })`): | Program / mint | Address | |---|---| | `ario_core` | `8Njx9wPkXiNzDCgjwVsJFRjpAEV34gGW3n8DzX3V23m1` | | `ario_gar` | `7WsDTrtZBsfKtnP33XkjuqXCY69JE7n4QVYpynqJCFxz` | | `ario_arns` | `6EZNezcg4rc5hnh8HG34vGquT3WpW5xXypzPb24uyEpp` | | `ario_ant` | `DbHbRwUD1oAn1mrDSqtWtvwGcNrmhWdD2g8L4xmeQ7NX` | | **ARIO mint** | `6vTw5CysRXQ4ybbHkDUiisHWVsBeMtUzYvJqs2iqHyaN` (6 decimals) | **Program IDs rotate.** The staging programs are periodically redeployed, so the program addresses above can change (the ARIO mint is stable). Always verify against the current values shown copy-ready on the [faucet](https://faucet.services.ar-io.dev), or read them from the ar.io SDK's devnet constants — see [advanced networks](/sdks/ar-io-sdk/advanced-networks). If ArNS resolutions return an unexpected `x-arns-ant-program-id`, you're likely on a stale program ID. The ar.io SDK also hardcodes the staging ARIO mint as its `DEVNET_ARIO_MINT`, and reads the payment mint from on-chain config automatically — so for the direct SDK path you generally only need the program IDs. ## Responses you'll see | Code | Where | Meaning | |---|---|---| | `200` `winc:"0"` | Upload | Uploaded, free | | `402 FREE_TIER_EXHAUSTED` | Upload | Out of free allowance → [top up credits](/build/testnet/uploading-and-credits#getting-credits) | | `413` | Upload | Data item over 10 MiB | | `400 Token not supported` | Payment | Non-testnet funding token | | `400 Name must be at least 8 characters…` | ArNS | Name too short (bundler route only) | | `302` → `*.ar-io.dev/` | Gateway | [Sandbox-subdomain redirect](/build/testnet/accessing-data#fetching-your-data) (follow it) | | `status: recorded` | ArNS poll | Purchase settled | ## Caveats recap - **Ephemeral:** data purges after **~3 days**. A name keeps resolving (the record lives on the devnet registry), but the data behind it disappears once purged. - **Testnet only:** don't send real tokens or real cards; they're rejected, and there is no mainnet settlement. - **Shared and rate-limited:** short ArNS names are reserved, the faucet is gated and rate-limited, and free-tier allowances are per-wallet and per-IP. ## Related # Uploading & Credits (/build/testnet/uploading-and-credits) Uploads go through the sandbox **Turbo bundler**. Data items are **ANS-104** — use the [Turbo SDK](/sdks/turbo-sdk) pointed at the sandbox endpoints, or `POST` a signed data item directly. ## Uploading with the Turbo SDK Point the SDK at the sandbox upload and payment services and sign with your **devnet Solana** keypair: ```ts const signer = new HexSolanaSigner(bs58.encode(secretKey)); // your devnet Solana keypair const turbo = TurboFactory.authenticated({ signer, token: 'solana', gatewayUrl: 'https://api.devnet.solana.com', // Solana RPC — used to VERIFY funding txs, NOT the ar.io gateway uploadServiceConfig: { url: 'https://upload.services.ar-io.dev' }, paymentServiceConfig: { url: 'https://payment.services.ar-io.dev' }, }); const { id, winc } = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream(path), fileSizeFactory: () => size, }); // Read the bytes back from the ar.io gateway, not from Turbo: // https://ar-io.dev/raw/ ``` **Watch the `gatewayUrl`.** Here it's the **Solana RPC** — how Turbo verifies your funding transactions — *not* `ar-io.dev`. You fetch uploaded data from the [gateway](/build/testnet/accessing-data) separately. Funding with `base-eth` instead? Use `token: 'base-eth'` and an EVM signer. ### Raw POST If you've already signed an ANS-104 data item, post the raw bytes: ```http POST https://upload.services.ar-io.dev/v1/tx Content-Type: application/octet-stream ``` A `200` returns `{ id, winc, dataCaches, ... }`. **`winc: "0"` means the upload was free.** ## Limits & responses - **Hard max size: 10 MiB** per data item. Larger uploads return **`413`** (`Data item is too large…`). - **Free tier** (no payment required): - Up to **105 KiB per item** is eligible to be free. - **10 MiB total free per wallet** (lifetime) **and** **10 MiB per IP /24 subnet** (lifetime) — an upload must fit under **both** or it isn't free. - Over the free allowance, or an item too big to be free, returns **`402`** `{ code: "FREE_TIER_EXHAUSTED", topUpUrl, byteCount }` → [top up credits](#getting-credits). ## Getting credits Credits (Turbo **"winc"**) are what the bundler debits for **paid uploads** and **ArNS purchases**. Fund them with **testnet** value only. **Accepted funding tokens:** | Token | Network | Notes | |---|---|---| | `ario` | ARIO staging (Solana devnet) | Mint `6vTw5CysRXQ4ybbHkDUiisHWVsBeMtUzYvJqs2iqHyaN`, 6 decimals — **fee-free** | | `solana` | Solana devnet | | | `base-eth` | Base Sepolia | | Mainnet tokens (`arweave`, `ethereum`, `matic`/`pol`) are **rejected** with `Token not supported`. There is no mainnet settlement here. ### Crypto top-up Submit your funding transaction to `POST /v1/account/balance/:token`. Get the per-token receiving addresses from `GET /v1/info` (the `addresses` map). The transaction is verified on the **testnet** RPC for that token, then your balance is credited. ### Stripe (test cards) The fiat top-up flow runs in **test mode**. Use Stripe test cards — e.g. `4242 4242 4242 4242` with any future expiry and CVC. No real money moves. ### x402 (USDC) The unsigned x402 upload path runs on **Base Sepolia** testnet USDC: `POST /x402/upload/unsigned`, then pay the returned `402` quote via an `X-PAYMENT` header. See [x402 uploading](/build/upload/x402-uploading-to-turbo) for the full flow. ## Next steps # Advanced Uploading with Turbo (/build/upload/advanced-uploading-with-turbo) Learn how to upload data to Arweave using the **Turbo SDK** for a streamlined upload experience with multiple payment options and authentication methods. ## What You'll Learn - How to install and authenticate with the Turbo SDK - Different authentication methods (Arweave, Ethereum, Solana, etc.) - How to purchase Turbo Credits - How to upload files, strings, binary data, and entire folders to Arweave - Browser and Node.js implementation examples - Using the versatile `upload` method for all data types ## Prerequisites - Node.js environment or modern web browser - Wallet for authentication (Arweave, Ethereum, Solana, etc.) - Basic understanding of JavaScript/TypeScript ## Quick Start ### Install the Turbo SDK ```bash # For Node.js npm install @ardrive/turbo-sdk # For Yarn users yarn add @ardrive/turbo-sdk ``` ### Authenticate with Your Wallet Choose your preferred authentication method: ```typescript // Load your Arweave JWK file const jwk = JSON.parse(fs.readFileSync('wallet.json', 'utf-8')) const turbo = await TurboFactory.authenticated({ privateKey: jwk, // ArweaveJWK type token: 'arweave', // Default token type }) ``` ```typescript // Your Ethereum private key (with 0x prefix) const privateKey = '0x1234...' // EthPrivateKey type // Create an Ethereum signer instance const signer = new EthereumSigner(privateKey) const turbo = await TurboFactory.authenticated({ signer, token: 'ethereum', }) ``` ```typescript // Your Solana secret key (as Uint8Array) const secretKey = new Uint8Array([...]) // SolSecretKey type const turbo = await TurboFactory.authenticated({ privateKey: bs58.encode(secretKey), token: 'solana' }) ``` ```typescript // Your Polygon private key (with 0x prefix) const privateKey = '0x1234...' // EthPrivateKey type // Create an Ethereum signer instance for Polygon const signer = new EthereumSigner(privateKey) const turbo = await TurboFactory.authenticated({ signer, token: 'matic', // or 'pol' }) ``` ```typescript async function initializeTurbo() { await window.arweaveWallet.connect([ 'ACCESS_ADDRESS', 'ACCESS_PUBLIC_KEY', 'SIGN_TRANSACTIONS', 'SIGN_MESSAGE', 'SIGNATURE', ]) const turbo = await TurboFactory.authenticated({ signer: new ArConnectSigner(window.arweaveWallet), }) } ``` ```typescript // Global variables for Wagmi config and connector let config = null let connector = null let turboInstance = null // Function to set up Wagmi configuration config = wagmiConfig connector = wagmiConnector } // Function to initialize Turbo with Wagmi try { if (!config || !connector) { throw new Error( 'Wagmi config and connector not set. Call setWagmiConfig first.', ) } console.log('Initializing Turbo client...') // Create a provider that uses wagmi's signMessage const provider = { getSigner: () => ({ signMessage: async (message) => { const arg = message instanceof String ? message : { raw: message } const ethAccount = getAccount(config) return await signMessage(config, { message: arg, account: ethAccount.address, connector: connector, }) }, }), } // Create the Turbo signer const signer = new InjectedEthereumSigner(provider) // Set up the public key signer.setPublicKey = async () => { const message = 'Sign this message to connect to Turbo' const ethAccount = getAccount(config) const signature = await signMessage(config, { message: message, account: ethAccount.address, connector: connector, }) const hash = await hashMessage(message) const recoveredKey = await recoverPublicKey({ hash, signature, }) signer.publicKey = Buffer.from(toBytes(recoveredKey)) } // Initialize the signer await signer.setPublicKey() turboInstance = await TurboFactory.authenticated({ signer: signer, token: 'base-eth', // Can be changed to 'ethereum' or 'matic', etc. }) console.log('Turbo client initialized successfully') return turboInstance } catch (error) { console.error('Error initializing Turbo client:', error) turboInstance = null throw error } } ``` ```typescript if (!window.ethereum) { throw new Error('Please install MetaMask to use this application') } try { const accounts = await window.ethereum.request({ method: 'eth_requestAccounts', }) const metaMaskProvider = window.ethereum.providers?.find( (p) => p.isMetaMask, ) const provider = new BrowserProvider(metaMaskProvider ?? window.ethereum) const signer = await provider.getSigner() const turbo = TurboFactory.authenticated({ signer: new InjectedEthereumSigner({ getSigner: () => signer }), token: 'ethereum', }) return { turbo, address: accounts[0] } } catch (error) { console.error('Connection failed:', error) throw error } } ``` ```typescript try { // Check if Phantom is installed if (window.solana) { const provider = window.solana const publicKey = new PublicKey((await provider.connect()).publicKey) const wallet: SolanaWalletAdapter = { publicKey, signMessage: async (message: Uint8Array) => { // Call Phantom's signMessage method const { signature } = await provider.signMessage(message) return signature }, } solanaTurboInstance = TurboFactory.authenticated({ token: 'solana', walletAdapter: wallet, }) } } catch (err) { console.error(err) } } ``` ### Purchase Turbo Credits [Turbo Credits](/build/upload/turbo-credits) are the payment medium used by the Turbo Upload Service. Each Credit represents a 1:1 conversion from the upload power of the Arweave native token (AR). - **Fiat Currency**: Credit/debit cards via the [Turbo Top Up App](https://turbo-topup.com/) - **Cryptocurrencies**: AR, ETH, SOL, MATIC, ARIO, USDC, ETH (BASE) - **Multiple Wallets**: Ethereum, Solana, and Arweave wallets supported ```typescript // Initialize authenticated client const turbo = await TurboFactory.authenticated({ privateKey: jwk }) // Top up with AR tokens const topUpResult = await turbo.topUpWithTokens({ tokenAmount: WinstonToTokenAmount(100_000_000), // 0.0001 AR }) ``` ```typescript // Initialize authenticated client const turbo = await TurboFactory.authenticated({ signer: new EthereumSigner(privateKey), token: 'ethereum', }) // Top up with ETH tokens const topUpResult = await turbo.topUpWithTokens({ tokenAmount: 0.001, // 0.001 ETH }) ``` ```typescript // Initialize authenticated client const turbo = await TurboFactory.authenticated({ privateKey: bs58.encode(secretKey), token: 'solana' }) // Top up with SOL tokens const topUpResult = await turbo.topUpWithTokens({ tokenAmount: 0.1, // 0.1 SOL }) ``` ```typescript // Initialize authenticated client const turbo = await TurboFactory.authenticated({ signer: new EthereumSigner(privateKey), token: 'matic', }) // Top up with MATIC tokens const topUpResult = await turbo.topUpWithTokens({ tokenAmount: 1.0, // 1.0 MATIC }) ``` ### Upload Your First File ```typescript // Upload a single file using the versatile upload method const result = await turbo.upload({ data: file, // Can be File, Blob, Buffer, Uint8Array, ArrayBuffer, or string dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }); console.log("File uploaded!", { id: result.id, url: `https://turbo-gateway.com/${result.id}`, owner: result.owner, dataCaches: result.dataCaches, }); ``` ## Uploading Files ### Basic File Upload ```typescript // Upload a single file using the versatile upload method const result = await turbo.upload({ data: file, // Can be File, Blob, Buffer, Uint8Array, ArrayBuffer, or string dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }); console.log("File uploaded!", { id: result.id, url: `https://turbo-gateway.com/${result.id}`, owner: result.owner, dataCaches: result.dataCaches, }); ``` ### Upload with Custom Tags ```typescript const result = await turbo.upload({ data: file, dataItemOpts: { tags: [ { name: "Content-Type", value: "application/json" }, { name: "App-Name", value: "MyApp-v1.0" }, { name: "App-Version", value: "1.0.0" }, { name: "Description", value: "My application data" }, ], }, }); ``` ### Upload Strings ```typescript // Upload a string const stringResult = await turbo.upload({ data: "Hello, Arweave!", dataItemOpts: { tags: [ { name: "Content-Type", value: "text/plain" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }); ``` ### Upload JSON Data ```typescript // Upload a JSON object const jsonData = { message: "Hello", timestamp: Date.now() }; const jsonResult = await turbo.upload({ data: JSON.stringify(jsonData), dataItemOpts: { tags: [ { name: "Content-Type", value: "application/json" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }); ``` ### Upload Binary Data ```typescript // Upload binary data const binaryData = new Uint8Array([1, 2, 3, 4, 5]); const binaryResult = await turbo.upload({ data: binaryData, dataItemOpts: { tags: [ { name: "Content-Type", value: "application/octet-stream" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }); ``` ### Upload Multiple Files ```typescript const files = [file1, file2, file3]; const uploadPromises = files.map((file) => turbo.upload({ data: file, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream", }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }) ); const results = await Promise.all(uploadPromises); console.log("All files uploaded!", results); ``` ### Upload an Entire Folder (Node.js) ```typescript const folderResult = await turbo.uploadFolder({ folderPath: "./my-website", dataItemOpts: { tags: [ { name: "App-Name", value: "MyWebsite-v1.0" }, { name: "Content-Type", value: "application/x.arweave-manifest+json" }, ], }, manifestOptions: { indexFile: "index.html", fallbackFile: "404.html", }, }); console.log("Folder uploaded!", { manifestId: folderResult.manifestResponse?.id, fileCount: folderResult.fileResponses.length, manifest: folderResult.manifest, }); ``` ### Upload Multiple Files as a Folder (Browser) ```typescript const files = [file1, file2, file3]; const webFolderResult = await turbo.uploadFolder({ files: files, dataItemOpts: { tags: [{ name: "App-Name", value: "MyWebsite-v1.0" }], }, manifestOptions: { indexFile: "index.html", }, }); ``` ## Browser Implementation Examples ### File Input with Drag & Drop ```html Turbo Upload Example .drop-zone { border: 2px dashed #ccc; border-radius: 10px; padding: 20px; text-align: center; margin: 20px 0; } .drag-over { border-color: #007bff; background-color: #f8f9fa; } Drag and drop files here or click to select // Your Turbo initialization code here // ... (authentication code from above) const fileInput = document.getElementById("file-input"); const dropZone = document.getElementById("drop-zone"); // File input handler fileInput.addEventListener("change", async (event) => { const files = Array.from(event.target.files); for (const file of files) { await uploadFile(file); } }); // Drag and drop handlers dropZone.addEventListener("dragover", (e) => { e.preventDefault(); e.stopPropagation(); dropZone.classList.add("drag-over"); }); dropZone.addEventListener("dragleave", (e) => { e.preventDefault(); e.stopPropagation(); dropZone.classList.remove("drag-over"); }); dropZone.addEventListener("drop", async (e) => { e.preventDefault(); e.stopPropagation(); dropZone.classList.remove("drag-over"); const files = Array.from(e.dataTransfer.files); for (const file of files) { await uploadFile(file); } }); async function uploadFile(file) { try { const result = await turbo.upload({ data: file, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream", }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, }); console.log("File uploaded!", { id: result.id, url: `https://turbo-gateway.com/${result.id}`, name: file.name, size: file.size, }); } catch (error) { console.error("Upload failed:", error); } } ``` ## Advanced Features ### Check Upload Costs ```typescript // Get upload cost for specific file size const costs = await turbo.getUploadCosts({ bytes: [file.size], }); console.log(`Upload cost: ${costs[0].winc} Winston Credits`); console.log(`USD cost: $${costs[0].usd}`); ``` ### Check Balance ```typescript // Get current balance const balance = await turbo.getBalance(); console.log(`Available credits: ${balance.controlledWinc} Winston Credits`); ``` ### Upload with Progress Tracking ```typescript const result = await turbo.upload({ data: file, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, events: { onUploadProgress: (progress) => { console.log( `Upload progress: ${Math.round((progress.processedBytes / progress.totalBytes) * 100)}%` ); }, onSigningProgress: (progress) => { console.log( `Signing progress: ${Math.round((progress.processedBytes / progress.totalBytes) * 100)}%` ); }, }, }); ``` ### Upload with Error Handling ```typescript try { const result = await turbo.upload({ data: file, dataItemOpts: { tags: [ { name: "Content-Type", value: file.type || "application/octet-stream" }, { name: "App-Name", value: "MyApp-v1.0" }, ], }, events: { onUploadError: (error) => { console.error("Upload failed:", error); }, onSigningError: (error) => { console.error("Signing failed:", error); }, }, }); console.log("Upload successful:", result); } catch (error) { console.error("Upload error:", error); // Handle error appropriately } ``` ## Benefits of Using Turbo - **Versatile upload method** - Upload files, strings, binary data, or entire folders with a single method - **Multiple payment options** - Pay with fiat, crypto, or AR tokens - **Easy integration** - Simple SDK for both Node.js and browsers - **Automatic retry** - Built-in retry logic for failed uploads - **Cost transparency** - See upload costs before confirming - **Fast uploads** - Optimized for speed and reliability - **Folder support** - Upload entire directories with automatic manifest generation ## Next Steps } > Buy Turbo Credits with fiat or crypto }> Discover best practices for organizing your data } > Learn how to organize files with manifests # Getting Started with Turbo (/build/upload/bundling-services) Upload data to Arweave using Turbo, a managed gateway service designed for reliability, scale, and ease of use. Turbo handles the infrastructure so you can focus on shipping. ## What is Turbo? Turbo is a managed ar.io gateway service that simplifies uploading data to Arweave. It abstracts away funding, performance tuning, and operational complexity, giving teams a fast, dependable path to permanent storage. Turbo supports flexible payment options, including fiat and major cryptocurrencies, and is built to handle high-throughput workloads with consistent performance. Whether you’re prototyping or running production systems, Turbo provides a reliable upload layer backed by enterprise-grade infrastructure. If you’re looking for a managed gateway service tailored to your use case — including custom limits, performance, or integration support — book a call to learn how we can get one set up for you: [Contact Us!](https://ar.io/contact) ## Get Started ### Install the SDK ```bash npm install @ardrive/turbo-sdk ``` ### Set Up Your Wallet Create a new wallet or use an existing one: ```bash # Create a new wallet (easy way) npx permaweb/wallet > key.json ``` Then load it in your code: ```js // Load your wallet const jwk = JSON.parse(fs.readFileSync("./key.json", "utf-8")); const signer = new ArweaveSigner(jwk); // Initialize Turbo const turbo = TurboFactory.authenticated({ signer }); ``` ### Get Turbo Credits Purchase Turbo Credits to pay for uploads. When you upload, credits are automatically used and Turbo handles the payment to Arweave. **Option 1: Via the Web Interface** - Go to [turbo-topup.com](https://turbo-topup.com) - Pay with fiat currencies (credit cards) or crypto tokens (ARIO, USDC, SOL, MATIC, AR) **Option 2: Via the SDK** ```js // Purchase credits programmatically const fundResult = await turbo.topUpWithTokens({ tokenAmount: TOKEN_AMOUNT, tokenType: "solana", // or 'ethereum', 'matic', 'arweave' }); ``` **Check Your Balance** ```js const balance = await turbo.getBalance(); console.log(`Balance: ${balance.winc} Winston Credits`); ``` ### Upload Your Data ```js tab="Upload File" const fileData = fs.readFileSync("./myfile.jpg"); const result = await turbo.upload({ data: fileData, dataItemOpts: { tags: [ { name: "Content-Type", value: "image/jpeg" }, { name: "Title", value: "My Image" }, ], }, }); console.log("Upload ID:", result.id); console.log("Owner:", result.owner); ```` ```js tab="Upload Folder" const folderResult = await turbo.uploadFolder({ folderPath: "./my-folder", dataItemOpts: { tags: [ { name: "Bundle-Format", value: "binary" }, { name: "Bundle-Version", value: "2.0.0" }, ], }, }); console.log("Folder Upload ID:", folderResult.id); console.log("Manifest ID:", folderResult.manifestId); ```` ```js tab="Upload Raw Data" const data = JSON.stringify({ message: "Hello Arweave!", timestamp: Date.now(), }); const result = await turbo.upload({ data: data, dataItemOpts: { tags: [ { name: "Content-Type", value: "application/json" }, { name: "App-Name", value: "MyApp" }, ], }, }); console.log("Upload ID:", result.id); console.log("Owner:", result.owner); ``` ## Advanced Features ### Turbo Credits System Learn about our flexible payment system that supports multiple currencies and payment methods. → [Understanding Turbo Credits](/build/upload/turbo-credits) ### Data Organization - [**Tagging**](/build/upload/tagging) - Organize your data with metadata - [**Manifests**](/build/upload/manifests) - Create folder structures and bundles - [**Encryption**](/build/upload/encryption) - Secure your sensitive data - [**ArFS**](/build/advanced/arfs) - File system protocol for structured storage ## Production Ready Turbo implements the **[ANS-104 bundling specification](https://github.com/ArweaveTeam/arweave-standards/blob/master/ans/ANS-104.md)**, providing enterprise-grade infrastructure for permanent data storage. **Perfect for:** Developers, production applications, high-volume uploads, and any project needing reliable permanent storage with flexible payment options. | Feature | Turbo Bundling | Alternative Options | | -------------------- | -------------------------------- | ---------------------------- | | **Payment Options** | Fiat, ARIO, USDC, SOL, MATIC, AR | AR tokens only | | **Implementation** | Simple SDK integration | Manual transaction handling | | **Performance** | Optimized bundling & retry logic | Depends on implementation | | **Reliability** | Built-in redundancy | Manual error handling | | **Cost** | Optimized for large uploads | Higher per-transaction costs | | **Setup Complexity** | Easy with SDK | Complex protocol knowledge | **Need help deciding?** Most developers should use Turbo for its simplicity and payment flexibility. Only consider alternatives for specialized use cases requiring maximum control. ## Ready to Get Started? } > Start building with Turbo's powerful bundling service. }> Explore the full SDK documentation and examples. } > Organize your data with metadata and tags. # Encryption (/build/upload/encryption) **Arweave has no built-in encryption.** All encryption and decryption must be handled client-side before uploading data to the network. Arweave is completely data-agnostic - it stores whatever data you provide without any knowledge of whether it's encrypted or not. ## How Encryption Works on Arweave **Critical Points:** - **No native encryption**: Arweave provides no encryption services whatsoever - **Client-side only**: You must encrypt data before uploading - **Data-agnostic storage**: Arweave stores any data type, including encrypted data - **Your responsibility**: You handle all encryption, key management, and decryption - **Permanent security**: Once encrypted and stored, data remains secure forever ## Encryption Options ### 1. Manual Client-Side Encryption Encrypt your data before uploading with Turbo: ```js // Encrypt sensitive data const data = "Sensitive information"; const secretKey = "your-secret-key"; const encryptedData = CryptoJS.AES.encrypt(data, secretKey).toString(); // Upload encrypted data const result = await turbo.upload({ data: encryptedData, dataItemOpts: { tags: [ { name: "Content-Type", value: "application/octet-stream" }, { name: "Encrypted", value: "true" }, { name: "Cipher", value: "AES-256-GCM" }, { name: "Cipher-IV", value: "YWJjZGVmZ2hpams=" }, // 12 byte initialization vector as Base64 ], }, }); ``` ## Encryption Standards ### Encryption Methods - **AES-256-GCM**: Authenticated encryption (recommended) - **AES-256-CTR**: Stream cipher for large files - **Any encryption method**: Arweave supports any encryption you choose (must be indicated in `Cipher` tag for ArFS compliance) ### Required Tags When uploading encrypted data, include these tags: ```js { name: "Content-Type", value: "application/octet-stream" // Required for encrypted data }, { name: "Cipher", value: "AES-256-GCM" // Specify encryption method }, { name: "Cipher-IV", value: "base64-encoded-iv" // Initialization vector } ``` ## ArFS Protocol (Optional Standardization) The [Arweave File System (ArFS)](/build/advanced/arfs) protocol provides optional standardization for encrypted storage: - **Private Drives**: Encrypt entire file systems - **File-level encryption**: Each file has its own encryption key - **Selective sharing**: Share individual files without exposing the entire drive - **Key derivation**: Uses HKDF-SHA256 with wallet signatures - **Completely optional**: You can use any encryption method you prefer **ArDrive Web App:** Data uploaded through the ArDrive web app to Private Drives is encrypted for you using the standards set in the ArFS protocol. ArDrive is simply a web application that implements ArFS - there is no separate "ArDrive Encryption Service." **ArFS Privacy:** To learn more about ArFS encryption schema, key derivation, and private drive management, see our detailed [ArFS Privacy & Encryption documentation](/build/advanced/arfs/privacy). ## Getting Started For most users, the ArDrive web app provides the easiest way to encrypt and store data using ArFS standards: **Create a private drive** in the ArDrive web app **Set a strong password** for your drive **Upload files** - they're automatically encrypted using ArFS **Access files** using your password and wallet For developers who need custom encryption: **Choose an encryption library** (Crypto-JS, Web Crypto API) **Encrypt your data** before uploading **Add proper tags** to indicate encryption **Store keys securely** for decryption ## Security Considerations **Important:** Never store encryption keys in your code or public repositories. Use secure key management practices and consider hardware security modules for production applications. **Best Practices:** - Use strong, randomly generated keys - Implement proper key rotation - Store keys securely (not in code) - Use authenticated encryption (AES-GCM) - Validate data integrity after decryption ## Next Steps } > Use the ArDrive web app for easy encrypted file storage using ArFS. }> Explore the Arweave File System protocol for structured storage. } > Purchase credits for programmatic uploads. # Upload Data (/build/upload) import { CreditCard, Upload, Code, Tag, Shield, FolderOpen, Zap, Check, Image, } from "lucide-react"; Arweave enables **permanent data storage** with a single payment. Unlike traditional cloud storage that requires ongoing fees, your data is preserved forever. ## Upload Methods There are multiple ways to upload data to Arweave. Each has its own attributes and characteristics to help you decide which is best for your use case. Turbo{" "} Recommended } description="Production-ready bundling service with enterprise features • Pay with credit cards, AR, ETH, SOL, MATIC • Free uploads under 100 KiB • Automatic retry & confirmation" href="/build/upload/bundling-services" icon={} /> } /> } /> ## Why Developers Choose Turbo **Cost Effective** - Pay per byte, not empty chunks - only pay for actual data uploaded - Free uploads under 100 KiB - subsidized small file uploads - No failed upload charges - automatic retry without extra costs **Developer Experience** - TypeScript & CLI support - choose your preferred tools - Simple 3-line integration - get started in minutes - Comprehensive documentation - extensive guides and examples **Enterprise Ready** - High Availability - reliable service for production apps - Handles millions of uploads daily - battle-tested infrastructure used by ArDrive - Open source infrastructure - fully auditable and transparent ## Get Started in Minutes With Turbo, uploading to Arweave is as simple as using any cloud storage API: ```typescript const turbo = TurboFactory.authenticated({ privateKey }); const uploadResult = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream("./my-file.pdf"), }); // Your file is now permanently stored! ``` ## Organize Your Data Before uploading, learn best practices for structuring and tagging your data for optimal retrieval and organization. ## Additional Resources # Manifests (/build/upload/manifests) Manifests enable friendly-path-name routing for data on Arweave, greatly improving the programmability of data relationships. Instead of accessing data with complex transaction IDs, manifests allow you to organize files with readable paths and relative links. ## What are Manifests? Manifests, also known as "Path Manifests" or "Arweave Manifests," are JSON objects that connect various Arweave data items and define relational paths for easy navigation. A common use case is permanently hosting websites on Arweave by linking all necessary files together. ### The Problem Manifests Solve Without manifests, accessing data on Arweave looks like this: ``` http:///cG7Hdi_iTQPoEYgQJFqJ8NMpN4KoZ-vH_j7pG4iP7NI (txID of a website's index.html) http:///3zFsd7bkCAUtXUKBQ4XiPiQvpLVKfZ6kiLNt2XVSfoV (txID of its js/style.css) http:///or0_fRYFcQYWh-QsozygI5Zoamw_fUsYu2w8_X1RkYZ (txID of its assets/img/logo.png) ``` With manifests, the same data becomes: ``` http:/// (resolves to the txID of index.html) http:////js/style.css http:////assets/img/logo.png ``` ## Manifest Structure Manifests are JSON objects that define how data items are connected and accessed through friendly paths. ### Sample Manifest ```json { "manifest": "arweave/paths", "version": "0.2.0", "index": { "path": "index.html" }, "fallback": { "id": "iXo3LSfVKVtXUKBzfZ4d7bkCAp6kiLNt2XVUFsPiQvQ" }, "paths": { "index.html": { "id": "cG7Hdi_iTQPoEYgQJFqJ8NMpN4KoZ-vH_j7pG4iP7NI" }, "404.html": { "id": "iXo3LSfVKVtXUKBzfZ4d7bkCAp6kiLNt2XVUFsPiQvQ" }, "js/style.css": { "id": "3zFsd7bkCAUtXUKBQ4XiPiQvpLVKfZ6kiLNt2XVSfoV" }, "css/style.css": { "id": "sPiQvpAUXLVK3zF6iXSfo7bkCVQkiLNt24dVtXUKBfZ" }, "css/mobile.css": { "id": "fZ4d7bkCAUiXSfo3zFsPiQvpLVKVtXUKB6kiLNt2XVQ" }, "assets/img/logo.png": { "id": "or0_fRYFcQYWh-QsozygI5Zoamw_fUsYu2w8_X1RkYZ" }, "assets/img/icon.png": { "id": "0543SMRGYuGKTaqLzmpOyK4AxAB96Fra2guHzYxjRGo" } } } ``` ### How it Works A resolver, typically an ar.io gateway, resolves URLs requesting content based on a manifest transaction ID to the corresponding path key in the `paths` object. The URL schema for this type of request is `https:////`. ### Example Usage Assume the manifest above is uploaded to Arweave with the transaction ID `UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk`. The below table shows https requests to the ar.io gateway `turbo-gateway.com`: | Request Path | Manifest Path | Data served from txID | | ---------------------------------------------------------------------------- | ------------- | ------------------------------------------- | | https://turbo-gateway.com/UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk | index | cG7Hdi_iTQPoEYgQJFqJ8NMpN4KoZ-vH_j7pG4iP7NI | | https://turbo-gateway.com/UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk/index.html | index.html | cG7Hdi_iTQPoEYgQJFqJ8NMpN4KoZ-vH_j7pG4iP7NI | | https://turbo-gateway.com/UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk/js/style.css | js/style.css | 3zFsd7bkCAUtXUKBQ4XiPiQvpLVKfZ6kiLNt2XVSfoV | | https://turbo-gateway.com/UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk/foobar | fallback | iXo3LSfVKVtXUKBzfZ4d7bkCAp6kiLNt2XVUFsPiQvQ | ## Creating Manifests with Turbo Turbo makes it easy to create manifests automatically when uploading folders, or you can create custom manifests manually. ### Folder Upload with Manifest ```js const folderResult = await turbo.uploadFolder({ folderPath: "./my-website", dataItemOpts: { tags: [ { name: "Bundle-Format", value: "binary" }, { name: "Bundle-Version", value: "2.0.0" }, { name: "App-Name", value: "Website" }, ], }, }); console.log("Folder Upload ID:", folderResult.id); console.log("Manifest ID:", folderResult.manifestId); ``` ### Custom Manifest Creation ```js const manifest = { manifest: "arweave/paths", version: "0.2.0", index: { path: "index.html", }, fallback: { id: "fallback-tx-id", }, paths: { "index.html": { id: "abc123...def789", }, "css/style.css": { id: "def456...ghi012", }, }, }; const manifestResult = await turbo.upload({ data: JSON.stringify(manifest), dataItemOpts: { tags: [ { name: "Content-Type", value: "application/x.arweave-manifest+json" }, { name: "App-Name", value: "CustomManifest" }, ], }, }); ``` ## Manifest Specifications ### Required Transaction Tags Manifests must be uploaded with specific tags so that ar.io gateways can recognize and properly resolve them: ```json { "name": "Content-Type", "value": "application/x.arweave-manifest+json" } ``` **Important:** This tag must be attached to the upload transaction, NOT placed inside the JSON object. Failure to provide this tag will result in resolvers not recognizing the manifest. ### Required JSON Attributes #### manifest ```json "manifest": "arweave/paths" ``` Must have the value `arweave/paths` for gateways to resolve the manifest. #### version ```json "version": "0.2.0" ``` Defines the version of manifest schema being used. #### index ```json "index": { "path": "index.html" } ``` or ```json "index": { "id": "cG7Hdi_iTQPoEYgQJFqJ8NMpN4KoZ-vH_j7pG4iP7NI" } ``` Defines the base or 'starting' data item. Accepts either `path` (key in paths object) or `id` (specific transaction ID). If both are defined, `id` overrides `path`. #### fallback ```json "fallback": { "id": "iXo3LSfVKVtXUKBzfZ4d7bkCAp6kiLNt2XVUFsPiQvQ" } ``` Defines a fallback data item for when requested paths don't exist (like a 404 page). #### paths ```json "paths": { "index.html": { "id": "cG7Hdi_iTQPoEYgQJFqJ8NMpN4KoZ-vH_j7pG4iP7NI" }, "css/style.css": { "id": "3zFsd7bkCAUtXUKBQ4XiPiQvpLVKfZ6kiLNt2XVSfoV" } } ``` Defines the URL paths that a manifest can resolve to. Each path maps to a specific Arweave transaction ID. ## Relative Path Routing Ar.io gateways support relative path routing, making it easy to develop and maintain websites hosted on Arweave. Instead of using fully qualified URLs: ```html ``` You can use relative paths: ```html ``` This makes HTML more readable and ensures links remain valid even if the hosting domain changes. ## Best Practices ### File Organization - Use descriptive file paths - Organize files in logical folders - Keep manifest files small - Use consistent naming conventions ### Performance Considerations - Minimize manifest size - Use relative paths - Avoid deep nesting - Consider file size limits ## Next Steps } > Capture timestamped upload attestations for chronology and audits. } > Secure your sensitive data with encryption. } > Advanced file organization with ArFS. } > Complete upload guide with Turbo. # Receipts (/build/upload/receipts) Turbo upload receipts provide a durable record of what was uploaded, who uploaded it, and what upload cost was assessed in Winston Credits (`winc`). They are useful for provenance, compliance, incident response, and operational debugging. ## What Are Turbo Receipts? A Turbo receipt is a signed upload attestation returned by Turbo upload flows when a data item is successfully accepted and processed. In practice, receipts help you: - Link app-level records to immutable data item IDs - Track upload ownership and storage cost (`winc`) - Keep machine-readable proof metadata for investigations and audits Receipts are proof of Turbo upload acceptance and signed receipt metadata for a specific upload event. They are not a full substitute for your own retrieval checks, gateway checks, or finality policies. ## Why Time and Ordering Matter Receipt timestamps are especially valuable because they let you prove when upload events occurred and how related uploads were ordered. - A receipt `timestamp` supports evidence of event time in incident and audit workflows. - Ordered receipts create chronology across related data items (for example, original item, revision, and derived artifacts). - Chronology improves chain-of-custody reconstruction and post-incident analysis. For production systems, preserving upload order can be as important as preserving data IDs. ## When Receipts Are Created Turbo receipts are created in three common contexts: 1. Standard SDK uploads: `turbo.upload()` and `turbo.uploadFile()` return upload result payloads that include receipt metadata such as `id`, `owner`, `winc`, `dataCaches`, and `fastFinalityIndexes`. 2. Multipart uploads: for larger uploads that use chunking, Turbo finalizes the upload and returns a finalized receipt payload when multipart status reaches `FINALIZED`. 3. x402 uploads: Turbo still returns an upload receipt, and x402 tooling can also provide a separate payment settlement receipt. Regardless of flow, store receipt timestamps and your own sequence metadata so ordered events can be reconstructed later. Turbo does not store and retrieve your receipts for you. If you need receipts later, persist the exact payload returned by Turbo in your own storage. You can also optionally store receipt records on Arweave for long-term archival. See [Capturing Receipts in Turbo SDK](#capturing-receipts-in-turbo-sdk) for an implementation example. ## Receipt Anatomy | Field | Meaning | Notes | | ----- | ------- | ----- | | `id` | Data item transaction ID | Primary key to join with your app records | | `owner` | Normalized address that signed/owns the data item | Useful for audit and policy checks | | `winc` | Upload cost measured in Winston Credits | Useful for billing and reporting. See [Paying for Uploads](/build/upload/turbo-credits) for unit and payment context | | `dataCaches` | Caches that accepted the data item | Operational visibility for upload path | | `fastFinalityIndexes` | Fast finality indexes that accepted the data item | Useful for observability | | `timestamp` | Receipt creation time in milliseconds | Core field for chronology, ordering, and evidence of event time | | `version` | Receipt schema/version identifier | Determines how verification inputs should be interpreted | | `deadlineHeight` | Deadline block height recorded in receipt payload | Useful as additional upload context | | `public` | Public key that signed the receipt | Needed to verify signature validity | | `signature` | Base64URL receipt signature | Required to verify receipt authenticity | Some fields vary by upload path and service response shape. Do not assume every SDK return object includes every signed receipt field in all flows. ## Capturing Receipts in Turbo SDK Capture the upload response as your receipt record and persist it alongside your own object IDs. Below is a simple example, in practice you might chose to store this alongside your existing logs in S3, Sentry or similar: ```typescript const turbo = TurboFactory.authenticated({ privateKey }); const receipt = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream("./report.json"), fileSizeFactory: () => fs.statSync("./report.json").size, dataItemOpts: { tags: [ { name: "Content-Type", value: "application/json" }, { name: "App-Name", value: "AnalyticsPipeline-v1.0" }, ], }, }); await fetch("https://api.yourdomain.com/upload-receipts", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ appObjectId: "report-2026-02-16", capturedAt: new Date().toISOString(), receipt, }), }); console.log("Stored Turbo receipt:", { id: receipt.id, timestamp: receipt.timestamp, owner: receipt.owner, winc: receipt.winc, }); ``` Turbo CLI upload commands print JSON output that can be captured as a receipt record in scripts and CI pipelines. ## Why Receipts Matter Across ar.io Use Cases | AR.IO Use Case | Receipt Value | | -------------- | ------------- | | [File Storage](https://ar.io/use-cases/file-storage/) | Map internal file objects to immutable upload IDs and timestamped upload events | | [Websites & Apps](https://ar.io/use-cases/websites-and-apps/) | Track deployment artifacts and ordering of publish history over time | | [Apps & Game Assets](https://ar.io/use-cases/apps-and-game-assets/) | Prove that specific asset versions were accepted and in which sequence they were released | | [Media Provenance](https://ar.io/use-cases/media-provenance/) | Establish chain-of-custody metadata for original and derivative media with event ordering | | [Verifiable AI Data](https://ar.io/use-cases/verifiable-ai-data/) | Attach signed upload evidence and chronology to datasets, prompts, and model outputs | | [Verifiable Computing](https://ar.io/use-cases/verifiable-computing/) | Bind compute inputs/outputs to durable upload receipts for reproducibility and timeline checks | | [Durable Financial Data](https://ar.io/use-cases/durable-financial-data/) | Maintain publish-time evidence for disclosures, reports, and compliance records | ## Deep Dive: Media Provenance Receipts strengthen provenance workflows by giving each media upload a signed event record with a timestamp and ordering context. - Preserve the receipt for original media ingestion - Store receipts for edited or transformed derivatives - Link parent and derivative records in your metadata model - Use receipt IDs, timestamps, and signed fields as part of publishing audit trails This gives teams a clearer chain-of-custody model for authenticity claims, moderation workflows, and external verification requests. ## Deep Dive: Verifiable AI Data AI pipelines benefit from receipts because reproducibility depends on stable, attributable inputs and outputs over time. - Store receipts for datasets, prompts, and generated artifacts - Link receipts to model versions, run IDs, and evaluation jobs - Preserve ordered receipt timelines to reconstruct dataset-to-output lineage - Use receipts as evidence in regulated or policy-bound AI workflows This makes lineage more transparent and reduces ambiguity when debugging model behavior or validating published results. ## Verifying Receipts Later Delayed verification is valuable for audits, disputes, compliance reviews, and reproducibility checks that happen long after initial upload. ### Verification Checklist 1. Load the stored receipt payload. 2. Validate required fields (`id`, `version`, `public`, `signature`, and contextual fields you depend on). 3. Verify the signature against the public key using receipt-version-specific hashing/signing rules. 4. Confirm data item status and availability via Turbo or gateway status endpoints. 5. Compare verified receipt data with your internal upload metadata and flag mismatches. Example verification workflow: ```typescript type StoredReceipt = { id: string; version?: string; public?: string; signature?: string; timestamp?: number; owner?: string; }; async function verifyStoredReceipt(receipt: StoredReceipt) { if (!receipt.id || !receipt.version || !receipt.public || !receipt.signature) { throw new Error("Receipt is missing required verification fields."); } // Implement this in your backend with version-aware receipt rules. const signatureValid = await verifyReceiptSignatureForVersion(receipt); if (!signatureValid) { throw new Error(`Invalid receipt signature for ${receipt.id}`); } const statusResponse = await fetch( `https://upload.ardrive.io/v1/tx/${receipt.id}/status`, ); if (!statusResponse.ok) { throw new Error(`Unable to retrieve status for ${receipt.id}`); } const status = await statusResponse.json(); return { id: receipt.id, signatureValid, status, }; } ``` Verification inputs are receipt-version dependent. Do not assume every response field is signature-bound in every version, and do not assume a high-level SDK helper exists unless it is explicitly documented for your target version. ## Best Practices - Persist raw receipt payloads server-side and keep them immutable - Store receipts with your own object IDs, pipeline IDs, and environment metadata - Index by data item `id` for fast traceability across systems - Preserve upload chronology (for example sequence numbers, parent/child links, and timestamps) - Capture both upload receipts and payment receipts when using x402 - Re-check status/finality in stricter workflows using service or gateway checks Treat a receipt as proof of Turbo upload acceptance and signing for that event, not as a blanket guarantee for every downstream retrieval state. ## Next Steps }> Connect receipt evidence to structured path-based content organization. }> Protect sensitive content before uploading to permanent storage. } > Add just-in-time payment flows to your upload pipeline. } > Understand Turbo Credits, top ups, and funding workflows. # Tagging (/build/upload/tagging) Tags are key-value pairs that provide metadata about your uploaded data on Arweave. They enable discoverability, proper content serving, and integration with various protocols. ## Essential Tags Every upload should include these tags: - **Content-Type**: Required - tells gateways how to serve your data - **App-Name**: Best practice - identifies your application for discoverability ```js const result = await turbo.upload({ data: fileData, dataItemOpts: { tags: [ { name: "Content-Type", value: "image/jpeg" }, { name: "App-Name", value: "MyApp-v1.0" }, { name: "Title", value: "My Image" }, ], }, }); ``` ## Common Tag Types ### Content Types - `image/jpeg`, `image/png` - Images - `application/json` - JSON data - `text/html` - HTML pages - `video/mp4` - Videos - `application/pdf` - Documents ### App-Specific Tags - `App-Name` - Your application identifier (e.g., "MyApp-v1.0", "PhotoGallery-2024") - `Title` - Human-readable title - `Description` - Content description - `Author` - Content creator - `Version` - Application version ### Protocol Tags - `License` - Universal Data License (UDL) transaction ID - `License-Fee` - Fee for UDL licensing **UDL Integration:** Learn about the [Universal Data License](https://mirror.xyz/0x64eA438bd2784F2C52a9095Ec0F6158f847182d9/AjNBmiD4A4Sw-ouV9YtCO6RCq0uXXcGwVJMB5cdfbhE) for monetizing your data. ## Advanced Tagging ### Folder Uploads ```js const folderResult = await turbo.uploadFolder({ folderPath: "./my-website", dataItemOpts: { tags: [ { name: "Bundle-Format", value: "binary" }, { name: "Bundle-Version", value: "2.0.0" }, { name: "App-Name", value: "MyWebsite-v2.1" }, { name: "Version", value: "2.1.0" }, ], }, }); ``` ### Licensed Content ```js const licensedTags = [ { name: "Content-Type", value: "image/jpeg" }, { name: "App-Name", value: "ArtGallery-v3.2" }, { name: "Version", value: "3.2.1" }, { name: "License", value: "udl-tx-id-here" }, { name: "License-Fee", value: "1000000" }, // Fee in Winston ]; ``` ## App-Name Best Practices ### Naming Convention Use descriptive, versioned App-Name values for better organization: - **Include version**: `MyApp-v1.0`, `PhotoGallery-2024` - **Be specific**: `EcommerceStore-v2.1` instead of just `Store` - **Use consistent format**: `ProjectName-vMajor.Minor` - **Include year for time-based apps**: `YearlyReport-2024` ## Tag Limitations - **4KB total** for bundled data items (Turbo) - **2KB total** for direct L1 uploads - **No maximum number of tags** (limited by total size) - Tag names are case-sensitive - No duplicate tag names allowed **Important:** Total tag size is limited to 4KB (bundled) or 2KB (L1). For larger metadata, store it in the data payload instead. ## Querying Data by Tags Once you've tagged your data, you can use GraphQL to search and filter based on those tags. This enables powerful discovery and retrieval of your stored content. ## Next Steps } > Organize files with manifests for better structure. }> Secure your sensitive data with encryption. } > Advanced file organization with ArFS. # Paying for Uploads (/build/upload/turbo-credits) import { Bot, Code, CreditCard, FileText, FolderOpen, Tag, Upload, Zap, } from "lucide-react"; Data storage on Arweave is paid for in Arweave's native [AR](https://www.coingecko.com/en/coins/arweave) token — divisible to the "Winston" (10¹² per AR). Turbo makes paying for data storage on Arweave faster and even simpler via Turbo Credits. ## What are Turbo Credits? Turbo Credits are the payment medium used by Turbo's upload service, providing a 1:1 representation of upload power on the Arweave network, and enabling precise pricing for uploads of any size. Unlike traditional crypto tokens, Turbo Credits cannot be traded or exchanged on secondary markets - they exist solely for uploading data to Arweave. However, credit holders can [share their credits](#credit-sharing) with other wallets, enabling organizational and collaborative use cases. **Important:** Turbo Credits are non-refundable and cannot be withdrawn or exchanged for other cryptocurrencies. ## How to Purchase Credits There are two important choices to make when purchasing credits: 1. Pay in either fiat via Stripe or via crypto tokens on a supported network 2. The address of a [supported crypto wallet](#supported-payment-wallets) type to make the primary holder of the credits When paying in fiat or crypto, the destination wallet address can be one from any [supported crypto wallet](#supported-payment-wallets). It is optional, however, when paying in crypto; the payer wallet is simply used as the default choice in its absence. ### Payment Methods Add credits to your wallet and they will be deducted automatically on each upload. For faster payment networks, it's possible to pay [just-in-time](#just-in-time-payments) with each upload. #### Fiat Payments Top up any address of a [supported wallet type](#supported-wallet-types) by paying with a credit or debit card via Stripe. #### Crypto Tokens Top up any address of a [supported wallet type](#supported-wallet-types) by paying in any of these supported crypto tokens: | Network | Tokens | | ----------- | -------------- | | Arweave | [AR](https://www.coingecko.com/en/coins/arweave) | | Solana | [ARIO](https://www.coingecko.com/en/coins/ar-io-network), [SOL](https://www.coingecko.com/en/coins/solana) | | Base | [ARIO](https://basescan.org/token/0x138746adfA52909E5920def027f5a8dc1C7EfFb6), [ETH](https://www.coingecko.com/en/coins/ethereum), [USDC](https://basescan.org/token/0x833589fcd6edb6e08f4c7c32d4f71b54bda02913) | | Ethereum | [ETH](https://www.coingecko.com/en/coins/ethereum), [USDC](https://etherscan.io/token/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48) | | Polygon | [POL](https://www.coingecko.com/en/coins/polygon), [USDC](https://polygonscan.com/address/0x3c499c542cef5e3811e1192ce70d8cc03d5c3359) | ### Supported Payment Wallets | Network Type | Signature Types | Supported Wallets | | ------------ | -------------- | ------------------------------------------------------ | | Arweave | RSA | JWK Keyfile, [Wander](https://www.wander.app/), [Beacon](https://getbeaconapp.com/) | | EVM | ECDSA (secp256k1) | JSON Keystore File, [MetaMask](https://metamask.io/), [Rainbow](https://rainbow.me/), [Brave](https://brave.com/wallet/), [WalletConnect](https://walletconnect.com/), [Privy](https://www.privy.io/) | | Solana | ED25519 | JSON Keypair File, [Phantom](https://phantom.com/), [Solflare](https://www.solflare.com/) | | Cosmos | secp256k1 | Keyfile | ### Purchasing Interfaces #### Console App Visit [console.ar.io](https://console.ar.io) to purchase credits with fiat or crypto using a connected wallet. #### ArDrive App Visit [app.ardrive.io](https://app.ardrive.io) to purchase credits with fiat for Arweave wallets. #### Turbo Typescript SDK ```js const turbo = TurboFactory.authenticated({ signer, token: "base-usdc" }); // Top up with crypto const result = await turbo.topUpWithTokens({ tokenAmount: USDCToTokenAmount(1), destinationAddress: "optional-different-wallet-address", }); // Check balance const { winc } = await turbo.getBalance(); ``` See the [Turbo SDK documentation](/sdks/turbo-sdk) and [GitHub repository](https://github.com/ardriveapp/turbo-sdk) for complete examples. #### Turbo CLI ```bash # Top up with crypto turbo top-up --token base-usdc --amount 1 # Check balance turbo balance ``` See the [Turbo Node.js CLI documentation](https://github.com/ardriveapp/turbo-sdk/?tab=readme-ov-file#cli) for installation and usage. #### HTTP API Direct API integration is available at [payment.ardrive.io/api-docs](https://payment.ardrive.io/api-docs) for custom implementations. #### Payment Interfaces By Token | Payment Method | Turbo SDK | Turbo CLI | Turbo API | Console App | ArDrive App | x402 | | ---------------------------- | --------- | --------- | --------- | --------- | ----------- | ---- | | **Fiat (credit/debit card)** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | | **AR** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **ETH (L1)** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **ETH (Base)** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **SOL** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **POL** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **ARIO (Solana)** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **ARIO (Base)** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **USDC (L1)** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | **USDC (Base)** | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | | **USDC (Polygon)** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ## Just-in-Time Payments Just-in-time (JIT) payments allow you to pay for uploads at the moment of upload rather than pre-funding a credit balance. This is ideal for agentic workflows, automated pipelines, and systems that prefer pay-per-upload over balance management. JIT payments are available via the Turbo SDK, CLI, or direct HTTP API integration for tokens with sufficiently fast confirmation times: | Token / Currency | Network | JIT Supported | | ---------------------- | -------- | ------------- | | ARIO | Solana | ✅ | | ARIO | Base | ✅ | | SOL | Solana | ✅ | | ETH | Base | ✅ | | USDC | Base | ✅ | | POL | Polygon | ✅ | | USDC | Polygon | ✅ | | AR | Arweave | ❌ | | ETH | L1 | ❌ | | Fiat | Stripe | ❌ | ### x402 Protocol [x402](https://www.x402.org/) is a protocolized approach to JIT payments that leverages the HTTP 402 "Payment Required" status code. Turbo currently supports x402 payments made in **USDC on Base**. The following Turbo features can utilize x402: **Note:** x402 facilitator fees may be applicable for each x402 payment. - **Uploads:** pay for storage at the time of upload - **Top up:** useful when x402 is the preferred payment medium, but facilitator fees on individual uploads are undesirable Example SDK usage: ```js const turbo = TurboFactory.authenticated({ signer, token: "base-usdc" }); const result = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream("./myfile.jpg"), fileSizeFactory: () => fs.statSync("./myfile.jpg").size, fundingMode: new X402Funding({ maxMUSDCAmount: 1_000_000 }), // Max 1 USDC }); ``` See the [Turbo SDK documentation](/sdks/turbo-sdk) and [HTTP API docs](https://payment.ardrive.io/api-docs) for complete integration details. ## Credit Sharing Turbo Credits can be shared with other users to fund their uploads while maintaining total control over who may spend them, how many they may spend, and for how long they can do so. This is perfect for organizations, collaborations, and teams. Credits can be shared via the Console App at [console.ar.io](https://console.ar.io) or programmatically via the [Turbo CLI](https://github.com/ardriveapp/turbo-sdk/?tab=readme-ov-file#cli), [SDK](/sdks/turbo-sdk), or [API](https://payment.ardrive.io/api-docs). ### How Credit Sharing Works - **Authorize Users**: Grant specific wallets access to your credits - **Set Limits**: Control how much each user can spend - **Time Limits**: Set expiration dates for access - **Revoke Anytime**: Regain control of shared credits instantly ### Use Cases - **Organizational Funds**: Central wallet shares credits with employees - **Onboarding**: Give new users free upload power for trials - **Collaboration**: Share credits with project contributors - **Educational Programs**: Provide students with controlled access ## Pricing & Fees - Turbo top up fees cover the costs of managing infrastructure and, in the case of crypto-based top ups, token liquidity. - No additional fees are applied when credits are spent. - Credits maintain a 1:1 peg of storage purchasing power as Arweave's data storage rates and token price fluctuate. - **Uploads under 100 KiB are completely free and do not require a prior top up.** | Currency | Top Up Fee Percentage | | ----------------- | --------------------- | | ARIO (Solana or Base) | NO FEE | | All others | 35% | ## Getting Started Ready to start using Turbo Credits? Choose your path: } > Buy credits instantly with credit cards or crypto } > Learn how to upload data with your new credits }> Integrate credit sharing and advanced features } > Pay-per-upload for automated workflows ## Next Steps } > Complete upload guide with Turbo. }> Organize with metadata and tags. } > Create folder structures with manifests. # x402 Uploading To Turbo (/build/upload/x402-uploading-to-turbo) ## x402 Uploads To Turbo Turbo provides multiple ways to upload data using the x402 Protocol. Users can upload data using signed data items or by leveraging the unsigned raw data API. Tooling is provided within the Turbo SDK and CLI to facilitate uploads using both methods. Integrators can also leverage existing x402 ecosystem tooling to upload data to Turbo with minimal code. ### Signed Data vs Unsigned Raw Data Users can choose to send data via signed data items or by using the unsigned raw data API and allow Turbo to handle signing the data items on their behalf. - `upload.ardrive.io/v1/x402/data-item/signed` - This endpoint accepts signed data items. Users are responsible for signing the ANS-104 data items before sending them to Turbo. This method provides users with full ownership and control over the data being uploaded. - `upload.ardrive.io/v1/x402/data-item/unsigned` - This endpoint accepts raw data payloads. Users send the raw data to Turbo, which then creates and signs the ANS-104 data items on behalf of the user. This method simplifies the upload process by offloading the signing responsibility to Turbo. ### Turbo CLI Signed Data Item Upload: ```sh turbo upload-file -f ../path/to/file --wallet-file ../path/to/evm/wallet.json --x402 --token base-usdc ``` Unsigned Raw Data Upload: ```sh turbo x402-unsigned-upload -f ../path/to/file --wallet-file ../path/to/evm/wallet.json --token base-usdc --tags 'App-Name' 'My-Cool-App' 'My-Custom-Tag' 'My-Custom-Value' ``` ### Turbo SDK Signed Data Item Upload: ```typescript const turbo = TurboFactory.authenticated({ privateKey, token: "base-usdc", }); await turbo.upload({ data: "The contents of my file!", fundingMode: new X402Funding({ maxMUSDCAmount: 1_000_000 }), // Max 1 USDC }); ``` Unsigned Raw Data Upload: ```typescript const turbo = TurboFactory.authenticated({ privateKey, token: "base-usdc", }); await turbo.uploadRawX402Data({ data: myRawData, maxMUSDCAmount: 1_000_000, // Max 1 USDC. Opt out if too expensive }); ``` NOTE: For free uploads under 100 KiB, this method of upload currently does not require a signature and can be used with an unauthenticated client. ```ts // Unsigned free upload of raw data under 100 KiB const turbo = TurboFactory.unauthenticated({ token: "base-usdc" }); await turbo.uploadRawX402Data({ data: myRawData, }); ``` **Receipt context:** x402 integrations can surface both an upload receipt and a payment settlement receipt. For how to capture and store Turbo upload receipts, see [Receipts](/build/upload/receipts#verifying-receipts-later). ### x402 Ecosystem Tooling Using the raw data API, developers can upload data to Turbo with minimal code using existing x402 ecosystem tooling. - x402 Fetch: ```ts // Create x402 signer const signer = privateKeyToAccount( process.env.EVM_PRIVATE_KEY as `0x${string}`, ); // Create x402 client and register schemes const client = new x402Client(); registerExactEvmScheme(client, { signer }); // Wrap fetch with payment handling const fetchWithPayment = wrapFetchWithPayment(fetch, client); const customTags = [ { name: "App-Name", value: "My-Cool-App" }, { name: "My-Custom-Tag", value: "My-Custom-Value" }, ]; // Make request - payment is handled automatically const response = await fetchWithPayment( "https://upload.ardrive.io/v1/x402/data-item/unsigned", { method: "POST", body: readFileSync("../path/to/file"), headers: { "Content-Type": "application/octet-stream", // Set appropriate content type "x-data-item-tags": JSON.stringify(customTags), // Include custom tags as a header }, ); ``` - x402 Axios: ```ts import { x402Client, withPaymentInterceptor, x402HTTPClient, } from "@x402/axios"; // Create signer const signer = privateKeyToAccount( process.env.EVM_PRIVATE_KEY as `0x${string}`, ); // Create x402 client and register schemes const client = new x402Client(); registerExactEvmScheme(client, { signer }); // Create an Axios instance with payment handling const api = withPaymentInterceptor( axios.create({ baseURL: "https://upload.ardrive.io" }), client, ); const customTags = [ { name: "App-Name", value: "My-Cool-App" }, { name: "My-Custom-Tag", value: "My-Custom-Value" }, ]; // Make request - payment is handled automatically const response = await api.post( "/x402/data-item/unsigned", readFileSync("../path/to/file"), { headers: { "Content-Type": "application/octet-stream", // Set appropriate content type "x-data-item-tags": JSON.stringify(customTags), // Include custom tags as a header }, }, ); console.log("Response:", response.data); // Get payment receipt const httpClient = new x402HTTPClient(client); const paymentResponse = httpClient.getPaymentSettleResponse( (name) => response.headers[name.toLowerCase()], ); console.log("Payment settled:", paymentResponse); ``` # Verifiable AI with ar.io (/build/verifiable-ai) import { Database, FileCheck, ShieldCheck, } from "lucide-react"; ## Verifiable MLflow lifecycle The verifiable AI stack is built around two projects: - [`ar-io-mlflow`](https://github.com/ar-io/ar-io-mlflow): a Python MLflow plugin that signs compact lifecycle proofs and anchors them to Arweave through ar.io Turbo. - [`ar-io-verify`](https://github.com/ar-io/ar-io-verify): an ar.io gateway sidecar that independently verifies Arweave data and can issue operator-signed attestations. This workflow keeps MLflow as the system of record and uses Arweave as a public, permanent witness. The plugin is currently alpha. Its proof format, verification flow, and API surface are stable enough for evaluation, but production deployments should use dedicated wallets, explicit signing keys, monitoring, and ar.io Verify attestations. Verifiable AI builds on ar.io's broader verification model for gateway claims, client-side verification, and network accountability. For the network-level concepts, see [Verification and Accountability](/learn/verification). ## Architecture The lifecycle is: 1. Train a model in MLflow and anchor the run. 2. Register the model and anchor the registry event. 3. Load the model through an integrity-checking wrapper before inference. 4. Anchor prediction proofs without publishing raw inputs or outputs. 5. Verify and attest the Arweave transactions through an ar.io gateway. Each lifecycle event produces a small signed envelope on Arweave. The envelope contains event metadata, a SHA-256 hash of canonical payload bytes, the previous proof hash for that event chain, the public key, and an Ed25519 signature. The canonical payload stays in MLflow as an `ario/payload.json` artifact, or as `ario/predictions//payload.json` for inference. A verifier fetches the envelope from an ar.io gateway, re-hashes the MLflow payload, re-derives the live MLflow record, and verifies the signature. Nothing sensitive needs to be written to Arweave. Source data, model artifacts, inputs, outputs, metrics, and traces remain in your MLflow store. Arweave stores the commitment that lets an auditor prove whether those records changed after anchoring. ## What you will build } /> } /> } /> ## Verification checks All verification paths reduce to the same core checks: 1. **Proof found**: the signed envelope is retrievable from an ar.io gateway for the recorded transaction ID. 2. **Record matches**: MLflow's stored payload re-hashes to the envelope's `payload_hash`, and a separate live MLflow surface re-derives the same canonical bytes. 3. **Signature confirmed**: the envelope signature verifies against the embedded Ed25519 public key. 4. **ar.io attestation**: optional gateway-operator verification that can be signed independently by the operator. This gives teams a tamper-evident record for training, registration, promotion, and inference without making Arweave the place where private AI data lives. # Model and Decision Proofs (/build/verifiable-ai/model-decision-proofs) The model registry is where training provenance becomes runtime protection. `ar-io-mlflow` provides a drop-in MLflow client that anchors registration and promotion events, plus a `VerifiedModel` wrapper that checks model artifact integrity before loading the underlying pyfunc model. ## Prerequisites - Completed [Training Provenance](/build/verifiable-ai/training-provenance) - A model logged to MLflow with `ario_mlflow.anchor()` - MLflow model registry access - Optional `ARIO_MLFLOW_ARIO_VERIFY_URL` for ar.io gateway attestations ## Anchor model registration ### Register with ArioMlflowClient Use `ArioMlflowClient` where you would normally use `mlflow.tracking.MlflowClient`. Registration returns immediately; anchoring runs in a background thread. ```python from ario_mlflow import ArioMlflowClient client = ArioMlflowClient() model_version = client.create_model_version( name="credit-risk-scorer", source=f"runs:/{run_id}/model", ) client.wait_for_anchor( "registration", "credit-risk-scorer", str(model_version.version), timeout=30, ) status = client.anchor_status( "registration", "credit-risk-scorer", str(model_version.version), ) print(status) ``` The status is one of `anchoring`, `anchored`, `signed`, `failed`, or `unknown`. ### Verify the model version ```bash MLFLOW_TRACKING_URI=file:///absolute/path/to/mlruns \ ar-io-mlflow verify model credit-risk-scorer/1 ``` Model verification re-hashes the registered model artifact, checks the registration proof, and confirms that the proof chains back to the training event where possible. ### Audit lineage Use the audit command when you want a registry-oriented view of the proof chain. ```bash MLFLOW_TRACKING_URI=file:///absolute/path/to/mlruns \ ar-io-mlflow audit credit-risk-scorer/1 ``` ## Guard inference with VerifiedModel `VerifiedModel` verifies artifact integrity before loading the model. If the model bytes in MLflow were swapped after registration, the constructor raises `IntegrityError` before user model code can execute. ```python from ario_mlflow import IntegrityError, VerifiedModel try: model = VerifiedModel("models:/credit-risk-scorer/1") except IntegrityError as exc: # Treat this as a security incident in production. raise RuntimeError("Registered model artifact failed verification") from exc result = model.predict([78000, 0.18, 0.22, 72, 745]) print("decision_id:", result.decision_id) print("proof_status:", result.proof_status) print("tx_id:", result.tx_id) ``` Prediction anchoring is asynchronous. The prediction returns as soon as the model has produced an output, and the proof upload completes in the background. When you need the transaction ID before responding to a caller, wait explicitly: ```python result.wait_for_anchor(timeout=10) print(result.tx_id, result.anchor_error) ``` ## Verify an inference proof Each `VerifiedModel.predict()` call writes prediction proof data to MLflow trace tags and `ario/predictions//payload.json`. ```bash MLFLOW_TRACKING_URI=file:///absolute/path/to/mlruns \ ar-io-mlflow verify trace ``` The verification flow checks: 1. The prediction envelope exists on ar.io for the recorded transaction ID. 2. The prediction payload artifact hashes to the envelope's `payload_hash`. 3. The trace tag mirror, `ario.payload_json`, re-derives the same canonical payload. 4. The envelope signature is valid. 5. Optional ar.io Verify attestation passes if configured. ## What gets written On model versions, the client writes tags such as: - `ario.artifact_verified` - `ario.registration_tx` - `ario.promotion_tx` - `ario.arweave_url` On prediction traces, `VerifiedModel` writes tags such as: - `ario.decision_id` - `ario.model_name` - `ario.model_version` - `ario.input_hash` - `ario.output_hash` - `ario.payload_hash` - `ario.proof_status` - `ario.prediction_tx` - `ario.payload_json` Raw inference inputs and outputs are not written to Arweave by the plugin. The proof stores hashes and lineage metadata. Verification proves that the model artifact and recorded decision metadata match what was anchored. It does not prove that the model is accurate, unbiased, or semantically correct for the task. ## Failure behavior - Registration and promotion still succeed if the background anchor fails. Inspect `anchor_status()` and alert on `signed` or `failed`. - `VerifiedModel(...)` fails closed on artifact hash mismatch by raising `IntegrityError`. - `predict()` returns even if proof anchoring later fails. Inspect `result.proof_status` and `result.anchor_error`. - If registration anchoring has not completed when a model is loaded, early predictions may chain to `GENESIS` instead of the model version registration transaction. ## Next steps Continue to [Operator Attestations](/build/verifiable-ai/operator-attestations) to add independent gateway verification, PDFs, attestations, and batch audits. # Operator Attestations (/build/verifiable-ai/operator-attestations) [`ar-io-verify`](https://github.com/ar-io/ar-io-verify) is a verification sidecar for ar.io gateways. It fetches data through a gateway, reconstructs the cryptographic proof, reports the strongest verification level reached, and can sign the result as an operator attestation. For verifiable AI, this gives an independent party a way to verify the envelopes written by `ar-io-mlflow` without trusting the training or inference service. For the broader network model behind gateway claims, client-side checks, and economic accountability, see [Verification and Accountability](/learn/verification). ## Verification levels | Level | Name | What it proves | | --- | --- | --- | | 1 | Existence confirmed | The transaction exists on Arweave and is associated with a confirmed block. | | 2 | Partially verified | The sidecar downloaded the bytes and computed a matching SHA-256 fingerprint. | | 3 | Verified | The sidecar verified the original Arweave or ANS-104 signature against the signed data. | When the operator configures a wallet, the sidecar signs the verification result with RSA-PSS SHA-256. That attestation can be checked later using the operator public key. ## Deploy the sidecar ### Start from the reference repo ```bash git clone https://github.com/ar-io/ar-io-verify.git cd ar-io-verify pnpm install ``` ### Run locally for development In one terminal, run the server: ```bash pnpm run dev ``` In another terminal, run the standalone web UI: ```bash pnpm --filter verify-web run dev ``` ### Run beside an ar.io gateway The Docker deployment expects an existing gateway on the `ar-io-network` Docker network. ```bash cd deploy cp .env.example .env ``` Edit `.env`: ```bash VERIFY_IMAGE=ar-io-verify:local GATEWAY_URL=http://ar-io-node-envoy-1:3000 GATEWAY_HOST=example.com PUBLIC_GATEWAY_URL=https://example.com VERIFY_PORT=4001 NODE_ENV=production WALLET_FILE=/absolute/path/to/operator-wallet.json ``` Start the sidecar and nginx proxy: ```bash bash start.sh ``` ## Connect ar-io-mlflow to ar.io Verify Set the plugin's Verify endpoint to the sidecar base URL exposed by your operator. ```bash ``` After this is configured, `ar-io-mlflow verify run`, `verify model`, and `verify trace` include the ar.io attestation row and write attestation metadata back to MLflow tags when available: - `ario.attestation_level` - `ario.report_url` - `ario.attested_by` - `ario.attested_at` The configured base URL must expose `GET /health` and `POST /api/v1/verify`. If you mount the sidecar under a path prefix, make sure those routes are still available under that same prefix. ## Single-transaction API Submit a transaction ID for verification: ```bash curl -X POST http://localhost:4001/api/v1/verify \ -H "Content-Type: application/json" \ -d '{"txId":""}' ``` Fetch the cached result: ```bash curl http://localhost:4001/api/v1/verify/ ``` Download the PDF certificate: ```bash curl -L http://localhost:4001/api/v1/verify//pdf \ -o verification.pdf ``` Fetch the programmatic attestation: ```bash curl http://localhost:4001/api/v1/verify//attestation ``` The same API is used by the standalone `/verify/` UI and by ar.io Console integrations. ## Batch verification jobs Use jobs for periodic audits, upload reconciliation, or verifier workflows that need to check many transaction IDs. ```bash curl -X POST http://localhost:4001/api/v1/jobs \ -H "Content-Type: application/json" \ -H "X-Tenant-Id: audit-team" \ -H "Idempotency-Key: run-2026-05-14" \ -d '{"txIds":["",""]}' ``` Check status: ```bash curl -H "X-Tenant-Id: audit-team" \ http://localhost:4001/api/v1/jobs/ ``` Fetch the signed verification bundle: ```bash curl -H "X-Tenant-Id: audit-team" \ http://localhost:4001/api/v1/jobs//report ``` The bundle is canonical JSON. To verify it offline, re-serialize the bundle without `signature` and `payloadHash`, compute SHA-256, compare that to `payloadHash`, then verify the RSA-PSS SHA-256 signature against `operatorPublicKey`. ## Operations The sidecar exposes: - `GET /health` for liveness - `GET /ready` for DB and gateway readiness - `GET /metrics` for Prometheus metrics - `GET /api-docs/` for Swagger UI Production deployments should restrict `/metrics` and administrative routes at the proxy layer, inject `X-Tenant-Id` from authenticated infrastructure for batch jobs, and treat the attestation wallet as a production signing secret. ## Production checklist - Configure `GATEWAY_URL` to a gateway reachable from the sidecar container. - Set `GATEWAY_HOST` to the public operator hostname included in attestations. - Set `PUBLIC_GATEWAY_URL` when browser previews need a public gateway URL. - Mount `WALLET_FILE` read-only if operator-signed attestations are required. - Monitor readiness failures, gateway request failures, job duration, and verification outcomes. - Configure `ARIO_MLFLOW_ARIO_VERIFY_URL` in MLflow verification environments. With these pieces in place, training, model, and inference proofs anchored by `ar-io-mlflow` can be independently verified and attested by an ar.io gateway operator. # Training Provenance (/build/verifiable-ai/training-provenance) Training provenance starts with a simple question: can someone prove that a model's current MLflow record matches what was recorded when training finished? `ar-io-mlflow` answers that by writing a canonical payload to MLflow, signing a compact commitment envelope, and anchoring only that envelope to Arweave. The plugin can also anchor MLflow dataset records so downstream trainers can refer to an immutable dataset proof without publishing rows or raw files. ## Prerequisites - Python 3.10 or newer - MLflow 2.14 or newer - A local or remote MLflow tracking store - Optional Arweave JWK wallet for production identity If no wallet is configured, the plugin generates one at `~/.ario-mlflow/wallet.json` and reuses it. That is convenient for evaluation. In production, set `ARIO_MLFLOW_ARWEAVE_WALLET` to a dedicated wallet file from your secrets manager. ## Install Install [`ar-io-mlflow`](https://github.com/ar-io/ar-io-mlflow) from source. ```bash git clone https://github.com/ar-io/ar-io-mlflow.git cd ar-io-mlflow pip install -e . ``` For the quickstart example, install scikit-learn as well: ```bash pip install scikit-learn ``` ## Anchor a training run ### Configure MLflow Point MLflow at the tracking store you want to use. This can be a local folder for development or your normal remote tracking URI. ```python from pathlib import Path import mlflow tracking_dir = Path("./mlruns").resolve() mlflow.set_tracking_uri(f"file://{tracking_dir}") mlflow.set_experiment("verifiable-ai") ``` ### Log a dataset and model Log the dataset through MLflow so the plugin can include dataset provenance in the training proof. The dataset proof commits to the dataset name, source, digest, and schema hash, not to the row contents. ```python import mlflow.data from sklearn.datasets import load_iris from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split X, y = load_iris(return_X_y=True) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, ) with mlflow.start_run() as run: dataset = mlflow.data.from_numpy( X_train, targets=y_train, source="https://archive.ics.uci.edu/dataset/53/iris", name="iris-train", ) mlflow.log_input(dataset, context="training") model = LogisticRegression(max_iter=200).fit(X_train, y_train) mlflow.log_params({"max_iter": 200, "random_state": 42}) mlflow.log_metric("accuracy", model.score(X_test, y_test)) mlflow.sklearn.log_model(model, name="model") ``` ### Anchor the proof Call `ario_mlflow.anchor()` before the run exits. The call hashes model artifacts, writes `ario/payload.json`, signs the envelope, uploads it through Turbo, and writes `ario.*` tags back to MLflow. ```python import ario_mlflow with mlflow.start_run() as run: # Fit model, log params, log metrics, log dataset, log model... result = ario_mlflow.anchor( metadata={"service_name": "credit-risk-training"}, ) print("run_id:", run.info.run_id) print("payload_hash:", result["payload_hash"]) print("training_tx:", result["tags"].get("ario.training_tx")) print("verify_status:", result["tags"]["ario.verify_status"]) ``` If the Arweave upload fails, the run still succeeds and the envelope remains signed locally. In that case `ario.verify_status` is `signed` and `ario.training_tx` is absent. ### Verify the run later Run the CLI against the same MLflow tracking store. ```bash MLFLOW_TRACKING_URI=file:///absolute/path/to/mlruns \ ar-io-mlflow verify run ``` Verification checks that the envelope exists on ar.io, the MLflow payload still hashes to the anchored commitment, the live MLflow run still re-derives the same canonical bytes, and the Ed25519 signature is valid. ## Standalone dataset proofs Dataset publishers can anchor a dataset proof without an active training run and hand the transaction ID to downstream teams. ```python import mlflow import ario_mlflow dataset = mlflow.data.from_pandas( df, source="s3://example-bucket/training/q1.parquet", name="credit-risk-q1", ) result = ario_mlflow.anchor(dataset=dataset) print(result["tx_id"]) ``` This pattern is useful when a data platform team publishes approved datasets and model teams consume them later. The proof records an immutable commitment to the dataset descriptor, while the source data remains in S3, a lakehouse, or another controlled system. ## What gets written On the MLflow run, the plugin writes tags such as: - `ario.enabled` - `ario.version` - `ario.public_key` - `ario.verify_status` - `ario.artifact_hash` - `ario.payload_hash` - `ario.training_tx` - `ario.arweave_url` - `ario.wallet_mode` It also writes `ario/payload.json` as the canonical payload artifact. Arweave receives only the compact signed envelope, usually hundreds of bytes rather than the source data or model artifact. ## Production notes - Use a dedicated wallet per environment through `ARIO_MLFLOW_ARWEAVE_WALLET`. - Set `ARIO_MLFLOW_SIGNING_KEY` explicitly if you need controlled key rotation. - Configure `ARIO_MLFLOW_GATEWAYS` with at least two gateways for fetch fallback. - Monitor runs where `ario.verify_status = signed`, because those were signed but not anchored. - Keep MLflow artifacts backed up independently. Arweave preserves the envelope, but MLflow still holds the canonical payload used for full record matching. ## Next steps After training proofs are anchored, continue to [Model and Decision Proofs](/build/verifiable-ai/model-decision-proofs) to anchor registry events and enforce artifact integrity before inference. # Glossary (/glossary) ## Ar.io Name System (ArNS) A decentralized naming system in the ar.io network that maps human-readable names to Arweave transaction IDs and application/content routes. ## Ar.io Name Token (ANT) An NFT on Solana associated with a registered ArNS name. ANT ownership controls name-level configuration such as records, undernames, and transfer operations. ## CPI (Cross-Program Invocation) The mechanism by which Solana programs call into each other. The ar.io architecture uses CPI for token operations — ario-gar and ario-arns call into ario-core for SPL Token transfers. ## Cranker A permissionless actor (bot or user) that submits transactions to drive lazy protocol operations, including the epoch pipeline and other state advancement tasks. Any participant can crank if protocol conditions are met. ## Epoch Pipeline The 6-step process for observation, reward calculation, and distribution each epoch: create_epoch → tally_weights → prescribe_epoch → save_observations → distribute_epoch → close_epoch. ## Gateway Address Registry (GAR) The protocol-level registry of joined gateways and their metadata. GAR powers gateway discovery and is a core input to observer selection, rewards, and network operations. ## Metaplex Core The NFT standard used for Ar.io Name Tokens (ANTs) on Solana. Metaplex Core assets are tradeable on standard NFT marketplaces like Tensor and Magic Eden. ## Native Address An address format that uses the raw public key bytes directly, without additional encoding or transformation. ## Normalized Address A standardized address format that ensures consistent representation across different systems and contexts. Normalized addresses help prevent issues with address matching and lookup operations. ## Observer A gateway selected to evaluate peer gateways during an epoch. Observers test assigned ArNS resolutions and submit reports used by the protocol for performance evaluation and rewards. ## Optimistic Indexing A data indexing strategy where new data is immediately made available for queries while verification processes continue in the background. This approach improves performance while maintaining data integrity through eventual consistency. ## PDA (Program Derived Address) Deterministic Solana account addresses derived from a program ID and a set of seeds. The ar.io protocol uses PDAs extensively — for gateways, delegations, withdrawals, vaults, ArNS records, ANT configs, and more. ## Program ID The unique onchain address of a deployed Solana program. Ar.io protocol interactions target specific program IDs for instructions and state access. ## Protocol Balance A protocol-controlled ARIO token pool that accumulates relevant inflows (for example, protocol fees) and serves as a source for incentive distributions according to protocol rules. ## Public Key A cryptographic key that can be shared publicly and is used to verify digital signatures or encrypt data. In the ar.io context, Solana public keys (Ed25519, base58-encoded) are used to identify wallet addresses and verify transactions. ## SPL Token Solana Program Library token standard. ARIO is implemented as an SPL Token on Solana, making it compatible with the full Solana wallet and DeFi ecosystem. ## Transaction (Solana) A signed unit of interaction with Solana containing one or more instructions. Instructions in a transaction are processed atomically: all succeed, or the transaction fails without applying state changes. ## Vault A protocol-level token lock with a start time, optional end time, and token balance. Vaults are used to hold ARIO over time and may be revocable depending on how they were created. ## Winston Winston is the smallest unit of Arweave's native AR token. One AR equals 10^12 Winston. ## Winston Credits (winc) Winston Credits (`winc`) are the unit used by Turbo to represent upload purchasing power. In Turbo contexts, receipt and balance fields often return values in `winc`. # What are Bundles? (/learn/(introduction)/ans-104-bundles) ANS-104 bundles are **data packaging standards** that efficiently bundle multiple data items and submit them to Arweave as single transactions, reducing transaction overhead and improving network efficiency. ## The Problem ANS-104 Solves **Individual Arweave transactions have inherent limitations:** - **Transaction overhead** - Each transaction requires separate processing and storage - **Network inefficiency** - Multiple small transactions consume more network resources - **Indexing complexity** - Individual transactions are harder to organize and query - **Storage fragmentation** - Related data items are stored separately **ANS-104 provides:** - **Reduced transaction overhead** by batching multiple data items - **Improved network efficiency** through consolidated transactions - **Better indexing capabilities** with structured data item format - **Standardized data format** for interoperability across applications ## How ANS-104 Bundling Works ### The ANS-104 Standard ANS-104 is the [official specification](https://github.com/ArweaveTeam/arweave-standards/blob/master/ans/ANS-104.md) for bundling data on Arweave: - **Data Items** - Individual pieces of data with standardized binary format - **Bundle** - Single Arweave transaction containing multiple data items - **Binary Serialization** - Consistent format for data item structure - **Standardized Format** - Ensures interoperability across applications ### How ANS-104 Works 1. **Data Item Creation** - Create individual data items with ANS-104 format 2. **Bundle Assembly** - Combine multiple data items into a single bundle 3. **Transaction Creation** - Submit bundle as one Arweave transaction 4. **Network Processing** - Miners process the single bundle transaction 5. **Data Retrieval** - Individual data items can be extracted and indexed ## Key Benefits of ANS-104 **Reduced Overhead** - Bundle multiple data items into a single transaction to reduce processing overhead **Network Efficiency** - Consolidate multiple uploads into fewer network transactions **Standardized Format** - Consistent binary serialization ensures interoperability across applications **Better Indexing** - Structured data item format enables more efficient data retrieval and querying ## Why ANS-104 Matters for the Permaweb ANS-104 bundles are essential for building scalable applications on the permaweb because they: - **Enable efficient data storage** by reducing transaction overhead for multiple data items - **Improve network performance** through consolidated transactions - **Support better data organization** with standardized data item formats - **Enable scalable applications** that need to store many related data items efficiently ## Explore Bundling } title="Upload Data" description="Learn how to upload data to Arweave using bundling" href="/build/upload" /> } title="Run a Bundler" description="Deploy your own bundling infrastructure" href="/build/extensions/bundler" /> } title="Gateway Extensions" description="Integrate bundling with your ar.io gateway" href="/build/extensions" /> } title="Turbo SDK" description="Use Turbo SDK for easy data uploads and bundling" href="/sdks/turbo-sdk" /> # Introduction (/learn/(introduction)) import { BookOpen, Wrench, Package, Code, Server, Globe, ArrowRight, Zap, Shield, Infinity, } from "lucide-react"; Ar.io is a data infrastructure solution for long-term access. We help enterprises, institutions, and platforms ensure that critical information remains accessible, verifiable, and intact over time — even as systems, providers, technologies, or attacks disrupt the environments around them. **For AI and LLM users**: Start at llms.txt for a machine-readable index, SKILL.md for code-generation recipes, or llms-full.txt for the complete documentation as plain text. See [AI Agents & LLMs](/build/agents) for the full toolkit. ## Explore the Documentation } title="What is ar.io?" description="Learn about the decentralized gateway protocol and how it powers the permanent web" href="/learn/what-is-ario" /> } title="Build" description="Get started building applications, running gateways, and uploading data" href="/build" /> } title="SDKs" description="Integrate ar.io services into your applications with our developer SDKs" href="/sdks" /> } title="API Reference" description="Complete API documentation for ar.io Node and Turbo services" href="/apis" /> ## Quick Start Guides } title="Upload Data to Arweave" description="Learn how to permanently store files and data using Turbo SDK" href="/build/upload" /> } title="Run a Gateway" description="Deploy your own ar.io gateway and participate in the network" href="/build/run-a-gateway" /> } title="Register ArNS Names" description="Get human-readable names for your permanent applications" href="/learn/arns" /> ## ar.io Ecosystem } title="ArNS Registry" description="Register and manage permanent names for your applications" href="https://arns.ar.io" /> } title="Network Portal" description="Monitor gateway performance and network statistics" href="https://gateways.ar.io" /> } title="ArDrive" description="User-friendly permanent storage for files and folders" href="https://ardrive.io" /> ## Join the Community Connect with developers, gateway operators, and the ar.io team. Get help, share ideas, and stay updated on the latest developments. - [Join Discord](https://discord.com/invite/HGG52EtTc2) - [View Guides](/build/guides) # Protocol Architecture (/learn/(introduction)/protocol-architecture) ## Ar.io Protocol Architecture The ar.io protocol operates through four Solana programs (3+1) that work together via cross-program invocation (CPI). This architecture organizes protocol responsibilities into a coordinated set of programs with clear boundaries. For the deployed Solana mainnet program IDs and ARIO token mint, see [Mainnet Addresses](/learn/token#mainnet-addresses). ```mermaid graph TD subgraph CORE["ario-core"] direction TB BAL[SPL Token / Balances] VAULTS[Vaults] PRIMARY[Primary Names] end subgraph GAR["ario-gar"] direction TB GW_REG[Gateway Registry] STAKING[Staking & Delegation] EPOCHS[Epochs & Rewards] end subgraph ARNS["ario-arns"] direction TB NAME_REG[Name Registry] DEMAND[Demand Factor] PRICING[Pricing & Auctions] end subgraph ANT["ario-ant"] direction TB NFT[Metaplex Core NFTs] RECORDS[DNS-like Records] CTRL[Controllers] end CORE --> GAR CORE --> ARNS ARNS --> ANT ARNS -.->|gateway data| GAR classDef coreProgram fill:#e3f2fd classDef garProgram fill:#e8f5e8 classDef arnsProgram fill:#fff3e0 classDef antProgram fill:#f3e5f5 class BAL,VAULTS,PRIMARY coreProgram class GW_REG,STAKING,EPOCHS garProgram class NAME_REG,DEMAND,PRICING arnsProgram class NFT,RECORDS,CTRL antProgram ``` ## Programs ### ario-core The core program manages the ARIO SPL Token, vaults, and primary names: - **SPL Token Operations**: ARIO is a standard SPL Token (6 decimals, 1 ARIO = 1,000,000 mARIO). Transfers, balances, and token accounts follow the SPL Token standard. - **Vaults**: Time-locked token deposits for various purposes, including ecosystem programs and other protocol-managed incentives. Other programs (ario-gar, ario-arns) call into ario-core via CPI for all token operations. ### ario-gar (Gateway Address Registry) Manages the network's gateway infrastructure, staking, delegation, and the epoch reward pipeline: - **Gateway Registry**: An onchain registry of network gateways. Each gateway is a PDA storing operator address, stake, observer address, settings, and performance stats. - **Staking & Delegation**: Operator stakes, delegated stakes (separate PDA per gateway-delegator pair), withdrawals, redelegation, and allowlists. - **Observer Address Uniqueness**: An ObserverLookup PDA enforces that no two gateways share the same observer address. - **Epoch Pipeline**: A 6-step permissionless pipeline driven by [cranker bots](/learn/oip/epoch-pipeline): 1. `create_epoch` — Initialize epoch, compute reward rate 2. `tally_weights` — Batched weight computation 3. `prescribe_epoch` — Select observers and prescribed names via weighted roulette 4. `save_observations` — Observers submit pass/fail reports 5. `distribute_epoch` — Batched reward distribution 6. `close_epoch` — Reclaim rent from completed epoch accounts - **Gateway Pruning**: Gateways that repeatedly fail observation are removed from the network and subject to stake slashing. ### ario-arns (ArNS Registry) Manages the Ar.io Name System — name registration, leasing, pricing, and returned names: - **Name Registry**: An onchain registry of ArNS domains. Each name is an ArnsRecord PDA storing the owner, ANT mint address, lease type, and expiration. - **Pricing**: Dynamic pricing via a DemandFactor PDA. - **Returned Name Auctions**: When a name expires or is released, it enters a Dutch auction. - **Validation**: Names must meet certain formatting criteria. - **Cost Simulation**: `get_token_cost` view instruction available via `simulateTransaction` for fee estimation. ### ario-ant (Ar.io Name Tokens) ANTs are [Metaplex Core](https://developers.metaplex.com/core) NFTs that represent ownership of ArNS names: - **NFT Standard**: Each ANT is a Metaplex Core asset — tradeable on Tensor, Magic Eden, and other NFT marketplaces. - **DNS-like Records**: Each ANT stores routing records (AntRecord PDAs) mapping undernames to Arweave transaction IDs or IPFS CIDs, with configurable TTL values. - **Controllers**: Controllers can manage records without holding the NFT itself. - **Lazy Reconciliation**: When an ANT is transferred via a marketplace (outside the ar.io protocol), controllers are cleared on the next write operation, ensuring the new owner has full control. ## State Model All protocol state is stored in Solana accounts using Program Derived Addresses (PDAs): | Account Type | Program | Derivation Seeds | Purpose | |-------------|---------|-----------------|---------| | ArioConfig | ario-core | `["config"]` | Global token configuration | | Vault | ario-core | `["vault", owner, vault_id]` | Time-locked token deposit | | PrimaryName | ario-core | `["primary_name", name_hash]` | Name → address mapping | | PrimaryNameReverse | ario-core | `["primary_name_reverse", owner]` | Address → name reverse lookup | | GatewayRegistry | ario-gar | `["gateway_registry"]` | Zero-copy gateway slot array | | Gateway | ario-gar | `["gateway", operator]` | Individual gateway state | | Delegation | ario-gar | `["delegation", gateway, delegator]` | Per-pair delegation | | Withdrawal | ario-gar | `["withdrawal", owner, id]` | Pending withdrawal | | ObserverLookup | ario-gar | `["observer_lookup", observer]` | Observer uniqueness check | | Epoch | ario-gar | `["epoch", epoch_index]` | Epoch state and rewards | | NameRegistry | ario-arns | `["name_registry"]` | Zero-copy name slot array | | ArnsRecord | ario-arns | `["arns_record", name_hash]` | Individual name record | | DemandFactor | ario-arns | `["demand_factor"]` | Current pricing multiplier | | AntConfig | ario-ant | `["ant_config", mint]` | ANT metadata and settings | | AntControllers | ario-ant | `["ant_controllers", mint]` | Controller list (max 10) | | AntRecord | ario-ant | `["ant_record", mint, undername_hash]` | DNS-like routing record | ## Security Model The protocol relies on a combination of Solana runtime guarantees, deterministic account ownership, and economic incentives: - **Signer and account checks**: Solana enforces transaction signatures, account ownership, and program execution rules. - **Program Derived Addresses (PDAs)**: Protocol state lives in deterministic accounts controlled by the relevant ar.io programs. - **Economic accountability**: Gateway operators stake ARIO, and repeated failure can lead to removal and slashing. - **Permissionless verification**: Gateway observations and cranking keep network operations open and independently verifiable. # Ar.io for Solana Developers (/learn/(introduction)/solana-developers) If you build on Solana, you already know most of what you need to use ar.io. This page maps familiar Solana concepts to ar.io equivalents and gets you to your first permanent deployment fast. ## What ar.io Does Ar.io is the **access and naming layer for Arweave** — the permanent storage blockchain. It gives you: - **Permanent hosting** — Upload your frontend once, it lives forever. No hosting bills. - **Human-readable URLs** — `yourapp.ar.io` instead of a 43-character transaction ID - **Decentralized CDN** — Gateways worldwide serve your content, no single point of failure - **On-chain naming** — ArNS names are Metaplex Core NFTs tradeable on Tensor/Magic Eden The ar.io protocol runs on Solana. You interact with it using your existing Solana wallet, SOL for fees, and ARIO tokens for protocol operations. ## Concept Mapping If you know Solana, you already understand ar.io: | Solana Concept | ar.io Equivalent | Details | |---|---|---| | **SPL Token** | ARIO token | The protocol token. 6 decimals, 1B supply. Used for staking, ArNS names, delegation. | | **Metaplex Core NFT** | ANT (Ar.io Name Token) | Each ArNS name is backed by an NFT. Trade on Tensor/Magic Eden. Holds DNS-like records. | | **Anchor Program** | ario-core, ario-gar, ario-arns, ario-ant | Four programs that manage the protocol. Interact via SDK or CPI. | | **PDA** | All protocol state | Gateways, delegations, vaults, name records, ANT configs — all PDAs. | | **CPI** | Cross-program token ops | ario-gar and ario-arns call into ario-core for SPL Token transfers. | | **Transaction fee (SOL)** | Same | Every protocol operation costs a small SOL fee ( You DO still need an Arweave wallet (or SOL via Turbo) to **upload data** to Arweave. Turbo accepts SOL directly — see the upload section below. **Want to try before spending real ARIO?** Run this whole flow — upload, buy an ArNS name, and resolve it — for free on the **[Testnet Sandbox](/build/testnet)**, which runs the full ar.io stack on Solana **devnet** with a faucet-funded staging token. It's also the ideal target for AI coding agents and CI. When you're ready for mainnet, the code below is the same minus the sandbox endpoints. ## Quick Start: 5 Minutes to Permanent ### 1. Install ```bash npm install @ar.io/sdk @solana/kit @ardrive/turbo-sdk ``` ### 2. Upload Your App to Arweave Use Turbo to upload your build folder. Turbo accepts SOL directly — no Arweave wallet needed: ```typescript // Authenticate with your Solana private key const turbo = TurboFactory.authenticated({ privateKey: bs58.encode(solanaSecretKey), token: 'solana', }); // Upload your build folder const result = await turbo.uploadFolder({ folderPath: './dist', dataItemOpts: { tags: [{ name: 'App-Name', value: 'my-cool-app' }], }, }); console.log('Manifest TX:', result.manifestResponse.id); // This is the Arweave Transaction ID — your app is now permanent ``` ### 3. Register an ArNS Name ```typescript // Create signer from Solana keypair const keypairBytes = new Uint8Array( JSON.parse(fs.readFileSync('./solana-keypair.json', 'utf-8')), ); const signer = await createKeyPairSignerFromBytes(keypairBytes); const ario = ARIO.mainnet({ signer }); // Buy the name (mints an ANT as a Metaplex Core NFT) const result = await ario.buyRecord({ name: 'my-cool-app', type: 'lease', years: 1, }); ``` ### 4. Point Your Name to Your App ```typescript // Get your ANT's mint address from the ArNS record const record = await ario.getArNSRecord({ name: 'my-cool-app' }); // Initialize the ANT and set the record const ant = ANT.init({ signer, processId: record.processId }); await ant.setRecord({ undername: '@', transactionId: result.manifestResponse.id, // from the Turbo upload ttlSeconds: 3600, }); // Your app is now live at: // https://my-cool-app.ar.io // https://my-cool-app. ``` ## How ArNS URLs Work Every ar.io gateway resolves ArNS names as subdomains: ``` https://my-cool-app.ar.io → served by the ar.io gateway https://my-cool-app. → served by that ar.io gateway ``` All network gateways serve the same content. If one goes down, users access through another. Your app is truly decentralized. Undernames use underscores: ``` https://docs_my-cool-app.ar.io → the "docs" undername https://api_my-cool-app.ar.io → the "api" undername ``` ## Costs at a Glance | Operation | Cost | |---|---| | Upload ( Use `ario.getTokenCost()` to check exact pricing before any purchase. ArNS prices adjust dynamically based on demand. ## What to Build Ar.io is ideal for: - **Permanent frontends** — Deploy your React/Next.js/Vue app with zero hosting costs forever - **Immutable assets** — Store game assets, NFT metadata, or config files that can never be altered - **Decentralized publishing** — Blog platforms, documentation sites, content archives - **Censorship-resistant apps** — Apps that no single entity can take down - **Version-controlled deployments** — Use undernames for staging, production, and rollbacks ## Next Steps } /> } /> } /> } /> } /> # What is ar.io? (/learn/(introduction)/what-is-ario) Ar.io is the first permanent cloud network. A decentralized infrastructure layer built on Arweave for permanent data storage, with protocol execution on Solana. Think of it as the gateway to Arweave's permaweb, turning its tamper-proof storage into a fully functional, user-friendly ecosystem for apps, websites, and data. The permaweb is the permanent web of files, applications, and identities built on Arweave. Ar.io makes that permanent data usable by providing fast gateway access, indexing, human-readable names, and routing across a decentralized network. ## Features of ar.io ### Gateways Ar.io operates a network of [gateways](/learn/gateways) — nodes that serve as entry points to Arweave’s data. These gateways fetch and deliver data quickly, supporting everything from static files to dynamic web apps. ### Ar.io Name System (ArNS) The [Ar.io Name System (ArNS)](/learn/arns) is a decentralized naming system for Arweave. It allows users to register and resolve human-readable names to Arweave transaction IDs. ### Data Access Ar.io offers a range of tools for accessing and querying data on Arweave, including: - [HTTP Requests](/build/access/fetch-data) via gateways - [GraphQL Queries](/build/access/find-data) for finding data by tags and metadata - [ArNS](/learn/arns) for human-readable URLs - [Wayfinder](/learn/wayfinder) for decentralized content discovery ## The Problem Arweave stores data forever, but accessing and organizing that data isn't always straightforward. Without efficient tools, retrieving files, serving websites, or finding specific content on Arweave's blockweave can be slow or complex, limiting its potential for developers and users. ## The Solution Ar.io builds on Arweave's permanent storage to create a decentralized, scalable access layer. It provides gateways, domain names, and indexing services, making it easy to interact with permaweb content as seamlessly as the traditional web. ### How It Works - **Decentralized Gateways**: ar.io operates a network of gateways—nodes that serve as entry points to Arweave’s data. These gateways fetch and deliver data quickly, supporting everything from static files to dynamic web apps. - **ArNS (Ar.io Name System)**: ar.io introduces decentralized domain names (e.g., yourname.arweave), mapping human-readable names to Arweave’s data IDs. This makes content easy to find and share, like URLs on the traditional web. - **Indexing and Querying**: ar.io enables efficient data indexing, allowing developers to search and retrieve specific content from Arweave’s vast storage without scanning the entire blockweave. - **Routing and verification**: ar.io 's ar://wayfinder Protocol intelligently routes requests to available gateways in the network and verifies the data's authenticity. - **Observation and incentives**: ARIO's [Observation and Incentive Protocol (OIP)](/learn/oip), ensures gateway operators are serving the right data and rewards them in the protocol native token, $ARIO, to create a secure and self-sustaining network. **In Simple Terms**: Imagine Arweave as a massive, unerasable library. Ar.io is the librarian who organizes the shelves, provides a catalog, and hands you the books you need—fast. ## Why It Matters - **Accessible**: Gateways make permaweb content load as quickly as traditional websites. - **Discoverable**: ArNS provides user-friendly domain names, simplifying navigation. - **Scalable**: Supports growing permaweb usage, from small apps to global platforms. - **Decentralized**: No single entity controls access, ensuring censorship resistance. ## Building on ar.io Ar.io empowers developers to create permaweb apps with tools for hosting, querying, and monetizing content, all while leveraging Arweave's permanent storage. **In Simple Terms**: Arweave locks data forever; ar.io makes it ready for the world to use. ## Ready to Dive Deeper? Ar.io transforms Arweave into a vibrant permaweb ecosystem. Ready to start building? Explore our comprehensive guides and start creating on the permanent web. ## Explore ar.io } /> } /> } /> } /> # What is Arweave? (/learn/(introduction)/what-is-arweave) Arweave is a decentralized storage network that ensures data is **permanent**, **affordable**, and **scalable**. Think of it as a global, tamper-proof hard drive where your files—photos, documents, or apps—stay accessible forever. It's the foundation for [ar.io](https://ar.io), powering a "permaweb" where data never disappears. Below, we break down Arweave's core features in a simple, beginner-friendly way. ## A Datachain for Permanent Storage Arweave is like Bitcoin, but for data. It solves one problem really well: **storing data permanently**. Once uploaded, your data—whether a tweet, NFT, or website—is immutable and preserved indefinitely. ### How Does It Work? - **Blockweave Architecture**: Unlike a blockchain's single chain, Arweave's blockweave links each new data block to the previous one and a random older block. Data is split into 256 KiB chunks in a secure Merkle tree, ensuring miners keep all data to add new blocks. - **Succinct Proofs of Random Access (SPoRA)**: Miners prove they store multiple data copies by accessing random chunks, verified efficiently with Verifiable Delay Functions (VDFs). This combines proof-of-work and proof-of-storage, making data loss nearly impossible. **In Simple Terms**: Picture a library where new books reference older ones, and librarians must keep every book to add more. SPoRA ensures they prove they’ve got the books, keeping your data safe forever. ## Pay Once, Store Forever: No Recurring Fees Pay a one-time fee to upload data, and it's stored "forever"—no subscriptions or renewals. ### How Does It Work? - **Endowment Fund**: Your fee, based on 200 years of storage for 20 replicas, goes mostly into a fund that slowly pays miners in AR tokens to maintain data. It assumes storage costs drop over time, making the fund sustainable. **In Simple Terms**: It's a “forever stamp” for data. Your payment funds a pot that keeps paying storage keepers, lasting longer as tech gets cheaper. ## Practically Unlimited Storage Arweave can practically store unlimited data, from small files to entire digital archives, without hitting a ceiling. The theoretical limit is 2^256 bytes which for scale is more atoms than there are in the universe. ### How Does It Work? - **Layer 1 Transactions**: Data is stored as 256 KiB chunks on the blockweave, replicated across many nodes. As more nodes join with standard hardware, storage capacity grows limitlessly. - **Bundling with [ar.io](https://ar.io) and Turbo**: Bundling packs multiple files into one transaction, reducing costs and congestion. Ar.io and Turbo (a Layer 2 tool) optimize this, enabling fast, cheap uploads of large datasets like websites. ## What Arweave doesn't solve well? Access Arweave solve's one problem and solve's it well. Storing your data for a very long-time. It doesn't, however, incentivise the indexing and access for data. ## Ready to Dive Deeper? Arweave powers a permaweb where apps, websites and data live forever. For ar.io developers, it's the bedrock for unstoppable decentralized applications. Learn more in the next section: [What is ar.io](/learn/what-is-ario). ## Explore Arweave } /> } /> } /> # Ar.io Name Tokens (ANTs) (/learn/arns/ants) To establish ownership of a record in the ArNS Registry, each record contains both a friendly name and a reference to an Ar.io Name Token (ANT). ANTs are [Metaplex Core](https://developers.metaplex.com/core) NFTs on Solana that give their owners the ability to update the Arweave transaction IDs their associated friendly names point to. ## What is an ANT? An ANT is a Metaplex Core NFT managed by the `ario-ant` Solana program. It implements the Ar.io Name Token specification required by ar.io gateways to resolve ArNS names to Arweave transaction IDs. The program handles record updates, controller management, metadata changes, and ownership reconciliation after transfers. Name Tokens have an owner, who can transfer the token and control its modifiable settings. These settings include record targets, address-resolution TTL values, the ANT name and ticker, and controller permissions. ## Ownership and Control Controllers can manage records, name, and ticker settings, but cannot transfer the ANT or assign additional controllers. ANTs are minted in accordance with network standards by an end user who can transfer ownership or assign controllers as needed. Because ANT logic lives in the `ario-ant` program rather than inside each individual token, protocol-level updates can apply consistently across ANTs. When an ANT is transferred through a marketplace outside the ar.io app, the `ario-ant` program reconciles ownership on the next interaction and clears previously assigned controllers so the new owner has clean control. Loss of a private key for a permanently purchased name can result in the name being inaccessible. ### Undername Ownership Undernames can have an `owner` set on them. This owner is empowered to set that undername as their primary name, remove that undername as their primary name, and control that undername's metadata, such as: - Transaction ID - the data the record resolves to. - TTL seconds - the time to live in seconds the data is cached for by clients. - Owner - the owner of the record. - Description - the description of the record. - Display Name - the display name for the owner of the record. - Keywords - the keywords for the record. - Logo - the logo of the record. They do *NOT* have control over the `priority` of the undername, which is restricted to the ANT Controllers and Owner. ## ANT Interactions The table below indicates some of the possible interactions with the ArNS registry, corresponding ANTs, and who can perform them: | Type | ANT Owner | ANT Controller | Undername Owner | Any ARIO Token Holder | | ----------------------------------------- | ----------- | ---------------- | ----------------- | ----------------------- | | Transfer ownership | ✔ | | | | | Add / remove controllers | ✔ | | | | | Approve/Remove Primary name | ✔ | | ✔ | | | Reassign name to new ANT | ✔ | | | | | Return a permanent name | ✔ | | | | | Set records (pointers, record metadata) | ✔ | ✔ | ✔ | | | Update records, name, ticker | ✔ | ✔ | | | | Update descriptions and keywords | ✔ | ✔ | | | | Create and assign undernames | ✔ | ✔ | | | | Extend / renew lease | ✔ | ✔ | ✔ | ✔ | | Increase undernames | ✔ | ✔ | ✔ | ✔ | | Convert lease to permanent | ✔ | ✔ | ✔ | ✔ | ## Undernames ANT owners and controllers can configure subdomains for their registered ArNS name, known as undernames. These undernames are assigned at registration or added later. Undernames use an underscore (`_`) in place of a dot (`.`) to separate the subdomain from the main ArNS domain. ## Secondary Markets Since ANTs are standard Metaplex Core NFTs, they are tradeable on any compatible NFT marketplace, including **Tensor** and **Magic Eden**. When an ANT is sold on a marketplace, lazy reconciliation clears the existing controllers on the next write operation, ensuring the new owner gets clean control. Additionally, tertiary markets could be created that support the leasing of these friendly names to other users. ## Next Steps } /> } /> } /> # Ar.io Name System (ArNS) (/learn/arns) ## What is ArNS? Arweave URLs and transaction IDs are long, difficult to remember, and occasionally categorized as spam. The Ar.io Name System (ArNS) aims to resolve these problems in a decentralized manner. ArNS is a **censorship-resistant naming system** powered by [ARIO tokens](/learn/token), enabled through [ar.io gateway](/learn/gateways) domains, and used to connect friendly domain names to permaweb apps, web pages, data, and identities. It's an open, permissionless, domain name registrar that doesn't rely on a single TLD. ## How ArNS Works This system works similarly to traditional DNS services, where users can purchase a name in a registry and DNS Name servers resolve these names to IP addresses. The system is flexible and allows users to purchase names permanently or lease them for a defined duration based on their use case. With ArNS, the registry is managed on Solana by the `ario-arns` program, making ownership and registration state globally verifiable. Names are controlled by Ar.io Name Tokens (ANTs), which point to Arweave transaction IDs so gateways can route friendly names to permanent data. ```mermaid graph TD User[User requests friendly name] --> Gateway[ar.io Gateway] Gateway --> Registry[ArNS Registryario-arns] Registry --> ANT[Ar.io Name TokenMetaplex Core NFT] ANT --> Target[Arweave Transaction ID] Gateway --> Content[Permaweb app, page, or data] Target --> Content ``` ## Name Resolution Users can register a name, like `ardrive`, within the ArNS Registry. Ownership is represented by an Ar.io Name Token (ANT), a Metaplex Core NFT on Solana used by ArNS to track control over the name. ANTs allow the owner to set a mutable pointer to any type of permaweb data, like a page, app or file, via its Arweave transaction ID. Each ar.io gateway acts as an ArNS name resolver. Gateways fetch the latest state of both the ArNS Registry and associated ANTs from Solana and use that state to route users to the right Arweave transaction. Ar.io gateways resolve names as gateway subdomains, e.g., `https://ardrive.ar.io`, and proxy requests to the associated Arweave transaction ID. This means an ArNS name can work across ar.io gateways that support ArNS. Users can easily reference these friendly names in their browsers, and other applications and infrastructure can build rich solutions on top of these ArNS primitives. ```mermaid sequenceDiagram participant User participant Gateway as ar.io Gateway participant Registry as ArNS Registry participant ANT as Ar.io Name Token participant Arweave User->>Gateway: Request ardrive.ar.io Gateway->>Registry: Query "ardrive" record Registry-->>Gateway: Returns ANT address Gateway->>ANT: Get target TxID ANT-->>Gateway: Returns TxID (abc123...) Gateway->>Gateway: Check cache for TxID alt TxID not in cache Gateway->>Arweave: Fetch data from TxID Arweave-->>Gateway: Returns permaweb content end Gateway-->>User: Serves content ``` ## Key Benefits - **Human-readable URLs** instead of complex transaction IDs - **Censorship-resistant** and decentralized - **Permanent data routing** to Arweave transaction IDs - **Cross-gateway compatibility** - works on all ar.io gateways - **Flexible ownership** - lease names for a defined period or buy them permanently ## Explore ArNS } /> } /> } /> } /> # Name Registration (/learn/arns/name-registration) ArNS names can be registered as leases or permanent purchases, depending on how long the user needs the name and how much they want to commit up front. ## Registration Types ### Lease Registration A name may be **leased on a yearly basis**. Leases lower the barrier to entry, support temporary projects, and allow inactive names to eventually return to public availability. ### Permanent Registration (Permabuy) A name may be **purchased for an indefinite duration** with no expiration date. This is useful for long-lived apps, identities, and data references that should remain associated with a stable friendly name. Registering a name requires spending ARIO tokens corresponding to the name's character length and purchase type. ## Name Registry The ArNS Registry is a list of registered names and their associated ANT mint addresses, managed by the `ario-arns` Solana program. Key rules embedded within the protocol include: - **Genesis Prices**: Set within the contract as starting conditions - **Dynamic Pricing**: Varies based on name length, purchase type (lease vs buy), lease duration, and current Demand Factor - **Name Records**: Include a pointer to the Ar.io Name Token (ANT) mint address, lease end time (if applicable), and undername allocation - **Reassignment**: Name registrations can be reassigned from one ANT to another - **Lease Extension**: Anyone with available ARIO Tokens can extend any name's active lease - **Lease to Permanent Buy**: Anyone with available ARIO Tokens can convert a name's lease to a permanent buy - **Undername Capacity**: Additional undername capacity can be purchased for actively registered names - **Name Removal**: Name records can only be removed from the registry if a lease expires, or a permanent name is returned to the protocol ## Name Validation Rules All names registered must meet the following criteria: 1. **Valid characters**: Only numbers 0-9, characters a-z and dashes 2. **Dash placement**: Dashes cannot be leading or trailing characters 3. **Single character domains**: Dashes cannot be used in single character domains 4. **Length limits**: 1 character minimum, 51 characters maximum 5. **43-character prohibition**: Names exactly 43 characters long are prohibited to prevent Arweave transaction ID collisions 6. **Lowercase enforcement**: Names must be lowercase at submission 7. **Reserved names**: Cannot be an invalid name predesignated to prevent unintentional use/abuse such as `www` ## Lease Management ### Lease Expirations When a lease term ends, there is a grace period where the lease can be renewed or converted to a permanent purchase before it fully expires. If this grace period elapses, the name is considered expired and returns to the protocol for public registration. Once expired, a name's associated undername registrations and capacity also expire. A recently expired name enters a **Returned Name Dutch Auction**, where it starts at a premium and decays back toward standard pricing over time. Revenue from returned name purchases is split between the protocol and the previous owner. ### Lease to Permabuy Conversions An actively leased name may be converted to a permanent registration. The price for this conversion is treated as if it were a new permanent name purchase. This functionality allows users to transition from leasing to permanent ownership based on changing needs and available resources. It generates additional protocol revenue through conversion fees, contributing to the ecosystem's financial health and reward system. ### Permanent Name Return Users have the option to "return" their permanently registered names back to the protocol. This process allows users to relinquish their ownership, returning the name to the protocol for public re-registration. Only the Owner of a name can initiate a name return. When a permanent name is returned, the name is subject to a "Returned Name Premium", similar to expired leases. A key difference is that if the name is repurchased during the premium window, the proceeds are split between the returning owner and the protocol balance. ## Primary Names The Ar.io Name System (ArNS) supports the designation of a "Primary Name" for users, simplifying how wallet addresses are displayed across applications. A Primary Name is a user-friendly alias that can replace complex wallet addresses, making interactions and profiles easier to manage and identify. Users can set one of their owned ArNS names as their Primary Name. The fee is equivalent to the cost of a single undername on a 51-character name of the same purchase type, adjusted by the current Demand Factor. Only one Primary Name can be set per wallet, and the same name cannot be the Primary Name for more than one wallet. The base name's ANT owner can remove any Primary Name set on one of its undernames. This allows applications to use a single, human-readable identifier for a wallet, improving user experience across the network. ## Next Steps Now that you understand name registration, learn about [Ar.io Name Tokens (ANTs)](/learn/arns/ants) to see how ownership and control work, or explore the [Pricing Model](/learn/arns/pricing-model) to understand how costs are calculated. # Pricing Model (/learn/arns/pricing-model) ## Addressing Variable Market Conditions The future market landscape is unpredictable, and ArNS is designed to adapt without relying on a centralized pricing oracle. Instead of fixing name prices to an external currency, the protocol adjusts pricing based on network activity and registration demand. This keeps ArNS self-contained while still allowing name prices to respond to changing market conditions over time. ArNS pricing is built from a few core ideas: 1. **Name length**: Shorter, more memorable names generally cost more. 2. **Registration type**: Names can be leased for a defined period or purchased permanently. 3. **Demand Factor**: A protocol multiplier adjusts prices based on recent ArNS activity. 4. **Returned Name Premium (RNP)**: Recently expired or returned names re-enter the market through a descending premium window. This approach lets ArNS remain predictable enough for users while still adapting to namespace demand. You can view current live pricing at [arns.ar.io](https://arns.ar.io/#/prices) to see these formulas in action. ## Key Definitions - **Demand Factor:** A protocol multiplier that adjusts prices based on recent registration activity. - **Base Fee:** The starting price for a name before dynamic adjustments. - **Lease:** A time-limited registration. - **Permabuy:** A permanent registration. - **Undername:** A subdomain-style record written with an underscore (`_`) instead of a dot (`.`). - **Protocol Revenue:** ARIO collected from ArNS actions, such as name registrations, lease extensions, and undername purchases. ## Dynamic Pricing Model ArNS uses an adaptive model to balance name availability, demand, and long-term sustainability. Prices are influenced by name length, whether the name is leased or bought permanently, undername capacity, and the current Demand Factor. The Demand Factor changes over time based on protocol activity. When demand is high, it can increase prices; when demand is low, it can decrease prices. If demand remains low for long enough, the protocol can step base fees downward so names stay accessible. For exact costs, users should rely on the live ArNS app or SDK cost simulation rather than copying formulas into their own applications. ## Returned Name Premiums (RNP) ArNS applies a **Returned Name Premium (RNP)** to names that re-enter the market after expiration or permanent return. The premium starts high and decreases over a return window until the name reaches standard pricing again. Returned name purchases split proceeds between the protocol balance and the previous owner. This discourages instant name sniping after expiry and gives owners a reason to release names they no longer need. For more detail, see [Returned Names](/learn/arns/returned-names). ## Gateway Operator ArNS Discount Gateway operators who demonstrate consistent, healthy participation in the network may be eligible for discounted ArNS interactions. This creates another incentive for gateways to provide reliable service while supporting ArNS usage. Discount eligibility requires a **Gateway Performance Ratio Weight (GPRW) of 0.9** and a **Tenure Weight (TW) of 1.0**. Eligible operators receive a **20% discount** on new ArNS name registrations, lease extensions, lease upgrades, and undername purchases. ## Next Steps Congratulations! You now understand the complete ArNS pricing system. Ready to get started? }> See current ArNS pricing in real-time with the live pricing chart. }> Visit arns.ar.io to register your first name and explore the pricing in action. }> Learn about ar.io gateways and how they integrate with ArNS. }> Start building applications that leverage ArNS for decentralized naming. # Returned Names (/learn/arns/returned-names) ## Overview When an ArNS name expires or is voluntarily returned to the protocol, it enters a **Returned Name Dutch Auction** before becoming available for standard registration. This mechanism prevents name squatting at expiry and provides fair pricing through a time-decaying premium. ## Dutch Auction Mechanics Returned names start at a high premium and decay to the base price over a return window: - **Starting price**: premium above the base registration price - **Ending price**: standard registration price - **Decay**: decreases over time until standard pricing resumes ```mermaid graph LR A[Name Expiredor Released] --> B[Dutch AuctionStarts at 50x] B --> C[Price DecaysOver Time] C --> D[ReachesStandard Price] D --> E[StandardRegistration] ``` ## Revenue Split When a returned name is purchased during the Dutch auction: - **50%** goes to the **protocol balance** (funds epoch rewards) - **50%** goes to the **previous owner** (the ANT holder at the time of return) This incentivizes name owners to voluntarily release names they no longer need, since they receive half the resale value. ## How Names Enter the Returned Pool ### Lease Expiration When a leased name's term ends and the grace period elapses without renewal or conversion to permanent ownership, the name enters the returned pool. ### Voluntary Release Permanent name owners can voluntarily release their name back to the protocol. This places the name in the returned pool. ## Lifecycle | Phase | Duration | Price | Action | |-------|----------|-------|--------| | **Active lease** | As registered | N/A | Name is in use | | **Grace period** | After expiry | N/A | Owner can renew or convert | | **Dutch auction** | Return window | Premium → standard price | Anyone can purchase | | **Standard registration** | Indefinite | Base price | Normal ArNS purchase | ## Querying Returned Names Use the SDK to check available returned names and their current auction price: ```typescript const ario = ARIO.mainnet(); // Get all active returned names const returnedNames = await ario.getArNSReturnedNames({ limit: 100, sortBy: 'endTimestamp', sortOrder: 'asc', }); // Get a specific returned name const name = await ario.getArNSReturnedName({ name: 'example' }); // Check current cost (includes auction premium) const cost = await ario.getTokenCost({ intent: 'Buy-Name', name: 'example', type: 'permabuy', }); ``` # Architecture (/learn/gateways/architecture) Ar.io gateways are data access layers built on top of Arweave. They make permanent data easier to retrieve, cache, index, and serve through standard web interfaces. ```mermaid graph TB subgraph Gateway ["ar.io Gateway"] ENVOY[Envoy ProxyLoad Balancer & Routing] API[Core ServiceGateway API] subgraph "Data Layer" DB1[(Chain IndexSQLite)] DB2[(Bundle IndexSQLite)] DB3[(Data IndexSQLite)] DB4[(Config & MetadataSQLite)] REDIS[(Redis CacheHigh-Speed Layer)] end subgraph "Storage Layer" FS[File System StorageLocal Cache] end end subgraph External ["External Network"] MEMPOOLS[MempoolsTransaction Pool] ARWEAVE[(Arweave NodesBlockchain Network)] MEMPOOLS --> ARWEAVE end ENVOY --> API API DB1 API DB2 API DB3 API DB4 API REDIS API FS API ARWEAVE API MEMPOOLS classDef database fill:#3b82f6,stroke:#1d4ed8,stroke-width:2px,color:#fff classDef service fill:#10b981,stroke:#059669,stroke-width:2px,color:#fff classDef proxy fill:#f59e0b,stroke:#d97706,stroke-width:2px,color:#fff classDef external fill:#8b5cf6,stroke:#7c3aed,stroke-width:2px,color:#fff classDef storage fill:#ef4444,stroke:#dc2626,stroke-width:2px,color:#fff class DB1,DB2,DB3,DB4,REDIS database class API service class ENVOY proxy class ARWEAVE,MEMPOOLS external class FS storage ``` ## Core Technology Stack Ar.io gateways are built using modern, scalable technologies designed for high-performance data operations: ### Runtime and Language - **Node.js**: The primary runtime environment for all gateway services - **TypeScript**: Core services written with flexible interfaces for customization - **Event-driven architecture**: Enables efficient handling of concurrent operations ### Data Storage - **SQLite**: Four specialized databases handle different aspects of gateway operations: - Chain data indexing - Bundle transaction processing - Data item management - Configuration and metadata - **Redis**: High-speed caching layer for frequently accessed data - **File system storage**: Local caching for frequently accessed data ### Processing Model - **Worker-based concurrency**: Specialized workers handle different background tasks - **Event-driven processing**: Loosely coupled components communicate via events - **Streaming data handling**: Minimizes memory overhead for large data operations ## Key Architectural Decisions Several important design decisions shape how ar.io gateways operate: ### Data Retrieval Strategy Ar.io gateways use a **hierarchical fallback system** for data retrieval: 1. **Trusted gateways**: Prioritize data from verified, high-performance peers 2. **Ar.io network**: Leverage the broader network of ar.io gateways 3. **Chunks data items**: Reconstruct data from individual chunks when needed 4. **Transaction data**: Fall back to raw Arweave transaction data This approach improves availability while optimizing for speed and reliability. ### Verification and Trust Model - **Multi-level cryptographic verification**: Data integrity is verified at multiple points - **Trust hierarchy**: Cached verified data → trusted cached data → network streams - **Self-healing mechanisms**: Automatic recovery and re-verification of corrupted data - **Verification headers**: HTTP headers indicate the verification status of returned data ### Serving Capabilities Gateways expose a serving layer for applications and end users. This layer includes byte-range requests, signed and verifiable responses, x402 paid access, peer routing, and content moderation hooks. These capabilities let operators tune how data is delivered while preserving verifiability and local operator choice. ### Worker Specialization Different background workers handle specific responsibilities: - **Block synchronization workers**: Keep the gateway aligned with Arweave blocks - **Bundle processing workers**: Extract and index ANS-104 data items - **Data verification workers**: Check cached data integrity - **Maintenance workers**: Perform cleanup and optimization tasks ## Scalability and Configuration Ar.io gateways are designed to scale from small personal deployments to large enterprise installations: ### Modular Architecture Gateway services can be independently configured or disabled based on operator needs: - **Data serving**: Serve cached data to applications - **Data indexing**: Index and process new Arweave data - **Bundle processing**: Handle Layer 2 bundled transactions - **ArNS routing**: Provide Ar.io Name System resolution ## Core Philosophy: Builder Independence A fundamental principle of ar.io gateway architecture is **empowering builders to do the things they care about** without relying on any centralized resource to leverage Arweave. This philosophy manifests in several key ways: ### Extensibility Through Modularity Gateways are designed as extensible platforms that operators can customize through **[Extensions](/build/extensions/)**, sidecar services, and plugin architectures for specialized functionality. ### Data Sovereignty Operators maintain complete control through **[Data Retrieval](/learn/gateways/data-retrieval)** strategies and **[Data Verification](/learn/gateways/data-verification)** systems that ensure independence from trusted intermediaries. ### Network Resilience The modular design creates a resilient ecosystem where distributed infrastructure and customizable trust models prevent single points of failure. This architecture helps builders use Arweave without depending on a single infrastructure provider. ## Explore Gateway Capabilities } /> } /> } /> } /> # Data Retrieval (/learn/gateways/data-retrieval) Ar.io gateways retrieve and serve Arweave data from multiple sources. They prefer fast local or trusted sources when available, then fall back to broader network peers, chunks, or Arweave nodes as needed. ## How Gateways Retrieve Data When a gateway needs to serve data, it follows a hierarchical retrieval pattern, trying each source in order until the data is successfully retrieved: ```mermaid graph TD REQUEST[Data Request] --> CACHE{Local Cache?} CACHE -->|Hit| SERVE[Serve Data] CACHE -->|Miss| SOURCES[Try Data Sources] SOURCES --> TRUSTED[Trusted Gateways] SOURCES --> NETWORK[ar.io] SOURCES --> CHUNKS[Chunk Assembly] SOURCES --> ARWEAVE[Arweave Nodes] TRUSTED -->|Success| VALIDATE NETWORK -->|Success| VALIDATE CHUNKS -->|Success| VALIDATE ARWEAVE -->|Success| VALIDATE TRUSTED -->|Fail| NETWORK NETWORK -->|Fail| CHUNKS CHUNKS -->|Fail| ARWEAVE VALIDATE{Valid?} -->|Yes| STORE[Cache & Serve] VALIDATE -->|No| NEXT[Try Next Source] classDef source fill:#2563eb,stroke:#1d4ed8,stroke-width:2px,color:#fff classDef process fill:#16a34a,stroke:#15803d,stroke-width:2px,color:#fff class TRUSTED,NETWORK,CHUNKS,ARWEAVE source class VALIDATE,SERVE,STORE process ``` ## Data Sources Ar.io gateways can retrieve data from multiple sources, each with different characteristics: ### 1. Trusted Gateways - **Purpose**: Peer-to-peer data sharing between verified ar.io gateways - **Benefits**: Distributed redundancy, load balancing, network resilience - **Trust Mechanism**: Operator-defined trust settings, observed performance, and reciprocity - **Selection**: Prioritized based on local gateway configuration ### 2. ar.io (Untrusted Peers) - **Purpose**: Broader network of ar.io gateways without established trust - **Benefits**: Geographic distribution, expanded data availability - **Selection**: Chosen based on availability, configuration, and routing strategy - **Validation**: Verification is important because the peer may not be trusted ### 3. Chunk Assembly - **Purpose**: Direct reconstruction from Arweave chunks via known offsets - **Benefits**: Data integrity guarantee, no intermediary trust required - **Process**: Fetches individual chunks efficiently and assembles them into complete data - **Optimization**: Uses offset awareness for faster chunk retrieval ### 4. TX Data - **Purpose**: Direct access to transaction data from Arweave nodes - **Benefits**: Authoritative data source, complete historical access - **Trade-off**: Higher latency but guaranteed availability - **Use Case**: Final fallback when other sources fail ## Retrieval Strategies Gateways employ different strategies based on the use case: ### On-Demand Retrieval Optimized for user requests with emphasis on speed: 1. **Priority order**: Trusted Gateways → Untrusted Peers (ar.io) → Chunks Assembly → Arweave 2. **Aggressive timeouts**: Quick fallback to next source 3. **Parallel attempts**: May query multiple sources simultaneously 4. **Response streaming**: Begin serving data as soon as available ### Background Retrieval Used specifically for unbundling and verification processes: 1. **Unbundling operations**: Extracting individual data items from ANS-104 bundles 2. **Data verification**: Comprehensive validation of retrieved data integrity 3. **Integrity focus**: Prefers authoritative sources for accurate processing 4. **Relaxed timeouts**: Allows for slower but reliable retrieval during verification 5. **Verification priority**: Extensive validation before caching verified data ## Trust and Validation ### Peer Trust Management Gateways can maintain trust relationships with peer gateways: ```mermaid graph TD PEER[Peer Gateway] --> METRICS[Performance Metrics] METRICS --> LATENCY[Response Time] METRICS --> SUCCESS[Success Rate] METRICS --> VALIDITY[Data Validity] LATENCY --> SCORE[Trust Score] SUCCESS --> SCORE VALIDITY --> SCORE SCORE --> SELECTION{Peer Selection} SELECTION -->|High Trust| PREFER[Preferred] SELECTION -->|Medium Trust| NORMAL[Normal] SELECTION -->|Low Trust| AVOID[Avoided] classDef metric fill:#7c3aed,stroke:#6d28d9,stroke-width:2px,color:#fff class LATENCY,SUCCESS,VALIDITY metric ``` Trust factors include: - **Response performance**: Latency and throughput metrics - **Success rates**: Percentage of successful requests - **Data validity**: Cryptographic verification results - **Reciprocity**: Mutual data sharing behavior ### Data Validation Process Every piece of retrieved data undergoes validation: 1. **Hash Verification**: Computed hash must match expected value 2. **Merkle Proof Validation**: Chunks proven against transaction root 3. **Signature Verification**: Transaction signatures validated 4. **Size Confirmation**: Data size matches header declaration ## Why Multi-Source Retrieval Matters ### For Gateway Operators - **Reduced infrastructure costs**: Leverage peer resources - **Improved reliability**: Multiple fallback options - **Better performance**: Optimal source selection - **Network effects**: Benefit from collective infrastructure ### For Users - **Faster access**: Data served from optimal source - **High availability**: Multiple paths to data - **Geographic optimization**: Nearby sources preferred - **Consistent experience**: Transparent source selection --- The data retrieval system is central to ar.io's mission of providing reliable, performant access to the permaweb. Multiple retrieval paths help keep permanent data accessible through a distributed gateway network. ## Related Gateway Concepts } /> } /> } /> } /> # Data Verification (/learn/gateways/data-verification) Ar.io gateways verify that retrieved and cached data matches what was committed to Arweave. Verification helps users receive authentic, uncorrupted data without trusting a single gateway operator. Gateway data verification is one layer of ar.io's broader verification architecture. For how gateway verification composes with signed response claims, client-side verification, and OIP accountability, see [Verification and Accountability](/learn/verification). ## How Gateways Verify Data Data verification uses Arweave data roots, hashes, and Merkle proofs to check that cached data matches what was originally stored. A gateway can verify data before serving it or re-import data when verification fails: ```mermaid sequenceDiagram participant Scheduler participant Worker as DataVerificationWorker participant ContigIndex as ContiguousDataIndex participant RootTxIndex as DataItemRootTxIndex participant DataRootComp as DataRootComputer participant DataSource as ContiguousDataSource participant Importer as DataImporter participant Bundler as BundleQueue Note over Scheduler,Bundler: Data Discovery Phase Scheduler->>Worker: Triggers queueRootTx() periodically Worker->>ContigIndex: getVerifiableDataIds() ContigIndex-->>Worker: Returns list of data IDs loop For each dataId Worker->>RootTxIndex: getRootTxId(dataId) RootTxIndex-->>Worker: Returns rootTxId Worker->>Worker: Enqueue rootTxId if not processed end Note over Scheduler,Bundler: Verification Phase Worker->>ContigIndex: getDataAttributes(rootTxId) ContigIndex-->>Worker: Returns attributes (indexedDataRoot, hash) alt indexedDataRoot is present Worker->>DataRootComp: computeDataRoot(rootTxId) DataRootComp->>DataSource: getData(rootTxId) DataSource-->>DataRootComp: Returns data stream DataRootComp-->>Worker: Returns computedDataRoot alt computedDataRoot matches indexedDataRoot Worker->>ContigIndex: saveVerificationStatus(rootTxId) ContigIndex-->>Worker: ✓ Verification Success Note over Worker: Data is now verified and cached for serving else computedDataRoot does NOT match Worker->>Importer: queueItem({id: rootTxId}, priority=true) Importer-->>Worker: Queued for re-import from Arweave end else indexedDataRoot is MISSING Worker->>Bundler: queueBundle({id: rootTxId}) Bundler-->>Worker: Queued for bundle unbundling end ``` **The Verification Workflow:** At a high level, verification moves through discovery, retrieval, cryptographic computation, comparison, and recovery: **1. Discovery Phase** - Periodically scan for unverified data items - Priority-based queue management (higher priority items first) - Track retry attempts for failed verifications **2. Data Retrieval** - Fetch data attributes from gateway storage - Retrieve the complete data stream - Gather metadata needed for verification **3. Cryptographic Computation** - Calculate Merkle data root from actual data stream - Generate cryptographic proofs using the same algorithm as Arweave - Create verifiable hash chains **4. Root Comparison** - Compare computed root against indexed root in database - Verify data hasn't been corrupted or altered - Validate chunk integrity against Merkle proofs **5. Action Based on Results** - **Success**: Mark data as verified with timestamp - **Failure**: Trigger re-import from Arweave or unbundle from parent - **Error**: Increment retry counter and requeue for later ## Verification Types Ar.io gateways handle different types of data verification based on the data's origin: ### Transaction Data Verification For individual Arweave transactions: - **Direct root validation** against transaction data roots stored onchain - **Complete data reconstruction** from chunks to ensure availability - **Cryptographic proof** that data matches what was originally stored ### Bundle Data Verification For ANS-104 data bundles (collections of data items): - **Bundle integrity checks** to verify the container is valid - **Individual item verification** within each bundle - **Recursive unbundling** when verification fails to re-extract items - **Nested bundle support** for bundles containing other bundles ### Chunk-Level Validation At the most granular level: - **Merkle proof validation** for individual data chunks - **Sequential integrity** ensuring chunks form complete data - **Parallel verification** of multiple chunks for performance ## Why Verification Matters ### Cryptographic Trust Foundation - **Mathematical Proof**: Merkle tree cryptography provides irrefutable proof of data integrity - **Independent Validation**: Multiple gateways verify the same data independently ### Data Integrity Guarantees - **Tamper Detection**: Any alteration to data is immediately detectable - **Corruption Recovery**: Automatic healing of corrupted data through re-import ### Gateway Reliability - **Continuous Monitoring**: Ongoing verification catches issues before users encounter them - **Self-Healing System**: Automatic recovery mechanisms maintain data availability - **Transparent Operations**: Verification status and timestamps provide audit trails ## Explore Gateway Systems } /> } /> } /> } /> # Gateway Registry (/learn/gateways/gateway-registry) ## Overview The Gateway Address Registry (GAR) is the public registry of ar.io gateways. It records which gateways have joined the network, how they can be reached, and the metadata applications need to discover and evaluate them. Gateway registry state is maintained by the ar.io Solana programs. Registered gateways are identified by Solana addresses and publish service information such as hostname, protocol settings, staking state, and operator-provided metadata. The gateways.ar.io portal displays gateways currently in the network, including stake, performance, and operational metadata. ## Joining the Registry To join the ar.io network, a gateway operator registers their gateway and locks ARIO as operator stake. Registration connects the gateway's public endpoint, operator address, observer address, and service metadata to the onchain registry. Once registered, a gateway can become eligible for network incentives, delegation, and observation through OIP. Gateway operators also need enough SOL to pay Solana transaction fees for network interactions. The registry has a maximum capacity of **3,000 gateways**. Each gateway requires a **20,000 ARIO** minimum network-join stake, and each gateway can have up to **10,000** unique delegated stakers. ## What the Registry Enables ### Discovery Apps, users, and other gateways can use the registry to find gateways by endpoint, service metadata, stake, observed performance, and supported capabilities. ### Incentive Participation The registry connects gateway identity to staking, delegated stake, performance history, and reward eligibility. OIP uses this information to evaluate gateways and distribute rewards. ### Network Transparency Gateway information is publicly visible through Solana state and network tooling such as [gateways.ar.io](https://gateways.ar.io). This makes participation, performance, and configuration easier to inspect. ### Operator Choice Gateways can specialize. Some may focus on fast public access, some on indexing, some on private infrastructure, some on paid access, and some on specific moderation or compliance policies. ## Relationship to OIP The registry is the set of gateways that OIP can observe and evaluate. Registered gateways are periodically checked for availability, correctness, and ArNS resolution behavior. Reliable gateways can earn rewards, while gateways that repeatedly fail can lose eligibility and eventually be removed through pruning. ## Recap - The Gateway Address Registry is the public source of network gateway metadata. - Gateways join by registering service details and locking operator stake. - Registered gateways can participate in OIP, receive delegations, and become eligible for rewards. - Apps can use registry data to discover, filter, and route through gateways. - Poorly performing gateways can be pruned from the network after sustained failure. ## Explore the Gateway Ecosystem } /> } /> } /> } /> # ar.io Gateways (/learn/gateways) ## What are Gateways? Ar.io gateways are infrastructure nodes that make Arweave data easy to access from web apps, APIs, and users. They retrieve data from Arweave, cache and index it, resolve ArNS names, and expose standard HTTP interfaces for the permaweb. Gateways do not replace Arweave. Arweave provides permanent storage and data availability guarantees; gateways make that data fast, discoverable, and practical to use. ## Core Responsibilities ### Data Access - **Retrieve permanent data** from Arweave, peer gateways, local cache, or chunk-level sources - **Serve content over HTTP** using familiar web patterns - **Optimize delivery** through caching, streaming, and source selection ### Indexing and Discovery - **Index transactions and bundled data items** so apps can search and retrieve content efficiently - **Resolve ArNS names** into the data records and routes they point to - **Expose gateway metadata** so users and applications can discover available infrastructure ### Network Participation - **Register in the Gateway Address Registry** to become discoverable as an ar.io network gateway - **Stake ARIO** to participate in network incentives and signal operational commitment - **Participate in OIP** by being observed, reporting observations when selected, and earning rewards for reliable service ## What Gateways Are Not ### Not Storage Providers - **They do not create Arweave permanence**: permanence comes from Arweave itself - **They do not need to store everything**: operators choose what to cache, index, and serve - **They do not control user data**: data ownership remains with the original publisher and underlying protocols ### Not Application Compute - **They do not run protocol logic**: ar.io protocol state, staking, ArNS, and incentives are coordinated by Solana programs - **They do not process arbitrary app logic**: gateways focus on data access, indexing, routing, and delivery ### Not a Single Service - **Anyone can run one**: the gateway network is open to independent operators - **Apps can choose among gateways**: routing can consider availability, geography, performance, policy, and payment requirements - **Operators can specialize**: gateways may differ in scale, indexing choices, moderation policy, payment setup, and extensions ## Explore Gateways } /> } /> } /> } /> } /> # x402 Payments (/learn/gateways/x402-payments) Ar.io gateways integrate the x402 payment protocol to enable frictionless cryptocurrency payments for data access. This creates a sustainable monetization model that allows gateways to cover infrastructure costs while providing premium access tiers for users who need higher bandwidth or priority service. ## What is x402? **x402** is an open-source payment protocol built by Coinbase that leverages the HTTP 402 "Payment Required" status code to enable frictionless cryptocurrency payments for web APIs. The protocol is designed to eliminate traditional friction points in web payments: ### Key Features - **Zero protocol fees**: No intermediaries or payment processing costs - **Fast settlement**: Payments settle in approximately 2 seconds - **Minimal integration**: Simple HTTP status codes and headers - **Privacy-focused**: No account creation or personal information required - **Chain-agnostic**: Works with multiple blockchain networks - **Token-agnostic**: Supports various cryptocurrencies ### How x402 Works The x402 protocol uses a simple HTTP-based flow: ```mermaid sequenceDiagram participant Client participant Gateway participant Payment as Payment Facilitator participant Blockchain Client->>Gateway: Request data Gateway->>Client: 402 Payment Required Note over Gateway,Client: Includes payment requirements in headers Client->>Payment: Initiate payment Payment->>Blockchain: Process transaction Blockchain-->>Payment: Transaction confirmed Payment-->>Client: Payment receipt Client->>Gateway: Retry request with payment proof Gateway->>Gateway: Verify payment Gateway-->>Client: Serve requested data ``` ## ar.io Gateway Integration Ar.io gateways implement x402 using **USDC (USD Coin) on the Base blockchain** to monetize data egress with per-byte pricing. This integration works seamlessly with the gateway's rate limiting system to provide a complete traffic management and monetization solution. ### Payment Flow in ar.io Gateways 1. **Free Tier Access**: Users consume regular tokens from their rate limit buckets 2. **Rate Limit Exceeded**: When limits are reached, gateway sends 402 Payment Required response 3. **Payment Processing**: Users can make USDC payments to continue access 4. **Premium Access**: Payments add paid tokens with configurable multiplier (default 10x capacity) 5. **Priority Consumption**: Regular tokens consumed first, then paid tokens 6. **Resource Bypass**: Paid requests bypass per-resource limits (only IP limits apply) ### Browser Paywall & Onramp Integration For mainnet deployments, gateways can integrate Coinbase Onramp to enable easy USDC purchases directly in the browser paywall. This requires Coinbase Developer Platform (CDP) API keys, which are separate from payment facilitator authentication. The Onramp integration allows users to purchase USDC without leaving the gateway interface, creating a seamless payment experience. **Note**: CDP keys are optional for testnet but required for mainnet when using Coinbase facilitators. These keys enable the browser-based purchasing experience and are not related to payment verification or settlement. ## Network Options Ar.io gateways support two Base blockchain networks for x402 payments: | Feature | Base Sepolia (Testnet) | Base (Mainnet) | | ----------------------- | --------------------------------- | --------------------------------- | | **USDC** | Free testnet USDC (faucet) | Real USDC (costs money) | | **CDP API Key** | Not required | Required for official facilitator | | **Default Facilitator** | https://x402.org/facilitator | Must configure | | **Use Case** | Development, testing | Production monetization | | **Configuration** | `X_402_USDC_NETWORK=base-sepolia` | `X_402_USDC_NETWORK=base` | ## Rate Limited Endpoints The x402 payment system applies to data egress endpoints where gateways serve actual content: ### Data Serving Endpoints - **Transaction/Data Item requests**: `/:txid` and `/:txid/path` - **Raw data requests**: `/raw/:txid` - **ArNS resolved content**: All requests resolved through ArNS names - **Farcaster frames**: `/local/farcaster/frame/:txid` - **Chunk requests**: `GET /chunk/:offset` (uses fixed size pricing) ### Not Rate Limited - GraphQL queries (`/graphql`) - Chunk POST requests (`POST /chunk`) - Administrative endpoints (`/ar-io/*`) ## Benefits for Gateway Operators ### Sustainable Infrastructure - **Cost Recovery**: Monetize data egress to cover operational expenses - **Premium Services**: Offer higher bandwidth tiers for power users - **Flexible Pricing**: Per-byte pricing scales with actual usage - **Revenue Generation**: Create sustainable business models ### Traffic Management - **Fair Access**: Free tier for casual users with reasonable limits - **Premium Access**: Power users can pay for additional capacity - **DDoS Protection**: Rate limiting prevents abuse while allowing legitimate paid access - **Resource Optimization**: Paid requests bypass resource limits for better performance ## Benefits for Users ### Frictionless Payments - **No Account Required**: Payments work without registration or OAuth - **Fast Settlement**: 2-second payment confirmation - **Privacy Preserved**: No personal information collection - **Transparent Pricing**: Clear per-byte costs with minimum/maximum limits ### Flexible Access - **Free Tier**: Reasonable free access for casual users - **Pay-as-you-go**: Only pay when you need additional capacity - **Predictable Costs**: Clear pricing structure with configurable limits - **Multiple Payment Methods**: Browser-based and programmatic payment options ## Use Cases ### Content Delivery Networks - **Media Streaming**: Monetize high-bandwidth video and audio content - **File Downloads**: Charge for large file transfers - **API Access**: Premium API tiers with higher rate limits ### Developer Platforms - **Data APIs**: Monetize access to indexed blockchain data - **Archive Services**: Charge for historical data retrieval - **Real-time Feeds**: Premium access to live data streams ### Enterprise Services - **Bulk Data Access**: Corporate clients paying for high-volume access - **Priority Support**: Premium service tiers with guaranteed performance - **Custom Limits**: Tailored rate limits for specific use cases ## Explore Gateway Monetization } /> } /> } /> } /> # Epoch Pipeline (/learn/oip/epoch-pipeline) ## Overview On Solana, the ar.io epoch lifecycle is broken into six discrete, permissionless steps. Each step is a separate instruction that can be executed by anyone. All steps are **idempotent** (safe to run multiple times) and **permissionless** (anyone can crank them). This design ensures the protocol cannot be halted by a single point of failure. ## Pipeline Steps ```mermaid graph LR A[create_epoch] --> B[tally_weights] B --> C[prescribe_epoch] C --> D[save_observations] D --> E[distribute_epoch] E --> F[close_epoch] ``` ### 1. create_epoch **Initializes the epoch account and computes the reward rate.** - Creates the epoch account - Computes the epoch reward allocation from the protocol balance ### 2. tally_weights **Batched computation of gateway weights for observer selection.** - Computes composite weights for gateways: - **Stake weight**: Based on total stake (operator + delegated) - **Tenure weight**: Based on how long the gateway has been in the network - **Gateway performance**: Based on pass rate across recent epochs - **Observer performance**: Based on observation submission history - Batches work so large gateway sets can be processed safely on Solana ### 3. prescribe_epoch **Selects observers and prescribed ArNS names via weighted roulette.** - Selects observers using weighted random selection - Selects prescribed ArNS names that observers use as common test targets ### 4. save_observations **Observers submit their pass/fail observation reports.** - Each selected observer submits compact pass/fail results for tested gateways - This is the only step that requires a specific signer (the selected observer) - Observations are stored on the Epoch account ### 5. distribute_epoch **Batched reward distribution to gateways and their delegates.** - Functional gateways receive the Base Gateway Reward (BGR) - Functional observers receive the Base Observer Reward (BOR) - Deficient observers do not receive observer rewards - Operator rewards auto-compound into operator stake - Delegate rewards are tracked via the reward-per-share accumulator (settled lazily) - Leaving gateways receive 0 rewards ### 6. close_epoch **Reclaims rent from completed epoch accounts.** - Recovers SOL rent from completed epoch accounts - Keeps onchain state lean over time ## Timing The pipeline steps can be executed as the epoch progresses: | Step | When | Batched? | |------|------|----------| | create_epoch | After previous epoch ends | No | | tally_weights | After create_epoch | Yes | | prescribe_epoch | After all weights tallied | No | | save_observations | During observation window | No (per observer) | | distribute_epoch | After observation window | Yes | | close_epoch | After epoch state is no longer needed | No | ## Who Cranks? A cranker is a permissionless actor that executes the epoch pipeline on Solana. Since Solana programs cannot execute on a timer, an external wallet must call each instruction to advance the epoch lifecycle. The key property is that cranking is **permissionless**: any wallet with SOL for transaction fees can run it. No ARIO tokens, gateway registration, or special authorization is required. Without crankers, the epoch pipeline would stall. Observations would not be prescribed, rewards would not be distributed, and completed epoch state would not be closed. Multiple independent crankers provide redundancy so the network is not dependent on a single operator. Cranking can run as a standalone bot that watches epoch state and submits whichever pipeline instruction is needed next. It can also be embedded directly in ar.io observers: ```bash # In your observer's .env file ENABLE_EPOCH_CRANKING=true ``` Multiple crankers can run at the same time without coordination. The first successful transaction advances the pipeline, and later attempts see that the step is already complete. Because the instructions are idempotent, duplicate calls do not double-distribute rewards or corrupt epoch state. # Observation & Incentive Protocol (/learn/oip) ## Overview The Observation and Incentive Protocol (OIP) helps maintain gateway quality through peer monitoring and performance-based rewards. Gateways are incentivized to serve data reliably while also acting as observers that evaluate their peers. The protocol runs in epochs. During each epoch, selected gateways observe other gateways, submit reports, and participate in an onchain pass/fail voting process. Functional gateways and observers become eligible for ARIO rewards, while deficient gateways and observers miss rewards and can lose future selection weight. ## Architecture Overview The protocol follows a repeatable flow: ```mermaid sequenceDiagram participant SC as ar.io Protocol participant OBS as Observer Gateway participant GW as Target Gateway participant AR as Arweave Network Note over SC,AR: Epoch Start SC->>SC: Select observers(weighted random) SC->>SC: Prescribe ArNS names SC->>OBS: Notify selection & provide names Note over SC,AR: Observation Phase loop For each gateway to test OBS->>GW: Test ArNS resolution GW-->>OBS: Response data OBS->>OBS: Score: Pass/Fail end Note over SC,AR: Reporting Phase OBS->>AR: Upload detailed JSON report AR-->>OBS: Confirm storage OBS->>SC: Submit compact pass/fail results SC-->>OBS: Confirm interaction Note over SC,AR: Evaluation & Rewards SC->>SC: Tally all observer votes SC->>SC: Tally observer votes SC->>SC: Determine rewards SC->>OBS: Distribute observer rewards SC->>GW: Distribute gateway rewards(if functional) ``` ## Epoch Cycle Each epoch follows a structured process with specific responsibilities for gateways and observers: ### Epoch Start - **ar.io protocol**: Selects observers using weighted random selection - **ar.io protocol**: Prescribes common ArNS names for observers to test - **Selected Observers**: Receive notification of selection and prescribed names ### Observation Phase - **Observers**: Test assigned gateways for ArNS resolution and response quality - **Observers**: Document pass/fail findings and failure reasons - **Target Gateways**: Respond to resolution requests and serve the requested content ### Reporting Phase - **Observers**: Upload detailed JSON reports to Arweave for transparency - **Observers**: Submit compact onchain pass/fail results for consensus voting ### Evaluation and Distribution - **ar.io protocol**: Tallies observer votes to classify gateways - **ar.io protocol**: Distributes rewards based on performance - **Functional Gateways/Observers**: Become eligible for ARIO rewards ## Key Features - **Decentralized Monitoring**: Peer-to-peer evaluation ensures no single point of failure - **Consensus-Based Scoring**: Observer submissions determine gateway functionality - **Performance Incentives**: Only functional gateways and observers receive ARIO token rewards - **Transparent Accountability**: All reports permanently stored on Arweave and viewable at [gateways.ar.io](https://gateways.ar.io) - **Sustainable Funding**: Protocol balance funded by ArNS name purchases, aligning rewards with network usage ## What Observers Evaluate Observer criteria can evolve over time without requiring every detail to live in the protocol itself. At a high level, observers evaluate whether gateways can: - Resolve prescribed ArNS names correctly - Return expected transaction IDs and response data - Serve data with acceptable availability and responsiveness - Produce reports that can be independently inspected by users and applications --- **View Live Data**: See current observers and performance metrics at [gateways.ar.io](https://gateways.ar.io) ## Explore the Protocol } /> } /> } /> } /> # Observer Selection (/learn/oip/observer-selection) ## Epochs and Selection Timeline The ar.io network operates in epochs, during which the observer selection and evaluation process takes place. At the start of each epoch: - Observers are selected to monitor the network - Prescribed ArNS names are chosen for all observers to test - Observers may evaluate additional names or criteria as defined by the current observation process This creates a consistent evaluation framework where all observers test the same baseline names while leaving room for the observation process to evolve over time. ## Selection Process Gateways are selected as observers using weighted random selection. The process combines randomness with gateway-specific weights so observer duties are unpredictable, but still influenced by stake, tenure, and historical performance. The entropy provides cryptographic randomness for selecting: - **Observer Gateways**: Gateways chosen to perform observations - **Prescribed ArNS Names**: Common names all observers must evaluate This approach prevents manipulation while maintaining weighted probabilities based on gateway performance and commitment. ![Current epoch observers showing their observation chance (normalized composite weight) and report status](/content/observers.png) gateways.ar.io/#/observers {" "} shows the current epoch prescribed observers and arns names, as well as their submission status ## Weighted Selection Criteria Observer selection is based on **normalized composite weights** that combine multiple performance and commitment factors. These weights determine each gateway's probability of being selected as an observer for the epoch. The selection considers four key factors that are multiplied together to create a composite weight (CW): - **Stake Weight (SW)**: Financial commitment to the network - **Tenure Weight (TW)**: Length of network participation - **Gateway Performance Ratio Weight (GPRW)**: Historical gateway performance - **Observer Performance Ratio Weight (OPRW)**: Historical observer performance These weights are then normalized across all eligible gateways to create selection probabilities. For more on how performance affects those weights, see [Performance Evaluation](/learn/oip/performance-evaluation). ## Random Selection The selection process maps random values onto normalized gateway weight ranges. Gateways with higher weights have proportionally better chances of selection, while randomness prevents predictable or easily manipulated assignments. ## Fairness and Meritocracy This system ensures: - **Meritocratic Selection**: Higher-performing gateways have better selection odds - **Fair Opportunity**: All gateways maintain non-zero selection probability - **Tamper Resistance**: Entropy prevents predictable observer assignment - **Consistent Standards**: Prescribed names create common evaluation baseline The selection is saved in the contract state at epoch start to ensure that activities during the epoch do not affect selection or reward distribution. --- ## Next Steps Ready to understand how performance is evaluated? Learn about [Performance Evaluation](/learn/oip/performance-evaluation) to see how gateways are scored, or explore [Reward Distribution](/learn/oip/reward-distribution) to understand how rewards are calculated and distributed. # Performance and Weights (/learn/oip/performance-evaluation) ## Gateway Classifications Consider the following classifications: - **Functional or Passed Gateways**: are gateways that meet or surpass the network's performance and quality standards, including ArNS resolution and chunk/offset validation (if selected). - **Deficient or Failed Gateways**: are gateways that fall short of the network's performance expectations, including failures in ArNS resolution or chunk/offset validation. - **Functional or Submitted Observers**: are selected observers who diligently perform their duties and submit observation reports and onchain observations. - **Deficient or Failed Observers**: are selected observers who do not fulfill their duty of submitting observation reports and onchain observations. ## Evaluation Process At the end of an epoch, the ar.io protocol processes observer submissions to determine gateway performance through a consensus-based vote tallying system. This evaluation transforms individual observer reports into network-wide performance assessments. ### Vote Tallying and Gateway Classification After observers submit their detailed reports (see [Reporting](/learn/oip/reporting) for submission details), the protocol performs consensus calculation: **Vote Processing:** - **Data Collection**: All observer onchain observations for each gateway are collected - **Vote Counting**: Each observer submission contributes either a PASS or FAIL vote - **Majority Determination**: If enough submitted observer interactions indicate PASS, the gateway is considered Functional - **Binary Classification**: Gateways are classified as either Functional (eligible for rewards) or Deficient (ineligible for rewards) **Consensus Mechanism:** - Multiple observers evaluate each gateway independently, ensuring reliable assessment - A majority-style threshold requires broad observer agreement for positive performance determination - Binary scoring provides clear, unambiguous performance classification - Vote tallying occurs after submissions are finalized ## Weight Impact on Gateway Performance Gateway performance directly affects multiple weighted factors that influence future observer selection and overall network participation: ### Gateway Performance Ratio Weight (GPRW) A gateway's evaluation results directly impact their Gateway Performance Ratio Weight, which affects their likelihood of being selected as an observer in future epochs. - **Functional Gateways**: Increase their passed epochs count, improving their GPRW - **Deficient Gateways**: Decrease their GPRW as participated epochs increase without corresponding passes - **Observer Selection**: Higher GPRW increases chances of being selected as an observer ### Observer Performance Ratio Weight (OPRW) For gateways selected as observers, their performance in submitting reports affects future selection. - **Functional Observers**: Who submit reports increase their OPRW - **Deficient Observers**: Who fail to submit reports see their OPRW decrease - **Future Selection**: Higher OPRW improves chances of future observer selection ### Composite Weight Performance factors combine to determine overall network influence: - **Stake Weight**: Financial commitment - **Tenure Weight**: Network longevity - **Gateway Performance Ratio Weight**: Gateway reliability history - **Observer Performance Ratio Weight**: Observation duty history **Long-term Effects:** - Consistently functional gateways accumulate higher composite weights - Poor performers see diminishing influence and selection chances - Performance history creates compounding effects on network participation ## Evaluation Timeline Rewards are distributed **at the end of each epoch** by the ar.io protocol based on the tallied observer votes. The protocol processes observer submissions and distributes rewards to functional gateways and observers based on their performance during the epoch. ## Key Features - **Majority Rule**: Gateway performance is determined by majority vote from observers - **Binary Scoring**: Simple pass/fail system for clear performance assessment - **Network Confirmation**: Evaluation happens after observer submissions are finalized - **Transparent Process**: All evaluations are based on onchain data ## Consequences of Performance ### Functional Gateways - Eligible for gateway rewards - Maintain good standing in the network - Continue to be considered for observer selection ### Deficient Gateways - Ineligible for gateway rewards - Risk being marked as deficient for multiple epochs - May face additional penalties for prolonged poor performance ### Observer Performance - Functional observers receive observer rewards - Deficient observers forfeit observer rewards - Deficient observers who are also functional gateways can have their gateway reward reduced for that epoch --- ## Next Steps Ready to understand how rewards are distributed? Learn about [Reward Distribution](/learn/oip/reward-distribution), or go back to [Observer Selection](/learn/oip/observer-selection) to review the selection process. # Gateway Pruning (/learn/oip/pruning) ## Overview Gateway pruning is the mechanism that removes gateways that continue to fail network performance checks. It helps keep the Gateway Address Registry focused on infrastructure that is reachable, useful, and aligned with the network's quality expectations. Pruning is based on OIP performance results. A gateway is not removed for a single bad result, but sustained failure can make it eligible for removal. ## How Pruning Works When a gateway remains deficient for a sustained period: 1. **The gateway becomes prunable** after repeated failed observations. 2. **A permissionless instruction can remove it** from the Gateway Address Registry. 3. **The operator's required network-join stake is slashed** to the protocol balance. 4. **Eligible excess and delegated stake follow the normal withdrawal process** instead of being slashed. ```mermaid graph TD A[Gateway participates in OIP] --> B{Performance result} B -->|Pass| C[Failure history resets] B -->|Fail| D[Failure history accumulates] D --> E{Sustained failure?} E -->|No| A E -->|Yes| F[Gateway becomes prunable] F --> G[Gateway removed from registry] G --> H[Required operator stake slashed] G --> I[Eligible excess and delegated stake withdrawable] ``` ## Impact on Operators Pruning is designed to make gateway operation economically accountable. Operators are expected to keep their gateways online, correctly configured, able to resolve ArNS names, and able to participate in observation duties when selected. When a gateway is pruned, the operator loses the required stake associated with joining the network. Any eligible excess stake follows the normal withdrawal flow. ## Impact on Delegators Delegated stake is not the target of pruning slashing. If a gateway is pruned, delegated stake follows the normal withdrawal process for leaving gateways. Delegators should still monitor gateway performance because pruning can interrupt reward eligibility and require withdrawal or redelegation decisions. ## Prevention Gateway operators can reduce pruning risk by: - Maintaining reliable uptime and public reachability - Keeping ArNS resolution working correctly - Monitoring OIP performance results and gateway health - Funding the observer wallet with SOL for required transaction fees - Reviewing gateway status in [gateways.ar.io](https://gateways.ar.io) ## Related Concepts - [Gateway Registry](/learn/gateways/gateway-registry) - [Observation and Incentive Protocol](/learn/oip) - [Staking](/learn/oip/staking) # Reporting (/learn/oip/reporting) ## Observer Responsibilities Selected observers have specific duties each epoch: test gateways, document results, and submit findings through two channels. Proper completion of these responsibilities determines observer rewards and future selection chances. ## Dual Submission Process Observers must submit their findings through both channels to fulfill their duties: ### 1. Detailed Reports to Arweave - **Format**: Comprehensive JSON reports with full evaluation data - **Purpose**: Permanent audit trail and transparency - **Content**: Complete test results, timing data, and failure details ### 2. Onchain Observations to the ar.io Protocol - **Format**: List of failed gateways - **Purpose**: Efficient vote tallying for consensus - **Content**: Binary pass/fail determinations for each gateway tested ## Observer Evaluations Observers test assigned gateways against prescribed ArNS names and document their findings: ![Observer Report Overview showing multiple gateway evaluations](/content/observer-report.png) ### Evaluation Results Passing Report: Gateway successfully resolves ArNS names with expected status codes, transaction IDs, and response data. Failing Report: Gateway fails ArNS resolution tests due to ownership issues, timeouts, or missing content. Observers evaluate gateways based on: - **Gateway Wallet Ownership**: Verifies correct wallet address - **ArNS Resolution**: Tests successful name-to-transaction resolution - **Content Hash Verification**: Ensures data integrity - **Response Times**: Measures performance within limits ## Observer Rewards and Penalties Observer performance directly impacts rewards and future participation: ### Successful Observer Performance - **Observer Reward**: Observers who submit both reports and onchain observations receive the Observer Reward - **Future Selection**: Successful reporting improves Observer Performance Ratio Weight (OPRW) - **Increased Chances**: Higher OPRW increases likelihood of future observer selection and more reward opportunities ### Failed Observer Performance - **No Observer Reward**: Observers who fail to submit required reports forfeit their Observer Reward - **Gateway Penalty**: If the deficient observer is also a functional gateway, their gateway reward can be reduced for that epoch - **Reduced Selection**: Failed submissions decrease OPRW, diminishing future observer selection chances - **Lost Opportunities**: Lower selection probability means fewer chances to earn Observer Rewards ## Observer Accountability The system tracks observer performance to ensure network quality: - **Submission Tracking**: Both Arweave reports and onchain observations must be submitted - **Performance History**: Observer submission record affects future selection probability - **Reward Impact**: Consistent reporting builds credibility and increases earning potential --- ## Next Steps Ready to understand how these reports are processed? Learn about [Performance Evaluation](/learn/oip/performance-evaluation) to see how reports become votes and determine gateway rewards, or explore [Reward Distribution](/learn/oip/reward-distribution) to understand the complete incentive structure. # Distributions (/learn/oip/reward-distribution) ## Protocol Balance and Funding The ar.io network maintains a protocol balance that funds all gateway and observer rewards. This balance is primarily funded through ArNS name purchases, ensuring sustainable network incentives aligned with usage. ## Epoch Allocation Each epoch, a portion of the protocol balance is earmarked for distribution as rewards. The protocol uses this allocation to reward functional gateways and observers. ### Funding Sources - **ArNS Name Purchases**: Primary funding mechanism - fees from ArNS name registrations and renewals - **Network Genesis Allocation**: Initial ARIO tokens allocated at network launch - **Undistributed Rewards**: Rewards not claimed due to poor performance roll forward to future epochs From this allocation, two distinct reward categories are derived: ## Base Rewards ### Base Gateway Reward (BGR) This is the portion of the epoch reward allocation distributed to functional gateways. ### Base Observer Reward (BOR) Observers, due to their additional responsibilities, have a separate reward category for successfully performing observation duties. ## Distribution Based on Performance The reward distribution is contingent on the performance classifications derived from the Performance Evaluation: - **Functional Gateways**: Gateways that meet the performance criteria receive the Base Gateway Reward. - **Deficient Gateways**: Gateways falling short in performance do not receive any gateway rewards. - **Functional Observers**: Observers that fulfilled their duty receive the Base Observer Reward. - **Deficient Observers**: Observers failing to meet their responsibilities do not receive observer rewards. If they are also functional gateways, their gateway reward can be reduced for that epoch as a consequence for not performing their observation duty. ![Epoch reward distributions showing eligible vs distributed ARIO tokens](/content/epoch-distributions.png) Epoch reward distributions showing the relationship between eligible rewards (total available) and distributed rewards (actually paid out) across epochs. The difference represents rewards not distributed due to gateway or observer deficiencies. ## Epoch Pipeline On Solana, reward distribution is driven by a permissionless 6-step [epoch pipeline](/learn/oip/epoch-pipeline) rather than a single atomic operation: 1. **create_epoch** — Initialize epoch, compute reward rate 2. **tally_weights** — Batched weight computation 3. **prescribe_epoch** — Select observers and prescribed names via weighted roulette 4. **save_observations** — Observers submit pass/fail reports 5. **distribute_epoch** — Batched reward distribution 6. **close_epoch** — Reclaim rent from completed epoch accounts All steps are permissionless and idempotent — anyone can crank them, and running them multiple times is safe. ## Operator Rewards Operator rewards always **auto-compound** into the operator's stake. There is no toggle — operators must call `decrease_operator_stake` to realize rewards as liquid tokens. ## Distribution to Delegates Delegate rewards use a **reward-per-share accumulator** pattern. Rather than distributing rewards to each delegate individually each epoch, the protocol tracks a cumulative reward-per-token on each gateway. Pending rewards are settled when the delegator interacts with the protocol (e.g., delegate more, withdraw, or claim rewards). Leaving gateways receive 0 rewards for any epoch in which they have initiated withdrawal. Delegate reward distribution considers the gateway's total reward, the gateway's delegate reward share setting, and each delegate's proportional stake. Delegated rewards are added to the delegate's existing stake for that gateway and can later be withdrawn subject to normal withdrawal rules. ## Undistributed Rewards In cases where rewards are not distributed, either due to the inactivity or deficiency of gateways or observers, the allocated tokens shall remain in the protocol balance and carry forward to the next epoch. This mechanism is in place to discourage observers from frivolously marking their peers as offline in hopes of attaining a higher portion of the reward pool. Note that if a gateway (and its delegates) leaves the network or a delegate fully withdraws stake from a gateway, they become ineligible to receive rewards within the corresponding epoch and the earmarked rewards will not be distributed. ## Handling Deficient Gateways To maintain network efficiency and reduce state bloat, gateways that remain deficient for a sustained period can be removed from the network. When this happens, their minimum network-join stake is slashed to the protocol balance, while eligible excess and delegated stake follow the standard withdrawal process. --- ## Next Steps Congratulations! You now understand the complete OIP system. Ready to learn more? - **Explore Gateways** → [Gateway Documentation](/learn/gateways/) for technical details - **Learn about ArNS** → [ArNS Documentation](/learn/arns/) for naming system details - **Back to Introduction** → [OIP Introduction](/learn/oip/) to review the basics # Staking (/learn/oip/staking) ## Overview Staking tokens within ar.io serves a dual primary purpose: it signifies a public commitment by gateway operators and qualifies them and their delegates for reward distributions. In the ar.io ecosystem, "staking" refers to the process of locking ARIO tokens into protocol-controlled vaults on Solana. This act signifies an opportunity cost for the staker, acting both as a motivator and a public pledge to uphold the network's collective interests. Once staked, tokens remain locked until the staker initiates an unstake or withdrawal action, or reaches the end of the vault's lock period. It is important to note that the ARIO Token is non-inflationary, distinguishing ar.io's staking mechanism from yield-generation tools found in other protocols. Staking in this context is about eligibility for potential rewards rather than direct token yield. By staking tokens, gateway operators (and their delegates) demonstrate their commitment to the network, thereby gaining eligibility for protocol-driven rewards and access to the network's shared resources. SOL is required for Solana transaction fees when staking, delegating, redelegating, or withdrawing ARIO. ## Gateway Staking A gateway operator must stake ARIO to join their gateway to the network. This makes them eligible for protocol rewards and promotes network reliability by giving operators an economic commitment to the network's performance. The minimum network-join stake is **20,000 ARIO**. The gateway registry can hold up to **3,000 gateways**; when the registry is full, new gateways cannot join until existing gateways leave. Operators may increase their stake above the minimum required to join the network. This additional amount is known as **excess stake**. A gateway's total stake impacts the following epoch once excess stake is added or removed. Operator rewards auto-compound into operator stake. To realize rewards, operators must withdraw excess stake. If an excess stake withdrawal is still within its withdrawal period, the operator can cancel it and immediately restake the amount to the original gateway. Operators cannot self-delegate. Operator stake and delegated stake are managed through separate mechanisms. ### Gateway Pruning Gateways that repeatedly fail observation can be pruned from the network. When a gateway is removed for sustained poor performance, its minimum operator stake is slashed to the protocol balance. Any excess operator stake and delegated stake enter the standard withdrawal process. ## Delegated Staking To promote participation from a wider audience, the network allows anyone with available ARIO tokens to partake in delegated staking. Users can choose to take part in the risk and rewards of gateway operations by staking their tokens with an active gateway (or multiple gateways) through an act known as delegating. Delegators can select which gateways to stake with in gateways.ar.io — maximize their potential rewards based on operator performance, stakes, and weights ### How Delegated Staking Works **Delegated staking allows you to participate in ar.io's reward system without running your own gateway.** By staking your ARIO tokens on existing gateways, you can earn rewards while supporting network infrastructure. When you delegate stake to a gateway, you're increasing that gateway's total stake. This can improve the gateway's chances of being selected as an observer, which means more potential rewards for both the gateway operator and you as a delegator. Delegate rewards use a **reward-per-share accumulator** pattern. Rather than distributing rewards to each delegate individually each epoch, the protocol tracks a cumulative reward-per-token on each gateway. Your pending rewards are settled when you interact with the protocol (e.g., delegate more, withdraw, or claim rewards). Gateway operators can choose whether to accept delegated staking and can configure their own delegation settings within protocol limits. They may also offer non-protocol incentives, such as faster access, privileged access, or other off-chain perks. Delegated staking has these protocol limits: - Global minimum delegated stake: **10 ARIO** - Gateway-selected delegate reward share: **0% to 95%** - Maximum unique delegates per gateway: **10,000** - Delegated stake withdrawal duration: **30 days** ### Benefits - **Potential Rewards**: Become eligible for rewards without running infrastructure - **Network Participation**: Support ar.io's growth - **Flexibility**: Redelegate to different gateways as conditions change - **Low Barrier to Entry**: No technical expertise required - **Transparent Rewards**: Clear visibility into reward distribution ### Getting Started **Get ARIO Tokens** You'll need ARIO tokens in a Solana wallet to delegate. See our comprehensive guide on [How to Get ARIO Tokens](/learn/token/get-the-token) for detailed information about acquiring tokens through exchanges, swaps, and network participation. **Choose a Gateway** Research gateways on the [Gateway Portal](https://gateways.ar.io/#/staking) to find one that matches your preferences for reward sharing and performance. Look for gateways with strong uptime, competitive reward sharing percentages, and reliable operation history. **Delegate Your Stake** Use the [Gateway Portal](https://gateways.ar.io/#/staking) to delegate your tokens. Connect your Solana wallet (Phantom, Solflare, or Backpack) and the process is straightforward. **Monitor Your Rewards** Track your delegation performance and rewards through the portal's dashboard. Your rewards accumulate via the reward-per-share accumulator and are settled when you interact with the protocol. ### Important Considerations - **Gateway Performance**: Your rewards depend on the gateway's performance and observer selection - **Reward Sharing**: Gateway operators set the percentage of rewards shared with delegators - **Delegation Settings**: Gateway operators can update delegation settings; changes take effect in the following epoch - **Redelegation**: You can move your stake between gateways as network conditions change - **Withdrawal Delays**: There is a 30 day standard delay when withdrawing your delegated stake - **Gateway Exit or Removal**: If a gateway leaves or is removed from the network, delegates must claim their stake through the protocol, subject to withdrawal delays ## Stake Redelegation This feature enables existing stakers to reallocate their staked tokens between gateways, known as redelegation. Both delegated stakers and gateway operators with excess stake (stake above the minimum network-join requirement) can take advantage of this feature. Redelegation is intended to offer users flexibility and the ability to respond to changing network conditions. To balance flexibility with network stability, users are entitled to one free redelegation during each seven-day window from their last redelegation action. Additional redelegations within that window incur increasing protocol fees, capped at 60%, and fees are sent to the protocol balance. All redelegations must respect minimum stake requirements at both the source and destination gateways. Gateway minimum network-join stakes are not eligible for redelegation, but operator excess stake and eligible withdrawal vaults may be redelegated. ## Redeeming Stake for ArNS Staked tokens generally have restricted liquidity to maintain a healthy degree of stability in the network. However, an exception allows delegated stakers and gateway operators to redeem eligible staked tokens to fund specific ArNS-related services. By leveraging their stake, participants can further engage with ArNS, strengthening the name system's utilization and impact across the network. Eligible ArNS interactions include: - Purchasing a name - Extending a lease - Upgrading a lease to a permabuy - Increasing undername capacity - Requesting or setting a primary name Each ArNS purchase is funded from a single source selected by the user: liquid balance, delegated stake, a delegated stake withdrawal vault, or eligible operator excess stake. If the selected source does not cover the full purchase, the transaction fails and no balances or stakes are modified. Using eligible staked tokens for ArNS interactions does not impose an additional fee, but source gateway minimum stake requirements still apply. ## Expedited Withdrawal Fees Gateway operators and delegated stakers can shorten the standard withdrawal delay period after initiating a withdrawal or being placed into an automatic withdrawal by protocol mechanisms. This action is subject to a dynamic fee paid to the protocol balance. At any point during the withdrawal delay, users can expedite access to pending withdrawal tokens by paying a fee based on how much sooner they want to receive their funds. Once triggered, the tokens are returned immediately to the user's wallet. A gateway's minimum operator stake is not withdrawable while the gateway is active; only excess stake can be placed into withdrawal during normal operation. If a gateway voluntarily leaves the network, the minimum stake becomes an exit vault and can follow the same expedited withdrawal flow. If a gateway is removed for sustained poor performance, the minimum stake is slashed and cannot be recovered. ## Explore Staking } /> } /> } /> } /> # Get the Token (/learn/token/get-the-token) ## Acquiring ARIO Tokens ARIO exists on multiple networks: - **Solana ARIO (canonical):** The native SPL Token that powers ar.io — staking, rewards, ArNS, and governance. - **Base ARIO (bridged):** A representation of Solana ARIO on the Base L2 for EVM-based liquidity, tools, and integrations. Choose between mainnet and testnet tokens based on your requirements. ## Mainnet Tokens #### Market Trackers - **[CoinGecko](https://www.coingecko.com/en/coins/ar-io-network)** - **[CoinMarketCap](https://coinmarketcap.com/currencies/ar-io-network/)** Pick the network you want to use: ### Token Details - **Mint Address:** `DcNnMuFxwhgV4WY1HVSaSEgr92bv2b1vUvEKiNxWqHdF` - **Token Standard:** SPL Token - **Decimals:** 6 - **Explorer:** [View on Solscan](https://solscan.io/token/DcNnMuFxwhgV4WY1HVSaSEgr92bv2b1vUvEKiNxWqHdF) --- ### Exchanges #### Decentralized Exchanges (DEXs) You can swap ARIO on Solana DEXs: - **[Jupiter](https://jup.ag)** — Solana's leading DEX aggregator - **[Raydium](https://raydium.io)** — AMM and liquidity on Solana When trading on a DEX, always confirm that the token mint address matches the official ARIO mint above. --- ### Wallets ARIO is a standard SPL Token — any Solana-compatible wallet can hold it: - **[Phantom](https://phantom.app)** — Most popular Solana wallet - **[Solflare](https://solflare.com)** — Full-featured Solana wallet with staking support - **[Backpack](https://backpack.app)** — Multi-chain wallet with Solana support Most Solana wallets auto-detect ARIO once you receive tokens. To add manually, import the mint address above. See our [Wallet Setup Guide](/learn/token/wallets) for detailed instructions on configuring your wallet for ARIO. ### Token Details - **Contract ID:** `0x138746adfA52909E5920def027f5a8dc1C7EfFb6` - **Explorer:** https://basescan.org/token/0x138746adfa52909e5920def027f5a8dc1c7effb6 --- You can obtain ARIO on Base by swapping on a DEX or by bridging it from Solana. ### Bridge Bridge from Solana to Base (and vice versa) via the **[ARIO Bridge](https://swap.ar.io)**. ### Decentralized Exchanges (DEXs) Acquire Base ARIO on Base-native DEXs: - **[Hydrex](https://www.hydrex.fi/swap?tokenIn=0x833589fcd6edb6e08f4c7c32d4f71b54bda02913&tokenOut=0x138746adfa52909e5920def027f5a8dc1c7effb6)** - **[Coinbase DEX](https://www.coinbase.com/dex)** (via Coinbase mobile app or Base integration) Important: Always verify the contract address before swapping. DEXs are permissionless, and imposters can appear. ### Wallets Use your favorite Base network-enabled EVM wallet to interact with ARIO on Base. --- ### Network Participation ARIO on Solana can be earned by supporting the network through direct operation or delegated staking. **Note:** Only ARIO on Solana can be used for joining a gateway to the network and delegated staking. SOL is also required for Solana transaction fees. #### Gateway Operation Operate an ar.io gateway and earn protocol rewards: 1. **Deploy a Gateway:** Follow open-source setup instructions. 2. **Stake ARIO:** Meet the minimum stake requirement (20,000 ARIO). 3. **Serve the Network:** Provide reliable data retrieval, caching, and indexing services. 4. **Earn Rewards:** Receive ARIO for contributing to network performance and uptime. #### Delegated Staking Support a gateway operator without running infrastructure: 1. **Select a Gateway:** Use the Gateway Address Registry (GAR) to find a trusted operator. 2. **Delegate Tokens:** Stake your ARIO with them to increase their total stake. 3. **Earn Rewards:** Share in their rewards proportionally to your delegation. 4. **Unstake Anytime:** Withdraw after the standard protocol delay period. --- ### Ecosystem and Community Programs Earn ARIO through ecosystem initiatives that reward contribution and participation. #### Grants and Bounties The **ar.io Foundation** funds and supports community builders through: - **Developer Grants:** Build tools, integrations, and applications for the network. - **Bug Bounties:** Help improve network security by identifying vulnerabilities. - **Community Programs:** Contribute to education, content creation, and outreach. #### Ecosystem Growth From app development to creative projects, contributors across the permaweb can earn ARIO for helping expand and improve the network ecosystem. ## Testnet Tokens For building and testing, ar.io runs a free **[Testnet Sandbox](/build/testnet)** on Solana devnet using a **staging ARIO** token — no mainnet, no real money. - **Staging ARIO** is dispensed by the **[ar.io faucet](https://faucet.services.ar-io.dev)** (GitHub sign-in, 2,500 ARIO per claim) on Solana devnet — mint `6vTw5CysRXQ4ybbHkDUiisHWVsBeMtUzYvJqs2iqHyaN`. - You'll also need a little **devnet SOL** for transaction fees, from [faucet.solana.com](https://faucet.solana.com) or `solana airdrop 1 --url devnet`. Use staging ARIO to pay for uploads and buy devnet ArNS names in the sandbox. See **[Funds & Faucet](/build/testnet/funds-and-faucet)** to get started. --- ## Summary - **ARIO powers ar.io** — enabling staking, naming, and incentives. - **Get ARIO** on DEXs or Bridging. - **Earn ARIO** by running gateways, delegating, or contributing to ecosystem growth. # Token (/learn/token) ## Overview ARIO is the native token of the [ar.io network](https://ar.io), implemented as an [SPL Token](https://spl.solana.com/token) on Solana. It powers the network's decentralized gateway infrastructure, ArNS naming system, and incentive mechanisms. The token uses 6 decimal places (1 ARIO = 1,000,000 mARIO). ## Mainnet Addresses Use these canonical Solana mainnet addresses when verifying ARIO in wallets, explorers, integrations, or gateway configuration. ### Token | Item | Address | | --- | --- | | ARIO SPL Token mint | `DcNnMuFxwhgV4WY1HVSaSEgr92bv2b1vUvEKiNxWqHdF` | ### ar.io Programs | Program | Program ID | | --- | --- | | `ario-core` | `73YoECm6NKXpVRoe5f1Q9BcP5DJGPFUjnFy6AxBE5Nvh` | | `ario-gar` | `89fNiiwgpFSPHKuqfNUkgYTYjtAJAhyqHjXmgXeppGpf` | | `ario-arns` | `2yCUx5edFvUrkibYaUa2ZXWyx9kuJkS8CwyzsgHPWdZZ` | | `ario-ant` | `2MWexMHfMhGJwMHv9Qm9YAVCqjUFUJwDJAysW4oCUGk5` | | `ario-ant-escrow` | `5HZhe9UqKL5zAsdz81nuuaxV41h8bFhudzxxBigAQndM` | ## How ARIO Is Used - **Gateway participation**: Gateway operators stake ARIO to join the network and become eligible for protocol rewards. - **Delegated staking**: Token holders can delegate ARIO to gateway operators and participate in network incentives without running infrastructure. - **ArNS names**: ARIO is used for ArNS name registration, lease extensions, undername capacity, and related naming operations. - **Protocol incentives**: ARIO rewards help coordinate gateway performance, observation, and network reliability. ## Solana Compatibility As an SPL Token, ARIO works with Solana-compatible wallets and applications. This makes it easier to hold, transfer, swap, and integrate ARIO across the Solana ecosystem while the ar.io protocol continues to serve Arweave's permanent data layer. Common places users interact with ARIO include: - Solana wallets such as Phantom, Solflare, and Backpack - Solana exchanges and liquidity tools such as Jupiter and Raydium - ar.io applications such as the Gateway Portal and ArNS app ## Network Incentives ARIO is non-inflationary. Staking is not a generic yield product; it is a protocol mechanism for aligning gateway operators, delegators, and users around reliable permanent cloud infrastructure. For the deeper mechanics, see [Staking & Delegation](/learn/oip/staking), [Protocol Architecture](/learn/protocol-architecture), and the [Observation and Incentive Protocol](/learn/oip). ## Explore the Token } /> } /> } /> } /> # Wallet Setup (/learn/token/wallets) ## Adding ARIO to Your Wallet ARIO is a standard SPL Token on Solana. Most Solana wallets auto-detect ARIO once you receive tokens, but you can also add it manually using the mint address. ## Supported Wallets ARIO tokens are support by any Solana-compatible wallet - below are some examples: ### Phantom [Phantom](https://phantom.app) is the most popular Solana wallet, available as a browser extension and mobile app. ### Install Phantom Download from [phantom.app](https://phantom.app) for your browser or mobile device. Create a new wallet or import an existing one. ### Receive or Add ARIO ARIO will appear automatically once you receive tokens. To add manually: 1. Open your Phantom wallet 2. Click **"Manage Token List"** or search for tokens 3. Search for **"ARIO"** or paste the mint address: `DcNnMuFxwhgV4WY1HVSaSEgr92bv2b1vUvEKiNxWqHdF` 4. Toggle the token on ### Verify Your Balance Your ARIO balance appears alongside your other Solana tokens. You can send, receive, and swap ARIO directly within Phantom. ### Solflare [Solflare](https://solflare.com) is a full-featured Solana wallet. ### Install Solflare Download from [solflare.com](https://solflare.com) as a browser extension, web app, or mobile app. ### Add ARIO Token 1. Navigate to your token list 2. Search for **"ARIO"** or use the mint address: `DcNnMuFxwhgV4WY1HVSaSEgr92bv2b1vUvEKiNxWqHdF` 3. Add the token to your portfolio ## Next Steps 1. **Stake ARIO**: Delegate your tokens to gateway operators at [gateways.ar.io](https://gateways.ar.io/#/staking) 2. **Buy an ArNS Name**: Register a permanent name at [arns.ar.io](https://arns.ar.io) 3. **Join the Network**: [Run a gateway](/build/run-a-gateway) and earn protocol rewards # Claims, Proofs, and Accountability (/learn/verification/claims-proofs-accountability) Not every verification signal has the same meaning. Ar.io uses claims, signed claims, cryptographic proofs, and economic accountability together because each one catches a different failure mode. ## Quick Reference | Mechanism | Type | Meaning | | --- | --- | --- | | `X-AR-IO-Verified` | Claim | The gateway says its local cache has verified the data. | | RFC 9421 `Signature` | Signed claim | The gateway is accountable for the response headers it signed. | | `Content-Digest` | Verifiable digest | The client can hash the response body and compare it to the header. | | Arweave signature verification | Proof | The client verifies the bytes match the signed commitment for a known ID. | | Observer reports and slashing | Economic accountability | Persistent network-visible failure can affect rewards or stake. | ## Claims A claim is a statement from a gateway. For example, `X-AR-IO-Verified: true` says the gateway believes the returned data was verified in its local system. That is useful operational metadata, but it is not a cryptographic proof. A malicious gateway controls its own software and database. It can set false headers. ## Signed Claims Signed response headers make claims attributable. If a gateway signs `X-AR-IO-Verified`, `X-AR-IO-Stable`, ArNS resolution headers, or a `Content-Digest`, the gateway cannot later deny that it made those claims. Signed claims are evidence. They are not automatically truth. A false signed claim is still possible, but it becomes attributable to a signing key bound to a gateway operator identity. ## Proofs A proof is something the client can verify independently. For known transaction or data item IDs, the strongest path is Arweave signature verification: 1. Fetch the data and the signed metadata needed for verification. 2. Recompute the data root or ANS-104 deep hash. 3. Verify the transaction or data item signature. 4. Confirm that `SHA-256(signature)` matches the requested ID. If this chain verifies, the bytes match the commitment for that ID. The result does not depend on trusting the gateway's local verification header. ## Accountability Network accountability is handled through OIP. Observers sample gateway behavior, upload detailed reports to Arweave, and submit compact results to the protocol. Gateways that perform poorly can miss rewards and, after sustained failure, become eligible for pruning. OIP is intentionally not duplicated here. For mechanics, see: - [Observation & Incentive Protocol](/learn/oip) - [Observer Selection](/learn/oip/observer-selection) - [Reporting](/learn/oip/reporting) - [Performance Evaluation](/learn/oip/performance-evaluation) - [Pruning](/learn/oip/pruning) ## How They Compose These mechanisms are not redundant: - Gateway local verification catches cache corruption and stale imports. - Signed claims create portable evidence about what a gateway said. - Client-side verification detects bad data at request time. - OIP handles long-running public behavior across the network. For high-integrity applications, use client-side verification for request-time correctness and signed claims or observer reports for later accountability. # Client-Side Verification (/learn/verification/client-side-verification) Client-side verification lets applications decide how much trust they place in a gateway response. In ar.io, this is primarily exposed through Wayfinder verification strategies. This page focuses on the verification model. For integration examples and SDK usage, see [Wayfinder](/learn/wayfinder). ## Verification Strategies | Strategy | What it checks | Trust profile | | --- | --- | --- | | Remote verification | Accepts the serving gateway's verification claim. | Trusts the serving gateway. | | Hash verification | Hashes the received data and compares it to hashes from reference gateways. | Trusts the reference set. | | Data root verification | Computes the Merkle data root and compares it to a known or referenced root. | Stronger for L1 transactions when root data is available. | | Signature verification | Reconstructs the signed commitment and verifies the transaction or data item signature. | Strongest path for known IDs. | ## Known IDs For requests by known transaction ID or data item ID, signature verification can establish data integrity without trusting the serving gateway. For ANS-104 data items, the verifier reconstructs the deep hash over the item fields and data, verifies the signature, and confirms that the item ID is derived from that signature. For Arweave L1 transactions, the verifier computes the data root from the served bytes, verifies the transaction signature over the committed fields, and confirms that the transaction ID is derived from the signature. ## ArNS Names Name resolution adds a separate question: did the name resolve to the correct ID? After a name resolves to a transaction ID, the data can be verified against that ID. But content integrity does not by itself prove that the gateway resolved the ArNS name correctly or freshly. Applications that need stronger ArNS guarantees can compare resolution across gateways, check protocol state directly, or require signed ArNS resolution headers from gateways that support response signing. ## Failure Handling Applications need to choose whether verification failures are fail-open or fail-closed: - **Fail-closed**: reject the response and surface an error. - **Fail-open**: return the response but emit a warning or telemetry event. High-integrity workflows should prefer fail-closed behavior. General browsing experiences may choose fail-open behavior for compatibility and performance. Verification is a policy choice. Ar.io provides the mechanisms, but applications decide the verification strategy, reference set, and failure behavior that match their risk model. ## Related Docs - [Wayfinder Protocol](/learn/wayfinder) - [Wayfinder Integration](/learn/wayfinder/integration) - [Gateway Data Verification](/learn/gateways/data-verification) - [Signed Gateway Claims](/learn/verification/signed-gateway-claims) # Verification and Accountability (/learn/verification) Arweave stores data permanently. Ar.io focuses on the delivery layer: retrieving that data, serving it through independent gateways, and giving clients ways to detect or hold gateways accountable when delivery is wrong. The key distinction is that **storage integrity** and **delivery integrity** are different problems. Arweave can prove that data was committed to the storage layer. A client still needs to know whether a gateway served the right bytes, made honest claims about those bytes, and resolved mutable names correctly. ## The Five Layers | Layer | What it does | Where to learn more | | --- | --- | --- | | Arweave base commitments | Bind transaction and data item IDs to signed data commitments. | [What is Arweave?](/learn/what-is-arweave) | | Gateway local verification | Recompute data roots, verify bundles, and re-import corrupted data. | [Gateway data verification](/learn/gateways/data-verification) | | Signed gateway claims | Sign trust-relevant response headers so gateway claims are attributable. | [Signed Gateway Claims](/learn/verification/signed-gateway-claims) | | Client-side verification | Let applications verify responses instead of trusting one gateway. | [Client-Side Verification](/learn/verification/client-side-verification) | | Network observation and incentives | Sample gateway behavior, record reports, and apply economic consequences. | [Observation & Incentive Protocol](/learn/oip) | ## The Core Property When a client knows the transaction ID or data item ID it wants, the strongest verification path is cryptographic: verify the signature chain from the served bytes back to the requested identifier. For Arweave L1 transactions, the identifier is derived from the transaction signature, and the signature commits to the transaction data root. For ANS-104 data items, the item ID is derived from the data item signature, and the signature commits to the item fields and data. This means a gateway cannot fabricate different valid bytes for a known ID without the original signer's private key. Other parts of the ar.io architecture make that property easier to use in production. ## What This Does Not Prove Verification is powerful, but the limits matter: - A signed gateway response proves that the gateway made a claim, not that the claim was true. - There is no universal proof that a specific client received a complete response body. - Client-side verification is opt-in and can be configured fail-open or fail-closed. - Observer-based accountability is sampling-based and is designed for persistent public misbehavior, not every targeted attack. - ArNS name correctness is separate from content integrity. A client can verify bytes for a transaction ID while still needing to verify that a name resolved to the right ID. ## Explore } /> } /> } /> } /> } /> # Signed Gateway Claims (/learn/verification/signed-gateway-claims) Gateway verification headers are useful, but unsigned headers can be modified, stripped, or fabricated. Ar.io gateways can opt in to signing trust-relevant response headers with [RFC 9421 HTTP Message Signatures](https://www.rfc-editor.org/rfc/rfc9421.html). ## What Gets Signed A gateway signs the trust-relevant headers that are present on a response. These can include: | Category | Examples | | --- | --- | | Data identity and status | `x-ar-io-data-id`, `x-ar-io-verified`, `x-ar-io-stable`, `x-ar-io-trusted` | | Body integrity | `content-type`, `content-digest` | | ArNS resolution | `x-arns-name`, `x-arns-resolved-id`, `x-arns-ttl-seconds`, `x-arns-process-id` | | Bundled data | `x-ar-io-root-transaction-id` | When gateways chain requests through one another, each gateway signs its own assessment. Upstream signature headers are stripped before the serving gateway signs its response. ## Signing Keys and Operator Identity The response-signing key is a hot Ed25519 key used for per-response signatures. It is separate from the operator wallet. The operator wallet signs an attestation document that binds the Ed25519 response-signing key to the operator's gateway identity. This creates a chain: 1. The gateway signs response headers with Ed25519. 2. The Ed25519 public key is attested by the operator wallet. 3. The operator identity is connected to network state through the Gateway Address Registry. That chain makes signed claims attributable to a gateway operator for the time period in which the attestation and registry state are valid. ## Body Integrity HTTP signatures sign headers, not necessarily the full response body. Body integrity depends on whether a signed `Content-Digest` is present. The architecture is designed around a performance tradeoff: - Cached responses, `HEAD` requests, and buffered responses under the gateway's body-digest threshold can include `Content-Digest`. - Large streamed responses may omit body binding to preserve streaming performance. - When no body digest is signed, clients that need integrity should use client-side Arweave verification. Signed headers provide non-repudiation for gateway claims. They do not by themselves prove that the response body was correct. ## Limitations - A malicious gateway can still sign a false claim. - Missing signatures should be treated as unsigned mode, not as proof of failure. - If a response-signing key is compromised, historical signatures from that key remain cryptographically valid until clients apply rotation or revocation policy. - Partial content responses need special care because signing the data ID does not necessarily bind a specific byte range. ## Related Docs - [Claims, Proofs, and Accountability](/learn/verification/claims-proofs-accountability) - [Gateway Registry](/learn/gateways/gateway-registry) - [Client-Side Verification](/learn/verification/client-side-verification) # Wayfinder Protocol (/learn/wayfinder) ## The Problem: Centralized Gateway Reliance Today, most Arweave content is accessed through a single gateway: `arweave.net`. This creates a critical centralization risk: - **Single point of failure** - If arweave.net goes down, content becomes inaccessible - **Censorship vulnerability** - A single gateway can block or filter content - **Performance bottlenecks** - All traffic flows through one gateway - **No content verification** - Users must trust the gateway to serve authentic content ## What is Wayfinder? The Wayfinder protocol solves these problems by enabling **decentralized access** to Arweave content through any gateway in the ar.io network. It's a [URI scheme](https://wikipedia.org/wiki/Uniform_Resource_Identifier) that transforms centralized URLs like `https://arweave.net/txid` into decentralized `ar://` URLs that can be resolved by any participating gateway. Key capabilities: - **Multi-gateway routing** - Access content through any ar.io gateway - **Built-in verification** - Verify content authenticity regardless of which gateway serves it - **Automatic failover** - If one gateway is down, requests route to another - **User control** - Choose routing strategies based on speed, trust, or randomization ## How Wayfinder Works The Wayfinder protocol consists of three core components that work together to resolve and serve Arweave content: ```mermaid graph TB subgraph "Centralized Access" U1[Users] --> AW[arweave.net] AW --> AR1[Arweave] style AW fill:#ffcccc,stroke:#ff0000,stroke-width:3px,color:#333 style U1 fill:#fff,stroke:#333,stroke-width:2px,color:#333 style AR1 fill:#fff,stroke:#333,stroke-width:2px,color:#333 end ``` vs. ```mermaid graph TB subgraph "Decentralized Access" U2[Users] --> WF[ar:// Protocol] WF --> G1[Gateway 1] WF --> G2[Gateway 2] WF --> G3[Gateway 3] WF --> GN[Gateway N] G1 --> AR2[Arweave] G2 --> AR2 G3 --> AR2 GN --> AR2 end style U2 fill:#fff,stroke:#333,stroke-width:2px,color:#333 style AR2 fill:#fff,stroke:#333,stroke-width:2px,color:#333 style WF fill:#b3d9ff,stroke:#333,stroke-width:3px,color:#333 style G1 fill:#b3d9ff,stroke:#333,stroke-width:2px,color:#333 style G2 fill:#b3d9ff,stroke:#333,stroke-width:2px,color:#333 style G3 fill:#b3d9ff,stroke:#333,stroke-width:2px,color:#333 style GN fill:#b3d9ff,stroke:#333,stroke-width:2px,color:#333 ``` Wayfinder enables: 1. **Decentralized Routing**: Select from multiple gateways instead of relying on arweave.net 2. **Redundant Retrieval**: If one gateway fails, automatically failover to another 3. **Trust-minimized Verification**: Verify content authenticity regardless of which gateway serves it ### Transaction ID Resolution To access content tied to an Arweave Transaction ID (TxId), simply append the TxId to `ar://`: ``` ar://qI19W6spw-kzOGl4qUMNp2gwFH2EBfDXOFsjkcNyK9A ``` Inputting this into a WayFinder-equipped browser will route your request through the right ar.io Gateway, translating it as per your `Routing Method` settings. ### ArNS Name Resolution Fetching content via an Ar.io Name System (ArNS) name is straightforward. Attach the ArNS name to `ar://`: ``` ar://good-morning ``` The Wayfinder protocol, along with the Wayfinder app, distinguishes between TxIDs and ArNS names. Once the suitable `https://` request is formulated, the chosen gateway resolves the ArNS name through the current ar.io network state. ## Detailed Flow ```mermaid sequenceDiagram participant User participant Wayfinder participant Gateway as ar.io Gateway participant Arweave User->>Wayfinder: ar://ardrive or ar://txid Wayfinder->>Wayfinder: Select a gateway alt ArNS Name (ar://ardrive) Wayfinder->>Gateway: Request content via ArNS name Gateway->>Gateway: Resolve ArNS to TxID Gateway->>Gateway: Check cache for content alt Content not cached Gateway->>Arweave: Fetch from network Arweave-->>Gateway: Return content end Gateway-->>Wayfinder: Return content else Transaction ID (ar://txid) Wayfinder->>Gateway: Request content via TxID Gateway->>Gateway: Check cache for content alt Content not cached Gateway->>Arweave: Fetch from network Arweave-->>Gateway: Return content end Gateway-->>Wayfinder: Return content end Wayfinder->>Wayfinder: Verify content integrity Wayfinder-->>User: Deliver verified content ``` ## Why Decentralized Access Matters ### Resilience Against Censorship With centralized gateways like arweave.net, content can be blocked or filtered at a single point. Wayfinder distributes access across multiple independent gateways, making censorship significantly more difficult. ### Always-Available Content When arweave.net experiences downtime or congestion, all content becomes inaccessible. Wayfinder automatically routes around failed gateways, ensuring your content remains available. ### Trust Through Verification Centralized gateways require blind trust - you can't verify if the content served matches what's stored on Arweave. Wayfinder includes built-in verification capabilities, allowing clients to cryptographically verify content authenticity from any gateway. ### Performance Through Competition Multiple gateways create a competitive ecosystem where gateways optimize for speed and reliability. Users benefit from automatic routing to the fastest available gateway. ## Verification: Trust but Verify Wayfinder supports content verification at multiple levels: 1. **Transaction verification** - Verify that content matches the requested transaction ID 2. **Data integrity checks** - Ensure content hasn't been tampered with during transmission 3. **Manifest validation** - For bundled content, verify all components are authentic 4. **ArNS resolution verification** - Confirm ArNS names resolve to the correct transaction IDs This verification happens transparently, giving users confidence that they're receiving authentic Arweave content regardless of which gateway serves it. For the trust model behind these strategies, including the difference between gateway claims, signed claims, and cryptographic proofs, see [Verification and Accountability](/learn/verification). ## Explore Wayfinder } /> } /> } /> } /> # Integration (/learn/wayfinder/integration) ## Getting Started **Get the Extension** The easiest way to use Wayfinder is the [Wayfinder Extension](https://chromewebstore.google.com/detail/ario-wayfinder/hnhmeknhajanolcoihhkkaaimapnmgil?hl=en-US), available in the Chrome Web Store. ### Wayfinder Extension The wayfinder-extension is a simple Chrome extension that supports the ar:// routing protocol and allows you to: - **Navigate ar:// URLs directly** in your browser - **Configure routing strategies** - Choose how requests are routed to gateways - **Set verification preferences** - Control content verification levels - **Monitor gateway performance** - See which gateways are serving your requests No coding required - just install the extension and start browsing ar:// URLs! ## Developer Integration Options For developers who want to integrate Wayfinder into their applications: ### Wayfinder Core The [wayfinder-core](/sdks/wayfinder/wayfinder-core) library is the core protocol implementation that accepts various configuration options for setting up Wayfinder. It provides: - **Gateway selection strategies** - Choose how to route requests - **Content verification** - Optionally verify content authenticity - **Telemetry collection** - Understand gateway performance across routing strategies - **Custom configurations** - Fine-tune behavior for your use case ### Wayfinder React Web developers will likely be interested in [wayfinder-react](/sdks/wayfinder/wayfinder-react), which provides: - **React Context Provider** - Easy integration with React apps - **Custom hooks** - Simplified data fetching and state management - **Component library** - Pre-built UI components for common patterns - **TypeScript support** - Full type safety out of the box ### Wayfinder Router For teams who want a **ready-to-deploy proxy service**, [Wayfinder Router](/build/run-wayfinder-router) provides: - **Single trusted endpoint** - Give your users one reliable URL - **Server-side verification** - All content verified before serving - **Built-in caching** - LRU cache with optional disk persistence - **Admin dashboard** - Web UI for monitoring and moderation - **Standalone binary** - No runtime dependencies, runs anywhere This is ideal when you want infrastructure-level Wayfinder without embedding the SDK in your application. ## Common Integration Pattern: Preferred with Fallback For most builds, teams use the "preferred with fallback" pattern. This routing strategy prioritizes your preferred gateway but automatically falls back to other gateways in the network if needed: ```typescript const wayfinder = new createWayfinderClient({ ario: ARIO.mainnet(), verification: 'hash', routing: 'preferred', preferredGateway: 'ar-io.net', }); ``` This pattern ensures: 1. **Primary traffic** goes to your preferred gateway (for example, one you run or trust) 2. **Automatic failover** if your gateway can't serve the data 3. **Network resilience** by finding another gateway that can serve the content For detailed routing strategy options, see the [routing strategies documentation](/sdks/wayfinder/wayfinder-core/routing-strategies). ## Verification: Optional but Encouraged While verification is optional, it's strongly encouraged when fetching from gateways: ```typescript const wayfinder = createWayfinderClient({ //...other settings, verification: 'hash', // hash based verification }); ``` Verification ensures you're receiving authentic content regardless of which gateway serves it. ## Telemetry for Performance Insights Enable telemetry to understand how Wayfinder performs across gateways: ```typescript const wayfinder = createWayfinderClient({ //...other settings, telemetry: { enabled: true, sampleRate: 0.1, // sample 10% of requests apiKey: 'your-api-key', // optional clientName: 'my-app', clientVersion: '1.0.0' } }); ``` This helps teams make data-driven decisions about gateway selection and optimization. ## React Integration Example Here's a complete example using wayfinder-react: ```tsx // Configure Wayfinder const wayfinderConfig = { gatewaysProvider: new NetworkGatewaysProvider({ ario: ARIO.mainnet(), }), routingSettings: { // use the fastest pinging strategy to select the fastest gateway for requests strategy: new FastestPingRoutingStrategy({ timeoutMs: 1000, }), } verificationSettings: { enabled: false } }; // Wrap your app function App() { return ( ); } // Use in components function WayfinderImage({ txId }: { txId: string }) { const { resolvedUrl, isLoading, error } = useWayfinderUrl({ txId }); if (error) { return Error resolving URL: {error.message}; } if (isLoading) { return Resolving URL...; } return ( ); } ``` ## Next Steps } /> } /> } /> } /> # Use Cases (/learn/wayfinder/use-cases) ## Decentralized Web Hosting with Flexible Access With Wayfinder, not only can websites be hosted on the Arweave network, but their accessibility is also enhanced. By using the Wayfinder Protocol, web developers can ensure that if a specific ar.io Gateway is down, the content can still be accessed through another gateway, offering a more reliable and resilient user experience. This is valuable for: - **Personal websites** that need to remain accessible - **Documentation sites** that must be always available - **Portfolio sites** for professionals and creators ## Digital Archives and Preservation Digitally archiving public domain works becomes easier to share with Wayfinder. Historical institutions, researchers, and enthusiasts can link directly to permanent documents or media without depending on a single gateway URL. Unlike hardcoded links which might break if a specific gateway goes offline, Wayfinder ensures that the content remains consistently accessible. This is ideal for: - **Historical documents** and public domain works - **Academic research** and scholarly articles - **Cultural preservation** projects - **Legal documents** that need permanent access ## Media Sharing Platforms with Consistent Content Delivery For platforms hosting user-generated content, the Wayfinder Protocol provides not just decentralized hosting but also a guarantee of content delivery. Even if a content piece becomes viral and one gateway gets congested, Wayfinder ensures that users can still access the content through another gateway, providing a seamless experience. Perfect for: - **Social media platforms** with user-generated content - **Video sharing sites** with viral content - **Image galleries** and art platforms - **Podcast hosting** and audio content ## Decentralized Applications (DApps) with Reliable Front-End Accessibility DApps, while benefiting from Arweave's permanent hosting, can further ensure their front-end remains consistently accessible to users by using Wayfinder. If a DApp's front-end is accessed frequently, causing strain on one gateway, Wayfinder can help ensure the load is distributed, and the DApp remains online and functional. This is essential for: - **DeFi applications** that need high availability - **NFT marketplaces** with high traffic - **Gaming platforms** with real-time requirements - **Collaborative tools** and productivity apps ## Branded Content Access Companies and individuals can brand their permaweb content, making it accessible through their domain, enhancing brand visibility and user trust. This is achieved through DNS TXT records that link domain names to Arweave content. ## Dynamic Content Updates Domain owners can easily update what Permaweb content their `ar://` URL resolves to, which is ideal for frequently updated resources like documents, blogs, and application interfaces. ## Educational and Informational Resources Educational institutions and information providers can make their resources permanently available on the permaweb, accessible through simple, memorable URLs. ## Next Steps Ready to get started with Wayfinder? Explore [Integration Methods](/learn/wayfinder/integration) to see how to implement Wayfinder in your applications, or go back to the [Overview](/learn/wayfinder) to review the basics. # Husky (Developers Only) (/sdks/(clis)/ardrive-cli/(build-and-run-from-source)/husky-developers-only) We use husky 6.x to manage the git commit hooks that help to improve the quality of our commits. Please run: ```shell yarn husky install ``` to enable git hooks for your local checkout. Without doing so, you risk committing non-compliant code to the repository. # Install Yarn 3 (/sdks/(clis)/ardrive-cli/(build-and-run-from-source)/install-yarn-3) Both the ArDrive CLI and ArDrive Core JS use Yarn 3 to manage dependencies and initiate workflows, so follow the [yarn installation instructions][yarn-install] in order to get the latest version. In most cases: ```shell brew install yarn npm install -g yarn ``` # Installing and Starting the CLI From Source (/sdks/(clis)/ardrive-cli/(build-and-run-from-source)/installing-and-starting-the-cli-from-source) Now that your runtime and/or development environment is set up, to install the package simply run: ```shell yarn && yarn build ``` And then start the CLI (always from the root of this repository): ```shell yarn ardrive ``` For convenience in the **non-developer case**, you can install the CLI globally on your system by performing the following step: ```shell yarn pack npm install i -g /path/to/package.tgz ardrive ``` # Recommended Visual Studio Code extensions (Developers Only) (/sdks/(clis)/ardrive-cli/(build-and-run-from-source)/recommended-visual-studio-code-extensions-developers-only) To ensure your environment is compatible, we also recommend the following VSCode extensions: - [ES-Lint][eslint-vscode] - [Editor-Config][editor-config-vscode] - [Prettier][prettier-vscode] - [ZipFS][zipfs-vscode] # Using a custom ArDrive-Core-JS (Optional) (/sdks/(clis)/ardrive-cli/(build-and-run-from-source)/using-a-custom-ardrive-core-js-optional) To test a with a custom version of the `ardrive-core-js` library on your local system, change the `"ardrive-core-js"` line in `package.json` to the root of your local `ardrive-core-js` repo: ```diff - "ardrive-core-js": "1.0.0" + "ardrive-core-js": "../ardrive-core-js/" ``` # Dealing With Network Congestion (/sdks/(clis)/ardrive-cli/(other-utility-operations)/dealing-with-network-congestion) Currently, Arweave blocks hold up to 1000 transactions per block. The "mempool", where pending transactions reside until they've been included into a block, will only hold a transaction for 50 blocks (~100-150 minutes) before it's discarded by the network resulting in no fees or data being transacted. During periods of network congestion (i.e. those where the mempool contains 1000 or more pending transactions), it may make sense to either: a) wait for congestion to dissipate before attempting your transactions. b) apply the fee boost multiplier to your transactions rewards with the --boost parameter during write operations in order to front-run some of the congestion. #### Check for network congestion before uploading ```shell ardrive get-mempool ardrive get-mempool | jq 'length' ``` #### Front-run Congestion By Boosting Miner Rewards ```shell ardrive upload-file --wallet-file /path/to/my/wallet.json --parent-folder-id "f0c58c11-430c-4383-8e54-4d864cc7e927" --local-path ./helloworld.txt --boost 1.5 ``` #### Send AR Transactions From a Cold Wallet The best cold wallet storage never exposes your seed phrase and/or private keys to the Internet or a compromised system interface. You can use the ArDrive CLI to facilitate cold storage and transfer of AR. If you need a new cold AR wallet, generate one from an air-gapped machine capable of running the ArDrive CLI by following the instructions in the [Wallet Operations](#wallet-operations) section. Fund your cold wallet from whatever external sources you'd like. NOTE: Your cold wallet won't appear on chain until it has received AR. The workflow to send the AR out from your cold wallet requires you to generate a signed transaction with your cold wallet on your air-gapped machine via the ArDrive CLI, and then to transfer the signed transaction (e.g. by a file on a clean thumb drive) to an Internet-connected machine and send the transaction to the network via the ArDrive CLI. You'll need two inputs from the Internet-connected machine: - the last transaction sent OUT from the cold wallet (or an empty string if none has ever been sent out) - the base fee for an Arweave transaction (i.e. a zero bye transaction). Note that this value could change if a sufficient amount of time passes between the time you fetch this value, create the transaction, and send the transaction. To get the last transaction sent from your cold wallet, use the `last-tx` command and specify your wallet address e.g.: ``` ardrive last-tx -a \ ``` To get the base transaction reward required for an AR transaction, use the `base-reward` function, optionally applying a reward boost multiple if you're looking to front-run network congestion: ``` ardrive base-reward --boost 1.5 ``` Write down or securely copy the values you derived from the Internet-connected machine and run the following commands on the airgapped machine, piping the outputted signed transaction data to a file in the process, e.g. `sendme.json` (if that's your signed transaction transfer medium preference): ``` ardrive create-tx -w /path/to/wallet/file.json -d \ -a \ --last-tx \ --reward "\" > sendme.json ``` Transport your signed transaction to the Internet-connected machine and run the following command to send your transaction to the Arweave network: ``` ardrive send-tx -x /path/to/sendme.json ``` # Monitoring Transactions (/sdks/(clis)/ardrive-cli/(other-utility-operations)/monitoring-transactions) Block time on Arweave is typically between 2-3 minutes in duration, so transactions can be mined within that time frame when [network congestion](#dealing-with-network-congestion) is low. Transactions, in the general case, proceed through the following set of states: - Pending: the transaction is waiting the "mempool" to be mined - Confirming: the transaction was mined on an Arweave Node, but has not yet been confirmed by at least 15 total nodes on the network - Confirmed: the transaction was mined on an Arweave Node and confirmed by at least 15 total nodes on the network - Not Found: the transaction is not available for any of the following reasons: - Insufficient reward to join the mempool - Insufficient reward to be mined within 50 blocks during a period of network congestion - Transaction is transitioning between states - Transaction ID is invalid Monitor any Arweave transaction's status via its transaction ID by performing: ```shell ardrive tx-status -t "ekSMckikdRJ8RGIkFa-X3xq3427tvM7J9adv8HP3Bzs" ``` Example output: ```shell ekSMckikdRJ8RGIkFa-X3xq3427tvM7J9adv8HP3Bzs: Mined at block height 775810 with 22439 confirmations ``` ```shell watch -n 10 ardrive tx-status -t "ekSMckikdRJ8RGIkFa-X3xq3427tvM7J9adv8HP3Bzs" ``` # Persistent Caching of ArFS Entity Metadata (/sdks/(clis)/ardrive-cli/(other-utility-operations)/persistent-caching-of-arfs-entity-metadata) To avoid redundant requests to the Arweave network for immutable ArFS entity metadata, a persistent file cache is created and maintained at: ``` Windows: /ardrive-caches/metadata Non-Windows: /.ardrive/caches/metadata ``` The `XDG_CACHE_HOME` environment variable is honored, where applicable, and will be used in place of `os.homedir()` in the scenarios described above. Metadata cache logging to stderr can be enabled by setting the `ARDRIVE_CACHE_LOG` environment variable to `1`. Cache performance is UNDEFINED for multi-process scenarios, but is presumed to be generally usable. The cache can be manually cleared safely at any time that any integrating app is not in operation. ```shell █████╗ ██████╗ ██████╗ ██████╗ ██╗██╗ ██╗███████╗ ██╔══██╗██╔══██╗██╔══██╗██╔══██╗██║██║ ██║██╔════╝ ███████║██████╔╝██║ ██║██████╔╝██║██║ ██║█████╗ ██╔══██║██╔══██╗██║ ██║██╔══██╗██║╚██╗ ██╔╝██╔══╝ ██║ ██║██║ ██║██████╔╝██║ ██║██║ ╚████╔╝ ███████╗ ╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═══╝ ╚══════╝ ██████╗██╗ ██╗ ██╔════╝██║ ██║ ██║ ██║ ██║ ██║ ██║ ██║ ╚██████╗███████╗██║ ╚═════╝╚══════╝╚═╝ Write ArFS =========== create-drive create-folder upload-file create-manifest move-file move-folder retry-tx Read ArFS =========== file-info folder-info drive-info list-folder list-drive list-all-drives download-file download-folder download-drive Wallet Ops =========== generate-seedphrase generate-wallet get-address get-balance send-ar get-drive-key get-file-key last-tx Arweave Ops =========== base-reward get-mempool create-tx send-tx tx-status ardrive \ --help ``` [ArDrive Community Discord][ardrive-discord] [ardrive]: https://ardrive.io [arweave]: https://ardrive.io/what-is-arweave/ [ardrive-github]: https://github.com/ardriveapp/ [arfs]: https://ardrive.atlassian.net/l/c/m6P1vJDo [ardrive-web-app]: https://app.ardrive.io [ardrive-core]: https://github.com/ardriveapp/ardrive-core-js [yarn-install]: https://yarnpkg.com/getting-started/install [nvm-install]: https://github.com/nvm-sh/nvm#installing-and-updating [wsl-install]: https://code.visualstudio.com/docs/remote/wsl [editor-config-vscode]: https://marketplace.visualstudio.com/items?itemName=EditorConfig.EditorConfig [prettier-vscode]: https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode [zipfs-vscode]: https://marketplace.visualstudio.com/items?itemName=arcanis.vscode-zipfs [eslint-vscode]: https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint [viewblock blockchain explorer]: https://viewblock.io/arweave/ [ardrive-discord]: https://discord.com/invite/ya4hf2H [arconnect]: https://arconnect.io/ [kb-wallets]: https://ardrive.atlassian.net/l/c/FpK8FuoQ [arweave-manifests]: https://github.com/ArweaveTeam/arweave/wiki/Path-Manifests [example-manifest-webpage]: https://arweave.net/qozq9YIUPEHfZhoTp9DkBpJuA_KNULBnfLiMroj5pZI [arlocal]: https://github.com/textury/arlocal [mozilla-mime-types]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types [viewblock]: https://viewblock.io/arweave/ [tx_anchors]: https://docs.arweave.org/developers/server/http-api#field-definitions [gql-guide]: https://gql-guide.vercel.app/#owners [ardrive-turbo]: https://ardrive.io/turbo/ # Using a Custom Arweave Gateway (/sdks/(clis)/ardrive-cli/(other-utility-operations)/using-a-custom-arweave-gateway) On each command that uses a gateway, it is possible to supply your own custom Arweave gateway using the flag `--gateway` or by setting an environment variable named `ARWEAVE_GATEWAY`. For example, you could test out that your ArFS transactions are working as expected on a local test network such as [ArLocal] with this flow: ```shell npx arlocal curl http://localhost:1984/mint/{ your public wallet address }/99999999999999 ardrive create-drive --gateway http://127.0.0.1:1984 -w /path/to/wallet -n 'my-test-drive' curl "$ARWEAVE_GATEWAY/mine" ardrive upload-file -F { root folder id from create drive } -l /path/to/file -w /path/to/wallet curl "$ARWEAVE_GATEWAY/mine" ardrive list-drive -d { drive id from create drive } ardrive download-file -f { file id from upload file } ``` # Git (/sdks/(clis)/ardrive-cli/(prerequisites)/git) Some of ArDrive's dependencies are transitively installed via Git. Install it, if necessary, and ensure that it's available within your terminal environment: [Download Git](https://git-scm.com/downloads) # NVM (Optional - Recommended) (/sdks/(clis)/ardrive-cli/(prerequisites)/nvm-optional-recommended) This project uses the Node Version Manager (NVM) and an `.nvmrc` file to lock the recommended Node version used by the latest version of `ardrive-core-js`. **Note for Windows: We recommend using WSL for setting up NVM on Windows using the [instructions described here][wsl-install]** Follow these steps to get NVM up and running on your system: 1. Install NVM using [these installation instructions][nvm-install]. 2. Navigate to this project's root directory 3. Ensure that the correct version of Node is installed by performing: `nvm install` 4. Use the correct version of Node, by performing: `nvm use` **IT IS STRONGLY RECOMMENDED THAT YOU AVOID GENERATING WALLETS VIA SEED PHRASE WITH THE CLI USING ANY NODE VERSION OTHER THAN THE ONE SPECIFIED IN `.nvmrc`.** # Creating Drives (/sdks/(clis)/ardrive-cli/(working-with-drives)/creating-drives) ```shell ardrive create-drive --wallet-file /path/to/my/wallet.json --drive-name "My Public Archive" ardrive create-drive --wallet-file /path/to/my/wallet.json --drive-name "Teenage Love Poetry" -P ``` # List Drive Pipeline Examples (/sdks/(clis)/ardrive-cli/(working-with-drives)/list-drive-pipeline-examples) You can utilize `jq` and the list commands to reshape the commands' output data into useful forms and stats for many use cases. Here are a few examples: ```shell ardrive list-drive -d a44482fd-592e-45fa-a08a-e526c31b87f1 | jq '.[] | select(.entityType == "file") | "https://app.ardrive.io/#/file/" + .entityId + "/view"' ``` Example output: ```shell "https://app.ardrive.io/#/file/1337babe-f000-dead-beef-ffffffffffff/view" "https://app.ardrive.io/#/file/cdbc9ddd-1cab-41d9-acbd-fd4328929de3/view" "https://app.ardrive.io/#/file/f19bc712-b57a-4e0d-8e5c-b7f1786b34a1/view" "https://app.ardrive.io/#/file/4f8e081b-42f2-442d-be41-57f6f906e1c8/view" "https://app.ardrive.io/#/file/0e02d254-c853-4ff0-9b6e-c4d23d2a95f5/view" "https://app.ardrive.io/#/file/c098b869-29d1-4a86-960f-a9e10433f0b0/view" "https://app.ardrive.io/#/file/4afc8cdf-4d27-408a-bfb9-0a2ec21eebf8/view" "https://app.ardrive.io/#/file/85fe488d-fcf7-48ca-9df8-2b39958bbf15/view" ... ``` ```shell ardrive list-drive -d 13c3c232-6687-4d11-8ac1-35284102c7db | jq ' map(select(.entityType == "file") | .size) | add' ``` ```shell ardrive list-drive -d 01ea6ba3-9e58-42e7-899d-622fd110211c | jq '[ .[] | select(.entityType == "file") ] | length' ``` # Listing Drives for an Address (/sdks/(clis)/ardrive-cli/(working-with-drives)/listing-drives-for-an-address) You can list all the drives associated with any Arweave wallet address, though the details of private drives will be obfuscated from you unless you provide the necessary decryption data. ```shell ardrive list-all-drives -w /path/to/my/wallet.json -P ardrive list-all-drives --address "HTTn8F92tR32N8wuo-NIDkjmqPknrbl10JWo5MZ9x2k" ``` # Listing Every Entity in a Drive (/sdks/(clis)/ardrive-cli/(working-with-drives)/listing-every-entity-in-a-drive) Useful notes on listing the contents of drives: - Listing a drive is effectively the same as listing its root folder. - You can control the tree depth of the data returned. - path, txPath, and entityIdPath properties on entities can provide useful handholds for other forms of data navigation ```shell ardrive list-drive -d "c7f87712-b54e-4491-bc96-1c5fa7b1da50" -w /path/to/my/wallet.json -P ardrive list-drive -d "c7f87712-b54e-4491-bc96-1c5fa7b1da50" -w /path/to/my/wallet.json -P --with-keys ardrive list-drive -d "c7f87712-b54e-4491-bc96-1c5fa7b1da50" --max-depth 2 ``` # Managing Drive Passwords (/sdks/(clis)/ardrive-cli/(working-with-drives)/managing-drive-passwords) The ArDrive CLI's private drive and folder functions all require either a drive password OR a drive key. Private file functions require either the drive password or the file key. **Keys and passwords are sensitive data, so manage the entry, display, storage, and transmission of them very carefully.** Drive passwords are the most portable, and fundamental, encryption facet, so a few options are available during private drive operations for supplying them: - Environment Variable - STDIN - Secure Prompt #### Supplying Your Password: Environment Variable ```shell read -rs TMP_ARDRIVE_PW ardrive \ -w /path/to/wallet.json -P ``` #### Supplying Your Password: STDIN ```shell cat /path/to/my/drive/password.txt | ardrive \ -w /path/to/wallet.json -P ardrive \ -w /path/to/wallet.json -P -w /path/to/wallet.json -P ? Enter drive password: › ******** ``` # Understanding Drive and File Keys (/sdks/(clis)/ardrive-cli/(working-with-drives)/understanding-drive-and-file-keys) Private Drives achieve privacy via end-to-end encryption facilitated by hash-derived "Keys". Drive Keys encrypt/decrypt Drive and Folder data, and File Keys encrypt/decrypt File Data. The relationships among your data and their keys is as follows: - Drive Key = functionOf(Wallet Signature, Randomly Generated Drive ID, User-specified Drive Password) - File Key = functionOf(Randomly Generated File ID, Drive Key) When you create private entities, the returned JSON data from the ArDrive CLI will contain the keys needed to decrypt the encrypted representation of your entity that is now securely and permanently stored on the blockweave. To derive the drive key again for a drive, perform the following: ```shell ardrive get-drive-key -w /path/to/my/wallet.json -d "6939b9e0-cc98-42cb-bae0-5888eca78885" -P ``` To derive the file key again for a file, perform the following: ```shell ardrive get-file-key --file-id "bd2ce978-6ede-4b0d-8f79-2d7bc235a0e0" --drive-id "6939b9e0-cc98-42cb-bae0-5888eca78885" --drive-key "yHdCjpCK3EcuhQcKNx2d/NN5ReEjoKfZVqKunlCnPEo" ``` # Understanding Drive Hierarchies (/sdks/(clis)/ardrive-cli/(working-with-drives)/understanding-drive-hierarchies) At the root of every data tree is a "Drive" entity. When a drive is created, a Root Folder is also created for it. The entity IDs for both are generated and returned when you create a new drive: ```shell ardrive create-drive --wallet-file /path/to/my/wallet.json --drive-name "Teenage Love Poetry" | tee created_drive.json | jq '[.created[] | del(.metadataTxId, .entityName, .bundledIn)]' [ { "type": "drive", "entityId": "6939b9e0-cc98-42cb-bae0-5888eca78885" } { "type": "folder", "entityId": "d1535126-fded-4990-809f-83a06f2a1118" } ] ``` The relationship between the drive and its root folder is clearly visible when retrieving the drive's info: ```shell ardrive drive-info -d "6939b9e0-cc98-42cb-bae0-5888eca78885" | jq '{driveId, rootFolderId}' { "driveId": "6939b9e0-cc98-42cb-bae0-5888eca78885", "rootFolderId": "d1535126-fded-4990-809f-83a06f2a1118" } ``` All file and folder entities in the drive will be anchored to it by a "Drive-ID" GQL Tag. And they'll each be anchored to a parent folder ID, tracked via the "Parent-Folder-ID" GQL tag, forming a tree structure whose base terminates at the Root Folder. # Dry Run (/sdks/(clis)/ardrive-cli/(working-with-entities)/dry-run) An important feature of the ArDrive CLI is the `--dry-run` flag. On each command that would write an ArFS entity, there is the option to run it as a "dry run". This will run all of the steps and print the outputs of a regular ArFS write, but will skip sending the actual transaction: ```shell ardrive \ \ --dry-run ``` This can be very useful for gathering price estimations or to confirm that you've copy-pasted your entity IDs correctly before committing to an upload. # Uploading to Turbo (BETA) (/sdks/(clis)/ardrive-cli/(working-with-entities)/uploading-to-turbo-beta) Users can optionally choose to send each ArFS entities created to [ArDrive Turbo][ardrive-turbo] using the `--turbo` flag. Instead of using AR from an Arweave wallet, you can use Turbo Credits or take advantage of free/discounted upload promotions. ```shell ardrive \ \ --turbo ``` This flag will skip any balance check on the CLI side. Turbo will check a user's balance and accept/reject a data item at the time of upload. The `--turbo` flag by default will send your files to `upload.ardrive.io` to be bundled. To change the Turbo destination, users can use the `--turbo-url` flag. # Download a Single file (BETA) (/sdks/(clis)/ardrive-cli/(working-with-files)/download-a-single-file-beta) By using the `download-file` command you can download a file on chain to a folder in your local storage specified by --local-path (or to your current working directory if not specified): ```shell ardrive download-file -w /path/to/wallet.json --file-id "ff450770-a9cb-46a5-9234-89cbd9796610" --local-path /my_ardrive_downloads/ ``` Specify a filename in the --local-path if you'd like to use a different name than the one that's used in your drive: ```shell ardrive download-file -w /path/to/wallet.json --file-id "ff450770-a9cb-46a5-9234-89cbd9796610" --local-path /my_ardrive_downloads/my_pic.png ``` # Downloading a Drive (/sdks/(clis)/ardrive-cli/(working-with-files)/downloading-a-drive) To download the whole drive you can use the `download-drive` command. ```shell ardrive download-drive -d "c0c8ba1c-efc5-420d-a07c-a755dc67f6b2" ``` This is equivalent to running the `download-folder` command against the root folder of the drive. # Downloading a Folder with Files (/sdks/(clis)/ardrive-cli/(working-with-files)/downloading-a-folder-with-files) You can download a folder from ArDrive to your local machine with the `download-folder` command. In the following examples, assume that a folder with ID "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" exists in your drive and is named "MyArDriveFolder". ```shell ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" ``` By specifying the `--local-path` option, you can choose the local parent folder into which the on-chain folder will be downloaded. When the parameter is omitted, its value defaults to the current working directory (i.e. `./`). ```shell ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" --local-path /my_ardrive_downloads/ ``` The `--max-depth` parameter lets you to choose a custom folder depth to download. When omitted, the entire subtree of the folder will be downloaded. In the following example, only the immediate children of the folder will be downloaded: ```shell ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" --max-depth 0 ``` The behaviors of `--local-path` are similar to those of `cp` and `mv` in Unix systems, e.g.: ```shell ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" --local-path "/existing_folder" ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" --local-path "/existing_folder/MyArDriveFolder" ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" --local-path "/existing_folder/non_existent_folder" ardrive download-folder -f "47f5bde9-61ba-49c7-b409-1aa4a9e250f6" --local-path "/non_existent_folder_1/non_existent_folder_2" ``` # Fetching the Metadata of a File Entity (/sdks/(clis)/ardrive-cli/(working-with-files)/fetching-the-metadata-of-a-file-entity) Simply perform the file-info command to retrieve the metadata of a file: ```shell ardrive file-info --file-id "e5ebc14c-5b2d-4462-8f59-7f4a62e7770f" ``` Example output: ```shell { "appName": "ArDrive-Web", "appVersion": "0.1.0", "arFS": "0.11", "contentType": "application/json", "driveId": "51062487-2e8b-4af7-bd81-4345dc28ea5d", "entityType": "file", "name": "2_depth.png", "txId": "CZKdjqwnmxbWchGA1hjSO5ZH--4OYodIGWzI-FmX28U", "unixTime": 1633625081, "size": 41946, "lastModifiedDate": 1605157729000, "parentFolderId": "a2c8a0cb-0ca7-4dbb-8bf8-93f75f308e63", "entityId": "e5ebc14c-5b2d-4462-8f59-7f4a62e7770f", "fileId": "e5ebc14c-5b2d-4462-8f59-7f4a62e7770f", "dataTxId": "Jz0WsWyAGVc0aE3UzACo-YJqG8OPrN3UucmDdt8Fbjc", "dataContentType": "image/png" } ``` # IPFS CID Tagging (/sdks/(clis)/ardrive-cli/(working-with-files)/ipfs-cid-tagging) Certain nodes on the Arweave network may be running the [IPFS+Arweave bridge](https://arweave.medium.com/arweave-ipfs-persistence-for-the-interplanetary-file-system-9f12981c36c3). Tagging your file upload transaction with its IPFS v1 CID value in the 'IPFS-Add' tag may allow you to take advantage of this system. It can also be helpful for finding data on Arweave via GQL based on its CID. To include the CID tag on your **PUBLIC** file uploads, you may use the '--add-ipfs-tag' flag: ```shell ardrive upload-file --add-ipfs-tag --local-path /path/to/file.txt --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` # Moving Files (/sdks/(clis)/ardrive-cli/(working-with-files)/moving-files) Files can be moved from one folder to another within the same drive. Moving a file is simply the process of uploading a new file metadata revision with an updated File ID Parent Folder ID relationship. The following command will move a file from its current location in a public drive to a new parent folder in that drive: ```shell ardrive move-file --file-id "e5ebc14c-5b2d-4462-8f59-7f4a62e7770f" --parent-folder-id "a2c8a0cb-0ca7-4dbb-8bf8-93f75f308e63" ``` # Name Conflict Resolution on Upload (/sdks/(clis)/ardrive-cli/(working-with-files)/name-conflict-resolution-on-upload) By default, the `upload-file` command will use the upsert behavior if existing entities are encountered in the destination folder tree that would cause naming conflicts. Expect the behaviors from the following table for each of these resolution settings: | Source Type | Conflict at Dest | `skip` | `replace` | `upsert` (default) | | ----------- | ---------------- | ------ | --------- | ------------------ | | File | None | Insert | Insert | Insert | | File | Matching File | Skip | Update | Skip | | File | Different File | Skip | Update | Update | | File | Folder | Skip | Fail | Fail | | Folder | None | Insert | Insert | Insert | | Folder | File | Skip | Fail | Fail | | Folder | Folder | Re-use | Re-use | Re-use | The default upsert behavior will check the destination folder for a file with a conflicting name. If no conflicts are found, it will insert (upload) the file. In the case that there is a FILE to FILE name conflict found, it will only update it if necessary. To determine if an update is necessary, upsert will compare the last modified dates of conflicting file and the file being uploaded. When they are matching, the upload will be skipped. Otherwise the file will be updated as a new revision. To override the upsert behavior, use the `--replace` option to always make new revisions of a file or the `--skip` option to always skip the upload on name conflicts: ```shell ardrive upload-file --replace --local-path /path/to/file.txt --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` ```shell ardrive upload-file --skip --local-path /path/to/file.txt --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` Alternatively, the upload-file commands now also supports the `--ask` conflict resolution option. This setting will always provide an interactive prompt on name conflicts that allows users to decide how to resolve each conflict found: ```shell ardrive upload-file --ask --local-file-path /path/to/file.txt --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json Destination folder has a file to file name conflict! File name: 2.png File ID: efbc0370-b69f-44d9-812c-0d272b019027 This file has a DIFFERENT last modified date Please select how to proceed: › - Use arrow-keys. Return to submit. ❯ Replace as new file revision Upload with a different file name Skip this file upload ``` # Progress Logging of Transaction Uploads (/sdks/(clis)/ardrive-cli/(working-with-files)/progress-logging-of-transaction-uploads) Progress logging of transaction uploads to stderr can be enabled by setting the `ARDRIVE_PROGRESS_LOG` environment variable to `1`: ```shell Uploading file transaction 1 of total 2 transactions... Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 0% Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 35% Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 66% Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 100% Uploading file transaction 2 of total 2 transactions... Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 0% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 13% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 28% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 42% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 60% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 76% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 91% Transaction nA1stCdTkuf290k0qsqvmJ78isEC0bwgrAi3D8Cl1LU Upload Progress: 100% ``` # Rename a Single File (/sdks/(clis)/ardrive-cli/(working-with-files)/rename-a-single-file) To rename an on-chain file you can make use of the `rename-file` command. The required parameters are the file ID and the new name, as well as the owner wallet or seed phrase. ```shell ardrive rename-file --file-id "290a3f9a-37b2-4f0f-a899-6fac983833b3" --file-name "My custom file name.txt" --wallet-file "wallet.json" ``` # Retrying a Failed File Data Transaction (Public Unbundled Files Only) (/sdks/(clis)/ardrive-cli/(working-with-files)/retrying-a-failed-file-data-transaction-public-unbundled-files-only) Arweave data upload transactions are split into two phases: transaction posting and chunks uploading. Once the transaction post phase has been completed, you've effectively "paid" the network for storage of the data chunks that you'll send in the next stage. If your system encounters an error while posting the transaction, you can retry posting the transaction for as long as your tx_anchor is valid ([learn more about tx_anchors here][tx_anchors]). You may retry and/or resume posting chunks at any time after your transaction has posted. The ArDrive CLI allows you to take advantage of this Arweave protocol capability. Using the CLI, when the transaction post has succeeded but the chunk upload step fails, the data transaction's ID could be lost. There are a few options to recover this ID. If the failed transaction is the most recent one sent from a wallet, the transaction ID can be recovered with the `ardrive last-tx -w /path/to/wallet` command AFTER the transaction's headers have been mined (It can take 5-10 minutes for the tx-id to become available with the last-tx approach). Other options for finding the partially uploaded transaction's ID include: - Using an Arweave gateway GQL http endpoint to search for transactions that belong to the wallet. See this [Arweave GQL Guide][gql-guide] for more info. - Browse the recent transactions associated with the wallet via a block explorer tool like [ViewBlock][viewblock]. In order to re-seed the chunks for an unbundled ArFS data transaction, a user must have the data transaction ID, the original file data, and either a destination folder ID or a valid file ID for the file. Supply that information to the `retry-tx` command like so: ```shell ardrive retry-tx --tx-id { Data Transaction ID } --parent-folder-id { Destination Folder ID } --local-path /path/to/file --wallet-file /path/to/wallet ``` **Note: Retry feature is currently only available for PUBLIC unbundled file transactions. It is also perfectly safe to mistakenly re-seed the chunks of a healthy transaction, the transaction will remain stable and the wallet balance will not be affected.** # Understanding Bundled Transactions (/sdks/(clis)/ardrive-cli/(working-with-files)/understanding-bundled-transactions) The ArDrive CLI currently uses two different methods for uploading transactions to the Arweave network: standard transactions and Direct to Network (D2N) bundled transactions. By default, the CLI will send a D2N bundled transaction for any action that would result in multiple transactions. This bundling functionality is currently used on the `upload-file` and `create-drive` commands. D2N bundled transactions come with several benefits and implications: - Bundling saves AR and enhances ArFS reliability by sending associated ArFS transactions up as one atomic bundle. - Bundled transactions are treated as a single data transaction by the Arweave network, but can be presented as separate transactions by the Arweave Gateway once they have been "unbundled". - Un-bundling can take anywhere from a few minutes up to an hour. During that time, the files in the bundle will neither appear in list- commands nor be downloadable. Similarly, they will not appear in the web app after syncs until un-bundling is complete. **This can negatively affect the accuracy of upsert operations**, so it's best to wait before retrying bulk uploads. - Bundling reliability on the gateway side degrades once bundles reach either 500 data items (or ~250 files) or 500 MiB, so the CLI will create and upload multiple bundles as necessary, or will send files that are simply too large for reliable bundling as unbundled txs. # Uploading a Custom Manifest (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-a-custom-manifest) Using the custom content type feature, it is possible for users to upload their own custom manifests. The Arweave gateways use this special content type in order to identify an uploaded file as a manifest: ```shell application/x.arweave-manifest+json ``` In addition to this content type, the manifest must also adhere to the [correct JSON structure](#manifest-json) of an Arweave manifest. A user can create their own manifest from scratch, or start by piping a generated manifest to a JSON file and editing it to their specifications: ```shell ardrive create-manifest -w /path/to/wallet -f "6c312b3e-4778-4a18-8243-f2b346f5e7cb" --dry-run | jq '{manifest}.manifest' > my-custom-manifest.json ``` After editing the generated manifest, simply perform an `upload-file` command with the custom Arweave manifest content type to any PUBLIC folder: ```shell ardrive upload-file --content-type "application/x.arweave-manifest+json" --local-path my-custom-manifest.json --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` The returned `dataTxId` field on the created `file` entity will be the endpoint that the manifest can be found on Arweave, just as explained in the [manifest sections](#uploading-manifests) above: ```shell https://arweave.net/{dataTxId} https://arweave.net/{dataTxId}/custom-file-1 https://arweave.net/{dataTxId}/custom-file-2 ``` # Uploading a Folder with Files (Bulk Upload) (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-a-folder-with-files-bulk-upload) Users can perform a bulk upload by using the upload-file command on a target folder. The command will reconstruct the folder hierarchy on local disk as ArFS folders on the permaweb and upload each file into their corresponding folders: ```shell ardrive upload-file --local-path /path/to/folder --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` # Uploading a Non-Bundled Transaction (NOT RECOMMENDED) (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-a-non-bundled-transaction-not-recommended) While not recommended, the CLI does provide the option to forcibly send all transactions as standard transactions rather than attempting to bundle them together. To do this, simply add the `--no-bundle` flag to the `upload-file` or `create-drive` command: ```shell ardrive upload-file --no-bundle --local-path /path/to/file --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` # Uploading a Single File (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-a-single-file) To upload a file, you'll need a parent folder id, the file to upload's file path, and the path to your wallet: ```shell ardrive upload-file --local-path /path/to/file.txt --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` Example output: ```shell { "created": [ { "type": "file", "entityName": "file.txt" "entityId": "6613395a-cf19-4420-846a-f88b7b765c05" "dataTxId": "l4iNWyBapfAIj7OU-nB8z9XrBhawyqzs5O9qhk-3EnI", "metadataTxId": "YfdDXUyerPCpBbGTm_gv_x5hR3tu5fnz8bM-jPL__JE", "bundledIn": "1zwdfZAIV8E26YjBs2ZQ4xjjP_1ewalvRgD_GyYw7f8", "sourceUri": "file:///path/to/file.txt" }, { "type": "bundle", "bundleTxId": "1zwdfZAIV8E26YjBs2ZQ4xjjP_1ewalvRgD_GyYw7f8" } ], "tips": [ { "txId": "1zwdfZAIV8E26YjBs2ZQ4xjjP_1ewalvRgD_GyYw7f8", "recipient": { "address": "3mxGJ4xLcQQNv6_TiKx0F0d5XVE0mNvONQI5GZXJXkt" }, "winston": "10000000" } ], "fees": { "1zwdfZAIV8E26YjBs2ZQ4xjjP_1ewalvRgD_GyYw7f8": 42819829 } } ``` NOTE: To upload to the root of a drive, specify its root folder ID as the parent folder ID for the upload destination. You can retrieve it like so: ```shell ardrive drive-info -d "c7f87712-b54e-4491-bc96-1c5fa7b1da50" | jq -r '.rootFolderId' ``` # Uploading Files with Custom MetaData (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-files-with-custom-metadata) ArDrive CLI has the capability of attaching custom metadata to ArFS File and Folder MetaData Transactions during the `upload-file` command. This metadata can be applied to either the GQL tags on the MetaData Transaction and/or into the MetaData Transaction's Data JSON. All custom metadata applied must ultimately adhere to the following JSON shapes: ```ts // GQL Tags type CustomMetaDataGqlTags = Record; // Data JSON Fields type CustomMetaDataJsonFields = Record; | string | number | boolean | null | { [member: string]: JsonSerializable } | JsonSerializable[]; ``` e.g: ```shell { IPFS-Add: 'MY_HASH' } { 'Custom Name': ['Val 1', 'Val 2'] } ``` When the custom metadata is attached to the MetaData Transaction's GQL tags, they will become visible on any Arweave GQL gateway and also third party tools that read GQL data. When these tags are added to the MetaData Transaction's Data JSON they can be read by downloading the JSON data directly from `https://arweave.net/METADATA_TX_ID`. To add this custom metadata to your file metadata transactions, CLI users can pass custom metadata these parameters: - `--metadata-file path/to/json/schema` - `--metadata-json '{"key": "val", "key-2": true, "key-3": 420, "key-4": ["more", 1337]}'` - `--metadata-gql-tags "Tag-Name" "Tag Val"` The `--metadata-file` will accept a file path to JSON file containing custom metadata: ```shell ardrive upload-file --metadata-file path/to/metadata/json # ... ``` This JSON schema object must contain instructions on where to put this metadata with the `metaDataJson` and `metaDataGqlTags` keys. e.g: ```json { "metaDataJson": { "Tag-Name": ["Value-1", "Value-2"] }, "metaDataGqlTags": { "GQL Tag Name": "Tag Value" } } ``` The `--metadata-gql-tags` parameter accepts an array of string values to be applied to the MetaData Tx GQL Tags. This method of CLI input does not support multiple tag values for a given tag name and the input must be an EVEN number of string values. (Known bug: String values starting with the `"-"` character are currently not supported. Use --metadata-file parameter instead.) e.g: ```shell upload-file --metadata-gql-tags "Custom Tag Name" "Custom Value" # ... ``` And the `--metadata-json` parameter will accept a stringified JSON input. It will apply all declared JSON fields directly to the MetaData Tx's Data JSON. e.g: ```shell upload-file --metadata-json ' { "json field": "value", "another fields": false } ' # ... ``` Custom metadata applied to files and/or folders during the `upload-file` command will be read back through all existing read commands. e.g: ```shell ardrive file-info -f 067c4008-9cbe-422e-b697-05442f73da2b { "appName": "ArDrive-CLI", "appVersion": "1.17.0", "arFS": "0.11", "contentType": "application/json", "driveId": "967215ca-a489-494b-97ec-0dd428d7be34", "entityType": "file", "name": "unique-name-9718", "txId": "sxg8bNu6_bbaHkJTxAINVVoz_F-LiFe6s7OnxzoJJk4", "unixTime": 1657655070, "size": 262148, "lastModifiedDate": 1655409872705, "dataTxId": "ublZcIff77ejl3m0uEA8lXEfnTWmSBOFoz-HibqKeyk", "dataContentType": "text/plain", "parentFolderId": "97bc4fb5-aca4-4ffe-938f-1285153d98ca", "entityId": "067c4008-9cbe-422e-b697-05442f73da2b", "fileId": "067c4008-9cbe-422e-b697-05442f73da2b", "IPFS-Add": "MY_HASH", "Tag-1": "Val", "Tag-2": "Val", "Tag-3": "Val", "Boost": "1.05" } ``` #### Applying Unique Custom MetaData During Bulk Workflows With some custom scripting and the `--metadata-file` parameter, the ArDrive CLI can be used to apply custom metadata to each file individually in a bulk workflow. For example, if you choose a numbered file naming pattern you can make use of a `for` loop: ```shell for i in {1..5} do ardrive upload-file -F f0c58c11-430c-4383-8e54-4d864cc7e927 --local-path "../uploads/test-file-$i.txt" -w "/path/to/wallet.json" --metadata-file "../custom/metadata-$i.json" --dry-run > "file-result-$i.json" done ``` # Uploading From a Remote URL (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-from-a-remote-url) You can upload a file from an existing url using the `--remote-path` flag. This must be used in conjunction with `--dest-file-name`. You can use a custom content type using the `--content-type` flag, but if this isn't used the app will use the content type from the response header of the request for the remote data. ```shell ardrive upload-file --remote-path "https://url/to/file" --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -d "example.jpg" -w /path/to/wallet.json ``` # Uploading Manifests (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-manifests) [Arweave Path Manifests][arweave-manifests] are are special `.json` files that instruct Arweave Gateways to map file data associated with specific, unique transaction IDs to customized, hosted paths relative to that of the manifest file itself. So if, for example, your manifest file had an arweave.net URL like: ```shell https://arweave.net/{manifest tx id} ``` Then, all the mapped transactions and paths in the manifest file would be addressable at URLs like: ```shell https://arweave.net/{manifest tx id}/foo.txt https://arweave.net/{manifest tx id}/bar/baz.png ``` ArDrive supports the creation of these Arweave manifests using any of your PUBLIC folders. The generated manifest paths will be links to each of the file entities within the specified folder. The manifest file entity will be created at the root of the folder. To create a manifest of an entire public drive, specify the root folder of that drive: ```shell ardrive create-manifest -f "bc9af866-6421-40f1-ac89-202bddb5c487" -w "/path/to/wallet" ``` You can also create a manifest of a folder's file entities at a custom depth by using the `--max-depth` option: ```shell ardrive create-manifest --max-depth 0 -f "867228d8-4413-4c0e-a499-e1decbf2ea38" -w "/path/to/wallet" ``` Creating a `.json` file of your manifest links output can be accomplished here with some `jq` parsing and piping to a file: ```shell ardrive create-manifest -w /path/to/wallet -f "6c312b3e-4778-4a18-8243-f2b346f5e7cb" | jq '{links}' > links.json ``` If you'd like to preview the contents of your manifest before uploading, you can perform a dry run and do some lightweight post processing to isolate the data: ```shell ardrive create-manifest -w /path/to/wallet -f "6c312b3e-4778-4a18-8243-f2b346f5e7cb" --dry-run | jq '{manifest}.manifest' ``` ```json { "manifest": "arweave/paths", "version": "0.1.0", "index": { "path": "index.html" }, "paths": { "hello_world.txt": { "id": "Y7GFF8r9y0MEU_oi1aZeD87vrmai97JdRQ2L0cbGJ68" }, "index.html": { "id": "pELonjVebHyBsdxVymvxbGTmHD96v9PuuUXj8GUHGoY" } } } ``` The manifest data transaction is tagged with a unique content-type, `application/x.arweave-manifest+json`, which tells the gateway to treat this file as a manifest. The manifest file itself is a `.json` file that holds the paths (the data transaction ids) to each file within the specified folder. When your folder is later changed by adding files or updating them with new revisions, the original manifest will NOT be updated on its own. A manifest is a permanent record of your files in their current state. However, creating a subsequent manifest with the same manifest name will create a new revision of that manifest in its new current state. Manifests follow the same name conflict resolution as outlined for files above (upsert by default). #### Hosting a Webpage with Manifest When creating a manifest, it is possible to host a webpage or web app. You can do this by creating a manifest on a folder that has an `index.html` file in its root. Using generated build folders from popular frameworks works as well. One requirement here to note is that the `href=` paths from your generated `index.html` file must not have leading a `/`. This means that the manifest will not resolve a path of `/dist/index.js` but it will resolve `dist/index.js` or `./dist/index.js`. As an example, here is a flow of creating a React app and hosting it with an ArDrive Manifest. First, generate a React app: ```shell yarn create react-app my-app ``` Next, add this field to the generated `package.json` so that the paths will resolve correctly: ```json "homepage": ".", ``` Then, create an optimized production build from within the app's directory: ```shell yarn build ``` Now, we can create and upload that produced build folder on ArDrive to any of your existing ArFS folder entities: ```shell ardrive upload-file -l "/build" -w "/path/to/wallet" --parent-folder-id "bc9af866-6421-40f1-ac89-202bddb5c487" ``` And finally, create the manifest using the generated Folder ID from the build folder creation: ```shell ardrive create-manifest -f "41759f05-614d-45ad-846b-63f3767504a4" -w "/path/to/wallet" ``` In the return output, the top link will be a link to the deployed web app: ```shell "links": [ "https://arweave.net/0MK68J8TqGhaaOpPe713Zn0jdpczMt2NGS2CtRYiuAg", "https://arweave.net/0MK68J8TqGhaaOpPe713Zn0jdpczMt2NGS2CtRYiuAg/asset-manifest.json", "https://arweave.net/0MK68J8TqGhaaOpPe713Zn0jdpczMt2NGS2CtRYiuAg/favicon.ico", "https://arweave.net/0MK68J8TqGhaaOpPe713Zn0jdpczMt2NGS2CtRYiuAg/index.html", # ... ``` This is effectively hosting a web app with ArDrive. Check out the ArDrive Price Calculator React App hosted as an [ArDrive Manifest][example-manifest-webpage]. # Uploading Multiple Files (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-multiple-files) To upload an arbitrary number of files or folders, pass a space-separated list of paths to `--local-paths`: ```shell ardrive upload-file -w wallet.json -F "6939b9e0-cc98-42cb-bae0-5888eca78885" --local-paths ./image.png ~/backups/ ../another_file.txt ardrive upload-file -w wallet.json -F "6939b9e0-cc98-42cb-bae0-5888eca78885" --local-paths ./*.json ``` # Uploading With a Custom Content Type (/sdks/(clis)/ardrive-cli/(working-with-files)/uploading-with-a-custom-content-type) Each file uploaded to the Arweave network receives a `"Content-Type"` GraphQL tag that contains the MIME type for the file. The gateway will use this content type to determine how to serve that file's data transaction at the `arweave.net/{data tx id}` endpoint. By default, the CLI will attempt to derive this content type from the file extension of the provided file. In most cases, the content type that is derived will be correct and the gateway will properly serve the file. The CLI also provides the option for users to upload files with a custom content type using the `--content-type` flag: ```shell ardrive upload-file --content-type "application/json" --local-path /path/to/file --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" -w /path/to/wallet.json ``` It is currently possible to set this value to any given string, but the gateway will still only serve valid content types. Check out this list of commonly used MIME types to ensure you're providing a valid content type: [Common MIME types][mozilla-mime-types]. Note: In the case of multi-file uploads or recursive folder uploads, setting this `--content-type` flag will set the provided custom content type on EVERY file entity within a given upload. # Creating Folders (/sdks/(clis)/ardrive-cli/(working-with-folders)/creating-folders) Creating folders manually is straightforward: ```shell ardrive create-folder --parent-folder-id "63153bb3-2ca9-4d42-9106-0ce82e793321" --folder-name "My Awesome Folder" -w /path/to/wallet.json ``` Example output: ```shell { "created": [ { "type": "folder", "metadataTxId": "AYFMBVmwqhbg9y5Fbj3Iasy5oxUqhauOW7PcS1sl4Dk", "entityId": "d1b7c514-fb12-4603-aad8-002cf63015d3", "key": "yHdCjpCKD2cuhQcKNx2d/XF5ReEjoKfZVqKunlCnPEk", "entityName": "My Awesome Folder" } ], "tips": [], "fees": { "AYFMBVmwqhbg9y5Fbj3Iasy5oxUqhauOW7PcS1sl4Dk": 1378052 } } ``` Note: Folders can also be created by supplying a folder as the --local-path of an upload-file command. In this case, the folder hierarchy on the local disk will be reconstructed on chain during the course of the recursive bulk upload. # Listing Contents of a Folder (/sdks/(clis)/ardrive-cli/(working-with-folders)/listing-contents-of-a-folder) Similar to drives, the `list-folder` command can be used to fetch the metadata of each entity within a folder. But by default, the command will fetch only the immediate children of that folder (`--max-depth 0`): ```shell ardrive list-folder --parent-folder-id "29850ab7-56d4-4e1f-a5be-cb86d5513940" ``` Example output: ```shell [ { "appName": "ArDrive-CLI", "appVersion": "2.0", "arFS": "0.11", "contentType": "application/json", "driveId": "01ea6ba3-9e58-42e7-899d-622fd110211a", "entityType": "folder", "name": "mytestfolder", "txId": "HYiKyfLwY7PT9NleTQoTiM_-qPVUwf4ClDhx1sjUAEU", "unixTime": 1635102772, "parentFolderId": "29850ab7-56d4-4e1f-a5be-cb86d5513940", "entityId": "03df2929-1440-4ab4-bbf0-9dc776e1ed96", "path": "/My Public Folder/mytestfolder", "txIdPath": "/09_x0X2eZ3flXXLS72WdTDq6uaa5g2LjsT-QH1m0zhU/HYiKyfLwY7PT9NleTQoTiM_-qPVUwf4ClDhx1sjUAEU", "entityIdPath": "/29850ab7-56d4-4e1f-a5be-cb86d5513940/03df2929-1440-4ab4-bbf0-9dc776e1ed96" }, { "appName": "ArDrive-CLI", "appVersion": "2.0", "arFS": "0.11", "contentType": "application/json", "driveId": "01ea6ba3-9e58-42e7-899d-622fd110211a", "entityType": "folder", "name": "Super sonic public folder", "txId": "VUk1B_vo1va2-EHLtqjsotzy0Rdn6lU4hQo3RD2xoTI", "unixTime": 1631283259, "parentFolderId": "29850ab7-56d4-4e1f-a5be-cb86d5513940", "entityId": "452c6aec-43dc-4015-9abd-20083068d432", "path": "/My Public Folder/Super sonic sub folder", "txIdPath": "/09_x0X2eZ3flXXLS72WdTDq6uaa5g2LjsT-QH1m0zhU/VUk1B_vo1va2-EHLtqjsotzy0Rdn6lU4hQo3RD2xoTI", "entityIdPath": "/29850ab7-56d4-4e1f-a5be-cb86d5513940/452c6aec-43dc-4015-9abd-20083068d432" }, { "appName": "ArDrive-CLI", "appVersion": "2.0", "arFS": "0.11", "contentType": "application/json", "driveId": "01ea6ba3-9e58-42e7-899d-622fd110211a", "entityType": "file", "name": "test-number-twelve.txt", "txId": "429zBqnd7ZBNzgukaix26RYz3g5SeXCCo_oIY6CPZLg", "unixTime": 1631722234, "size": 47, "lastModifiedDate": 1631722217028, "dataTxId": "vA-BxAS7I6n90cH4Fzsk4cWS3EOPb1KOhj8yeI88dj0", "dataContentType": "text/plain", "parentFolderId": "29850ab7-56d4-4e1f-a5be-cb86d5513940", "entityId": "e5948327-d6de-4acf-a6fe-e091ecf78d71", "path": "/My Public Folder/test-number-twelve.txt", "txIdPath": "/09_x0X2eZ3flXXLS72WdTDq6uaa5g2LjsT-QH1m0zhU/429zBqnd7ZBNzgukaix26RYz3g5SeXCCo_oIY6CPZLg", "entityIdPath": "/29850ab7-56d4-4e1f-a5be-cb86d5513940/e5948327-d6de-4acf-a6fe-e091ecf78d71" }, { "appName": "ArDrive-CLI", "appVersion": "2.0", "arFS": "0.11", "contentType": "application/json", "driveId": "01ea6ba3-9e58-42e7-899d-622fd110211a", "entityType": "file", "name": "wonderful-test-file.txt", "txId": "6CokwlzB81Fx7dq-lB654VM0XQykdU6eYohDmEJ2gk4", "unixTime": 1631671275, "size": 23, "lastModifiedDate": 1631283389232, "dataTxId": "UP8THwA_1gvyRqNRqYmTpWvU4-UzNWBN7SiX_AIihg4", "dataContentType": "text/plain", "parentFolderId": "29850ab7-56d4-4e1f-a5be-cb86d5513940", "entityId": "3274dae9-3487-41eb-94d5-8d5d3d8bc343", "path": "/My Public Folder/wonderful-test-file.txt", "txIdPath": "/09_x0X2eZ3flXXLS72WdTDq6uaa5g2LjsT-QH1m0zhU/6CokwlzB81Fx7dq-lB654VM0XQykdU6eYohDmEJ2gk4", "entityIdPath": "/29850ab7-56d4-4e1f-a5be-cb86d5513940/3274dae9-3487-41eb-94d5-8d5d3d8bc343" } ] ``` To list further than the immediate children, you can make use of the flags: `--all` and `--max-depth`. ```shell ardrive list-folder --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" --all ardrive list-folder --parent-folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" --max-depth 2 ``` In the case of private entitites, the `--with-keys` flag will make the command to include the keys in the output. ```shell ardrive list-folder --parent-folder-id "1b027047-4cfc-4eee-88a8-9af694f660c0" -w /my/wallet.json --with-keys ``` # Moving Folders (/sdks/(clis)/ardrive-cli/(working-with-folders)/moving-folders) Moving a folder is as simple as supplying a new parent folder ID. Note that naming collisions among entities within a folder are not allowed. ```shell ardrive move-folder --folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" --parent-folder-id "29850ab7-56d4-4e1f-a5be-cb86d5513921" -w /path/to/wallet.json ``` # Renaming Folders (/sdks/(clis)/ardrive-cli/(working-with-folders)/renaming-folders) In order to rename a folder you must provide a name different from its current one, and it must not create naming conflicts with its sibling entities. ```shell ardrive rename-folder --folder-id "568d5eba-dbf3-4a49-8129-1c58f7fd35bc" --folder-name "Folder with cool stuff" -w "./wallet.json" ``` # Viewing Folder Metadata (/sdks/(clis)/ardrive-cli/(working-with-folders)/viewing-folder-metadata) To view the metadata of a folder, users can use the `folder-info` command: ```shell ardrive folder-info --folder-id "9af694f6-4cfc-4eee-88a8-1b02704760c0" ``` # ArFS (/sdks/(clis)/ardrive-cli/arfs) [ArFS] is a data modeling, storage, and retrieval protocol designed to emulate common file system operations and to provide aspects of mutability to your data hierarchy on [Arweave]'s otherwise permanent, immutable data storage blockweave. # CLI Help (/sdks/(clis)/ardrive-cli/cli-help) Learn to use any command: ```shell ardrive --help ``` # CLI Version (/sdks/(clis)/ardrive-cli/cli-version) You can print out the version by running any of: ```shell ardrive --version ardrive -V ``` # Data Portability (/sdks/(clis)/ardrive-cli/data-portability) Data uploaded via the ArDrive CLI, once indexed by Arweave's Gateways and sufficiently seeded across enough nodes on the network, can be accessed via all other ArDrive applications including the [ArDrive Web application][ardrive-web-app] at https://app.ardrive.io. All transactions successfully executed by ArDrive can always be inspected in the [Viewblock blockchain explorer]. # ArDrive CLI (/sdks/(clis)/ardrive-cli) **For AI and LLM users**: Access the complete ArDrive CLI documentation in plain text format at llm.txt for easy consumption by AI agents and language models. # ArDrive CLI Please refer to the [source code](https://github.com/ardriveapp/ardrive-cli) for SDK details. # Intended Audience (/sdks/(clis)/ardrive-cli/intended-audience) This tool is intended for use by: - ArDrive power users with advanced workflows and resource efficiency in mind: bulk uploaders, those with larger storage demand, game developers, nft creators, storage/db admins, etc. - Automation tools - Services - Terminal aficionados - Extant and aspiring cypherpunks For deeper integrations with the [ArDrive] platform, consider using the [ArDrive Core][ardrive-core] (Node) library's configurable and intuitive class interfaces directly within your application. To simply install the latest version of the CLI to your local system and get started, follow the [Quick Start](#quick-start) instructions. To build and/or develop the CLI from source, follow the [Build and Run from Source](#build-and-run-from-source) instructions. In either case, be sure to satisfy the requirements in the [Prerequisites](#prerequisites) section. # Limitations (/sdks/(clis)/ardrive-cli/limitations) **Number of files in a bulk upload:** Theoretically unlimited **Max individual file size**: 2GB (Node.js limitation) **Max file name length**: 255 bytes **Max ANS-104 bundled transaction size:** 500 MiB per bundle. App will handle creating multiple bundles. **Max ANS-104 data item counts per bundled transaction:** 250 Files per bundle (500 Data Items). # Using the CLI # Wallet Operations (/sdks/(clis)/ardrive-cli/wallet-operations) Browsing of ArDrive public data is possible without the need for an [Arweave wallet][kb-wallets]. However, for all write operations, or read operations without encryption/decryption keys, you'll need a wallet. As you utilize the CLI, you can use either your wallet file or your seed phrase interchangeably. Consider the security implications of each approach for your particular use case carefully. If at any time you'd like to generate a new wallet altogether, start by generating a new seed phase. And if you'd like to use that seed phrase in the form of a wallet file, or if you'd like to recover an existing wallet via its seed phrase, use either or both of the following commands: ```shell ardrive generate-seedphrase "this is an example twelve word seed phrase that you could use" ardrive generate-wallet -s "this is an example twelve word seed phrase that you could use" > /path/to/wallet/file.json ``` Public attributes of Arweave wallets can be retrieved via their 43-character Arweave wallet address. You can retrieve the wallet address associated with [your wallet file or 12-word seed phrase][kb-wallets] (e.g. wallets generated by [ArConnect][arconnect]) like so: ```shell ardrive get-address -w /path/to/wallet/file.json ardrive get-address -s "this is an example twelve word seed phrase that you could use" HTTn8F92tR32N8wuo-NIDkjmqPknrbl10JWo5MZ9x2k ``` You'll need AR in your wallet for any write operations you perform in ArDrive. You can always check your wallet balance (in both AR and Winston units) by performing: ```shell ardrive get-balance -w /path/to/wallet/file.json ardrive get-balance -a "HTTn8F92tR32N8wuo-NIDkjmqPknrbl10JWo5MZ9x2k" 1500000000000 Winston 1.5 AR ``` If, at any time, you need to send AR out of your wallet to another wallet address, you may perform: ```shell ardrive send-ar -w /path/to/wallet/file.json --dest-address "HTTn8F92tR32N8wuo-NIDkjmqPknrbl10JWo5MZ9x2k" --ar-amount 2.12345 ``` # Add the Skill to Your Project (/sdks/(clis)/ario-deploy/(claude-code-integration)/add-the-skill-to-your-project) ```bash mkdir -p .claude/skills curl -o .claude/skills/deploy.md https://raw.githubusercontent.com/ar-io/ar-io-deploy/main/examples/claude-skill/deploy.md ``` Then in Claude Code, say: - "deploy to ar.io" - "deploy my app to arweave" - "set up CI/CD for ar.io deployment" Claude will build your project, detect the output folder, and run the deploy with the right flags. # What the Skill Does (/sdks/(clis)/ario-deploy/(claude-code-integration)/what-the-skill-does) 1. **Detects your build folder** (`./dist`, `./build`, `./out`) 2. **Checks for credentials** (`DEPLOY_KEY` env var or wallet file) 3. **Installs `@ar.io/deploy`** if not already available 4. **Runs the deployment** with appropriate flags 5. **Reports results** — transaction ID, Arweave URL, ArNS URL See [`examples/claude-skill/`](./examples/claude-skill/) for the full skill file and customization options. --- # Advanced Usage (/sdks/(clis)/ario-deploy/(commands)/advanced-usage) Deploy to an undername (subdomain) — the ArNS authority key is a Solana wallet: ```bash ario-deploy deploy --use-arns --arns-name my-app --wallet ./wallet.json --arns-wallet ./arns-id.json --undername staging ``` Deploy with a custom TTL: ```bash ario-deploy deploy --use-arns --arns-name my-app --wallet ./wallet.json --arns-wallet ./arns-id.json --ttl-seconds 7200 ``` Update ArNS on devnet (or against a custom RPC): ```bash ario-deploy deploy --use-arns --arns-name my-app --wallet ./wallet.json --arns-wallet ./arns-id.json --cluster devnet ario-deploy deploy --use-arns --arns-name my-app --wallet ./wallet.json --arns-wallet ./arns-id.json --rpc-url https://my-rpc.example.com ``` Upload using an Ethereum wallet (file): ```bash ario-deploy deploy --sig-type ethereum --wallet ./private-key.txt ``` Upload using a Solana wallet (base58 private key): ```bash ario-deploy deploy --sig-type solana --private-key "\" ``` # Direct Commands (/sdks/(clis)/ario-deploy/(commands)/direct-commands) Use flags for faster, scriptable deployments: ```bash ario-deploy deploy --wallet ./wallet.json ario-deploy deploy --use-arns --arns-name my-app --wallet ./wallet.json --arns-wallet ./arns-id.json ``` Deploy using private key directly: ```bash ario-deploy deploy --private-key "$(cat wallet.json)" ``` Deploy using environment variable: ```bash DEPLOY_KEY=$(base64 -i wallet.json) ario-deploy deploy --deploy-folder ./dist ``` Deploy a specific folder: ```bash ario-deploy deploy --wallet ./wallet.json --deploy-folder ./build ``` Deploy a single file: ```bash ario-deploy deploy --wallet ./wallet.json --deploy-file ./path/to/file.txt ``` # Interactive Mode (Easiest) (/sdks/(clis)/ario-deploy/(commands)/interactive-mode-easiest) Run the deploy command without arguments to be guided through all deployment options: ```bash ario-deploy deploy ``` When ArNS details aren't supplied via flags, `deploy` asks whether you want to update an ArNS name (defaulting to yes) and, if so, prompts for the details. It will guide you through: - Whether to update an ArNS name (and which one) - Wallet method (file, string, or environment variable) - What to deploy (folder or file) - Advanced options (optional: undername, TTL, Solana cluster) Pass `--arns-name` (or `--use-arns`) to skip the ArNS confirmation, or use the `upload` command for an upload-only run. In a non-interactive environment (CI, or no TTY) `deploy` does not prompt — supply everything via flags or `DEPLOY_KEY`. # Upload/deploy without ArNS (/sdks/(clis)/ario-deploy/(commands)/upload-deploy-without-arns) `deploy` uploads without updating ArNS by default. You can also use the `upload` command explicitly for the same Turbo upload, dedupe cache, and payment options as deploy, minus ArNS flags: ```bash ario-deploy deploy --wallet ./wallet.json --deploy-folder ./dist ario-deploy upload --wallet ./wallet.json --deploy-folder ./dist ario-deploy upload --wallet ./wallet.json --deploy-file ./dist/index.html DEPLOY_KEY=$(base64 -i wallet.json) ario-deploy upload --deploy-folder ./dist ``` # Basic Usage (/sdks/(clis)/ario-deploy/(github-action)/basic-usage) ```yaml - uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} # upload key (pays for the upload) arns-key: ${{ secrets.ARNS_KEY }} # Solana ArNS authority key arns-name: myapp deploy-folder: ./dist ``` # Disabling Deduplication (/sdks/(clis)/ario-deploy/(github-action)/disabling-deduplication) By default, the action caches transaction IDs to avoid re-uploading unchanged files. To disable this: ```yaml - name: Deploy without dedupe uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} deploy-folder: ./dist no-dedupe: 'true' ``` You can also limit the cache size: ```yaml - name: Deploy with limited cache uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} deploy-folder: ./dist dedupe-cache-max-entries: '1000' ``` --- # PR Preview Deployments (/sdks/(clis)/ario-deploy/(github-action)/pr-preview-deployments) Automatically deploy preview builds for each pull request. The `preview` mode auto-generates an undername from the PR number and posts a comment with the preview URL: ```yaml name: Deploy PR Preview on: pull_request: types: [opened, synchronize] jobs: deploy-preview: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '20' - name: Install dependencies run: npm ci - name: Build run: npm run build - name: Deploy Preview uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} arns-key: ${{ secrets.ARNS_KEY }} arns-name: myapp preview: 'true' github-token: ${{ secrets.GITHUB_TOKEN }} deploy-folder: ./dist ``` When `preview` is enabled, the action will: - Auto-generate an undername like `pr-123` from the PR number - Post a comment on the PR with the preview URL - Update the comment on subsequent pushes instead of creating new ones # Production Deployment (/sdks/(clis)/ario-deploy/(github-action)/production-deployment) Deploy to your base ArNS name when pushing to main: ```yaml name: Deploy to Production on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '20' - name: Install dependencies run: npm ci - name: Build run: npm run build - name: Deploy to Permaweb uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} arns-key: ${{ secrets.ARNS_KEY }} arns-name: myapp deploy-folder: ./dist ``` # Updating ArNS (Solana) (/sdks/(clis)/ario-deploy/(github-action)/updating-arns-solana) ArNS updates run against the Solana ARIO programs. Provide the Solana ArNS authority key via `arns-key` (a base58 Solana secret key); the upload is still paid for by `deploy-key`. Use `cluster` to target `mainnet` (default) or `devnet`, and `rpc-url` for a custom RPC endpoint. ```yaml - name: Deploy and update ArNS uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} # upload key arns-key: ${{ secrets.ARNS_KEY }} # Solana ArNS authority key arns-name: myapp deploy-folder: ./dist cluster: mainnet ``` # With On-Demand Payment (/sdks/(clis)/ario-deploy/(github-action)/with-on-demand-payment) ```yaml - name: Deploy with ARIO on-demand uses: ar-io/ar-io-deploy@v1.0.0 with: deploy-key: ${{ secrets.DEPLOY_KEY }} arns-key: ${{ secrets.ARNS_KEY }} arns-name: myapp deploy-folder: ./dist on-demand: ario max-token-amount: '2.0' ``` # ArNS authority key (ARNS_KEY) (/sdks/(clis)/ario-deploy/(prerequisites)/arns-authority-key-arns-key) Set a base58-encoded **Solana** secret key as `ARNS_KEY`, or pass a `solana-keygen` `id.json` file via `--arns-wallet` (or a base58 string via `--arns-private-key`). This key must control the ArNS name being updated. ⚠️ **Important:** Use dedicated wallets for deployments to minimize security risks. Ensure your upload wallet has sufficient Turbo Credits for uploads. # Upload key (DEPLOY_KEY) (/sdks/(clis)/ario-deploy/(prerequisites)/upload-key-deploy-key) 1. **Arweave signer (default):** Encode your Arweave wallet key in base64 and set it as `DEPLOY_KEY`: ```bash base64 -i wallet.json | pbcopy ``` 2. **Ethereum/Polygon signers:** Use your raw private key (no encoding needed) as `DEPLOY_KEY`. 3. **Solana signer:** Use a base58-encoded secret key as `DEPLOY_KEY`, or a `solana-keygen` `id.json` byte-array wallet file via `--wallet`. # Bundler service (/sdks/(clis)/ario-deploy/bundler-service) Uploads go through a bundler service that accepts signed data items and posts them to Arweave. By default, ario-deploy uses the [Turbo](https://docs.ardrive.io/docs/turbo/) API and ArDrive’s production bundler (`https://upload.ardrive.io`). **`--uploader`** sets the **base URL** of the bundler service to use (scheme + host; typically no path). | When to use | Example value | | ------------------------- | ------------------------------------------------------- | | **Default** (omit flag) | ArDrive production bundler — same as Turbo CLI defaults | | **Arweave bundler** | `https://turbo.ardrive.io` | | **Development / staging** | `https://upload.services.ar-io.dev` | | **Custom or self-hosted** | Your own base URL if it implements the Turbo API | **Examples:** ```bash ario-deploy deploy --wallet ./wallet.json --deploy-folder ./dist --uploader https://turbo.ardrive.io ario-deploy upload --wallet ./wallet.json --deploy-folder ./dist --uploader https://turbo.ardrive.io ``` **Notes:** - Turbo billing and signer behavior follow Turbo. - Use a **base URL only** (e.g. `https://turbo.ardrive.io`), not a path to a specific file or route. # CLI in GitHub Actions (/sdks/(clis)/ario-deploy/cli-in-github-actions) You can also use the CLI directly in your workflows: **Basic Workflow:** ```yaml name: Deploy to Permaweb on: push: branches: - main jobs: publish: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: pnpm/action-setup@v3 with: version: 9 - uses: actions/setup-node@v4 with: node-version: 20 cache: 'pnpm' - run: pnpm install - run: pnpm deploy env: DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }} ``` **With On-Demand Payment:** ```yaml name: Deploy to Permaweb with On-Demand Payment on: push: branches: - main jobs: publish: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: pnpm/action-setup@v3 with: version: 9 - uses: actions/setup-node@v4 with: node-version: 20 cache: 'pnpm' - run: pnpm install - run: pnpm build - name: Deploy with ARIO on-demand run: ario-deploy deploy --arns-name my-app --on-demand ario --max-token-amount 2.0 env: DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }} # upload key (pays for the upload) ARNS_KEY: ${{ secrets.ARNS_KEY }} # Solana ArNS authority key # Or upload with Ethereum and Base-ETH on-demand payment (upload only; ArNS requires Solana): # - name: Upload with Base-ETH on-demand # run: | # ario-deploy upload \ # --sig-type ethereum \ # --on-demand base-eth \ # --max-token-amount 0.2 # env: # DEPLOY_KEY: ${{ secrets.ETH_PRIVATE_KEY }} ``` # Command Options (/sdks/(clis)/ario-deploy/command-options) **`deploy`** (upload by default, optional ArNS update): - `--use-arns`: Update an ArNS/ANT record after upload. When ArNS details aren't supplied and you're in a TTY, `deploy` asks by default. - `--arns-name, -n`: The ArNS name to update. Required when using `--use-arns`; also implies ArNS mode. - `--cluster, -p`: Solana cluster for ArNS updates. Choices: `mainnet`, `devnet`. Default: `mainnet` - `--rpc-url`: Optional Solana RPC URL override for ArNS updates - `--deploy-folder, -d`: Folder to deploy. Default: `./dist` - `--deploy-file, -f`: Deploy a single file instead of a folder - `--undername, -u`: ANT undername to update. Default: `@` - `--ttl-seconds, -t`: TTL in seconds for the ANT record (60-86400). Default: `60` Upload key (pays for the upload): - `--sig-type, -s`: Signer type for the upload key. Choices: `arweave`, `ethereum`, `polygon`, `solana`. Default: `arweave` - `--wallet, -w`: Path to the upload wallet file (JWK for Arweave, private key for Ethereum/Polygon, `solana-keygen` `id.json` for Solana). Falls back to `DEPLOY_KEY`. - `--private-key, -k`: Upload private-key string (alternative to `--wallet`). JWK JSON for Arweave, hex for EVM chains, base58 secret key for Solana. ArNS authority key (controls the name, signs the update — always Solana): - `--arns-wallet`: Path to the Solana `solana-keygen` `id.json` wallet that controls the ArNS name. Falls back to `ARNS_KEY`. - `--arns-private-key`: Base58 Solana secret key for the ArNS authority (alternative to `--arns-wallet`). Falls back to `ARNS_KEY`. - `--on-demand`: Enable on-demand payment with specified token. Choices: `ario`, `base-eth` - `--max-token-amount`: Maximum token amount for on-demand payment (used with `--on-demand`) - `--no-dedupe`: Disable deduplication (do not cache or reuse previous uploads) - `--dedupe-cache-max-entries`: Maximum number of entries to keep in the dedupe cache (LRU). Default: `10000` - `--uploader`: Custom Turbo upload service base URL. See the **Bundler service** section. **`upload`** (explicit upload without ArNS): accepts `--deploy-folder`, `--deploy-file`, wallet/signer flags, `--uploader`, `--on-demand` / `--max-token-amount`, and dedupe flags only. # Deduplication (/sdks/(clis)/ario-deploy/deduplication) By default, ario-deploy caches your deployment log to prevent uploading duplicate (unchanged) files. This saves both time and upload costs by reusing existing data on Arweave. **How it works:** 1. When you deploy, ario-deploy hashes each file in your build 2. It checks the local cache for matching hashes from previous uploads 3. Files that haven't changed are skipped - the existing transaction ID is reused 4. Only new or modified files are uploaded to Arweave 5. The cache is stored locally in `.ario-deploy/transaction-cache.json` **Disable deduplication:** If you need to force a fresh upload of all files (e.g., for debugging or to ensure a completely new deployment): ```bash ario-deploy deploy --wallet ./wallet.json --no-dedupe ``` **Limit cache size:** The dedupe cache uses an LRU (Least Recently Used) eviction strategy. By default, it keeps up to 10,000 entries. You can adjust this limit: ```bash ario-deploy deploy --wallet ./wallet.json --dedupe-cache-max-entries 1000 ``` **Cache location:** The cache file is stored at `.ario-deploy/transaction-cache.json` in your project root. You can: - Add it to `.gitignore` if you don't want to share cache across team members - Commit it to share cached transaction IDs with your team (reduces duplicate uploads) - Delete it to start fresh: `rm -rf .ario-deploy/` # Dependencies (/sdks/(clis)/ario-deploy/dependencies) - **@ar.io/sdk** - For ANT operations and ArNS management on Solana - **@ardrive/turbo-sdk** - For fast file uploads to Arweave (and signer types) - **@solana/kit** - Solana RPC clients and transaction signers for ArNS updates - **bs58** - Base58 encoding/decoding for Solana keys - **@oclif/core** - CLI framework - **mime-types** - MIME type detection # Features (/sdks/(clis)/ario-deploy/features) - **Turbo SDK Integration:** Uses Turbo SDK for fast, reliable file uploads to Arweave - **On-Demand Payment:** Pay with ARIO or Base-ETH tokens on-demand during upload - **Arweave Manifest v0.2.0:** Creates manifests with fallback support for SPAs - **Optional ArNS Updates:** Updates ArNS records via ANT with new transaction IDs - **Automated Workflow:** Integrates with GitHub Actions for continuous deployment - **Git Hash Tagging:** In CI (GitHub Actions), tags uploaded data items with the deploying commit SHA - **404 Fallback Detection:** Automatically detects and sets 404.html as fallback - **Network Support:** ArNS updates run against the Solana ARIO programs on `mainnet` or `devnet`, with an optional custom RPC URL - **Flexible Deployment:** Supports deploying a folder or a single file - **Modern CLI:** Built with oclif for a robust command-line experience - **TypeScript:** Fully typed for better developer experience # On-Demand Payment (/sdks/(clis)/ario-deploy/on-demand-payment) Use on-demand payment to automatically fund uploads with ARIO or Base-ETH tokens when your Turbo balance is insufficient: Deploy with ARIO on-demand payment: ```bash ario-deploy deploy --wallet ./wallet.json --deploy-folder ./dist --on-demand ario --max-token-amount 1.5 ``` Deploy with Base-ETH on-demand payment (using Ethereum signer): ```bash ario-deploy deploy --sig-type ethereum --private-key "0x..." --on-demand base-eth --max-token-amount 0.1 ``` **On-Demand Payment Options:** - `--on-demand`: Token to use for on-demand payment (`ario` or `base-eth`) - `--max-token-amount`: Maximum token amount to spend (in native token units, e.g., `1.5` for 1.5 ARIO or `0.1` for 0.1 ETH) **How it works:** 1. Checks your Turbo balance before upload 2. If balance is insufficient, converts tokens to Turbo credits on-demand 3. Automatically adds a 10% buffer (`topUpBufferMultiplier: 1.1`) for reliability 4. Proceeds with upload once funded **Token compatibility:** - **ARIO**: Works with Arweave signer - **Base-ETH**: Works with Ethereum signer (Base Network) # Package.json Scripts (/sdks/(clis)/ario-deploy/package-json-scripts) Add deployment scripts to your `package.json`: ```json { "scripts": { "build": "vite build", "deploy": "pnpm build && ario-deploy deploy --arns-name \", "deploy:staging": "pnpm build && ario-deploy deploy --arns-name \ --undername staging", "deploy:devnet": "pnpm build && ario-deploy deploy --arns-name \ --cluster devnet", "deploy:on-demand": "pnpm build && ario-deploy deploy --arns-name \ --on-demand ario --max-token-amount 1.5" } } ``` These read the upload key from `DEPLOY_KEY` and the Solana ArNS authority key from `ARNS_KEY`. Deploy with: ```bash DEPLOY_KEY=$(base64 -i wallet.json) ARNS_KEY=\ pnpm deploy ``` Or with on-demand payment: ```bash DEPLOY_KEY=$(base64 -i wallet.json) ARNS_KEY=\ pnpm deploy:on-demand ``` # Security & Best Practices (/sdks/(clis)/ario-deploy/security-best-practices) - **Dedicated Wallet:** Always use a dedicated wallet for deployments to minimize security risks - **Wallet Encoding:** Arweave wallets must be base64 encoded to be used in the deployment script - **ArNS Name:** Required only when updating an ANT/ArNS target undername or root record - **Turbo Credits:** Ensure your wallet has sufficient Turbo Credits, or use on-demand payment for automatic funding - **On-Demand Limits:** Set reasonable `--max-token-amount` limits to prevent unexpected costs - **Secret Management:** Keep your `DEPLOY_KEY` secret secure and never commit it to your repository - **Build Security:** Always check your build for exposed environmental secrets before deployment, as data on Arweave is permanent # Troubleshooting (/sdks/(clis)/ario-deploy/troubleshooting) - **Error: "DEPLOY_KEY environment variable not set":** Verify your base64 encoded wallet is set as the `DEPLOY_KEY` environment variable - **Error: "deploy-folder does not exist":** Check that your build folder exists and the path is correct - **Error: "deploy-file does not exist":** Check that your build file exists and the path is correct - **Error: "ArNS name does not exist":** Verify the ArNS name is correct and exists in the specified network - **Upload timeouts:** Files have a timeout for upload. Large files may fail and require optimization - **Insufficient Turbo Credits:** Use `--on-demand` with `--max-token-amount` to automatically fund uploads when balance is low - **On-demand payment fails:** Ensure your wallet has sufficient tokens (ARIO or Base-ETH) and the token type matches your signer (`ario` with Arweave, `base-eth` with Ethereum) # Networks (/sdks/ar-io-sdk/(advanced)/advanced-networks) | Network | RPC | Programs | |---|---|---| | Mainnet | `https://api.mainnet-beta.solana.com` (mainnet-beta, default) | Not yet deployed — placeholder IDs in `src/solana/constants.ts` | | Devnet | `https://api.devnet.solana.com` | See `src/solana/constants.ts` for current devnet program IDs | | Localnet | Surfpool — `https://github.com/solana-foundation/surfpool` | Localnet harness in `solana-ar-io` monorepo | The migration tooling (snapshot exporter, batch importer, claim app) lives in the [`solana-ar-io`](https://github.com/ar-io/solana-ar-io) monorepo until cutover. # Automatic Retries (/sdks/ar-io-sdk/(advanced)/automatic-retries) All RPC **read** calls (account fetches, `getProgramAccounts`, etc.) automatically retry on transient transport errors with exponential back-off. Writes are **not** retried (to avoid double-sends). Retried errors: HTTP 429/5xx, `fetch failed`, `ECONNRESET`, `ETIMEDOUT`, `AbortError` / timeouts. Non-retryable errors (account not found, invalid params, deserialization) throw immediately. Defaults: **6 attempts**, 500 ms base delay, 5 s max delay. Override per-call with the exported `withRetry` helper: ```ts const result = await withRetry(() => rpc.getAccountInfo(addr).send(), { maxAttempts: 3, baseDelayMs: 1000, }); ``` # Circuit Breaker (/sdks/ar-io-sdk/(advanced)/circuit-breaker) The SDK ships an [opossum]-backed circuit breaker that wraps the RPC transport. When the primary endpoint starts failing (429 rate-limits, 5xx errors, network timeouts) the circuit opens and subsequent calls route transparently to a fallback RPC until the primary recovers. ```ts const rpc = createCircuitBreakerRpc({ primaryUrl: 'https://my-premium-rpc.example.com', fallbackUrl: 'https://api.mainnet-beta.solana.com', }); const ario = ARIO.init({ rpc }); ``` Use `defaultFallbackUrl()` to auto-pick mainnet or devnet based on the primary URL: ```ts import { createCircuitBreakerRpc, defaultFallbackUrl, } from '@ar.io/sdk'; const primaryUrl = 'https://my-premium-rpc.example.com'; const rpc = createCircuitBreakerRpc({ primaryUrl, fallbackUrl: defaultFallbackUrl(primaryUrl), // → mainnet public RPC }); ``` Tuning knobs (all optional): | Option | Default | Description | |---|---|---| | `timeout` | `10000` | ms before a single request is timed out (`false` to disable) | | `errorThresholdPercentage` | `50` | error % at which to open the circuit | | `resetTimeout` | `30000` | ms to wait before probing the primary again (half-open) | | `volumeThreshold` | `5` | minimum requests in the rolling window before the circuit can trip | # Generated instruction builders (/sdks/ar-io-sdk/(advanced)/generated-instruction-builders) For custom transaction building, import Codama-generated typed clients from [`@ar.io/solana-contracts`](https://www.npmjs.com/package/@ar.io/solana-contracts): ```ts import { getBuyNameInstructionAsync, ARIO_ARNS_PROGRAM_ADDRESS, } from '@ar.io/solana-contracts/arns'; ``` # RPC Configuration (/sdks/ar-io-sdk/(advanced)/rpc-configuration) The SDK accepts any `@solana/kit` RPC client. For read-only usage, only `rpc` is required. Write operations additionally need `rpcSubscriptions` (WebSocket) for transaction confirmation and a `signer`. #### Basic (read-only) ```ts const rpc = createSolanaRpc('https://api.mainnet-beta.solana.com'); const ario = ARIO.init({ rpc }); ``` #### With writes (signer + WebSocket subscriptions) ```ts import { createSolanaRpc, createSolanaRpcSubscriptions, createKeyPairSignerFromBytes, } from '@solana/kit'; const rpc = createSolanaRpc('https://api.mainnet-beta.solana.com'); const rpcSubscriptions = createSolanaRpcSubscriptions( 'wss://api.mainnet-beta.solana.com', ); const signer = await createKeyPairSignerFromBytes(/* ... */); const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); ``` > **Note:** `rpcSubscriptions` opens a WebSocket connection and is only > needed for writes. If your RPC provider doesn't expose a WebSocket > endpoint, omit it and use the SDK in read-only mode. # ARIO Integrations (/sdks/ar-io-sdk/(ant-contracts)/ario-integrations) `releaseName`, `reassignName`, `approvePrimaryNameRequest`, and `removePrimaryNames` were AO-only orchestration helpers and have been removed. Their on-chain equivalents on Solana live on the `ario-arns` program and are exposed through the `ARIO` write client (`upgradeRecord`, `setPrimaryName`, etc.) or — for permissionless maintenance — through `SolanaARIOWriteable`'s prune helpers. # Balances (/sdks/ar-io-sdk/(ant-contracts)/balances) #### getBalances() Returns all token balances for the ANT. ```typescript const balances = await ant.getBalances(); ``` **Output:** ```json { "ccp3blG__gKUvG3hsGC2u06aDmqv4CuhuDJGOIg0jw4": 1, "aGzM_yjralacHIUo8_nQXMbh9l1cy0aksiL_x9M359f": 0 } ``` #### getBalance() Returns the balance of a specific address. ```typescript const balance = await ant.getBalance({ address: "ccp3blG__gKUvG3hsGC2u06aDmqv4CuhuDJGOIg0jw4", }); ``` **Output:** ```json 1 ``` # Controllers (/sdks/ar-io-sdk/(ant-contracts)/controllers) #### addController() Adds a new controller to the list of approved controllers on the ANT. Controllers can set records and change the ticker and name of the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.addController( { controller: "aGzM_yjralacHIUo8_nQXMbh9l1cy0aksiL_x9M359f" }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### removeController() Removes a controller from the list of approved controllers on the ANT. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.removeController( { controller: "aGzM_yjralacHIUo8_nQXMbh9l1cy0aksiL_x9M359f" }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` # Initialize (/sdks/ar-io-sdk/(ant-contracts)/initialize) #### init() Factory that creates a read-only or writeable ANT client. Providing `signer` and `rpcSubscriptions` enables write methods (`setRecord`, `transfer`, `addController`, etc.). ```typescript import { createSolanaRpc, createSolanaRpcSubscriptions, } from '@solana/kit'; const rpc = createSolanaRpc('https://api.mainnet-beta.solana.com'); // Read-only const ant = await ANT.init({ processId: '\', rpc, }); // Read + write const antWrite = await ANT.init({ processId: '\', rpc, rpcSubscriptions: createSolanaRpcSubscriptions( 'wss://api.mainnet-beta.solana.com', ), signer, }); ``` # Metadata (/sdks/ar-io-sdk/(ant-contracts)/metadata) #### setName() Sets the name of the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.setName( { name: "My ANT" }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### setTicker() Sets the ticker of the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.setTicker( { ticker: "ANT-NEW-TICKER" }, // optional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### setDescription() Sets the description of the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.setDescription( { description: "A friendly description of this ANT" }, // optional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### setKeywords() Sets the keywords of the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.setKeywords( { keywords: ["Game", "FPS", "AO"] }, // optional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### getLogo() Returns the TX ID of the logo set for the ANT. ```typescript const logoTxId = await ant.getLogo(); ``` #### setLogo() Sets the Logo of the ANT - logo should be an Arweave transaction ID. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.setLogo( { txId: "U7RXcpaVShG4u9nIcPVmm2FJSM5Gru9gQCIiRaIPV7f" }, // optional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` # Records (/sdks/ar-io-sdk/(ant-contracts)/records) #### setBaseNameRecord() Adds or updates the base name record for the ANT. This is the top level name of the ANT (e.g. ardrive.ar.io). Supports undername ownership delegation and metadata. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript // get the ant for the base name const arnsRecord = await ario.getArNSRecord({ name: "ardrive" }); const ant = await ANT.init({ processId: arnsName.processId }); // Basic usage const { id: txId } = await ant.setBaseNameRecord({ transactionId: "432l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", ttlSeconds: 3600, }); // With ownership delegation and metadata const { id: txId } = await ant.setBaseNameRecord({ transactionId: "432l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", ttlSeconds: 3600, owner: "user-wallet-address-123...", // delegate ownership to another address displayName: "ArDrive", // display name logo: "logo-tx-id-123...", // logo transaction ID description: "Decentralized storage application", keywords: ["storage", "decentralized", "web3"], }); // ardrive.ar.io will now resolve to the provided transaction id and include metadata ``` #### setUndernameRecord() Adds or updates an undername record for the ANT. An undername is appended to the base name of the ANT (e.g. dapp_ardrive.ar.io). Supports undername ownership delegation and metadata. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ > Records, or `undernames` are configured with the `transactionId` - the arweave transaction id the record resolves - and `ttlSeconds`, the Time To Live in the cache of client applications. ```typescript const arnsRecord = await ario.getArNSRecord({ name: "ardrive" }); const ant = await ANT.init({ processId: arnsName.processId }); // Basic usage const { id: txId } = await ant.setUndernameRecord( { undername: "dapp", transactionId: "432l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", ttlSeconds: 900, }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); // With ownership delegation and metadata const { id: txId } = await ant.setUndernameRecord( { undername: "alice", transactionId: "432l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", ttlSeconds: 900, owner: "alice-wallet-address-123...", // delegate ownership to Alice displayName: "Alice's Site", // display name logo: "avatar-tx-id-123...", // avatar/logo transaction ID description: "Personal portfolio and blog", keywords: ["portfolio", "personal", "blog"], }, { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); // dapp_ardrive.ar.io will now resolve to the provided transaction id // alice_ardrive.ar.io will be owned by Alice and include metadata ``` #### removeUndernameRecord() Removes an undername record from the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.removeUndernameRecord( { undername: "dapp" }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); // dapp_ardrive.ar.io will no longer resolve to the provided transaction id ``` #### setRecord() Deprecated: Use `setBaseNameRecord` or `setUndernameRecord` instead. Adds or updates a record for the ANT process. The `undername` parameter is used to specify the record name. Use `@` for the base name record. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ > Records, or `undernames` are configured with the `transactionId` - the arweave transaction id the record resolves - and `ttlSeconds`, the Time To Live in the cache of client applications. ```typescript const { id: txId } = await ant.setRecord( { undername: '@', transactionId: '432l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM' ttlSeconds: 3600 }, // optional additional tags { tags: [{ name: 'App-Name', value: 'My-Awesome-App' }] }, ); ``` #### removeRecord() Deprecated: Use `removeUndernameRecord` instead. Removes a record from the ANT process. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const arnsRecord = await ario.getArNSRecord({ name: "ardrive" }); const ant = await ANT.init({ processId: arnsName.processId }); const { id: txId } = await ant.removeRecord( { undername: "dapp" }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); // dapp_ardrive.ar.io will no longer resolve to the provided transaction id ``` # Spawn (/sdks/ar-io-sdk/(ant-contracts)/spawn) #### ANT.spawn() Static factory that mints a new MPL Core asset and initializes the `ario-ant` PDAs in a single transaction. Returns `{ processId, mint, signature }`. ```typescript const { processId, signature } = await ANT.spawn({ rpc, rpcSubscriptions, signer, state: { name: 'My ANT', ticker: 'MYANT', description: 'My ANT token', uri: 'ar://\', }, }); ``` **CLI Usage:** ```bash ar.io spawn-ant \ --wallet-file wallet.json \ --name "My ANT" \ --ticker "MYANT" \ --metadata-uri "ar://\" ``` **Parameters:** - `state.name: string` — display name of the ANT - `state.ticker?: string` — ticker symbol - `state.description?: string` — short description - `state.uri: string` — `ar://` URI of the Metaplex Core asset's JSON metadata. Build via `buildAntMetadata` from `@ar.io/sdk` and upload to Arweave (e.g. via `@ardrive/turbo-sdk`). - `state.keywords?: string[]` - `state.logo?: string` — Arweave TX ID of the logo - `state.transactionId?: string` — initial `@` record target **Returns:** ```ts { processId: string; // the MPL Core asset mint pubkey mint: Address; signature: string; // the Solana tx signature } ``` # State (/sdks/ar-io-sdk/(ant-contracts)/state) #### getInfo() Retrieves the information of the ANT process. ```typescript const info = await ant.getInfo(); ``` **Output:** ```json { "Name": "ArDrive", "Owner": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "Ticker": "ANT-ARDRIVE", "Total-Supply": "1", "Description": "This is the ANT for the ArDrive decentralized web app.", "Keywords": ["File-sharing", "Publishing", "dApp"], "Logo": "Sie_26dvgyok0PZD_-iQAFOhOd5YxDTkczOLoqTTL_A", "Denomination": "0", "Handlers": [ "balance", "balances", "totalSupply", "info", "controllers", "record", "records", "state", "transfer", "addController", "removeController", "setRecord", "removeRecord", "setName", "setTicker", "setDescription", "setKeywords", "setLogo", "initializeState", "releaseName", "reassignName", "approvePrimaryName", "removePrimaryNames", "transferRecordOwnership", "_eval", "_default" ] } ``` #### getHandlers() Retrieves the handlers supported on the ANT ```typescript const handlers = await ant.getHandlers(); ``` **Output:** ```json [ "balance", "balances", "totalSupply", "info", "controllers", "record", "records", "state", "transfer", "addController", "removeController", "setRecord", "removeRecord", "setName", "setTicker", "setDescription", "setKeywords", "setLogo", "initializeState", "releaseName", "reassignName", "approvePrimaryName", "removePrimaryNames", "transferRecordOwnership", "_eval", "_default" ] ``` #### getState() Retrieves the state of the ANT process. ```typescript const state = await ant.getState(); ``` **Output:** ```json { "Name": "ar.io Foundation", "Ticker": "ANT-AR-IO", "Description": "A friendly description for this ANT.", "Keywords": ["keyword1", "keyword2", "keyword3"], "Denomination": 0, "Owner": "98O1_xqDLrBKRfQPWjF5p7xZ4Jx6GM8P5PeJn26xwUY", "Controllers": [], "Records": { "v1-0-0_whitepaper": { "transactionId": "lNjWn3LpyhKC95Kqe-x8X2qgju0j98MhucdDKK85vc4", "ttlSeconds": 900, "targetProtocol": 0 }, "@": { "transactionId": "2rMLb2uHAyEt7jSu6bXtKx8e-jOfIf7E-DOgQnm8EtU", "ttlSeconds": 3600, "targetProtocol": 0 }, "alice": { "transactionId": "kMk95k_3R8x_7d3wB9tEOiL5v6n8QhR_VnFCh3aeE3f", "ttlSeconds": 900, "targetProtocol": 0, "owner": "alice-wallet-address-123...", "displayName": "Alice's Portfolio", "logo": "avatar-tx-id-456...", "description": "Personal portfolio and blog", "keywords": ["portfolio", "personal", "blog"] }, "whitepaper": { "transactionId": "lNjWn3LpyhKC95Kqe-x8X2qgju0j98MhucdDKK85vc4", "ttlSeconds": 900, "targetProtocol": 0 } }, "Balances": { "98O1_xqDLrBKRfQPWjF5p7xZ4Jx6GM8P5PeJn26xwUY": 1 }, "Logo": "Sie_26dvgyok0PZD_-iQAFOhOd5YxDTkczOLoqTTL_A", "TotalSupply": 1, "Initialized": true } ``` #### getOwner() Returns the owner of the configured ANT process. ```typescript const owner = await ant.getOwner(); ``` **Output:** ```json "ccp3blG__gKUvG3hsGC2u06aDmqv4CuhuDJGOIg0jw4" ``` #### getName() Returns the name of the ANT (not the same as ArNS name). ```typescript const name = await ant.getName(); ``` **Output:** ```json "ArDrive" ``` #### getTicker() Returns the ticker symbol of the ANT. ```typescript const ticker = await ant.getTicker(); ``` **Output:** ```json "ANT-ARDRIVE" ``` #### getControllers() Returns the controllers of the configured ANT process. ```typescript const controllers = await ant.getControllers(); ``` **Output:** ```json ["ccp3blG__gKUvG3hsGC2u06aDmqv4CuhuDJGOIg0jw4"] ``` #### getRecords() Returns all records on the configured ANT process, including the required `@` record that resolve connected ArNS names. ```typescript const records = await ant.getRecords(); ``` **Output:** ```json { "@": { "transactionId": "UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk", "ttlSeconds": 3600, "targetProtocol": 0, "index": 0 }, "alice": { "transactionId": "kMk95k_3R8x_7d3wB9tEOiL5v6n8QhR_VnFCh3aeE3f", "ttlSeconds": 900, "targetProtocol": 0, "owner": "alice-wallet-address-123...", "displayName": "Alice's Portfolio", "logo": "avatar-tx-id-456...", "description": "Personal portfolio and blog", "keywords": ["portfolio", "personal", "blog"], "index": 1 }, "zed": { "transactionId": "-k7t8xMoB8hW482609Z9F4bTFMC3MnuW8bTvTyT8pFI", "ttlSeconds": 900, "targetProtocol": 0, "index": 2 }, "ardrive": { "transactionId": "-cucucachoodwedwedoiwepodiwpodiwpoidpwoiedp", "ttlSeconds": 900, "targetProtocol": 0, "index": 3 } } ``` #### getRecord() Returns a specific record by its undername. ```typescript const record = await ant.getRecord({ undername: "dapp" }); ``` **Output:** ```json { "transactionId": "432l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "ttlSeconds": 900, "targetProtocol": 0, "owner": "alice-wallet-address-123...", "displayName": "Alice's Site", "logo": "avatar-tx-id-456...", "description": "Personal portfolio and blog", "keywords": ["portfolio", "personal", "blog"] } ``` # Static Methods (/sdks/ar-io-sdk/(ant-contracts)/static-methods) `ANT.fork()` and the static `ANT.upgrade()` were AO-only (process forking + name reassignment). On Solana, schema migration is a per-asset CPI exposed as the instance method `ant.upgrade()` documented above; new ANTs are created with `ANT.spawn()`. # Transfer (/sdks/ar-io-sdk/(ant-contracts)/transfer) #### transfer() Transfers ownership of the ANT to a new target address. Target must be a Solana address. ANT transfers are standard Metaplex Core NFT transfers. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.transfer( { target: "aGzM_yjralacHIUo8_nQXMbh9l1cy0aksiL_x9M359f" }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` # Undername Ownership (/sdks/ar-io-sdk/(ant-contracts)/undername-ownership) NTs support ownership of undernames: 1. **ANT Owner** - Has full control over the ANT and all records 2. **Controllers** - Can manage records but cannot transfer ANT ownership 3. **Record Owners** - Can only update their specific delegated records When a record owner updates their own record, they **MUST** include their own address in the `owner` field. If the `owner` field is omitted or set to a different address, the record ownership will be transferred or renounced. #### transferRecord() Transfers ownership of a specific record (undername) to another address. This enables delegation of control for individual records within an ANT while maintaining the ANT owner's ultimate authority. The current record owner or ANT owner/controllers can transfer ownership. _Note: Requires `signer` to be provided on `ANT.init` to sign the transaction._ ```typescript const { id: txId } = await ant.transferRecord({ undername: "alice", // the subdomain/record to transfer recipient: "new-owner-address-123...", // address of the new owner }); // alice_ardrive.ar.io is now owned by the new owner address // The new owner can update the record but not other records in the ANT ``` **CLI Usage:** ```bash ar.io transfer-record \ --process-id "ANT_PROCESS_ID" \ --undername "alice" \ --recipient "new-owner-address-123..." \ --wallet-file "path/to/wallet.json" ``` #### Record Owner Workflow Examples **Checking Record Ownership:** ```typescript const record = await ant.getRecord({ undername: "alice" }); console.log(`Record owner: ${record.owner}`); console.log(`Transaction ID: ${record.transactionId}`); ``` **Record Owner Updating Their Own Record:** ```typescript // Alice (record owner) updating her own record const aliceAnt = await ANT.init({ processId: 'ANT_MINT_PUBKEY', rpc, rpcSubscriptions, signer: aliceSigner, // Alice's @solana/kit signer }); // ✅ CORRECT: Alice includes her own address as owner const { id: txId } = await aliceAnt.setUndernameRecord({ undername: "alice", transactionId: "new-content-tx-id-456...", ttlSeconds: 1800, owner: "alice-wallet-address-123...", // MUST be Alice's own address displayName: "Alice Updated Portfolio", description: "Updated personal portfolio and blog", }); // ❌ WRONG: Omitting owner field will renounce ownership const badUpdate = await aliceAnt.setUndernameRecord({ undername: "alice", transactionId: "new-content-tx-id-456...", ttlSeconds: 1800, // Missing owner field - this will renounce ownership! }); // ❌ WRONG: Setting different owner will transfer ownership const badTransfer = await aliceAnt.setUndernameRecord({ undername: "alice", transactionId: "new-content-tx-id-456...", ttlSeconds: 1800, owner: "someone-else-address-789...", // This transfers ownership to someone else! }); ``` **What Happens When Record Ownership is Renounced:** If a record owner updates their record without including the `owner` field, the record becomes owned by the ANT owner/controllers again: ```typescript // Before: alice record is owned by alice-wallet-address-123... const recordBefore = await ant.getRecord({ undername: "alice" }); console.log(recordBefore.owner); // "alice-wallet-address-123..." // Alice updates without owner field await aliceAnt.setUndernameRecord({ undername: "alice", transactionId: "new-tx-id...", ttlSeconds: 900, // No owner field = renounces ownership }); // After: record ownership reverts to ANT owner const recordAfter = await ant.getRecord({ undername: "alice" }); console.log(recordAfter.owner); // undefined (controlled by ANT owner again) ``` # Upgrade (/sdks/ar-io-sdk/(ant-contracts)/upgrade) #### upgrade() Migrates this ANT's on-chain state to the latest schema version (per- ANT data migration on Solana — no process forking, no name reassignment). Returns `{ id, needsMigration }`. ```typescript const result = await ant.upgrade(); if (result.needsMigration) { console.log(`Migrated: ${result.id}`); } ``` # Versions (/sdks/ar-io-sdk/(ant-contracts)/versions) #### getModuleId() Gets the module ID of the current ANT process by querying its spawn transaction tags. Results are cached after the first successful fetch. ```typescript const moduleId = await ant.getModuleId(); console.log(`ANT was spawned with module: ${moduleId}`); // With custom GraphQL URL and retries const moduleId = await ant.getModuleId({ graphqlUrl: "https://turbo-gateway.com/graphql", retries: 5, }); ``` **Output:** ```json "FKtQtOOtlcWCW2pXrwWFiCSlnuewMZOHCzhulVkyqBE" ``` #### getVersion() Gets the version string of the current ANT by matching its module ID with versions from the ANT registry. ```typescript const version = await ant.getVersion(); console.log(`ANT is running version: ${version}`); // With custom ANT registry const version = await ant.getVersion({ antRegistryId: "custom-ant-registry-id", }); ``` **Output:** ```json "23" ``` #### isLatestVersion() Checks if the current ANT version is the latest according to the ANT registry. ```typescript const isLatest = await ant.isLatestVersion(); if (!isLatest) { console.log("ANT can be upgraded to the latest version"); } ``` **Output:** ```json true ``` # ar.io Name System (ArNS) (/sdks/ar-io-sdk/(ario-contract)/ar-io-name-system-arns) #### resolveArNSName() Resolves an ArNS name to the underlying data id stored on the names corresponding ANT id. ##### Resolving a base name ```typescript const ario = ARIO.init({ rpc }); const record = await ario.resolveArNSName({ name: "ardrive" }); ``` **Output:** ```json { "name": "ardrive", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "txId": "kvhEUsIY5bXe0Wu2-YUFz20O078uYFzmQIO-7brv8qw", "type": "lease", "ttlSeconds": 3600, "undernameLimit": 100 } ``` ##### Resolving an undername ```typescript const ario = ARIO.init({ rpc }); const record = await ario.resolveArNSName({ name: "logo_ardrive" }); ``` **Output:** ```json { "name": "ardrive", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "txId": "kvhEUsIY5bXe0Wu2-YUFz20O078uYFzmQIO-7brv8qw", "type": "lease", "ttlSeconds": 3600, "undernameLimit": 100 } ``` #### buyRecord() Purchases a new ArNS record with the specified name, type, processId, and duration. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ **Arguments:** - `name` - _required_: the name of the ArNS record to purchase - `type` - _required_: the type of ArNS record to purchase - `processId` - _optional_: the process id of an existing ANT process. If not provided, a new ANT process using the provided `signer` will be spawned, and the ArNS record will be assigned to that process. - `years` - _optional_: the duration of the ArNS record in years. If not provided and `type` is `lease`, the record will be leased for 1 year. If not provided and `type` is `permabuy`, the record will be permanently registered. - `referrer` - _optional_: track purchase referrals for analytics (e.g. `my-app.com`) ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const record = await ario.buyRecord( { name: "ardrive", type: "lease", years: 1, processId: "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", // optional: assign to existing ANT process referrer: "my-app.com", // optional: track purchase referrals for analytics }, { // optional tags tags: [{ name: "App-Name", value: "ArNS-App" }], onSigningProgress: (step, event) => { console.log(`Signing progress: ${step}`); if (step === "spawning-ant") { console.log("Spawning ant:", event); } if (step === "registering-ant") { console.log("Registering ant:", event); } if (step === "verifying-state") { console.log("Verifying state:", event); } if (step === "buying-name") { console.log("Buying name:", event); } }, }, ); ``` #### upgradeRecord() Upgrades an existing leased ArNS record to a permanent ownership. The record must be currently owned by the caller and be of type "lease". _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const record = await ario.upgradeRecord( { name: "ardrive", referrer: "my-app.com", // optional: track purchase referrals for analytics }, { // optional tags tags: [{ name: "App-Name", value: "ArNS-App" }], }, ); ``` #### getArNSRecord() Retrieves the record info of the specified ArNS name. ```typescript const ario = ARIO.init({ rpc }); const record = await ario.getArNSRecord({ name: "ardrive" }); ``` **Output:** ```json { "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "startTimestamp": 1720720819969, "endTimestamp": 1752256702026, "type": "lease", "undernameLimit": 100, "purchasePrice": 75541282285 } ``` #### getArNSRecords() Retrieves all registered ArNS records of the ARIO process, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last ArNS name from the previous request. ```typescript const ario = ARIO.init({ rpc }); // get the newest 100 names const records = await ario.getArNSRecords({ limit: 100, sortBy: "startTimestamp", sortOrder: "desc", }); ``` Available `sortBy` options are any of the keys on the record object, e.g. `name`, `processId`, `endTimestamp`, `startTimestamp`, `type`, `undernames`. **Output:** ```json { "items": [ { "name": "ao", "processId": "eNey-H9RB9uCdoJUvPULb35qhZVXZcEXv8xds4aHhkQ", "purchasePrice": 75541282285, "startTimestamp": 1720720621424, "endTimestamp": 1752256702026, "type": "permabuy", "undernameLimit": 10 }, { "name": "ardrive", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "endTimestamp": 1720720819969, "startTimestamp": 1720720620813, "purchasePrice": 75541282285, "type": "lease", "undernameLimit": 100 }, { "name": "arweave", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "endTimestamp": 1720720819969, "startTimestamp": 1720720620800, "purchasePrice": 75541282285, "type": "lease", "undernameLimit": 100 }, { "name": "ar-io", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "endTimestamp": 1720720819969, "startTimestamp": 1720720619000, "purchasePrice": 75541282285, "type": "lease", "undernameLimit": 100 }, { "name": "fwd", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "endTimestamp": 1720720819969, "startTimestamp": 1720720220811, "purchasePrice": 75541282285, "type": "lease", "undernameLimit": 100 } // ...95 other records ], "hasMore": true, "nextCursor": "fwdresearch", "totalItems": 21740, "sortBy": "startTimestamp", "sortOrder": "desc" } ``` #### getArNSRecordsForAddress() Retrieves all registered ArNS records of the specified address according to the `ANTRegistry` access control list, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last ArNS name from the previous request. ```typescript const ario = ARIO.init({ rpc }); const records = await ario.getArNSRecordsForAddress({ address: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", limit: 100, sortBy: "startTimestamp", sortOrder: "desc", }); ``` Available `sortBy` options are any of the keys on the record object, e.g. `name`, `processId`, `endTimestamp`, `startTimestamp`, `type`, `undernames`. **Output:** ```json { "limit": 1, "totalItems": 31, "hasMore": true, "nextCursor": "ardrive", "items": [ { "startTimestamp": 1740009600000, "name": "ardrive", "endTimestamp": 1777328018367, "type": "permabuy", "purchasePrice": 0, "undernameLimit": 100, "processId": "hpF0HdijWlBLFePjWX6u_-Lg3Z2E_PrP_AoaXDVs0bA" } ], "sortOrder": "desc", "sortBy": "startTimestamp" } ``` #### increaseUndernameLimit() Increases the undername support of a domain up to a maximum of 10k. Domains, by default, support up to 10 undernames. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.increaseUndernameLimit( { name: "ar-io", qty: 420, referrer: "my-app.com", // optional: track purchase referrals for analytics }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### extendLease() Extends the lease of a registered ArNS domain, with an extension of 1-5 years depending on grace period status. Permanently registered domains cannot be extended. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.extendLease( { name: "ar-io", years: 1, referrer: "my-app.com", // optional: track purchase referrals for analytics }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### getTokenCost() Calculates the price in mARIO to perform the interaction in question, eg a 'Buy-Name' interaction, where args are the specific params for that interaction. ```typescript const price = await ario .getTokenCost({ intent: "Buy-Name", name: "ar-io", type: "permabuy", }) .then((p) => new mARIOToken(p).toARIO()); // convert to ARIO for readability ``` **Output:** ```json 1642.34 ``` #### getCostDetails() Calculates the expanded cost details for the interaction in question, e.g a 'Buy-Name' interaction, where args are the specific params for that interaction. The fromAddress is the address that would be charged for the interaction, and fundFrom is where the funds would be taken from, either `balance`, `stakes`, or `any`. On Solana, the result also includes a `gasEstimate` — the total SOL (in lamports) the wallet needs to execute the intent: transaction fees (quoted from recent on-chain prioritization fees) plus rent-exempt deposits for the accounts the flow creates. For `Buy-Name` that covers both transactions (ANT spawn + buy) and the rent for the spawned asset/PDAs and the ArNS record; first-time buyers with no ACL accounts yet are quoted the ACL bootstrap rent as well (pass `fromAddress` so that check can be made). The fee side is a conservative upper bound: the write path tightens the compute-unit limit from a pre-send simulation, so the landed fee is usually lower. ```typescript const costDetails = await ario.getCostDetails({ intent: "Buy-Name", fromAddress: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", fundFrom: "stakes", name: "ar-io", type: "permabuy", }); ``` **Output:** ```json { "tokenCost": 1907401818, "discounts": [ { "name": "Gateway Operator", "discountTotal": 476850455, "multiplier": 0.8 } ], "gasEstimate": { "totalLamports": 14679680, "feeLamports": 23000, "baseFeeLamports": 15000, "priorityFeeLamports": 8000, "rentLamports": 14656680, "rentReclaimedLamports": 0, "priorityFeeMicroLamports": 10000, "computeUnitLimit": 400000, "signatureCount": 3, "transactionCount": 2 } } ``` #### getDemandFactor() Retrieves the current demand factor of the network. The demand factor is a multiplier applied to the cost of ArNS interactions based on the current network demand. ```typescript const ario = ARIO.init({ rpc }); const demandFactor = await ario.getDemandFactor(); ``` **Output:** ```json 1.05256 ``` #### getArNSReturnedNames() Retrieves all active returned names of the ARIO process, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last returned name from the previous request. ```typescript const ario = ARIO.init({ rpc }); const returnedNames = await ario.getArNSReturnedNames({ limit: 100, sortBy: "endTimestamp", sortOrder: "asc", // return the returned names ending soonest first }); ``` **Output:** ```json { "items": [ { "name": "permalink", "startTimestamp": 1729775641349, "endTimestamp": 1730985241349, "initiator": "GaQrvEMKBpkjofgnBi_B3IgIDmY_XYelVLB6GcRGrHc", "premiumMultiplier": 50 } ], "hasMore": false, "totalItems": 1, "limit": 100, "sortBy": "endTimestamp", "sortOrder": "asc" } ``` #### getArNSReturnedName() Retrieves the returned name data for the specified returned name. ```typescript const ario = ARIO.init({ rpc }); const returnedName = await ario.getArNSReturnedName({ name: "permalink" }); ``` **Output:** ```json { "name": "permalink", "startTimestamp": 1729775641349, "endTimestamp": 1730985241349, "initiator": "GaQrvEMKBpkjofgnBi_B3IgIDmY_XYelVLB6GcRGrHc", "premiumMultiplier": 50 } ``` # Configuration (/sdks/ar-io-sdk/(ario-contract)/configuration) `ARIO.init` accepts a `@solana/kit` RPC client plus optional program ID overrides for non-mainnet clusters. See [Networks](#networks) above for the full shape. # Epochs (/sdks/ar-io-sdk/(ario-contract)/epochs) #### getCurrentEpoch() Returns the current epoch data. ```typescript const ario = ARIO.init({ rpc }); const epoch = await ario.getCurrentEpoch(); ``` **Output:** ```json { "epochIndex": 0, "startHeight": 0, "startTimestamp": 1720720621424, "endTimestamp": 1752256702026, "distributionTimestamp": 1752256702026, "observations": { "failureSummaries": { "-Tk2DDk8k4zkwtppp_XFKKI5oUgh6IEHygAoN7mD-w8": [ "Ie2wEEUDKoU26c7IuckHNn3vMFdNQnMvfPBrFzAb3NA" ] }, "reports": { "IPdwa3Mb_9pDD8c2IaJx6aad51Ss-_TfStVwBuhtXMs": "B6UUjKWjjEWDBvDSMXWNmymfwvgR9EN27z5FTkEVlX4" } }, "prescribedNames": ["ardrive", "ar-io", "arweave", "fwd", "ao"], "prescribedObservers": [ { "gatewayAddress": "2Fk8lCmDegPg6jjprl57-UCpKmNgYiKwyhkU4vMNDnE", "observerAddress": "2Fk8lCmDegPg6jjprl57-UCpKmNgYiKwyhkU4vMNDnE", "stake": 10000000000, "startTimestamp": 1720720621424, "stakeWeight": 1, "tenureWeight": 0.4494598765432099, "gatewayPerformanceRatio": 1, "observerPerformanceRatio": 1, "gatewayRewardRatioWeight": 1, "observerRewardRatioWeight": 1, "compositeWeight": 0.4494598765432099, "normalizedCompositeWeight": 0.002057032496835938 } ], "distributions": { "totalEligibleGateways": 1, "totalEligibleRewards": 100000000, "totalEligibleObserverReward": 100000000, "totalEligibleGatewayReward": 100000000 }, "arnsStats": { "totalReturnedNames": 0, "totalActiveNames": 0, "totalGracePeriodNames": 0, "totalReservedNames": 0 } } ``` #### getEpoch() Returns the epoch data for the specified block height. If no epoch index is provided, the current epoch is used. ```typescript const ario = ARIO.init({ rpc }); const epoch = await ario.getEpoch({ epochIndex: 0 }); ``` **Output:** ```json { "epochIndex": 0, "startHeight": 0, "startTimestamp": 1720720620813, "endTimestamp": 1752256702026, "distributionTimestamp": 1752256702026, "observations": { "failureSummaries": { "-Tk2DDk8k4zkwtppp_XFKKI5oUgh6IEHygAoN7mD-w8": [ "Ie2wEEUDKoU26c7IuckHNn3vMFdNQnMvfPBrFzAb3NA" ] }, "reports": { "IPdwa3Mb_9pDD8c2IaJx6aad51Ss-_TfStVwBuhtXMs": "B6UUjKWjjEWDBvDSMXWNmymfwvgR9EN27z5FTkEVlX4" } }, "prescribedNames": ["ardrive", "ar-io", "arweave", "fwd", "ao"], "prescribedObservers": [ { "gatewayAddress": "2Fk8lCmDegPg6jjprl57-UCpKmNgYiKwyhkU4vMNDnE", "observerAddress": "2Fk8lCmDegPg6jjprl57-UCpKmNgYiKwyhkU4vMNDnE", "stake": 10000000000, "startTimestamp": 1720720620813, "stakeWeight": 1, "tenureWeight": 0.4494598765432099, "gatewayPerformanceRatio": 1, "observerPerformanceRatio": 1, "gatewayRewardRatioWeight": 1, "observerRewardRatioWeight": 1, "compositeWeight": 0.4494598765432099, "normalizedCompositeWeight": 0.002057032496835938 } ], "distributions": { "totalEligibleGateways": 1, "totalEligibleRewards": 100000000, "totalEligibleObserverReward": 100000000, "totalEligibleGatewayReward": 100000000 }, "arnsStats": { "totalReturnedNames": 0, "totalActiveNames": 0, "totalGracePeriodNames": 0, "totalReservedNames": 0 } } ``` #### getEligibleEpochRewards() Returns the eligible epoch rewards for the specified block height. If no epoch index is provided, the current epoch is used. ```typescript const ario = ARIO.init({ rpc }); const rewards = await ario.getEligibleEpochRewards({ epochIndex: 0 }); ``` **Output:** ```json { "sortOrder": "desc", "hasMore": true, "totalItems": 37, "limit": 1, "sortBy": "cursorId", "items": [ { "cursorId": "xN_aVln30LmoCffwmk5_kRkcyQZyZWy1o_TNtM_CTm0_xN_aVln30LmoCffwmk5_kRkcyQZyZWy1o_TNtM_CTm0", "recipient": "xN_aVln30LmoCffwmk5_kRkcyQZyZWy1o_TNtM_CTm0", "gatewayAddress": "xN_aVln30LmoCffwmk5_kRkcyQZyZWy1o_TNtM_CTm0", "eligibleReward": 2627618704, "type": "operatorReward" } ], "nextCursor": "xN_aVln30LmoCffwmk5_kRkcyQZyZWy1o_TNtM_CTm0_xN_aVln30LmoCffwmk5_kRkcyQZyZWy1o_TNtM_CTm0" } ``` #### getObservations() Returns the epoch-indexed observation list. If no epoch index is provided, the current epoch is used. ```typescript const ario = ARIO.init({ rpc }); const observations = await ario.getObservations(); ``` **Output:** ```json { "failureSummaries": { "-Tk2DDk8k4zkwtppp_XFKKI5oUgh6IEHygAoN7mD-w8": [ "Ie2wEEUDKoU26c7IuckHNn3vMFdNQnMvfPBrFzAb3NA" ] }, "reports": { "IPdwa3Mb_9pDD8c2IaJx6aad51Ss-_TfStVwBuhtXMs": "B6UUjKWjjEWDBvDSMXWNmymfwvgR9EN27z5FTkEVlX4", "Ie2wEEUDKoU26c7IuckHNn3vMFdNQnMvfPBrFzAb3NA": "7tKsiQ2fxv0D8ZVN_QEv29fZ8hwFIgHoEDrpeEG0DIs", "osZP4D9cqeDvbVFBaEfjIxwc1QLIvRxUBRAxDIX9je8": "aatgznEvC_UPcxp1v0uw_RqydhIfKm4wtt1KCpONBB0", "qZ90I67XG68BYIAFVNfm9PUdM7v1XtFTn7u-EOZFAtk": "Bd8SmFK9-ktJRmwIungS8ur6JM-JtpxrvMtjt5JkB1M" } } ``` #### getDistributions() Returns the current rewards distribution information. If no epoch index is provided, the current epoch is used. ```typescript const ario = ARIO.init({ rpc }); const distributions = await ario.getDistributions({ epochIndex: 0 }); ``` **Output:** ```json { "totalEligibleGateways": 1, "totalEligibleRewards": 100000000, "totalEligibleObserverReward": 100000000, "totalEligibleGatewayReward": 100000000 } ``` #### saveObservations() Saves the observations of the current epoch. Requires `signer` to be provided on `ARIO.init` to sign the transaction. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.saveObservations( { reportTxId: "fDrr0_J4Iurt7caNST02cMotaz2FIbWQ4Kcj616RHl3", failedGateways: ["t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3"], }, { tags: [{ name: "App-Name", value: "My-Awesome-App" }], }, ); ``` #### getPrescribedObservers() Retrieves the prescribed observers of the ARIO process. To fetch prescribed observers for a previous epoch set the `epochIndex` to the desired epoch index. ```typescript const ario = ARIO.init({ rpc }); const observers = await ario.getPrescribedObservers({ epochIndex: 0 }); ``` **Output:** ```json [ { "gatewayAddress": "BpQlyhREz4lNGS-y3rSS1WxADfxPpAuing9Lgfdrj2U", "observerAddress": "2Fk8lCmDegPg6jjprl57-UCpKmNgYiKwyhkU4vMNDnE", "stake": 10000000000, "startTimestamp": 1720720620813, "stakeWeight": 1, "tenureWeight": 0.41453703703703704, "gatewayPerformanceRatio": 1, "observerPerformanceRatio": 1, "gatewayRewardRatioWeight": 1, "observerRewardRatioWeight": 1, "compositeWeight": 0.41453703703703704, "normalizedCompositeWeight": 0.0018972019546783507 } ] ``` #### crankEpochStep() High-level, permissionless epoch crank. Advances the epoch lifecycle by **one step per call** and returns the action it took — run it on a loop (this is what the standalone cranker and the observer-embedded cranker do). It owns the whole sequence so you don't orchestrate the individual instructions yourself: `create` → `tally` → `prescribe` → `distribute` → `close` — closing an epoch's observation PDAs first (`close_observation`) so `close_epoch` doesn't revert — plus an idle-tail of permissionless maintenance: `compound` delegate rewards, `update_demand_factor`, and `prune_returned_names`. The close path is non-wedging: a cleanup failure never blocks creation of the next epoch. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); // one step const result = await ario.crankEpochStep(); // → { action, epochIndex?, txId?, progress? } // action ∈ create | tally | prescribe | distribute | close // | close_observation | compound | update_demand_factor // | prune_returned_names | idle // or drive it on an interval setInterval(async () => { const r = await ario.crankEpochStep(); if (r.action !== 'idle') console.log(r.action, r.epochIndex, r.txId); }, 60_000); ``` All options are optional: `batchSize`, `enableClose`, `epochRetention`, `enableCompound`, `compoundMinPendingRewards`, `enableDemandFactorRoll`, `enablePrune`, `pruneBatchSize`, `nameRegistryAccount`. # Gateways (/sdks/ar-io-sdk/(ario-contract)/gateways) #### getGateway() Retrieves a gateway's info by its staking wallet address. ```typescript const ario = ARIO.init({ rpc }); const gateway = await ario.getGateway({ address: "-7vXsQZQDk8TMDlpiSLy3CnLi5PDPlAaN2DaynORpck", }); ``` **Output:** ```json { "observerAddress": "IPdwa3Mb_9pDD8c2IaJx6aad51Ss-_TfStVwBuhtXMs", "operatorStake": 250000000000, "totalDelegatedStake": 0, "settings": { "allowDelegatedStaking": true, "allowedDelegates": [], "autoStake": false, "delegateRewardShareRatio": 10, "minDelegatedStake": 100000000, "fqdn": "ar-io.dev", "label": "ar.io Test", "note": "Test Gateway operated by PDS for the ar.io ecosystem.", "port": 443, "properties": "raJgvbFU-YAnku-WsupIdbTsqqGLQiYpGzoqk9SCVgY", "protocol": "https" }, "startTimestamp": 1720720620813, "endTimestamp": 0, "stats": { "passedConsecutiveEpochs": 30, "failedConsecutiveEpochs": 0, "totalEpochCount": 31, "passedEpochCount": 30, "failedEpochCount": 1, "observedEpochCount": 30, "prescribedEpochCount": 31 }, "status": "joined", "weights": { "stakeWeight": 5.02400000024, "tenureWeight": 0.19444444444444, "gatewayPerformanceRatio": 1, "observerPerformanceRatio": 1, "gatewayRewardRatioWeight": 1, "observerRewardRatioWeight": 1, "compositeWeight": 0.97688888893556, "normalizedCompositeWeight": 0.19247316211083 } } ``` #### getGateways() Retrieves registered gateways of the ARIO process, using pagination and sorting by the specified criteria. The `cursor` used for pagination is the last gateway address from the previous request. ```typescript const ario = ARIO.init({ rpc }); const gateways = await ario.getGateways({ limit: 100, sortOrder: "desc", sortBy: "operatorStake", }); ``` Available `sortBy` options are any of the keys on the gateway object, e.g. `operatorStake`, `start`, `status`, `settings.fqdn`, `settings.label`, `settings.note`, `settings.port`, `settings.protocol`, `stats.failedConsecutiveEpochs`, `stats.passedConsecutiveEpochs`, etc. **Output:** ```json { "items": [ { "gatewayAddress": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "observerAddress": "IPdwa3Mb_9pDD8c2IaJx6aad51Ss-_TfStVwBuhtXMs", "operatorStake": 250000000000, "totalDelegatedStake": 0, "settings": { "allowDelegatedStaking": true, "allowedDelegates": [], "autoStake": false, "delegateRewardShareRatio": 10, "minDelegatedStake": 100000000, "fqdn": "ar-io.dev", "label": "ar.io Test", "note": "Test Gateway operated by PDS for the ar.io ecosystem.", "port": 443, "properties": "raJgvbFU-YAnku-WsupIdbTsqqGLQiYpGzoqk9SCVgY", "protocol": "https" }, "startTimestamp": 1720720620813, "endTimestamp": 0, "stats": { "passedConsecutiveEpochs": 30, "failedConsecutiveEpochs": 0, "totalEpochCount": 31, "passedEpochCount": 30, "failedEpochCount": 1, "observedEpochCount": 30, "prescribedEpochCount": 31 }, "status": "joined", "weights": { "stakeWeight": 5.02400000024, "tenureWeight": 0.19444444444444, "gatewayPerformanceRatio": 1, "observerPerformanceRatio": 1, "gatewayRewardRatioWeight": 1, "observerRewardRatioWeight": 1, "compositeWeight": 0.97688888893556, "normalizedCompositeWeight": 0.19247316211083 } } ], "hasMore": true, "nextCursor": "-4xgjroXENKYhTWqrBo57HQwvDL51mMdfsdsxJy6Y2Z_sA", "totalItems": 316, "limit": 100, "sortBy": "operatorStake", "sortOrder": "desc" } ``` #### getGatewayDelegates() Retrieves all delegates for a specific gateway, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last delegate address from the previous request. ```typescript const ario = ARIO.init({ rpc }); const delegates = await ario.getGatewayDelegates({ address: "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", limit: 3, sortBy: "startTimestamp", sortOrder: "desc", }); ``` **Output:** ```json { "nextCursor": "ScEtph9-vfY7lgqlUWwUwOmm99ySeZGQhOX0MFAyFEs", "limit": 3, "sortBy": "startTimestamp", "totalItems": 32, "sortOrder": "desc", "hasMore": true, "items": [ { "delegatedStake": 600000000, "address": "qD5VLaMYyIHlT6vH59TgYIs6g3EFlVjlPqljo6kqVxk", "startTimestamp": 1732716956301 }, { "delegatedStake": 508999038, "address": "KG8TlcWk-8pvroCjiLD2J5zkG9rqC6yYaBuZNqHEyY4", "startTimestamp": 1731828123742 }, { "delegatedStake": 510926479, "address": "ScEtph9-vfY7lgqlUWwUwOmm99ySeZGQhOX0MFAyFEs", "startTimestamp": 1731689356040 } ] } ``` #### joinNetwork() Joins a gateway to the ar.io network via its associated wallet. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.joinNetwork( { qty: new ARIOToken(10_000).toMARIO(), // minimum operator stake allowed autoStake: true, // auto-stake operator rewards to the gateway allowDelegatedStaking: true, // allows delegated staking minDelegatedStake: new ARIOToken(100).toMARIO(), // minimum delegated stake allowed delegateRewardShareRatio: 10, // percentage of rewards to share with delegates (e.g. 10%) label: "john smith", // min 1, max 64 characters note: "The example gateway", // max 256 characters properties: "FH1aVetOoulPGqgYukj0VE0wIhDy90WiQoV3U2PeY44", // Arweave transaction ID containing additional properties of the Gateway observerWallet: "0VE0wIhDy90WiQoV3U2PeY44FH1aVetOoulPGqgYukj", // wallet address of the observer, must match OBSERVER_WALLET on the observer fqdn: "example.com", // fully qualified domain name - note: you must own the domain and set the OBSERVER_WALLET on your gateway to match `observerWallet` port: 443, // port number protocol: "https", // only 'https' is supported }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### leaveNetwork() Sets the gateway as `leaving` on the ar.io network. Requires `signer` to be provided on `ARIO.init` to sign the transaction. The gateways operator and delegate stakes are vaulted and will be returned after leave periods. The gateway will be removed from the network after the leave period. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.leaveNetwork( // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### updateGatewaySettings() Writes new gateway settings to the callers gateway configuration. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.updateGatewaySettings( { // any other settings you want to update minDelegatedStake: new ARIOToken(100).toMARIO(), }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### increaseDelegateStake() Increases the callers stake on the target gateway. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.increaseDelegateStake( { target: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", qty: new ARIOToken(100).toMARIO(), }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### decreaseDelegateStake() Decreases the callers stake on the target gateway. Can instantly decrease stake by setting instant to `true`. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.decreaseDelegateStake( { target: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", qty: new ARIOToken(100).toMARIO(), }, { tags: [{ name: "App-Name", value: "My-Awesome-App" }], }, ); ``` Pay the early withdrawal fee and withdraw instantly. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.decreaseDelegateStake({ target: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", qty: new ARIOToken(100).toMARIO(), instant: true, // Immediately withdraw this stake and pay the instant withdrawal fee }); ``` #### getDelegations() Retrieves all active and vaulted stakes across all gateways for a specific address, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last delegationId (concatenated gateway and startTimestamp of the delgation) from the previous request. ```typescript const ario = ARIO.init({ rpc }); const vaults = await ario.getDelegations({ address: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", cursor: "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ_123456789", limit: 2, sortBy: "startTimestamp", sortOrder: "asc", }); ``` **Output:** ```json { "sortOrder": "asc", "hasMore": true, "totalItems": 95, "limit": 2, "sortBy": "startTimestamp", "items": [ { "type": "stake", "startTimestamp": 1727815440632, "gatewayAddress": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "delegationId": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ_1727815440632", "balance": 1383212512 }, { "type": "vault", "startTimestamp": 1730996691117, "gatewayAddress": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "delegationId": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ_1730996691117", "vaultId": "_sGDS7X1hyLCVpfe40GWioH9BSOb7f0XWbhHBa1q4-g", "balance": 50000000, "endTimestamp": 1733588691117 } ], "nextCursor": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ_1730996691117" } ``` #### instantWithdrawal() Instantly withdraws an existing vault on a gateway. If no `gatewayAddress` is provided, the signer's address will be used. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); // removes a delegated vault from a gateway const { id: txId } = await ario.instantWithdrawal( { // gateway address where delegate vault exists gatewayAddress: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", // delegated vault id to cancel vaultId: "fDrr0_J4Iurt7caNST02cMotaz2FIbWQ4Kcj616RHl3", }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }], }, ); // removes an operator vault from a gateway const { id: txId } = await ario.instantWithdrawal({ vaultId: "fDrr0_J4Iurt7caNST02cMotaz2FIbWQ4Kcj616RHl3", }); ``` #### cancelWithdrawal() Cancels an existing vault on a gateway. The vaulted stake will be returned to the callers stake. If no `gatewayAddress` is provided, the signer's address will be used. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); // cancels a delegated vault from a gateway const { id: txId } = await ario.cancelWithdrawal( { // gateway address where vault exists gatewayAddress: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", // vault id to cancel vaultId: "fDrr0_J4Iurt7caNST02cMotaz2FIbWQ4Kcj616RHl3", }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); // cancels an operator vault from a gateway const { id: txId } = await ario.cancelWithdrawal({ // operator vault id to cancel vaultId: "fDrr0_J4Iurt7caNST02cMotaz2FIbWQ4Kcj616RHl3", }); ``` #### getAllowedDelegates() Retrieves all allowed delegates for a specific address. The `cursor` used for pagination is the last address from the previous request. ```typescript const ario = ARIO.init({ rpc }); const allowedDelegates = await ario.getAllowedDelegates({ address: "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", }); ``` **Output:** ```json { "sortOrder": "desc", "hasMore": false, "totalItems": 4, "limit": 100, "items": [ "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM", "N4h8M9A9hasa3tF47qQyNvcKjm4APBKuFs7vqUVm-SI", "JcC4ZLUY76vmWha5y6RwKsFqYTrMZhbockl8iM9p5lQ", "31LPFYoow2G7j-eSSsrIh8OlNaARZ84-80J-8ba68d8" ] } ``` #### getGatewayVaults() Retrieves all vaults across all gateways for a specific address, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last vaultId from the previous request. ```typescript const ario = ARIO.init({ rpc }); const vaults = await ario.getGatewayVaults({ address: '"PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM', }); ``` **Output:** ```json { "sortOrder": "desc", "hasMore": false, "totalItems": 1, "limit": 100, "sortBy": "endTimestamp", "items": [ { "cursorId": "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM_1728067635857", "startTimestamp": 1728067635857, "balance": 50000000000, "vaultId": "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM", "endTimestamp": 1735843635857 } ] } ``` #### getAllGatewayVaults() Retrieves all vaults across all gateways, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last vaultId from the previous request. ```typescript const ario = ARIO.init({ rpc }); const vaults = await ario.getAllGatewayVaults({ limit: 1, sortBy: "endTimestamp", sortOrder: "desc", }); ``` **Output:** ```json { "sortOrder": "desc", "hasMore": true, "totalItems": 95, "limit": 1, "sortBy": "endTimestamp", "items": [ { "cursorId": "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM_E-QVU3dta36Wia2uQw6tQLjQk7Qw5uN0Z6fUzsoqzUc", "gatewayAddress": "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM", "startTimestamp": 1728067635857, "balance": 50000000000, "vaultId": "E-QVU3dta36Wia2uQw6tQLjQk7Qw5uN0Z6fUzsoqzUc", "endTimestamp": 1735843635857 } ], "nextCursor": "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM_E-QVU3dta36Wia2uQw6tQLjQk7Qw5uN0Z6fUzsoqzUc" } ``` #### getWithdrawals() Returns every pending stake withdrawal owned by `address` — covering both operator-stake decreases (`isDelegate: false`) and delegate-stake decreases (`isDelegate: true`). A withdrawal is claimable when `Date.now() >= endTimestamp`; call `claimWithdrawal({ withdrawalId: item.vaultId })` to release the tokens. This is the per-owner read needed to drive "you have X claimable withdrawals" UIs without fanning out across every gateway the wallet has interacted with. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const withdrawals = await ario.getWithdrawals({ address: "9xQeWvG816bUx9EPjHmaT23yvVM2ZWbrrpZb9PusVFin", }); const claimable = withdrawals.items.filter( (w) => Date.now() >= w.endTimestamp, ); ``` **Output:** ```json { "hasMore": false, "totalItems": 2, "limit": 100, "items": [ { "cursorId": "8CSdSjf7gXqQ5p1U2qfdwHzVw9sZRYHJpDpV87dnvb4d", "vaultId": "0", "gatewayAddress": "Bxz7Q2tWfqr9Q5T6cZjUnVxRk9CnHwShfgUaW5fY1Mvr", "balance": 50000000000, "startTimestamp": 1735843635857, "endTimestamp": 1738435635857, "isDelegate": true }, { "cursorId": "FmWUz4w7vSdLcz1nN8H1n2KkjJgrQQXR1n4kV3WqJ7Hf", "vaultId": "1", "gatewayAddress": "9xQeWvG816bUx9EPjHmaT23yvVM2ZWbrrpZb9PusVFin", "balance": 10000000000, "startTimestamp": 1735843835857, "endTimestamp": 1738435835857, "isDelegate": false } ] } ``` #### increaseOperatorStake() Increases the callers operator stake. Must be executed with a wallet registered as a gateway operator. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.increaseOperatorStake( { qty: new ARIOToken(100).toMARIO(), }, { tags: [{ name: "App-Name", value: "My-Awesome-App" }], }, ); ``` #### decreaseOperatorStake() Decreases the callers operator stake. Must be executed with a wallet registered as a gateway operator. Requires `signer` to be provided on `ARIO.init` to sign the transaction. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.decreaseOperatorStake( { qty: new ARIOToken(100).toMARIO(), }, { tags: [{ name: "App-Name", value: "My-Awesome-App" }], }, ); ``` #### redelegateStake() Redelegates the stake of a specific address to a new gateway. Vault ID may be optionally included in order to redelegate from an existing withdrawal vault. The redelegation fee is calculated based on the fee rate and the stake amount. Users are allowed one free redelegation every seven epochs. Each additional redelegation beyond the free redelegation will increase the fee by 10%, capping at a 60% redelegation fee. e.g: If 1000 mARIO is redelegated and the fee rate is 10%, the fee will be 100 mARIO. Resulting in 900 mARIO being redelegated to the new gateway and 100 mARIO being deducted back to the protocol balance. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.redelegateStake({ target: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", source: "HwFceQaMQnOBgKDpnFqCqgwKwEU5LBme1oXRuQOWSRA", stakeQty: new ARIOToken(1000).toMARIO(), vaultId: "fDrr0_J4Iurt7caNST02cMotaz2FIbWQ4Kcj616RHl3", }); ``` #### getRedelegationFee() Retrieves the fee rate as percentage required to redelegate the stake of a specific address. Fee rate ranges from 0% to 60% based on the number of redelegations since the last fee reset. ```typescript const ario = ARIO.init({ rpc }); const fee = await ario.getRedelegationFee({ address: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", }); ``` **Output:** ```json { "redelegationFeeRate": 10, "feeResetTimestamp": 1730996691117 } ``` #### getAllDelegates() Retrieves all delegates across all gateways, paginated and sorted by the specified criteria. The `cursor` used for pagination is a `cursorId` derived from delegate address and the gatewayAddress from the previous request. e.g `address_gatewayAddress`. ```typescript const ario = ARIO.init({ rpc }); const delegates = await ario.getAllDelegates({ limit: 2, sortBy: "startTimestamp", sortOrder: "desc", }); ``` **Output:** ```json { "sortOrder": "desc", "hasMore": true, "totalItems": 95, "limit": 2, "sortBy": "startTimestamp", "items": [ { "startTimestamp": 1734709397622, "cursorId": "9jfM0uzGNc9Mkhjo1ixGoqM7ygSem9wx_EokiVgi0Bs_E-QVU3dta36Wia2uQw6tQLjQk7Qw5uN0Z6fUzsoqzUc", "gatewayAddress": "E-QVU3dta36Wia2uQw6tQLjQk7Qw5uN0Z6fUzsoqzUc", "address": "9jfM0uzGNc9Mkhjo1ixGoqM7ygSem9wx_EokiVgi0Bs", "delegatedStake": 2521349108, "vaultedStake": 0 }, { "startTimestamp": 1734593229454, "cursorId": "LtV0aSqgK3YI7c5FmfvZd-wG95TJ9sezj_a4syaLMS8_M0WP8KSzCvKpzC-HPF1WcddLgGaL9J4DGi76iMnhrN4", "gatewayAddress": "M0WP8KSzCvKpzC-HPF1WcddLgGaL9J4DGi76iMnhrN4", "address": "LtV0aSqgK3YI7c5FmfvZd-wG95TJ9sezj_a4syaLMS8", "delegatedStake": 1685148110, "vaultedStake": 10000000 } ], "nextCursor": "PZ5vIhHf8VY969TxBPQN-rYY9CNFP9ggNsMBqlWUzWM_QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ" } ``` # General (/sdks/ar-io-sdk/(ario-contract)/general) #### init() Factory function that creates a read-only or writeable ARIO client. Providing `signer` plus `rpcSubscriptions` enables write methods (`joinNetwork`, `delegateStake`, `buyRecord`, etc.). Without a signer, the client is read-only. ```typescript import { createSolanaRpc, createSolanaRpcSubscriptions, createKeyPairSignerFromBytes, } from '@solana/kit'; const rpc = createSolanaRpc('https://api.mainnet-beta.solana.com'); // read-only client const ario = ARIO.init({ rpc }); // read-write client (needs rpcSubscriptions for sendAndConfirm) const rpcSubscriptions = createSolanaRpcSubscriptions( 'wss://api.mainnet-beta.solana.com', ); const signer = await createKeyPairSignerFromBytes(/* 64-byte secret key */); const arioWrite = ARIO.init({ rpc, rpcSubscriptions, signer }); ``` #### getInfo() Retrieves the information of the ARIO process. ```typescript const ario = ARIO.init({ rpc }); const info = await ario.getInfo(); ``` **Output:** ```json { "Name": "AR.IO", "Ticker": "ARIO", "Logo": "", "Denomination": 6, "Handlers": [], "LastCreatedEpochIndex": 0, "LastDistributedEpochIndex": 0, "totalSupply": 1000000000000000, "protocolBalance": 0, "epochSettings": { "durationMs": 86400000, "prescribedNameCount": 25, "maxObservers": 50 } } ``` > **Note**: `Handlers`, `LastCreatedEpochIndex`, and `LastDistributedEpochIndex` > are placeholders on Solana (returned for backwards-compatible field shape > with consumer code). `totalSupply` / `protocolBalance` are live reads from > the `ArioConfig` PDA; `epochSettings` is live from the `EpochSettings` > PDA. See `src/solana/io-readable.ts` for the exact projection. #### getTokenSupply() Retrieves the total supply of tokens, returned in mARIO. The total supply includes the following: - `total` - the total supply of all tokens - `circulating` - the total supply minus locked, withdrawn, delegated, and staked - `locked` - tokens that are locked in the protocol (a.k.a. vaulted) - `withdrawn` - tokens that have been withdrawn from the protocol by operators and delegators - `delegated` - tokens that have been delegated to gateways - `staked` - tokens that are staked in the protocol by gateway operators - `protocolBalance` - tokens that are held in the protocol's treasury. This is included in the circulating supply. ```typescript const ario = ARIO.init({ rpc }); const supply = await ario.getTokenSupply(); ``` **Output:** ```json { "total": 1000000000000000000, "circulating": 998094653842520, "locked": 0, "withdrawn": 560563387278, "delegated": 1750000000, "staked": 1343032770199, "protocolBalance": 46317263683761 } ``` #### getBalance() Retrieves the balance of the specified wallet address. ```typescript const ario = ARIO.init({ rpc }); // the balance will be returned in mARIO as a value const balance = await ario .getBalance({ address: "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", }) .then((balance: number) => new mARIOToken(balance).toARIO()); // convert it to ARIO for readability ``` **Output:** ```json 100000 ``` #### getBalances() Retrieves the balances of the ARIO process in `mARIO`, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last wallet address from the previous request. ```typescript const ario = ARIO.init({ rpc }); const balances = await ario.getBalances({ cursor: "-4xgjroXENKYhTWqrBo57HQwvDL51mMdfsdsxJy6Y2Z_sA", limit: 100, sortBy: "balance", sortOrder: "desc", }); ``` **Output:** ```json { "items": [ { "address": "-4xgjroXENKYhTWqrBo57HQwvDL51mMvSxJy6Y2Z_sA", "balance": 1000000 }, { "address": "-7vXsQZQDk8TMDlpiSLy3CnLi5PDPlAaN2DaynORpck", "balance": 1000000 } // ...98 other balances ], "hasMore": true, "nextCursor": "-7vXsQZQDk8TMDlpiSLy3CnLi5PDPlAaN2DaynORpck", "totalItems": 1789, "sortBy": "balance", "sortOrder": "desc" } ``` #### transfer() Transfers `mARIO` to the designated `target` recipient address. Requires `signer` to be provided on `ARIO.init` to sign the transaction. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.transfer({ target: 'RecipientSolanaPubkeyBase58', qty: new ARIOToken(1000).toMARIO(), }); ``` # Networks (/sdks/ar-io-sdk/(ario-contract)/networks) The SDK talks to whatever cluster your `@solana/kit` RPC client points at — mainnet-beta by default. For devnet or a local validator, override the RPC URL and (on any non-mainnet cluster) the per-program addresses: ```typescript const ario = ARIO.init({ rpc: createSolanaRpc('https://api.devnet.solana.com'), coreProgramId: address('\'), garProgramId: address('\'), arnsProgramId: address('\'), antProgramId: address('\'), }); ``` On localnet (Surfpool) source program IDs from `migration/localnet/out/localnet.env` in the `solana-ar-io` monorepo. # Primary Names (/sdks/ar-io-sdk/(ario-contract)/primary-names) #### getPrimaryNames() Retrieves all primary names paginated and sorted by the specified criteria. The `cursor` used for pagination is the last name from the previous request. ```typescript const ario = ARIO.init({ rpc }); const names = await ario.getPrimaryNames({ cursor: "ao", // this is the last name from the previous request limit: 1, sortBy: "startTimestamp", sortOrder: "desc", }); ``` **Output:** ```json { "sortOrder": "desc", "hasMore": true, "totalItems": 100, "limit": 1, "sortBy": "startTimestamp", "nextCursor": "arns", "items": [ { "name": "arns", "owner": "HwFceQaMQnOBgKDpnFqCqgwKwEU5LBme1oXRuQOWSRA", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "startTimestamp": 1719356032297 } ] } ``` #### getPrimaryName() Retrieves the primary name for a given name or address. ```typescript const ario = ARIO.init({ rpc }); const name = await ario.getPrimaryName({ name: "arns", }); // or const name = await ario.getPrimaryName({ address: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", }); ``` **Output:** ```json { "name": "arns", "owner": "HwFceQaMQnOBgKDpnFqCqgwKwEU5LBme1oXRuQOWSRA", "processId": "bh9l1cy0aksiL_x9M359faGzM_yjralacHIUo8_nQXM", "startTimestamp": 1719356032297 } ``` #### setPrimaryName() Sets an ArNS name already owned by the `signer` as their primary name. Note: `signer` must be the owner of the `processId` that is assigned to the name. If not, the transaction will fail. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); await ario.setPrimaryName({ name: 'my-arns-name' }); ``` #### requestPrimaryName() Requests a primary name for the `signer`'s address. The request must be approved by the new owner of the requested name via the `approvePrimaryNameRequest`[#approveprimarynamerequest-name-address-] API. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.requestPrimaryName({ name: "arns", }); ``` #### getPrimaryNameRequest() Retrieves the primary name request for a a wallet address. ```typescript const ario = ARIO.init({ rpc }); const request = await ario.getPrimaryNameRequest({ initiator: "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", }); ``` **Output:** ```json { "initiator": "t4Xr0_J4Iurt7caNST02cMotaz2FIbWQ4Kbj616RHl3", "name": "arns", "startTimestamp": 1728067635857, "endTimestamp": 1735843635857 } ``` # Vaults (/sdks/ar-io-sdk/(ario-contract)/vaults) #### getVault() Retrieves the locked-balance user vault of the ARIO process by the specified wallet address and vault ID. ```typescript const ario = ARIO.init({ rpc }); const vault = await ario.getVault({ address: "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", vaultId: "vaultIdOne", }); ``` **Output:** ```json { "balance": 1000000, "startTimestamp": 123, "endTimestamp": 4567 } ``` #### getVaults() Retrieves all locked-balance user vaults of the ARIO process, paginated and sorted by the specified criteria. The `cursor` used for pagination is the last wallet address from the previous request. ```typescript const ario = ARIO.init({ rpc }); const vaults = await ario.getVaults({ cursor: "0", limit: 100, sortBy: "balance", sortOrder: "desc", }); ``` **Output:** ```json { "items": [ { "address": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "vaultId": "vaultIdOne", "balance": 1000000, "startTimestamp": 123, "endTimestamp": 4567 }, { "address": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "vaultId": "vaultIdTwo", "balance": 1000000, "startTimestamp": 123, "endTimestamp": 4567 } // ...98 other addresses with vaults ], "hasMore": true, "nextCursor": "QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ", "totalItems": 1789, "sortBy": "balance", "sortOrder": "desc" } ``` #### vaultedTransfer() Transfers `mARIO` to the designated `recipient` address and locks the balance for the specified `lockLengthMs` milliseconds. The `revokable` flag determines if the vaulted transfer can be revoked by the sender. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.vaultedTransfer( { recipient: "-5dV7nk7waR8v4STuwPnTck1zFVkQqJh5K9q9Zik4Y5", quantity: new ARIOToken(1000).toMARIO(), lockLengthMs: 1000 * 60 * 60 * 24 * 365, // 1 year revokable: true, }, // optional additional tags { tags: [{ name: "App-Name", value: "My-Awesome-App" }] }, ); ``` #### revokeVault() Revokes a vaulted transfer by the recipient address and vault ID. Only the sender of the vaulted transfer can revoke it. _Note: Requires `signer` to be provided on `ARIO.init` to sign the transaction._ ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.revokeVault({ recipient: "-5dV7nk7waR8v4STuwPnTck1zFVkQqJh5K9q9Zik4Y5", vaultId: "IPdwa3Mb_9pDD8c2IaJx6aad51Ss-_TfStVwBuhtXMs", }); ``` #### createVault() Creates a vault for the specified `quantity` of mARIO from the signer's balance and locks it for the specified `lockLengthMs` milliseconds. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.createVault({ lockLengthMs: 1000 * 60 * 60 * 24 * 365, // 1 year quantity: new ARIOToken(1000).toMARIO(), }); ``` #### extendVault() Extends the lock length of a signer's vault by the specified `extendLengthMs` milliseconds. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.extendVault({ vaultId: "vaultIdOne", extendLengthMs: 1000 * 60 * 60 * 24 * 365, // 1 year }); ``` #### increaseVault() Increases the balance of a signer's vault by the specified `quantity` of mARIO. ```typescript const ario = ARIO.init({ rpc, rpcSubscriptions, signer }); const { id: txId } = await ario.increaseVault({ vaultId: "vaultIdOne", quantity: new ARIOToken(1000).toMARIO(), }); ``` # Escrow (/sdks/ar-io-sdk/escrow) Trustless, multi-protocol escrow for handing an asset to a recipient identified by an **Arweave** or **Ethereum** address, claimable once they hold a Solana wallet. Backed by the `ario-ant-escrow` program. Two clients: - `TokenEscrow` — escrow liquid **ARIO** (SPL) or a **time-locked vault**. - `ANTEscrow` — escrow an **ANT** (Metaplex Core NFT). Each supports **deposit → claim → cancel/refund → update-recipient**. Claims work three ways: **Arweave-attested** (an off-chain attestor re-signs the canonical claim with Ed25519, verified on-chain), **Ethereum** (on-chain `secp256k1_recover` + EIP-191), and **vault** (instruction introspection that preserves the remaining lock). ```typescript const escrow = new TokenEscrow({ rpc, rpcSubscriptions, signer, programId, coreProgram, }); // deposit 50 ARIO to an Ethereum recipient await escrow.depositTokens({ assetId, // 32-byte client-supplied id amount: 50_000_000n, arioMint, depositorTokenAccount, recipient: { protocol: 'ethereum', publicKey: ethAddress20 }, }); // the recipient claims (Ethereum path) once they have a Solana wallet await escrow.claimTokensEthereum({ depositor, assetId, claimant, claimantTokenAccount, escrowTokenAccount, signature, // recipient's EIP-191 signature over canonicalMessageV2(...) }); ``` Build the exact bytes a recipient signs with `canonicalMessage` / `canonicalMessageV2` — byte-identical to the on-chain program (and the off-chain attestor). See the contracts repo's escrow design + protocol spec for the full flow and the cross-language canonical-message vectors. # ar.io SDK (/sdks/ar-io-sdk) **For AI and LLM users**: Access the complete ar.io SDK documentation in plain text format at{" "} llm.txt for easy consumption by AI agents and language models. See [AI Agents & LLMs](/build/agents) for the full agent toolkit. The ar.io SDK provides comprehensive tools for interacting with ar.io and the Arweave ecosystem. Built with TypeScript, it offers type-safe interfaces for ArNS name management, gateway operations, and Solana program interactions. The SDK defaults to the Solana backend as of v3.23. ## Quick Start **Requires `@ar.io/sdk` version 3.23+** for Solana support. Write operations also need `@solana/kit` for signing — see the [Configuration](/sdks/ar-io-sdk/configuration) page for setup details. ### Install the SDK ```npm npm install @ar.io/sdk ``` ### Use the SDK ```javascript // Connect to mainnet const ario = ARIO.mainnet(); // Get paginated gateway registry const { items: gateways } = await ario.getGateways(); // Get paginated ArNS registry const { items: records } = await ario.getArNSRecords(); console.log('Network data:', { gateways, records }); ``` ### Install the SDK ```npm npm install @ar.io/sdk ``` ### Install polyfills (required for web environments) Polyfills are required for React web environments due to the use of `crypto`, `buffer` and `process` types in the SDK's dependencies. ```npm npm install --save-dev vite-plugin-node-polyfills ``` ```js // vite.config.js plugins: [ nodePolyfills({ globals: { Buffer: true, global: true, process: true, }, }), ], }); ``` Configure your bundler (Webpack, Vite, Rollup, etc.) to provide polyfills for `crypto`, `process`, and `buffer`. Refer to your bundler's documentation for polyfill configuration. ### Use the SDK ```javascript // Connect to mainnet const ario = ARIO.mainnet(); // Get paginated gateway registry const { items: gateways } = await ario.getGateways(); // Get paginated ArNS registry const { items: records } = await ario.getArNSRecords(); console.log('Network data:', { gateways, records }); ``` ```html ar.io SDK Example // Polyfills are included in the minimized web bundle, so not necessary to import directly // Connect to mainnet const ario = ARIO.mainnet(); // Function to load ar.io data async function loadArioData() { try { // Get gateway information const { items: gateways } = await ario.getGateways(); // Get ArNS records const { items: records } = await ario.getArNSRecords(); // Display results document.getElementById('results').innerHTML = ` Gateways: ${Object.keys(gateways).length} ArNS Records: ${Object.keys(records).length} ${JSON.stringify({ gateways, records }, null, 2)} `; } catch (error) { document.getElementById('results').innerHTML = ` Error: ${error.message} `; } } // Load data when page loads window.addEventListener('load', loadArioData); ar.io SDK Example Loading ar.io data... ``` ## API Reference & Documentation } title="API Reference" description="Complete API documentation for all SDK methods and classes" href="/apis/ar-io-node" /> } title="SDK Details" description="Detailed guides for ARIO contracts, ANT operations, and utilities" href="/sdks/ar-io-sdk/ar-io-name-system-arns" /> ## Core Features } title="ARIO Protocol Operations" description="ArNS management, gateway discovery, and network configuration" href="/sdks/ar-io-sdk/ar-io-name-system-arns" /> } title="ANT Contract Integration" description="Initialize, manage records, and transfer ANT ownership" href="/sdks/ar-io-sdk/initialize" /> } title="Pagination & Utilities" description="Handle large datasets and token conversions efficiently" href="/sdks/ar-io-sdk/overview" /> } title="Token Operations" description="ARIO token conversions and management utilities" href="/sdks/ar-io-sdk/converting-ario-to-mario" /> # Logging (/sdks/ar-io-sdk/logging) The library uses a lightweight console logger by default for both Node.js and web environments. The logger outputs structured JSON logs with timestamps. You can configure the log level via `setLogLevel()` API or provide a custom logger that satisfies the `ILogger` interface. #### Default Logger ```typescript // set the log level Logger.default.setLogLevel("debug"); // Create a new logger instance with a specific level const logger = new Logger({ level: "debug" }); ``` #### Custom Logger Implementation You can provide any custom logger that implements the `ILogger` interface: ```typescript // Custom logger example const customLogger: ILogger = { info: (message, ...args) => console.log(`[INFO] ${message}`, ...args), warn: (message, ...args) => console.warn(`[WARN] ${message}`, ...args), error: (message, ...args) => console.error(`[ERROR] ${message}`, ...args), debug: (message, ...args) => console.debug(`[DEBUG] ${message}`, ...args), setLogLevel: (level) => { /* implement level filtering */ }, }; // Set it as the default logger across the entire SDK — every class // (ARIO, ANT, ANTRegistry, etc.) will route logs through it. `ARIO.init` // does not accept a per-instance logger. Logger.default = customLogger; ``` # Pagination (/sdks/ar-io-sdk/pagination) #### Overview Certain APIs that could return a large amount of data are paginated using cursors. The SDK uses the `cursor` pattern (as opposed to pages) to better protect against changing data while paginating through a list of items. For more information on pagination strategies refer to [this article](https://www.getknit.dev/blog/api-pagination-best-practices#api-pagination-techniques-). Paginated results include the following properties: - `items`: the list of items on the current request, defaulted to 100 items. - `nextCursor`: the cursor to use for the next batch of items. This is `undefined` if there are no more items to fetch. - `hasMore`: a boolean indicating if there are more items to fetch. This is `false` if there are no more items to fetch. - `totalItems`: the total number of items available. This may change as new items are added to the list, only use this for informational purposes. - `sortBy`: the field used to sort the items, by default this is `startTimestamp`. - `sortOrder`: the order used to sort the items, by default this is `desc`. To request all the items in a list, you can iterate through the list using the `nextCursor` until `hasMore` is `false`. ```typescript let hasMore = true; let cursor: string | undefined; const gateaways = []; while (hasMore) { const page = await ario.getGateways({ limit: 100, cursor }); gateaways.push(...items); cursor = page.nextCursor; hasMore = page.hasMore; } ``` #### Filtering Paginated APIs also support filtering by providing a `filters` parameter. Filters can be applied to any field in the response. When multiple keys are provided, they are treated as AND conditions (all conditions must match). When multiple values are provided for a single key (as an array), they are treated as OR conditions (any value can match). Example: ```typescript const records = await ario.getArNSRecords({ filters: { type: "lease", processId: [ "ZkgLfyHALs5koxzojpcsEFAKA8fbpzP7l-tbM7wmQNM", "r61rbOjyXx3u644nGl9bkwLWlWmArMEzQgxBo2R-Vu0", ], }, }); ``` In the example above, the query will return ArNS records where: - The type is "lease" AND - The processId is EITHER "ZkgLfyHALs5koxzojpcsEFAKA8fbpzP7l-tbM7wmQNM" OR "r61rbOjyXx3u644nGl9bkwLWlWmArMEzQgxBo2R-Vu0" # Token Conversion (/sdks/ar-io-sdk/token-conversion) The ARIO process stores all values as mARIO (micro-ARIO) to avoid floating-point arithmetic issues. The SDK provides an `ARIOToken` and `mARIOToken` classes to handle the conversion between ARIO and mARIO, along with rounding logic for precision. **All process interactions expect values in mARIO. If numbers are provided as inputs, they are assumed to be in raw mARIO values.** #### Converting ARIO to mARIO ```typescript const arioValue = 1; const mARIOValue = new ARIOToken(arioValue).toMARIO(); const mARIOValue = 1_000_000; const arioValue = new mARIOToken(mARIOValue).toARIO(); ``` # Anonymous Operations (/sdks/ardrive-core-js/(advanced-features)/anonymous-operations) Use ArDrive without a wallet for read-only operations: ```typescript const anonymousArDrive = arDriveAnonymousFactory({}); // Read public data const publicFile = await anonymousArDrive.getPublicFile({ fileId }); const folderContents = await anonymousArDrive.listPublicFolder({ folderId }); ``` # Bundle Support (/sdks/ardrive-core-js/(advanced-features)/bundle-support) Large uploads are automatically bundled for efficiency: ```typescript // Bundling happens automatically for multiple files const bulkResult = await arDrive.uploadAllEntities({ entitiesToUpload: manyFiles // Bundling is handled internally }); ``` # Caching (/sdks/ardrive-core-js/(advanced-features)/caching) ArDrive Core maintains a metadata cache for improved performance: ```shell Windows: /ardrive-caches/metadata Non-Windows: /.ardrive/caches/metadata ``` Enable cache logging: ```bash ``` # Community Features (/sdks/ardrive-core-js/(advanced-features)/community-features) Send tips to the ArDrive community: ```typescript // Send community tip await arDrive.sendCommunityTip({ tokenAmount: new Winston(1000000000000), // 1 AR walletAddress, communityWalletAddress }); ``` # Incremental Drive Synchronization (/sdks/ardrive-core-js/(advanced-features)/incremental-drive-synchronization) ArDrive Core provides efficient incremental synchronization capabilities for tracking changes in drives over time. This feature enables applications to sync only new or modified content rather than fetching entire drive structures repeatedly. **Note:** The standard `arDriveFactory` creates an ArDrive instance with in-memory sync state caching (5-minute TTL). For persistent storage across sessions, see the [Persistent Storage](#persistent-storage-for-sync-state) section below. #### Basic Sync Operations ```typescript // Important: Requires ArFSDAOIncrementalSync for full functionality // The standard arDriveFactory may not support all sync features // For basic sync with in-memory caching, first create the DAO: const arfsDao = new ArFSDAOIncrementalSync( wallet, arweave, false // dryRun ); const arDrive = arDriveFactory({ wallet: myWallet, arfsDao }); // Now sync operations will work: const syncResult = await arDrive.syncPublicDrive(driveId); console.log(`Found ${syncResult.entities.length} total entities`); console.log(`Added: ${syncResult.changes.added.length}`); console.log(`Modified: ${syncResult.changes.modified.length}`); console.log(`Unreachable: ${syncResult.changes.unreachable.length}`); // Sync a private drive with decryption const privateSyncResult = await arDrive.syncPrivateDrive( driveId, driveKey ); ``` #### Incremental Sync with Previous State ```typescript // First sync - gets all entities const initialSync = await arDrive.syncPublicDrive(driveId); // Save the sync state for later const syncState = initialSync.newSyncState; // Later, sync only changes since last sync const incrementalSync = await arDrive.syncPublicDrive(driveId, undefined, { syncState: syncState // Pass previous state }); // Only new/modified entities since last sync console.log(`New entities: ${incrementalSync.changes.added.length}`); ``` #### Progress Tracking ```typescript // Track sync progress for large drives const result = await arDrive.syncPublicDrive(driveId, undefined, { onProgress: (processed, total) => { console.log(`Progress: ${processed}/${total} entities`); } }); ``` #### Advanced Sync Options ```typescript const syncOptions = { // Include all file revisions (not just latest) includeRevisions: true, // Batch size for GraphQL queries (default: 100, max: 100) batchSize: 50, // Stop early after finding N consecutive known entities (optimization) stopAfterKnownCount: 10, // Progress callback onProgress: (processed, total) => { console.log(`Syncing: ${processed}/${total}`); } }; const result = await arDrive.syncPublicDrive(driveId, owner, syncOptions); ``` #### Working with Sync Results ```typescript // Sync result structure const result = await arDrive.syncPublicDrive(driveId); // All entities in the drive (files and folders) result.entities.forEach(entity => { console.log(`${entity.entityType}: ${entity.name} (${entity.entityId})`); }); // Change detection result.changes.added.forEach(entity => { console.log(`New: ${entity.name}`); }); result.changes.modified.forEach(entity => { console.log(`Modified: ${entity.name} at block ${entity.blockHeight}`); }); result.changes.unreachable.forEach(entity => { console.log(`No longer accessible: ${entity.name}`); }); // Sync statistics console.log(`Processed from cache: ${result.stats.fromCache}`); console.log(`Fetched from network: ${result.stats.fromNetwork}`); console.log(`Block range: ${result.stats.lowestBlockHeight} - ${result.stats.highestBlockHeight}`); ``` #### Error Handling ```typescript try { const result = await arDrive.syncPublicDrive(driveId); } catch (error) { if (error instanceof IncrementalSyncError) { // Partial results are available even if sync failed console.log(`Sync failed but got ${error.partialResult.entities.length} entities`); console.log(`Error: ${error.message}`); // Can continue from partial state const partialState = error.partialResult.newSyncState; } } ``` # Manifest Creation (/sdks/ardrive-core-js/(advanced-features)/manifest-creation) Create Arweave manifests for web hosting: ```typescript // Create a manifest for a folder const manifest = await arDrive.uploadPublicManifest({ folderId, destManifestName: 'index.html', conflictResolution: 'upsert' }); // Access: https://arweave.net/{manifestId} ``` # Persistent Storage for Sync State (/sdks/ardrive-core-js/(advanced-features)/persistent-storage-for-sync-state) By default, sync state is only cached in memory for 5 minutes. To maintain sync state across application restarts, ArDrive Core provides storage adapters that automatically persist and restore sync state. #### Available Storage Adapters - **MemorySyncStateStore** - In-memory storage (default behavior) - **FileSystemSyncStateStore** - Persists to disk (Node.js) - **LocalStorageSyncStateStore** - Browser localStorage - **IndexedDBSyncStateStore** - Browser IndexedDB for larger datasets - **SQLiteSyncStateStore** - SQLite database (optional, see SQLite section below) #### Quick Start: Persistent Sync (Node.js) ```typescript import { arDriveFactory, ArFSDAOIncrementalSync, FileSystemSyncStateStore } from 'ardrive-core-js'; // 1. Create persistent storage adapter const syncStateStore = new FileSystemSyncStateStore('./.ardrive-cache'); // 2. Create DAO with storage adapter const arfsDao = new ArFSDAOIncrementalSync( wallet, arweave, false, // dryRun 'MyApp', '1.0.0', undefined, // use default settings for these undefined, undefined, syncStateStore // ← Pass storage adapter here ); // 3. Create ArDrive with the DAO const arDrive = arDriveFactory({ wallet, arfsDao }); // 4. Sync operations now persist state automatically const result = await arDrive.syncPublicDrive(driveId); // State is saved to disk and will be reused on next run ``` **Important:** The storage adapter must be passed to `ArFSDAOIncrementalSync`, not to `arDriveFactory`. #### Browser Storage Options ```typescript // Option 1: localStorage (simple, ~5-10MB limit) const syncStateStore = new LocalStorageSyncStateStore('ardrive-sync-'); // Option 2: IndexedDB (for larger datasets) const syncStateStore = new IndexedDBSyncStateStore('ardrive-sync-db'); // Use with ArFSDAOIncrementalSync same as Node.js example const arfsDao = new ArFSDAOIncrementalSync( wallet, arweave, false, 'MyApp', '1.0.0', undefined, undefined, undefined, syncStateStore ); ``` #### Working Example See `examples/persistent-sync-example.js` for a complete working example that demonstrates: - Setting up persistent storage - Performing initial full sync - Simulating app restart - Performing incremental sync from saved state - Managing stored sync states #### SQLite Storage (Optional) The SQLite adapter provides advanced features like statistics, cleanup, and backups. To use it: 1. Install the peer dependency: `yarn add better-sqlite3` 2. Copy `src/utils/sync_state_store_sqlite.ts.optional` to your project 3. Import and use like other storage adapters **Note:** SQLite adapter is not included in the default build to avoid forcing the peer dependency. #### Storage Management Methods All storage adapters implement these methods: ```typescript // List all drives with cached state const driveIds = await syncStateStore.list(); // Load specific drive state const state = await syncStateStore.load(driveId); // Clear specific drive await syncStateStore.clear(driveId); // Clear all cached states await syncStateStore.clearAll(); ``` #### Custom Storage Implementation Create your own storage adapter by implementing the `SyncStateStore` interface: ```typescript class CustomSyncStateStore implements SyncStateStore { async save(driveId: DriveID, state: DriveSyncState): Promise\ { // Your storage logic } async load(driveId: DriveID): Promise\ { // Your retrieval logic } async clear(driveId: DriveID): Promise\ { // Your deletion logic } async list(): Promise { // Return all stored drive IDs } async clearAll(): Promise\ { // Clear all stored states } } ``` #### Serialization for External Storage If you need to store sync state in an external system: ```typescript // Serialize state for storage const result = await arDrive.syncPublicDrive(driveId); const serialized = serializeSyncState(result.newSyncState); const jsonString = JSON.stringify(serialized); // Store in your backend await myAPI.saveSyncState(driveId, jsonString); // Later, retrieve and deserialize const stored = await myAPI.getSyncState(driveId); const parsed = JSON.parse(stored); const syncState = deserializeSyncState(parsed); // Use restored state for incremental sync const nextSync = await arDrive.syncPublicDrive(driveId, undefined, { syncState }); ``` # Progress Tracking (/sdks/ardrive-core-js/(advanced-features)/progress-tracking) Enable upload progress logging: ```bash ``` Progress will be logged to stderr: ``` Uploading file transaction 1 of total 2 transactions... Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 0% Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 35% Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 66% Transaction _GKQasQX194a364Hph8Oe-oku1AdfHwxWOw9_JC1yjc Upload Progress: 100% ``` # Turbo Integration (/sdks/ardrive-core-js/(advanced-features)/turbo-integration) Enable Turbo for optimized uploads: ```typescript // Node.js const arDriveWithTurbo = arDriveFactory({ wallet: myWallet, turboSettings: { turboUploadUrl: new URL('https://upload.ardrive.io') } }); // Browser const arDrive = arDriveFactory({ signer: myBrowserSigner, turboSettings: { turboUploadUrl: new URL('https://upload.ardrive.io') } }); // Uploads will automatically use Turbo const result = await arDriveWithTurbo.uploadAllEntities({ entitiesToUpload: [{ wrappedEntity, destFolderId }] }); ``` # Bulk Operations (/sdks/ardrive-core-js/(api-reference)/bulk-operations) #### Upload Multiple Files and Folders ```typescript // Prepare entities for upload const folder1 = wrapFileOrFolder('/path/to/folder1'); const folder2 = wrapFileOrFolder('/path/to/folder2'); const file1 = wrapFileOrFolder('/path/to/file1.txt'); // Upload everything in one operation const bulkUpload = await arDrive.uploadAllEntities({ entitiesToUpload: [ // Public folder { wrappedEntity: folder1, destFolderId: rootFolderId }, // Private folder { wrappedEntity: folder2, destFolderId: rootFolderId, driveKey: privateDriveKey }, // Public file { wrappedEntity: file1, destFolderId: someFolderId } ], conflictResolution: 'upsert' }); // Results include all created entities console.log('Created folders:', bulkUpload.created.length); console.log('Total cost:', bulkUpload.totalCost.toString()); ``` #### Create Folder and Upload Contents ```typescript // Create folder and upload all children const folderWithContents = await arDrive.createPublicFolderAndUploadChildren({ parentFolderId, wrappedFolder: wrapFileOrFolder('/path/to/folder'), conflictResolution: 'skip' }); ``` # Conflict Resolution (/sdks/ardrive-core-js/(api-reference)/conflict-resolution) Available strategies when uploading files/folders that already exist: ```typescript // Skip existing files await arDrive.uploadAllEntities({ entitiesToUpload: [...], conflictResolution: 'skip' }); // Replace all existing files await arDrive.uploadAllEntities({ entitiesToUpload: [...], conflictResolution: 'replace' }); // Update only if content differs (default) await arDrive.uploadAllEntities({ entitiesToUpload: [...], conflictResolution: 'upsert' }); // Rename conflicting files await arDrive.uploadAllEntities({ entitiesToUpload: [...], conflictResolution: 'rename' }); // Throw error on conflicts await arDrive.uploadAllEntities({ entitiesToUpload: [...], conflictResolution: 'error' }); // Interactive prompt (CLI only) await arDrive.uploadAllEntities({ entitiesToUpload: [...], conflictResolution: 'ask' }); ``` # Custom Metadata (/sdks/ardrive-core-js/(api-reference)/custom-metadata) Attach custom metadata to files: ```typescript const fileWithMetadata = wrapFileOrFolder('/path/to/file.txt', 'text/plain', { metaDataJson: { 'Custom-Field': 'Custom Value', Version: '1.0' }, metaDataGqlTags: { 'App-Name': ['MyApp'], 'App-Version': ['1.0.0'] }, dataGqlTags: { 'Content-Type': ['text/plain'] } }); // Upload with custom metadata await arDrive.uploadPublicFile({ parentFolderId, wrappedFile: fileWithMetadata }); ``` # Download Operations (/sdks/ardrive-core-js/(api-reference)/download-operations) #### Download Files ```typescript // Download public file const publicData = await arDrive.downloadPublicFile({ fileId }); // publicData is a Buffer/Uint8Array // Download private file (automatically decrypted) const privateData = await arDrive.downloadPrivateFile({ fileId, driveKey }); ``` #### Download Folders ```typescript // Download entire folder const folderData = await arDrive.downloadPublicFolder({ folderId, destFolderPath: '/local/download/path' }); // Download private folder const privateFolderData = await arDrive.downloadPrivateFolder({ folderId, driveKey, destFolderPath: '/local/download/path' }); ``` # Drive Operations (/sdks/ardrive-core-js/(api-reference)/drive-operations) #### Creating Drives ```typescript // Public drive const publicDrive = await arDrive.createPublicDrive({ driveName: 'My Public Drive' }); // Private drive with password const privateDrive = await arDrive.createPrivateDrive({ driveName: 'My Private Drive', drivePassword: 'mySecretPassword' }); ``` #### Reading Drive Information ```typescript // Get public drive const publicDriveInfo = await arDrive.getPublicDrive({ driveId }); // Get private drive (requires drive key) const privateDriveInfo = await arDrive.getPrivateDrive({ driveId, driveKey }); // Get all drives for an address const allDrives = await arDrive.getAllDrivesForAddress({ address: walletAddress, privateKeyData: wallet.getPrivateKey() }); ``` #### Renaming Drives ```typescript // Rename public drive await arDrive.renamePublicDrive({ driveId, newName: 'Updated Drive Name' }); // Rename private drive await arDrive.renamePrivateDrive({ driveId, driveKey, newName: 'Updated Private Name' }); ``` # Encryption & Security (/sdks/ardrive-core-js/(api-reference)/encryption-security) #### Key Derivation ```typescript // Derive drive key from password const driveKey = await deriveDriveKey('myPassword', driveId.toString(), JSON.stringify(wallet.getPrivateKey())); // File keys are automatically derived from drive keys const fileKey = await deriveFileKey(driveKey, fileId); ``` #### Manual Encryption/Decryption ```typescript // Encrypt data const { cipher, cipherIV } = await driveEncrypt(driveKey, data); // Decrypt data const decrypted = await driveDecrypt(cipherIV, driveKey, cipher); ``` # File Operations (/sdks/ardrive-core-js/(api-reference)/file-operations) #### Uploading Files ```typescript // Wrap file for upload const wrappedFile = wrapFileOrFolder('/path/to/file.pdf'); // Upload public file const publicUpload = await arDrive.uploadPublicFile({ parentFolderId, wrappedFile, conflictResolution: 'upsert' // skip, replace, upsert, or error }); // Upload private file const privateUpload = await arDrive.uploadPrivateFile({ parentFolderId, driveKey, wrappedFile }); ``` #### Reading File Information ```typescript // Get public file metadata const publicFile = await arDrive.getPublicFile({ fileId }); // Get private file metadata const privateFile = await arDrive.getPrivateFile({ fileId, driveKey }); ``` #### Moving and Renaming Files ```typescript // Move file await arDrive.movePublicFile({ fileId, newParentFolderId }); // Rename file await arDrive.renamePublicFile({ fileId, newName: 'renamed-file.pdf' }); ``` # Folder Operations (/sdks/ardrive-core-js/(api-reference)/folder-operations) #### Creating Folders ```typescript // Public folder const publicFolder = await arDrive.createPublicFolder({ folderName: 'Documents', driveId, parentFolderId }); // Private folder const privateFolder = await arDrive.createPrivateFolder({ folderName: 'Secret Documents', driveId, driveKey, parentFolderId }); ``` #### Listing Folder Contents ```typescript // List public folder const publicContents = await arDrive.listPublicFolder({ folderId, maxDepth: 2, // Optional: limit recursion depth includeRoot: true // Optional: include root folder in results }); // List private folder const privateContents = await arDrive.listPrivateFolder({ folderId, driveKey, maxDepth: 1 }); ``` #### Moving and Renaming Folders ```typescript // Move folder await arDrive.movePublicFolder({ folderId, newParentFolderId }); // Rename folder await arDrive.renamePublicFolder({ folderId, newName: 'New Folder Name' }); ``` # Pricing & Cost Estimation (/sdks/ardrive-core-js/(api-reference)/pricing-cost-estimation) ```typescript // Get price estimator const priceEstimator = arDrive.getArDataPriceEstimator(); // Estimate cost for data size const cost = await priceEstimator.getARPriceForByteCount( new ByteCount(1024 * 1024) // 1MB ); // Get base Winston price (without tips) const basePrice = await priceEstimator.getBaseWinstonPriceForByteCount( new ByteCount(5 * 1024 * 1024) // 5MB ); ``` # Entity IDs (/sdks/ardrive-core-js/(core-concepts)/entity-ids) Use the type-safe entity ID constructors: ```typescript // Generic entity ID const entityId = EID('10108b54a-eb5e-4134-8ae2-a3946a428ec7'); // Specific entity IDs const driveId = new DriveID('12345674a-eb5e-4134-8ae2-a3946a428ec7'); const folderId = new FolderID('47162534a-eb5e-4134-8ae2-a3946a428ec7'); const fileId = new FileID('98765432a-eb5e-4134-8ae2-a3946a428ec7'); ``` # Entity Types (/sdks/ardrive-core-js/(core-concepts)/entity-types) ArDrive uses a hierarchical structure: - **Drives**: Top-level containers (public or private) - **Folders**: Organize files within drives - **Files**: Individual files stored on Arweave Each entity has a unique ID (`DriveID`, `FolderID`, `FileID`) and can be either public (unencrypted) or private (encrypted). # Wallet Management (/sdks/ardrive-core-js/(core-concepts)/wallet-management) ```typescript // Create wallet from JWK const wallet = new JWKWallet(jwkKey); // Check wallet balance const balance = await wallet.getBalance(); ``` # ArDrive Core JS (/sdks/ardrive-core-js) **For AI and LLM users**: Access the complete ArDrive Core JS documentation in plain text format at llm.txt for easy consumption by AI agents and language models. The ArDrive Core JS SDK provides a comprehensive TypeScript library for building applications on ArDrive. It offers type-safe interfaces for drive management, file operations, encryption, and seamless integration with Arweave. ## Quick Start ### Install the SDK ```npm npm install ardrive-core-js ``` ### Initialize with a Wallet ```typescript // Load your Arweave wallet const wallet = readJWKFile('./wallet.json'); // Create an ArDrive instance const arDrive = arDriveFactory({ wallet }); ``` ### Create a Drive and Upload Files ```typescript // Create a new public drive const { driveId, rootFolderId } = await arDrive.createPublicDrive({ driveName: 'My-Drive' }); console.log('Drive created:', driveId.toString()); console.log('Root folder:', rootFolderId.toString()); // Upload a file to the drive const wrappedFile = wrapFileOrFolder('./my-file.pdf'); const uploadResult = await arDrive.uploadPublicFile({ parentFolderId: rootFolderId, wrappedFile }); console.log('File uploaded:', uploadResult.fileId.toString()); ``` ### Install the SDK ```npm npm install ardrive-core-js ``` ### Configure Polyfills Polyfills are required for web environments due to Node.js dependencies used by the SDK. ```npm npm install --save-dev vite-plugin-node-polyfills ``` ```js // vite.config.js plugins: [ nodePolyfills({ globals: { Buffer: true, global: true, process: true, }, }), ], }); ``` Configure your bundler (Webpack, Vite, Rollup, etc.) to provide polyfills for `crypto`, `process`, and `buffer`. Refer to your bundler's documentation for polyfill configuration. ### Use the SDK ```typescript // Initialize with a JWK wallet object const arDrive = arDriveFactory({ wallet: jwkWallet }); // Create a public drive const { driveId, rootFolderId } = await arDrive.createPublicDrive({ driveName: 'My-Drive' }); console.log('Drive created:', driveId.toString()); ``` ## Documentation } title="Source Code" description="View the complete source code and contribute on GitHub" href="https://github.com/ardriveapp/ardrive-core-js" /> } title="API Reference" description="Detailed documentation for all SDK methods and classes" href="/sdks/ardrive-core-js/drive-operations" /> ## Core Features } title="Drive Operations" description="Create and manage public and private drives" href="/sdks/ardrive-core-js/drive-operations" /> } title="Folder Operations" description="Create folders, list contents, and organize your data" href="/sdks/ardrive-core-js/folder-operations" /> } title="File Operations" description="Upload, download, and manage files on Arweave" href="/sdks/ardrive-core-js/file-operations" /> } title="Encryption & Security" description="End-to-end encryption for private drives and files" href="/sdks/ardrive-core-js/encryption-security" /> } title="Pricing & Cost Estimation" description="Estimate upload costs before committing transactions" href="/sdks/ardrive-core-js/pricing-cost-estimation" /> } title="Advanced Features" description="Turbo integration, bundling, manifests, and more" href="/sdks/ardrive-core-js/turbo-integration" /> # Introduction (/sdks) Build powerful applications with our comprehensive suite of SDKs designed for the ar.io ecosystem. } title="Upload data with the Turbo SDK" description="High-performance data upload service for Arweave with instant confirmation and transparent pricing" href="/sdks/turbo-sdk" /> } title="Interact with ar.io using the ar.io SDK" description="Access ar.io protocols, manage ArNS names, interact with ANTs, and integrate gateway services" href="/sdks/ar-io-sdk" /> } title="Decentralized access with Wayfinder SDK" description="Robust, censorship-resistant access to Arweave data through the distributed ar.io gateway network" href="/sdks/wayfinder" /> ## Choose Your SDK Each SDK serves a specific purpose in the ar.io ecosystem: - **Turbo SDK** - For applications that need fast, reliable data uploads to Arweave - **ar.io SDK** - For interacting with ar.io Solana programs and services - **Wayfinder SDK** - For decentralized data access with built-in verification and gateway routing All SDKs are available for both Node.js and browser environments, with TypeScript support included. ## Next Steps # TurboAuthenticatedClient (/sdks/turbo-sdk/(apis)/turboauthenticatedclient) #### getBalance() Issues a signed request to get the credit balance of a wallet measured in AR (measured in Winston Credits, or winc). ```typescript const { winc: balance } = await turbo.getBalance(); ``` #### signer.getNativeAddress() Returns the [native address][docs/native-address] of the connected signer. ```typescript const address = await turbo.signer.getNativeAddress(); ``` #### getWincForFiat() Returns the current amount of Winston Credits including all adjustments for the provided fiat currency, amount, and optional promo codes. ```typescript const { winc, paymentAmount, quotedPaymentAmount, adjustments } = await turbo.getWincForFiat({ amount: USD(100), promoCodes: ['MY_PROMO_CODE'], // promo codes require an authenticated client }); ``` #### createCheckoutSession() Creates a Stripe checkout session for a Turbo Top Up with the provided amount, currency, owner, and optional promo codes. The returned URL can be opened in the browser, all payments are processed by Stripe. Promo codes require an authenticated client. ```typescript const { url, winc, paymentAmount, quotedPaymentAmount, adjustments } = await turbo.createCheckoutSession({ amount: USD(10.0), // $10.00 USD owner: publicArweaveAddress, promoCodes: ['MY_PROMO_CODE'], // promo codes require an authenticated client }); // open checkout session in a browser window.open(url, '_blank'); ``` #### upload() The easiest way to upload data to Turbo. The `signal` is an optional [AbortSignal] that can be used to cancel the upload or timeout the request. `dataItemOpts` is an optional object that can be used to configure tags, target, and anchor for the data item upload. ```typescript const uploadResult = await turbo.upload({ data: 'The contents of my file!', signal: AbortSignal.timeout(10_000), // cancel the upload after 10 seconds dataItemOpts: { // optional }, events: { // optional }, }); ``` #### uploadFile() Signs and uploads a raw file. There are two ways to provide the file to the SDK: 1. Using a `file` parameter 2. Using a `fileStreamFactory` and `fileSizeFactory` ##### Using file` In Web with a file input: ```typescript const selectedFile = e.target.files[0]; const uploadResult = await turbo.uploadFile({ file: selectedFile, dataItemOpts: { tags: [{ name: 'Content-Type', value: 'text/plain' }], }, events: { onUploadProgress: ({ totalBytes, processedBytes }) => { console.log('Upload progress:', { totalBytes, processedBytes }); }, onUploadError: (error) => { console.log('Upload error:', { error }); }, onUploadSuccess: () => { console.log('Upload success!'); }, }, }); ``` In NodeJS with a file path: ```typescript const filePath = path.join(__dirname, './my-unsigned-file.txt'); const fileSize = fs.stateSync(filePath).size; const uploadResult = await turbo.uploadFile({ file: filePath, dataItemOpts: { tags: [{ name: 'Content-Type', value: 'text/plain' }], }, }); ``` ##### Using fileStreamFactory` and `fileSizeFactory` Note: The provided `fileStreamFactory` should produce a NEW file data stream each time it is invoked. The `fileSizeFactory` is a function that returns the size of the file. The `signal` is an optional [AbortSignal] that can be used to cancel the upload or timeout the request. `dataItemOpts` is an optional object that can be used to configure tags, target, and anchor for the data item upload. ```typescript const filePath = path.join(__dirname, './my-unsigned-file.txt'); const fileSize = fs.stateSync(filePath).size; const uploadResult = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream(filePath), fileSizeFactory: () => fileSize, }); ``` ##### Customize Multi-Part Upload Behavior By default, the Turbo upload methods will split files that are larger than 10 MiB into chunks and send them to the upload service multi-part endpoints. This behavior can be customized with the following inputs: - `chunkByteCount`: The maximum size in bytes for each chunk. Must be between 5 MiB and 500 MiB. Defaults to 5 MiB. - `maxChunkConcurrency`: The maximum number of chunks to upload concurrently. Defaults to 5. Reducing concurrency will slow down uploads, but reduce memory utilization and serialize network calls. Increasing it will upload faster, but can strain available resources. - `chunkingMode`: The chunking mode to use. Can be 'auto', 'force', or 'disabled'. Defaults to 'auto'. Auto behavior means chunking is enabled if the file would be split into at least three chunks. - `maxFinalizeMs`: The maximum time in milliseconds to wait for the finalization of all chunks after the last chunk is uploaded. Defaults to 1 minute per GiB of the total file size. ```typescript // Customize chunking behavior await turbo.upload({ ...params, chunkByteCount: 1024 * 1024 * 500, // Max chunk size maxChunkConcurrency: 1, // Minimize concurrency }); ``` ```typescript // Disable chunking behavior await turbo.upload({ ...params, chunkingMode: 'disabled', }); ``` ```typescript // Force chunking behavior await turbo.upload({ ...params, chunkingMode: 'force', }); ``` #### On Demand Uploads With the upload methods, you can choose to Top Up with selected crypto token on demand if the connected wallet does not have enough credits to complete the upload. This is done by providing the `OnDemandFunding` class to the `fundingMode` parameter on upload methods. The `maxTokenAmount` (optional) is the maximum amount of tokens in the token type's smallest unit value (e.g: Winston for arweave token type) to fund the wallet with. The `topUpBufferMultiplier` (optional) is the multiplier to apply to the estimated top-up amount to avoid underpayment during on-demand top-ups due to price fluctuations on longer uploads. Defaults to 1.1, meaning a 10% buffer. Note: On demand API currently only available for $ARIO (`ario`), $SOL (`solana`), $ETH on Base Network (`base-eth`) and $USDC on Base Network (`base-usdc`) token types. ```typescript const turbo = TurboFactory.authenticated({ signer: arweaveSignerWithARIO, token: 'ario', }); await turbo.upload({ ...params, fundingMode: new OnDemandFunding({ maxTokenAmount: ARIOToTokenAmount(500), // Max 500 $ARIO topUpBufferMultiplier: 1.1, // 10% buffer to avoid underpayment }), }); ``` #### x402 Uploads Another method of uploading files is via the x402 protocol. This method is optimized for agent workflows and allows for direct uploads to Arweave gateways that support the x402 protocol using an EVM wallet and base-usdc token type. ```typescript const turbo = TurboFactory.authenticated({ signer: ethereumSignerWithBaseUSDC, token: 'base-usdc', }); await turbo.uploadFile({ ...params, fundingMode: new X402Funding({ maxMUSDCAmount: 1_000_000 }), // Max 1 USDC. Opt out if too expensive }); ``` #### Raw x402 Data Uploads Using the x402 protocol, you can also upload raw data to Turbo without signing a data item. This method is ideal for quick agent workflows where the ownership of the data is not required to be tied to a specific wallet. The eventual data item on chain will be signed by Turbo's x402 EVM signer. ```typescript const turbo = TurboFactory.authenticated({ signer: ethereumSignerWithBaseUSDC, token: 'base-usdc', }); await turbo.uploadRawX402Data({ data: myRawData, maxMUSDCAmount: 1_000_000, // Max 1 USDC. Opt out if too expensive }); ``` NOTE: For free uploads under 100 KiB, this method of upload currently does not require a signature and can be used with an unauthenticated client. ```ts // Unsigned free upload of raw data under 100 KiB const turbo = TurboFactory.unauthenticated({ token: 'base-usdc' }); await turbo.uploadRawX402Data({ data: myRawData, }); ``` #### uploadFolder() Signs and uploads a folder of files. For NodeJS, the `folderPath` of the folder to upload is required. For the browser, an array of `files` is required. The `dataItemOpts` is an optional object that can be used to configure tags, target, and anchor for the data item upload. The `signal` is an optional [AbortSignal] that can be used to cancel the upload or timeout the request. The `maxConcurrentUploads` is an optional number that can be used to limit the number of concurrent uploads. The `throwOnFailure` is an optional boolean that can be used to throw an error if any upload fails. The `manifestOptions` is an optional object that can be used to configure the manifest file, including a custom index file, fallback file, or whether to disable manifests altogether. Manifests are enabled by default. ##### NodeJS Upload Folder ```typescript const folderPath = path.join(__dirname, './my-folder'); const { manifest, fileResponses, manifestResponse } = await turbo.uploadFolder({ folderPath, dataItemOpts: { // optional tags: [ { // User defined content type will overwrite file content type name: 'Content-Type', value: 'text/plain', }, { name: 'My-Custom-Tag', value: 'my-custom-value', }, ], // no timeout or AbortSignal provided }, manifestOptions: { // optional indexFile: 'custom-index.html', fallbackFile: 'custom-fallback.html', disableManifests: false, }, }); ``` ##### Browser Upload Folder ```html const folderInput = document.getElementById('folder'); folderInput.addEventListener('change', async (event) => { const selectedFiles = folderInput.files; console.log('Folder selected:', selectedFiles); const { manifest, fileResponses, manifestResponse } = await turbo.uploadFolder({ files: Array.from(selectedFiles).map((file) => file), }); console.log(manifest, fileResponses, manifestResponse); }); ``` ##### Upload Folder with Progress Events The `uploadFolder` method supports folder-level and per-file events for tracking upload progress. This is useful for building progress bars or providing feedback to users during folder uploads. ```typescript const folderPath = path.join(__dirname, './my-folder'); const { manifest, fileResponses, manifestResponse } = await turbo.uploadFolder({ folderPath, events: { // Per-file events onFileStart: ({ fileName, fileSize, fileIndex, totalFiles }) => { console.log( `Starting file ${ fileIndex + 1 }/${totalFiles}: ${fileName} (${fileSize} bytes)`, ); }, onFileProgress: ({ fileName, fileIndex, totalFiles, fileProcessedBytes, fileTotalBytes, step, }) => { const percentComplete = (fileProcessedBytes / fileTotalBytes) * 100; console.log( `File ${ fileIndex + 1 }/${totalFiles} (${fileName}) ${step}: ${percentComplete.toFixed(2)}%`, ); }, onFileComplete: ({ fileName, fileIndex, totalFiles, id }) => { console.log( `Completed file ${fileIndex + 1}/${totalFiles}: ${fileName} (${id})`, ); }, onFileError: ({ fileName, fileIndex, totalFiles, error }) => { console.error( `Error uploading file ${fileIndex + 1}/${totalFiles}: ${fileName}`, error, ); }, // Folder-level aggregate events onFolderProgress: ({ processedFiles, totalFiles, processedBytes, totalBytes, currentPhase, }) => { const percentComplete = (processedBytes / totalBytes) * 100; console.log( `Folder progress (${currentPhase}): ${processedFiles}/${totalFiles} files, ${percentComplete.toFixed( 2, )}%`, ); }, onFolderError: (error) => { console.error('Folder upload error:', error); }, onFolderSuccess: () => { console.log('Folder upload complete!'); }, }, }); ``` #### topUpWithTokens() Tops up the connected wallet with Credits by submitting a payment transaction for the token amount to the Turbo wallet and then submitting that transaction id to Turbo Payment Service for top up processing. - The `tokenAmount` is the amount of tokens in the token type's smallest unit value (e.g: Winston for arweave token type) to fund the wallet with. - The `feeMultiplier` (optional) is the multiplier to apply to the reward for the transaction to modify its chances of being mined. Credits will be added to the wallet balance after the transaction is confirmed on the given blockchain. Defaults to 1.0, meaning no multiplier. - The `turboCreditDestinationAddress` (optional) is the native address to credit the funds to. If not provided, the connected wallet's native address will be used. ##### Arweave (AR) Crypto Top Up ```typescript const turbo = TurboFactory.authenticated({ signer, token: 'arweave' }); const { winc, status, id, ...fundResult } = await turbo.topUpWithTokens({ tokenAmount: WinstonToTokenAmount(100_000_000), // 0.0001 AR feeMultiplier: 1.1, // 10% increase in reward for improved mining chances turboCreditDestinationAddress: '0xabc...123', // Any custom EVM / SOL / AR native destination address }); ``` ##### AR.IO Network (ARIO) Crypto Top Up ```typescript const turbo = TurboFactory.authenticated({ signer, token: 'ario' }); const { winc, status, id, ...fundResult } = await turbo.topUpWithTokens({ tokenAmount: ARIOToTokenAmount(100), // 100 $ARIO }); // ARIO on Base Network const { winc, status, id, ...fundResult } = await TurboFactory.authenticated({ signer, token: 'base-ario', }).topUpWithTokens({ tokenAmount: ARIOToTokenAmount(100), // 100 $ARIO }); ``` ##### USDC Crypto Top Up ```typescript // USDC on Ethereum Mainnet const { winc, status, id, ...fundResult } = await TurboFactory.authenticated({ signer, token: 'usdc', }).topUpWithTokens({ tokenAmount: USDCToTokenAmount(1), // 1 USDC }); // USDC on Base Network const { winc, status, id, ...fundResult } = await TurboFactory.authenticated({ signer, token: 'base-usdc', }).topUpWithTokens({ tokenAmount: USDCToTokenAmount(1), // 1 USDC }); // USDC on Polygon Network const { winc, status, id, ...fundResult } = await TurboFactory.authenticated({ signer, token: 'polygon-usdc', }).topUpWithTokens({ tokenAmount: USDCToTokenAmount(1), // 1 USDC }); ``` ##### Ethereum (ETH) Crypto Top Up ```typescript const turbo = TurboFactory.authenticated({ signer, token: 'ethereum' }); const { winc, status, id, ...fundResult } = await turbo.topUpWithTokens({ tokenAmount: ETHToTokenAmount(0.00001), // 0.00001 ETH }); ``` ##### Polygon (POL / MATIC) Crypto Top Up ```typescript const turbo = TurboFactory.authenticated({ signer, token: 'pol' }); const { winc, status, id, ...fundResult } = await turbo.topUpWithTokens({ tokenAmount: POLToTokenAmount(0.00001), // 0.00001 POL }); ``` ##### Eth on Base Network Crypto Top Up ```typescript const turbo = TurboFactory.authenticated({ signer, token: 'base-eth' }); const { winc, status, id, ...fundResult } = await turbo.topUpWithTokens({ tokenAmount: ETHToTokenAmount(0.00001), // 0.00001 ETH bridged on Base Network }); ``` ##### Solana (SOL) Crypto Top Up ```typescript const turbo = TurboFactory.authenticated({ signer, token: 'solana' }); const { winc, status, id, ...fundResult } = await turbo.topUpWithTokens({ tokenAmount: SOLToTokenAmount(0.00001), // 0.00001 SOL }); ``` #### shareCredits() Shares credits from the connected wallet to the provided native address and approved winc amount. This action will create a signed data item for the approval ```typescript const { approvalDataItemId, approvedWincAmount } = await turbo.shareCredits({ approvedAddress: '2cor...VUa', approvedWincAmount: 800_000_000_000, // 0.8 Credits expiresBySeconds: 3600, // Credits will expire back to original wallet in 1 hour }); ``` #### revokeCredits() Revokes all credits shared from the connected wallet to the provided native address. ```typescript const revokedApprovals = await turbo.revokeCredits({ revokedAddress: '2cor...VUa', }); ``` #### getCreditShareApprovals() Returns all given or received credit share approvals for the connected wallet or the provided native address. ```typescript const { givenApprovals, receivedApprovals } = await turbo.getCreditShareApprovals({ userAddress: '2cor...VUa', }); ``` # TurboFactory (/sdks/turbo-sdk/(apis)/turbofactory) #### unauthenticated() Creates an instance of a client that accesses Turbo's unauthenticated services. ```typescript const turbo = TurboFactory.unauthenticated(); ``` #### authenticated() Creates an instance of a client that accesses Turbo's authenticated and unauthenticated services. Requires either a signer, or private key to be provided. See the [Signers] section for all supported signers and authentication methods. ```typescript const signer = new ArweaveSigner(jwk); const turbo = TurboFactory.authenticated({ signer }); ``` #### Testnet Configuration For development and testing, you can configure the SDK to use blockchain testnets. This allows you to test your integration with free testnet tokens without spending real cryptocurrency. **Important**: The SDK defaults to mainnet. You must explicitly set the `gatewayUrl` parameter to use a testnet. ```typescript // Base Sepolia (recommended for testing) const turbo = TurboFactory.authenticated({ privateKey: process.env.BASE_SEPOLIA_PRIVATE_KEY, token: 'base-eth', gatewayUrl: 'https://sepolia.base.org', // Required for testnet paymentServiceConfig: { url: 'https://payment.services.ar-io.dev', // ar.io testnet sandbox }, uploadServiceConfig: { url: 'https://upload.services.ar-io.dev', // ar.io testnet sandbox } }); // Solana Devnet const turbo = TurboFactory.authenticated({ privateKey: bs58.encode(secretKey), token: 'solana', gatewayUrl: 'https://api.devnet.solana.com', paymentServiceConfig: { url: 'https://payment.services.ar-io.dev', }, uploadServiceConfig: { url: 'https://upload.services.ar-io.dev', } }); // Ethereum Sepolia const turbo = TurboFactory.authenticated({ privateKey: process.env.SEPOLIA_PRIVATE_KEY, token: 'ethereum', gatewayUrl: 'https://sepolia.gateway.tenderly.co', paymentServiceConfig: { url: 'https://payment.services.ar-io.dev', }, uploadServiceConfig: { url: 'https://upload.services.ar-io.dev', }, }); ``` These endpoints are the **ar.io Testnet Sandbox** — the full ar.io stack (upload, payment, ArNS, and gateway) running on testnet, with a faucet so nothing costs real money. Uploaded data is served from the sandbox gateway at `https://ar-io.dev` and is **ephemeral** (purged after ~3 days); it is never posted to mainnet Arweave. See [the ar.io Testnet Sandbox docs](https://docs.ar.io/build/testnet). **Supported Testnets**: - **ARIO staging** (`ario`) - Staging ARIO on Solana devnet; fee-free funding, claim from the [ar.io faucet](https://faucet.services.ar-io.dev) - **Base Sepolia** (`base-eth`) - Supports on-demand funding - **Solana Devnet** (`solana`) - Supports on-demand funding - **Ethereum Sepolia** (`ethereum`) - Manual top-up only - **Polygon Amoy** (`pol`) - Manual top-up only # TurboUnauthenticatedClient (/sdks/turbo-sdk/(apis)/turbounauthenticatedclient) #### getSupportedCurrencies() Returns the list of currencies supported by the Turbo Payment Service for topping up a user balance of AR Credits (measured in Winston Credits, or winc). ```typescript const currencies = await turbo.getSupportedCurrencies(); ``` #### getSupportedCountries() Returns the list of countries supported by the Turbo Payment Service's top up workflow. ```typescript const countries = await turbo.getSupportedCountries(); ``` #### getFiatToAR() Returns the current raw fiat to AR conversion rate for a specific currency as reported by third-party pricing oracles. ```typescript const fiatToAR = await turbo.getFiatToAR({ currency: 'USD' }); ``` #### getFiatRates() Returns the current fiat rates for 1 GiB of data for supported currencies, including all top-up adjustments and fees. ```typescript const rates = await turbo.getFiatRates(); ``` #### getWincForFiat() Returns the current amount of Winston Credits including all adjustments for the provided fiat currency. ```typescript const { winc, actualPaymentAmount, quotedPaymentAmount, adjustments } = await turbo.getWincForFiat({ amount: USD(100), }); ``` #### getWincForToken() Returns the current amount of Winston Credits including all adjustments for the provided token amount. ```typescript const { winc, actualTokenAmount, equivalentWincTokenAmount } = await turbo.getWincForToken({ tokenAmount: WinstonToTokenAmount(100_000_000), }); ``` #### getFiatEstimateForBytes() Get the current price from the Turbo Payment Service, denominated in the specified fiat currency, for uploading a specified number of bytes to Turbo. ```typescript const turbo = TurboFactory.unauthenticated(); const { amount } = await turbo.getFiatEstimateForBytes({ byteCount: 1024 * 1024 * 1024, currency: 'usd', // specify the currency for the price }); console.log(amount); // Estimated usd price for 1 GiB ``` **Output:** ```json { "byteCount": 1073741824, "amount": 20.58, "currency": "usd", "winc": "2402378997310" } ``` #### getTokenPriceForBytes() Get the current price from the Turbo Payment Service, denominated in the specified token, for uploading a specified number of bytes to Turbo. ```typescript const turbo = TurboFactory.unauthenticated({ token: 'solana' }); const { tokenPrice } = await turbo.getTokenPriceForBytes({ byteCount: 1024 * 1024 * 100, }); console.log(tokenPrice); // Estimated SOL Price for 100 MiB ``` #### getUploadCosts() Returns the estimated cost in Winston Credits for the provided file sizes, including all upload adjustments and fees. ```typescript const [uploadCostForFile] = await turbo.getUploadCosts({ bytes: [1024] }); const { winc, adjustments } = uploadCostForFile; ``` #### uploadSignedDataItem() Uploads a signed data item. The provided `dataItemStreamFactory` should produce a NEW signed data item stream each time is it invoked. The `dataItemSizeFactory` is a function that returns the size of the file. The `signal` is an optional [AbortSignal] that can be used to cancel the upload or timeout the request. The `events` parameter is an optional object that can be used to listen to upload progress, errors, and success (refer to the [Events] section for more details). ```typescript const filePath = path.join(__dirname, './my-signed-data-item'); const dataItemSize = fs.statSync(filePath).size; const uploadResponse = await turbo.uploadSignedDataItem({ dataItemStreamFactory: () => fs.createReadStream(filePath), dataItemSizeFactory: () => dataItemSize, signal: AbortSignal.timeout(10_000), // cancel the upload after 10 seconds events: { // track upload events only onUploadProgress: ({ totalBytes, processedBytes }) => { console.log('Upload progress:', { totalBytes, processedBytes }); }, onUploadError: (error) => { console.log('Upload error:', { error }); }, onUploadSuccess: () => { console.log('Upload success!'); }, }, }); ``` #### createCheckoutSession() Creates a Stripe checkout session for a Turbo Top Up with the provided amount, currency, owner. The returned URL can be opened in the browser, all payments are processed by Stripe. To leverage promo codes, see [TurboAuthenticatedClient]. ##### Arweave (AR) Fiat Top Up ```typescript const { url, winc, paymentAmount, quotedPaymentAmount, adjustments } = await turbo.createCheckoutSession({ amount: USD(10.0), // $10.00 USD owner: publicArweaveAddress, // promo codes require an authenticated client }); // Open checkout session in a browser window.open(url, '_blank'); ``` ##### Ethereum (ETH) Fiat Top Up ```typescript const turbo = TurboFactory.unauthenticated({ token: 'ethereum' }); const { url, winc, paymentAmount } = await turbo.createCheckoutSession({ amount: USD(10.0), // $10.00 USD owner: publicEthereumAddress, }); ``` ##### Solana (SOL) Fiat Top Up ```typescript const turbo = TurboFactory.unauthenticated({ token: 'solana' }); const { url, winc, paymentAmount } = await turbo.createCheckoutSession({ amount: USD(10.0), // $10.00 USD owner: publicSolanaAddress, }); ``` ##### Polygon (POL / MATIC) Fiat Top Up ```typescript const turbo = TurboFactory.unauthenticated({ token: 'pol' }); const { url, winc, paymentAmount } = await turbo.createCheckoutSession({ amount: USD(10.0), // $10.00 USD owner: publicPolygonAddress, }); ``` #### submitFundTransaction() Submits the transaction ID of a funding transaction to Turbo Payment Service for top up processing. The `txId` is the transaction ID of the transaction to be submitted. Use this API if you've already executed your token transfer to the Turbo wallet. Otherwise, consider using `topUpWithTokens` to execute a new token transfer to the Turbo wallet and submit its resulting transaction ID for top up processing all in one go ```typescript const turbo = TurboFactory.unauthenticated(); // defaults to arweave token type const { status, id, ...fundResult } = await turbo.submitFundTransaction({ txId: 'my-valid-arweave-fund-transaction-id', }); ``` # ANT custody: transfer & manage records (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/ant-custody-transfer-manage-records) Turbo can custody the ANT (Metaplex Core asset) backing your name. These methods let you take self-custody or manage resolution records. Each is authenticated with an **action-bound, single-use signature**: the wallet signs a canonical `arns\n\\n` message plus the UUID nonce, so a captured signature can't be replayed against a different operation. ```typescript // Self-custody exit: move the ANT to a Solana pubkey you control await turbo.transferArNSAnt({ antId: 'ant-id', target: 'your-solana-pubkey', }); // Set a resolution record (undername defaults to '@') await turbo.setArNSRecord({ antId: 'ant-id', undername: 'docs', // omit for the apex '@' record transactionId: 'arweave-tx-id', ttlSeconds: 900, }); // Remove a resolution record await turbo.removeArNSRecord({ antId: 'ant-id', undername: 'docs' }); ``` # Buying a name (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/buying-a-name) `buyArNSName(params)` is the `Buy-Name` convenience wrapper over `purchaseArNSName`. Optionally, `paidBy` delegates the charge to one or more addresses that have shared credits with you. **`processId` is optional**, and it selects who owns the ANT (Metaplex Core asset) the name resolves to: - **Omit `processId`** → **Turbo custodial provisioning** (Model A): Turbo spawns and _owns_ the ANT for you. You can take self-custody later via `transferArNSAnt` (see "ANT custody" below). - **Supply `processId`** → **user-owned ANT** (Model B): the name points at an ANT you already own; Turbo never takes custody. ```typescript // Custodial lease (Model A): omit processId → Turbo owns the ANT const receipt = await turbo.buyArNSName({ name: 'my-name', type: 'lease', years: 1, }); // Lease against your own ANT (Model B) for 1 year const receipt = await turbo.buyArNSName({ name: 'my-name', type: 'lease', years: 1, processId: 'ant-process-id', }); // Permanent buy, charged to a delegated payer const receipt = await turbo.buyArNSName({ name: 'my-name', type: 'permabuy', processId: 'ant-process-id', // optional — omit for Turbo custodial provisioning paidBy: '\', // or an array of addresses }); console.log(receipt.nonce); // capture this to poll status / retry idempotently ``` Full runnable example (buy → poll to terminal): ```typescript const turbo = TurboFactory.authenticated({ privateKey: arweaveJwk }); async function buyName() { try { const { nonce } = await turbo.buyArNSName({ name: 'my-name', type: 'lease', years: 1, processId: 'ant-process-id', }); // Poll until terminal (success => messageId, failure => failedDate) for (;;) { const status = await turbo.getArNSPurchaseStatus({ nonce }); if (status.messageId) { console.log('Purchased. ArNS write tx:', status.messageId); return status; } if (status.failedDate) { throw new Error(`Purchase failed at ${status.failedDate}`); } await new Promise((r) => setTimeout(r, 2000)); } } catch (err) { if (err instanceof InsufficientCreditsError) { console.error('Not enough Turbo Credits — top up and retry.'); } throw err; } } ``` # Connecting a signer (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/connecting-a-signer) ArNS purchases are authenticated per wallet. Construct the client with `TurboFactory.authenticated` using any supported identity — the credit balance is keyed to that wallet's native address: ```typescript // Arweave const turbo = TurboFactory.authenticated({ privateKey: arweaveJwk }); // Ethereum const turbo = TurboFactory.authenticated({ privateKey: ethHexadecimalPrivateKey, token: 'ethereum', }); // Solana — request nonces are signed with arbundles' HexSolanaSigner (ed25519) const turbo = TurboFactory.authenticated({ privateKey: bs58SolanaSecretKey, token: 'solana', }); ``` # Dependency note (@solana/codecs) (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/dependency-note-solana-codecs) ArNS/ARIO support pulls in `@solana/spl-token`, whose transitive `@solana/spl-token-metadata@0.1.6` imports `getDataEnumCodec` from `@solana/codecs@2.0.0-rc.1`. In `@solana/codecs@3+` that There is no single codecs version that satisfies both `spl-token-metadata` (needs the old `getDataEnumCodec`) and `@solana/kit` (needs `5.x`), and `spl-token-metadata` has no release that uses the renamed API — so the fix belongs at the app's dependency-resolution layer, **not** at symbol-aliasing: - **Recommended:** stop deduping `@solana/codecs` so `@solana/spl-token-metadata` keeps its own nested `2.0.0-rc.1` copy. In Vite, ensure `@solana/codecs` is **not** in `resolve.dedupe`; with pnpm/yarn, allow the nested version (avoid a hoisted-to-`6.x` override for that subtree). This is cleaner than the `getDataEnumCodec → getDiscriminatedUnionCodec` alias plugin some apps use today, and removes the need for that shim. - If you must keep a single hoisted codecs copy, a build-time alias mapping `getDataEnumCodec` to `getDiscriminatedUnionCodec` remains the fallback. # Error handling & retries (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/error-handling-retries) - **`InsufficientCreditsError`** (HTTP `402`) — the wallet (or delegated payer) doesn't hold enough Turbo Credits. Prompt the user to top up, then retry. It exposes `.status === 402` and is exported from the package root. - **`ProvidedInputError`** — thrown client-side (before any network call) when required per-intent params are missing/invalid (e.g. a lease `Buy-Name` without `years`, or `Extend-Lease` without a positive `years`). - **`FailedRequestError`** — any other non-2xx response; inspect `.status` (e.g. `401`, `503`). **Idempotency / retry guidance:** the `nonce` is the idempotency key. Capture `response.nonce` up front; if the network drops after the request is sent, re-poll `getArNSPurchaseStatus({ nonce })` rather than blindly re-buying. On a `402`, top up and issue a fresh purchase — the captured nonce still lets you reconcile status. ```typescript try { await turbo.buyArNSName({ name, type: 'permabuy', processId }); } catch (err) { if (err instanceof InsufficientCreditsError) { // surface a top-up flow to the user } else { throw err; } } ``` # Extend, increase undernames, upgrade (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/extend-increase-undernames-upgrade) Each intent has a typed wrapper that enforces its required fields: ```typescript // Extend an existing lease by N years await turbo.extendArNSLease({ name: 'my-name', years: 2 }); // Increase the undername limit await turbo.increaseArNSUndernameLimit({ name: 'my-name', increaseQty: 5 }); // Upgrade a lease to a permanent name await turbo.upgradeArNSName({ name: 'my-name' }); ``` All of them return the same `{ nonce, purchaseReceipt, arioWriteResult }` shape as `buyArNSName` and are polled the same way. `purchaseArNSName(params)` is the general form if you prefer to pass `intent` explicitly. # Polling purchase status (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/polling-purchase-status) `getArNSPurchaseStatus({ nonce })` is available on both the authenticated and unauthenticated clients: ```typescript const status = await turbo.getArNSPurchaseStatus({ nonce }); // status.messageId -> present on terminal success (Solana ArNS write tx id) // status.failedDate -> present on terminal failure ``` # Pricing a name (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/pricing-a-name) `getArNSPriceForName(params)` returns the cost in both Turbo Credits (`winc`) and `mARIO`. Params are validated client-side per intent (a `ProvidedInputError` is thrown for missing/invalid fields before any request is sent). ```typescript const { winc, mARIO } = await turbo.getArNSPriceForName({ intent: 'Buy-Name', name: 'my-name', type: 'lease', // 'lease' | 'permabuy' years: 1, // required for leases processId: 'ant-process-id', // the ANT the name resolves to }); ``` # Purchase lifecycle (/sdks/turbo-sdk/(arns-names-paid-with-turbo-credits)/purchase-lifecycle) Every purchase is identified by a client-minted **UUID `nonce`**. The nonce is: 1. **Signed** by your wallet and sent to the bundler (proving intent). 2. The **idempotency key** for the purchase. 3. The **status-lookup key** — poll `getArNSPurchaseStatus({ nonce })` until the purchase reaches a terminal state. `purchaseArNSName` returns the `nonce` on **both** `response.nonce` and `response.purchaseReceipt.nonce`. A purchase is **terminal-success** once its status carries a `messageId` (the Solana transaction id of the on-chain ArNS write) and **terminal-failure** once it carries a `failedDate`. ``` buyArNSName() ──▶ POST /arns/purchase ──▶ { nonce, purchaseReceipt, arioWriteResult } │ poll getArNSPurchaseStatus({ nonce }) │ ┌──────────────────────────────────────┴───────────────────────┐ messageId present (success) failedDate present (failure) ``` # File Upload Events (/sdks/turbo-sdk/(events)/file-upload-events) These events are available for `upload`, `uploadFile`, and `uploadSignedDataItem` methods: - `onProgress` - emitted when the overall progress changes (includes both upload and signing). Each event consists of the total bytes, processed bytes, and the step (upload or signing) - `onError` - emitted when the overall upload or signing fails (includes both upload and signing) - `onSuccess` - emitted when the overall upload or signing succeeds (includes both upload and signing) - this is the last event emitted for the upload or signing process - `onSigningProgress` - emitted when the signing progress changes. - `onSigningError` - emitted when the signing fails. - `onSigningSuccess` - emitted when the signing succeeds - `onUploadProgress` - emitted when the upload progress changes - `onUploadError` - emitted when the upload fails - `onUploadSuccess` - emitted when the upload succeeds ```typescript const uploadResult = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream(filePath), fileSizeFactory: () => fileSize, events: { // overall events (includes signing and upload events) onProgress: ({ totalBytes, processedBytes, step }) => { console.log('Overall progress:', { totalBytes, processedBytes, step }); }, onError: ({ error, step }) => { console.log('Overall error:', { error, step }); }, onSuccess: () => { console.log('Overall success!'); }, // signing events onSigningProgress: ({ totalBytes, processedBytes }) => { console.log('Signing progress:', { totalBytes, processedBytes }); }, onSigningError: (error) => { console.log('Signing error:', { error }); }, onSigningSuccess: () => { console.log('Signing success!'); }, // upload events onUploadProgress: ({ totalBytes, processedBytes }) => { console.log('Upload progress:', { totalBytes, processedBytes }); }, onUploadError: (error) => { console.log('Upload error:', { error }); }, onUploadSuccess: () => { console.log('Upload success!'); }, }, }); ``` # Folder Upload Events (/sdks/turbo-sdk/(events)/folder-upload-events) These events are available for the `uploadFolder` method: - `onFileStart` - emitted when a file in the folder starts uploading. Includes the file name, file size, file index, and total number of files - `onFileProgress` - emitted when a file's upload or signing progress changes. Includes the file name, file index, total files, processed bytes for the file, total bytes for the file, and the current step (signing or upload) - `onFileComplete` - emitted when a file successfully completes uploading. Includes the file name, file index, total files, and the data item ID - `onFileError` - emitted when a file upload fails. Includes the file name, file index, total files, and the error - `onFolderProgress` - emitted when the overall folder upload progress changes. Includes the number of processed files, total files, processed bytes across all files, total bytes across all files, and the current phase (files or manifest) - `onFolderError` - emitted when the overall folder upload fails - `onFolderSuccess` - emitted when the folder upload successfully completes (including manifest generation) - this is the last event emitted for the folder upload process ```typescript const uploadResult = await turbo.upload({ data: 'The contents of my file!', signal: AbortSignal.timeout(10_000), // cancel the upload after 10 seconds dataItemOpts: { // optional }, events: { // overall events (includes signing and upload events) onProgress: ({ totalBytes, processedBytes, step }) => { const percentComplete = (processedBytes / totalBytes) * 100; console.log('Overall progress:', { totalBytes, processedBytes, step, percentComplete: percentComplete.toFixed(2) + '%', // eg 50.68% }); }, onError: (error) => { console.log('Overall error:', { error }); }, onSuccess: () => { console.log('Signed and upload data item!'); }, // upload events onUploadProgress: ({ totalBytes, processedBytes }) => { console.log('Upload progress:', { totalBytes, processedBytes }); }, onUploadError: (error) => { console.log('Upload error:', { error }); }, onUploadSuccess: () => { console.log('Upload success!'); }, // signing events onSigningProgress: ({ totalBytes, processedBytes }) => { console.log('Signing progress:', { totalBytes, processedBytes }); }, onSigningError: (error) => { console.log('Signing error:', { error }); }, onSigningSuccess: () => { console.log('Signing success!'); }, }, }); ``` # Arweave (/sdks/turbo-sdk/(signers)/arweave) #### Arweave JWK ```typescript const jwk = await arweave.crypto.generateJWK(); const turbo = TurboFactory.authenticated({ privateKey: jwk }); ``` #### ArweaveSigner ```typescript const signer = new ArweaveSigner(jwk); const turbo = TurboFactory.authenticated({ signer }); ``` #### ArconnectSigner ```typescript const signer = new ArconnectSigner(window.arweaveWallet); const turbo = TurboFactory.authenticated({ signer }); ``` # Base (/sdks/turbo-sdk/(signers)/base) #### Base ETH Private Key ```typescript const turbo = TurboFactory.authenticated({ privateKey: ethHexadecimalPrivateKey, token: 'base-eth', }); ``` #### Base USDC Private Key ```typescript const turbo = TurboFactory.authenticated({ privateKey: ethHexadecimalPrivateKey, token: 'base-usdc', }); ``` #### Base ARIO Private Key ```typescript const turbo = TurboFactory.authenticated({ privateKey: ethHexadecimalPrivateKey, token: 'base-ario', }); ``` # Ethereum (/sdks/turbo-sdk/(signers)/ethereum) #### EthereumSigner ```typescript const signer = new EthereumSigner(privateKey); const turbo = TurboFactory.authenticated({ signer }); ``` #### Ethereum Private Key ```typescript const turbo = TurboFactory.authenticated({ privateKey: ethHexadecimalPrivateKey, token: 'ethereum', }); ``` #### POL (MATIC) Private Key ```typescript const turbo = TurboFactory.authenticated({ privateKey: ethHexadecimalPrivateKey, token: 'pol', }); ``` # Solana (/sdks/turbo-sdk/(signers)/solana) #### HexSolanaSigner ```typescript const signer = new HexSolanaSigner(bs58.encode(secretKey)); const turbo = TurboFactory.authenticated({ signer }); ``` #### Solana Web Wallet Adapter ```typescript const turbo = TurboFactory.authenticated({ walletAdapter: window.solana, token: 'solana', }); ``` #### Solana Secret Key ```typescript const turbo = TurboFactory.authenticated({ privateKey: bs58.encode(secretKey), token: 'solana', }); ``` # Turbo SDK (/sdks/turbo-sdk) **For AI and LLM users**: Access the complete Turbo SDK documentation in plain text format at{" "} llm.txt for easy consumption by AI agents and language models. See [AI Agents & LLMs](/build/agents) for the full agent toolkit. The Turbo SDK provides a high-level interface for uploading data to Arweave through Turbo's optimized infrastructure. Built with TypeScript, it offers seamless integration with built-in error handling, automatic retries, and transparent pricing. ## Quick Start ### Install the SDK ```npm npm install @ardrive/turbo-sdk ``` ### Use the SDK ```javascript // Create an authenticated client const turbo = TurboFactory.authenticated({ privateKey: yourPrivateKey }); // Upload data with automatic payment const result = await turbo.uploadFile({ fileStreamFactory: () => fs.createReadStream('./my-file.pdf'), fileSizeFactory: () => fs.statSync('./my-file.pdf').size, }); console.log('Upload successful:', result); ``` ### Install the SDK ```npm npm install @ardrive/turbo-sdk ``` ### Install polyfills (required for web environments) Polyfills are required for React web environments due to the use of `crypto`, `buffer` and `process` types in the SDK's dependencies. ```npm npm install --save-dev vite-plugin-node-polyfills ``` ```js // vite.config.js plugins: [ nodePolyfills({ globals: { Buffer: true, global: true, process: true, }, }), ], }); ``` Configure your bundler (Webpack, Vite, Rollup, etc.) to provide polyfills for `crypto`, `process`, and `buffer`. Refer to your bundler's documentation for polyfill configuration. ### Use the SDK ```javascript // Create an authenticated client const turbo = TurboFactory.authenticated({ privateKey: yourPrivateKey }); // Upload data with automatic payment const fileInput = document.querySelector('input[type="file"]'); const file = fileInput.files[0]; const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, }); console.log('Upload successful:', result); ``` ```html Turbo SDK Upload Example // Polyfills are included in the minimized web bundle, so not necessary to import directly // Function to handle file upload async function uploadFile() { const fileInput = document.getElementById('fileInput'); const file = fileInput.files[0]; const privateKeyInput = document.getElementById('privateKey'); if (!file) { alert('Please select a file'); return; } if (!privateKeyInput.value) { alert('Please enter your private key'); return; } try { // Show loading state document.getElementById('status').textContent = 'Uploading...'; // Create authenticated client const turbo = TurboFactory.authenticated({ privateKey: privateKeyInput.value }); // Upload file const result = await turbo.uploadFile({ fileStreamFactory: () => file.stream(), fileSizeFactory: () => file.size, }); // Show success document.getElementById('status').innerHTML = ` Upload successful! Transaction ID: ${result.id} Data Item ID: ${result.dataItemId} `; } catch (error) { document.getElementById('status').innerHTML = ` Error: ${error.message} `; } } // Add click handler window.addEventListener('load', () => { document.getElementById('uploadBtn').addEventListener('click', uploadFile); }); Turbo SDK Upload Example Private Key (JWK): Select File: Upload to Arweave ``` ## API Reference & Documentation } title="API Reference" description="Complete API documentation for all Turbo client methods" href="/apis/turbo" /> } title="SDK Details" description="Advanced features, events, logging, and credit sharing" href="/sdks/turbo-sdk" /> ## Core Features } title="Upload Management" description="Authenticated and unauthenticated upload clients with retry logic" href="/sdks/turbo-sdk/turboauthenticatedclient" /> } title="Events & Monitoring" description="Monitor upload progress and handle events in real-time" href="/sdks/turbo-sdk/file-upload-events/" /> } title="Credit Sharing" description="Manage shared credit pools for streamlined billing" href="/sdks/turbo-sdk/turbo-credit-sharing" /> } title="Logging & Configuration" description="Configure logging for debugging and monitoring uploads" href="/sdks/turbo-sdk/logging" /> # Logging (/sdks/turbo-sdk/logging) The SDK uses winston for logging. You can set the log level using the `setLogLevel` method. ```typescript TurboFactory.setLogLevel('debug'); ``` # Turbo Credit Sharing (/sdks/turbo-sdk/turbo-credit-sharing) Users can share their purchased Credits with other users' wallets by creating Credit Share Approvals. These approvals are created by uploading a signed data item with tags indicating the recipient's wallet address, the amount of Credits to share, and an optional amount of seconds that the approval will expire in. The recipient can then use the shared Credits to pay for their own uploads to Turbo. Shared Credits cannot be re-shared by the recipient to other recipients. Only the original owner of the Credits can share or revoke Credit Share Approvals. Credits that are shared to other wallets may not be used by the original owner of the Credits for sharing or uploading unless the Credit Share Approval is revoked or expired. Approvals can be revoked at any time by similarly uploading a signed data item with tags indicating the recipient's wallet address. This will remove all approvals and prevent the recipient from using the shared Credits. All unused Credits from expired or revoked approvals are returned to the original owner of the Credits. To use the shared Credits, recipient users must provide the wallet address of the user who shared the Credits with them in the `x-paid-by` HTTP header when uploading data. This tells Turbo services to look for and use Credit Share Approvals to pay for the upload before using the signer's balance. For user convenience, during upload the Turbo CLI will use any available Credit Share Approvals found for the connected wallet before using the signing wallet's balance. To instead ignore all Credit shares and only use the signer's balance, use the `--ignore-approvals` flag. To use the signer's balance first before using Credit shares, use the `--use-signer-balance-first` flag. In contrast, the Turbo SDK layer does not provide this functionality and will only use approvals when `paidBy` is provided. The Turbo SDK provides the following methods to manage Credit Share Approvals: - `shareCredits`: Creates a Credit Share Approval for the specified wallet address and amount of Credits. - `revokeCredits`: Revokes all Credit Share Approvals for the specified wallet address. - `listShares`: Lists all Credit Share Approvals for the specified wallet address or connected wallet. - `dataItemOpts: { ...opts, paidBy: string[] }`: Upload methods now accept 'paidBy', an array of wallet addresses that have provided credit share approvals to the user from which to pay, in the order provided and as necessary, for the upload. The Turbo CLI provides the following commands to manage Credit Share Approvals: - `share-credits`: Creates a Credit Share Approval for the specified wallet address and amount of Credits. - `revoke-credits`: Revokes all Credit Share Approvals for the specified wallet address. - `list-shares`: Lists all Credit Share Approvals for the specified wallet address or connected wallet. - `paidBy: --paid-by `: Upload commands now accept '--paid-by', an array of wallet addresses that have provided credit share approvals to the user from which to pay, in the order provided and as necessary, for the upload. - `--ignore-approvals`: Ignore all Credit Share Approvals and only use the signer's balance. - `--use-signer-balance-first`: Use the signer's balance first before using Credit Share Approvals. # Wayfinder SDK's (/sdks/wayfinder) **For AI and LLM users**: Access the complete Wayfinder documentation in plain text format at{" "} llm.txt for easy consumption by AI agents and language models. See [AI Agents & LLMs](/build/agents) for the full agent toolkit. Wayfinder leverages the decentralized ar.io Network to provide robust, censorship-resistant access to data stored on Arweave, removing reliance on centralized gateways. By routing requests through a distributed set of community-operated gateways, Wayfinder ensures high availability, redundancy, and improved performance for users and applications. The `ar://` protocol enables decentralized resolution and access to Arweave data using several flexible URL formats: - `ar://TRANSACTION_ID` — Direct access to a specific Arweave transaction - `ar://NAME` — Resolution of ArNS names (with optional path support) - `ar:///info` — Direct access to gateway endpoints (e.g., `/info`) To learn more about the Wayfinder protocol and how it works, visit [/learn/wayfinder](/learn/wayfinder). ## Getting Started Choose your environment to get started with Wayfinder: } title="@ar.io/wayfinder-core" description="Core SDK for Node.js applications and server environments" href="/sdks/wayfinder/wayfinder-core" /> } title="@ar.io/wayfinder-react" description="React hooks and components for browser applications" href="/sdks/wayfinder/wayfinder-react" /> } title="Wayfinder Router" description="Standalone proxy service with built-in verification and caching" href="/build/run-wayfinder-router" /> **Don't want to embed an SDK?** [Wayfinder Router](/build/run-wayfinder-router) is a standalone proxy you can deploy to provide a single verified endpoint for your users. ## Quick Examples ### Node.js ```javascript // Create a Wayfinder client const wayfinder = createWayfinderClient({ ario: ARIO.mainnet(), }); // Fetch data using the ar:// protocol const response = await wayfinder.request('ar://ardrive'); console.log(response); ``` ### React ```jsx // Wrap your app with the provider function App() { return ( ); } // Use the hook in your components function WayfinderImage({ txId }: { txId: string }) { const { resolvedUrl, isLoading, error } = useWayfinderUrl({ txId }); if (error) { return Error resolving URL: {error.message}; } if (isLoading) { return Resolving URL...; } return ( ); } ``` ## Next Steps } title="Learn about ar.io" description="Understanding the ar.io decentralized infrastructure" href="/learn/what-is-ario" /> } title="Decentralized Access" description="Learn how to access Arweave data in a decentralized way" href="/build/access" /> } title="Run Wayfinder Router" description="Deploy a standalone verified proxy for your users" href="/build/run-wayfinder-router" /> } title="Run a Gateway" description="Join the network by operating your own ar.io gateway" href="/build/run-a-gateway/quick-start" /> # Data Retrieval Strategies (/sdks/wayfinder/wayfinder-core/data-retrieval-strategies) Wayfinder supports multiple data retrieval strategies to fetch transaction data from AR.IO gateways. These strategies determine how data is requested and assembled from the underlying storage layer. | Strategy | Use Case | Requirements | | --------------------------------- | ------------------------------------------- | -------------------------------------- | | `ContiguousDataRetrievalStrategy`| Standard data fetching via direct GET | Gateway has the data cached or able to fetch from trusted peers | | `ChunkDataRetrievalStrategy` | Chunk-based data assembly | Gateway supports `/chunk/\/data` endpoint (r58) and has requested transactions indexed | #### ContiguousDataRetrievalStrategy The default strategy that fetches data using a direct GET request to the gateway. This is the most straightforward approach and works for most use cases. ```javascript const wayfinder = new Wayfinder({ dataRetrievalStrategy: new ContiguousDataRetrievalStrategy(), }); ``` #### ChunkDataRetrievalStrategy An advanced strategy that provides the easiest way to load chunks stored on Arweave nodes via the robust chunk API provided by AR.IO gateways. This approach is particularly useful for: - **Direct chunk access**: Efficiently retrieves data directly from the underlying chunk storage layer - **Bundled data items**: Seamlessly fetches data items from within ANS-104 bundles using calculated offsets - **x402 payment compatibility**: Both strategies support custom fetch clients for payment-enabled requests - **Large file handling**: More reliable for large transactions that may time out with direct requests **Requirements:** - Gateway must support the `/chunk/\/data` endpoint (added in [r58](https://github.com/ar-io/ar-io-node/releases/tag/r58)) - Gateway must have the requested transaction indexed (offsets are needed to fetch directly from chunks) ```javascript const wayfinder = new Wayfinder({ dataRetrievalStrategy: new ChunkDataRetrievalStrategy(), }); ``` **How it works:** 1. Makes a HEAD request to get transaction metadata (root transaction ID, data offset, content length) 2. Queries `/tx/{root-tx-id}/offset` to get the root transaction's absolute offset in the weave 3. Calculates the absolute offset for the requested data item 4. Fetches data in chunks using `/chunk/\/data` and assembles the complete data stream 5. Validates that chunks belong to the expected root transaction for security **Sequence Diagram:** ```mermaid sequenceDiagram participant Client participant Wayfinder participant Gateway as AR.IO Gateway participant Arweave as Arweave Nodes Client->>Wayfinder: request('ar://data-item-id') activate Wayfinder Wayfinder->>Gateway: HEAD /tx/{data-item-id} Note over Gateway: Lookup data item metadatafrom indexed bundles Gateway-->>Wayfinder: Headers:- x-root-tx-id (bundle ID)- x-data-offset- content-length Wayfinder->>Gateway: GET /tx/{root-tx-id}/offset Gateway->>Arweave: GET /tx/{root-tx-id}/offset Note over Arweave: Lookup transaction offsetin the weave Arweave-->>Gateway: Root transaction offset Gateway-->>Wayfinder: Root transaction offset in weave Note over Wayfinder: Calculate absolute offset:absolute = root_offset + data_offset loop For each chunk needed Wayfinder->>Gateway: GET /chunk/{absolute-offset}/data Note over Gateway: Serve chunk data fromindexed storage usingroot transaction ID Gateway-->>Wayfinder: Chunk data + validation headers(x-root-tx-id for security) Note over Wayfinder: Validate chunk belongsto expected root TX Wayfinder-->>Client: Stream chunk data end Wayfinder-->>Client: Complete response deactivate Wayfinder ``` **Example with createWayfinderClient:** ```javascript const wayfinder = createWayfinderClient({ dataRetrievalStrategy: new ChunkDataRetrievalStrategy(), }); // Fetch a data item from within an ANS-104 bundle const response = await wayfinder.request('ar://data-item-id'); ``` #### x402 Support Both data retrieval strategies support custom fetch implementations, allowing you to use x402-enabled fetch clients for paid gateway requests. ```javascript const x402Fetch = createX402Fetch({ /* payment config */ }); const wayfinder = createWayfinderClient({ fetch: x402Fetch, dataRetrievalStrategy: new ChunkDataRetrievalStrategy({ fetch: x402Fetch, }), }); ``` # Dynamic Routing (/sdks/wayfinder/wayfinder-core/dynamic-routing) Wayfinder supports a `resolveUrl` method which generates dynamic redirect URLs to a target gateway based on the provided routing strategy. This function can be used to directly replace any hard-coded gateway URLs, and instead use Wayfinder's routing logic to select a gateway for the request. #### ArNS names Given an ArNS name, the redirect URL will be the same as the original URL, but with the gateway selected by Wayfinder's routing strategy. ```javascript const redirectUrl = await wayfinder.resolveUrl({ arnsName: 'ardrive', }); // results in https://ardrive.\ ``` #### Transaction Ids Given a txId, the redirect URL will be the same as the original URL, but with the gateway selected by Wayfinder's routing strategy. ```javascript const redirectUrl = await wayfinder.resolveUrl({ txId: 'example-tx-id', }); // results in https://\/example-tx-id ``` #### Legacy URLs Given a legacy arweave.net or arweave.dev URL, the redirect URL will be the same as the original URL, but with the gateway selected by Wayfinder's routing strategy. ```javascript const redirectUrl = await wayfinder.resolveUrl({ originalUrl: 'https://arweave.net/example-tx-id', }); // results in https://\/example-tx-id ``` #### ar:// URLs Given an ar:// URL, the redirect URL will be the same as the original URL, but with the gateway selected by Wayfinder's routing strategy. ```javascript const redirectUrl = await wayfinder.resolveUrl({ originalUrl: 'ar://example-name/subpath?query=value', }); // results in https://\/example-name/subpath?query=value ``` # Events and Monitoring (/sdks/wayfinder/wayfinder-core/events-and-monitoring) #### Global Events ```javascript const wayfinder = createWayfinderClient({ routingSettings: { events: { onRoutingStarted: (event) => console.log('Routing started:', event), onRoutingSucceeded: (event) => console.log('Gateway selected:', event), }, }, verificationSettings: { events: { onVerificationProgress: (event) => { const percentage = (event.processedBytes / event.totalBytes) * 100; console.log(`Verification: ${percentage.toFixed(2)}%`); }, onVerificationSucceeded: (event) => console.log('Verified:', event.txId), }, }, }); ``` #### Request-Specific Events ```javascript const response = await wayfinder.request('ar://example', { verificationSettings: { events: { onVerificationProgress: (event) => { console.log(`This request: ${event.txId}`); }, }, }, }); ``` # Gateway Providers (/sdks/wayfinder/wayfinder-core/gateway-providers) Gateway providers supply the list of gateways for routing. **By default, `createWayfinderClient` uses a cached `TrustedPeersGatewaysProvider`**. | Provider | Description | Use Case | | ------------------------------ | ---------------------------------------------- | --------------------------------------- | | `NetworkGatewaysProvider` | Returns gateways from AR.IO Network | Leverage AR.IO Network with quality filtering | | `TrustedPeersGatewaysProvider` | Fetches from trusted gateway's peers | Dynamic gateway discovery (default) | | `StaticGatewaysProvider` | Returns a static list of gateways | Testing or specific gateways | | `SimpleCacheGatewaysProvider` | In-memory caching wrapper | Reduce API calls (used by default) | | `LocalStorageGatewaysProvider` | Browser localStorage caching | Persistent caching (used by default in browsers) | | `CompositeGatewaysProvider` | Chains multiple providers with fallback | Maximum resilience with multiple sources | #### NetworkGatewaysProvider Returns a list of gateways from the ARIO Network based on on-chain [Gateway Address Registry](https://docs.ar.io/learn/gateways/gateway-registry). You can specify on-chain metrics for gateways to prioritize the highest quality gateways. Requires `@ar.io/sdk` and `@solana/kit`. ```javascript const ario = ARIO.init({ rpc: createSolanaRpc('https://api.mainnet-beta.solana.com'), }); const gatewayProvider = new NetworkGatewaysProvider({ ario, sortBy: 'operatorStake', sortOrder: 'desc', limit: 10, filter: (gateway) => gateway.status === 'joined', }); ``` #### TrustedPeersGatewaysProvider Fetches a dynamic list of trusted peer gateways from an AR.IO gateway's `/ar-io/peers` endpoint. This provider is useful for discovering available gateways from a trusted source. ```javascript const gatewayProvider = new TrustedPeersGatewaysProvider({ trustedGateway: 'https://turbo-gateway.com', }); ``` #### CompositeGatewaysProvider Chains multiple gateway providers together, trying each in sequence until one succeeds. This is useful for building resilient gateway discovery with fallbacks. **How it works:** 1. Tries each provider in the order they're provided 2. If a provider returns a non-empty list of gateways, those gateways are used 3. If a provider throws an error or returns an empty list, moves to the next provider 4. If all providers fail, throws an error ```javascript import { CompositeGatewaysProvider, NetworkGatewaysProvider, StaticGatewaysProvider, TrustedPeersGatewaysProvider, } from '@ar.io/wayfinder-core'; const ario = ARIO.init({ rpc: createSolanaRpc('https://api.mainnet-beta.solana.com'), }); // Example: Network-first with static fallback const gatewayProvider = new CompositeGatewaysProvider({ providers: [ // Try fetching from AR.IO network first new NetworkGatewaysProvider({ ario, sortBy: 'operatorStake', limit: 10, }), // Fallback to trusted peers if network fetch fails new TrustedPeersGatewaysProvider({ trustedGateway: 'https://turbo-gateway.com', }), // Final fallback to static list new StaticGatewaysProvider({ gateways: ['https://turbo-gateway.com', 'https://g8way.io'], }), ], }); ``` # Wayfinder Core (/sdks/wayfinder/wayfinder-core) **Building for the web?** Consider using [@ar.io/wayfinder-react](/sdks/wayfinder/wayfinder-react) for React applications, which provides hooks and components optimized for browser environments. ## Quick Start ### Install ```npm npm install @ar.io/wayfinder-core @ar.io/sdk ``` ### Import and configure ```javascript const wayfinder = createWayfinderClient(); ``` ### Make your first request ```javascript // Use Wayfinder to fetch and verify data using ar:// protocol const response = await wayfinder.request('ar://ardrive'); console.log(response); ``` ### Use custom strategies ```javascript const customWayfinder = createWayfinderClient({ routingStrategy: createRoutingStrategy({ strategy: 'random' }) verificationStrategy: createVerificationStrategy({ strategy: 'hash', trustedGateways: ['https://turbo-gateway.com'] }) }); ``` ### Enable Telemetry (Optional) ```javascript const customWayfinder = createWayfinderClient({ telemetrySettings: { enabled: true, sampleRate: 0.1, // 10% sampling clientName: 'my-app', clientVersion: '1.0.0', } }); ``` ## Next Steps } title="Dynamic Routing" description="Use Wayfinder for dynamic routing" href="/sdks/wayfinder/wayfinder-core/dynamic-routing" /> } title="Gateway Providers" description="Learn about different gateway provider strategies" href="/sdks/wayfinder/wayfinder-core/networkgatewaysprovider" /> } title="Request Flow" description="Understanding how Wayfinder routes and verifies requests" href="/sdks/wayfinder/wayfinder-core/request-flow" /> } title="Monitoring & Events" description="Monitor performance and handle events" href="/sdks/wayfinder/wayfinder-core/global-events" /> # Request Flow (/sdks/wayfinder/wayfinder-core/request-flow) ```mermaid sequenceDiagram participant Client participant Wayfinder participant Gateways Provider participant Routing Strategy participant Selected Gateway participant Verification Strategy participant Trusted Gateways Client->>Wayfinder: request('ar://example') activate Wayfinder Wayfinder->>+Gateways Provider: getGateways() Gateways Provider-->>-Wayfinder: List of gateway URLs Wayfinder->>+Routing Strategy: selectGateway() Routing Strategy-->>-Wayfinder: Selected gateway Wayfinder->>+Selected Gateway: HTTP request Selected Gateway-->>-Wayfinder: Response data opt Verification enabled Wayfinder->>+Verification Strategy: verifyData() Verification Strategy->>Trusted Gateways: Get verification data Trusted Gateways-->>Verification Strategy: Verification headers Verification Strategy-->>-Wayfinder: Verification result end Wayfinder-->>Client: Response or error deactivate Wayfinder ``` # Resiliency (/sdks/wayfinder/wayfinder-core/resiliency) Wayfinder includes built-in resiliency features: - **Gateway retry**: If a gateway returns a 5xx error or a network failure occurs, Wayfinder automatically re-selects a different gateway and retries (up to 3 attempts). Client errors (4xx) are returned immediately without retry. - **Fetch timeouts**: All outbound requests include configurable timeouts — 10s for metadata (HEAD, peer list), 30s for data retrieval — to prevent indefinite hangs on slow or dead gateways. - **Smart pagination**: `NetworkGatewaysProvider` stops fetching from the on-chain registry once enough gateways pass the filter, avoiding unnecessary RPC calls. # Routing Strategies (/sdks/wayfinder/wayfinder-core/routing-strategies) Wayfinder supports multiple routing strategies to select target gateways for your requests. | Strategy | Description | Use Case | | ---------------------------- | ---------------------------------------------- | --------------------------------------- | | `RandomRoutingStrategy` | Selects a random gateway from a list | Good for load balancing and resilience | | `StaticRoutingStrategy` | Always uses a single gateway | When you need to use a specific gateway | | `RoundRobinRoutingStrategy` | Selects gateways in round-robin order | Good for load balancing and resilience | | `FastestPingRoutingStrategy` | Selects the fastest gateway based on ping time | Good for performance and latency | | `PreferredWithFallbackRoutingStrategy` | Uses a preferred gateway, with a fallback strategy if the preferred gateway is not available | Good for performance and resilience. Ideal for builders who run their own gateways. | | `CompositeRoutingStrategy` | Chains multiple routing strategies together, trying each sequentially until one succeeds | Good for complex fallback scenarios and maximum resilience | #### RandomRoutingStrategy Selects a random gateway from a list of gateways. ```javascript const strategy = new RandomRoutingStrategy({ gatewaysProvider: myGatewaysProvider, }); ``` #### FastestPingRoutingStrategy Selects the fastest gateway based on ping time. This strategy pings all available gateways and selects the one with the lowest latency. ```javascript const strategy = new FastestPingRoutingStrategy({ timeoutMs: 1000, gatewaysProvider: myGatewaysProvider, }); ``` #### PreferredWithFallbackRoutingStrategy Uses a preferred gateway, with a fallback strategy if the preferred gateway is not available. This is useful for builders who run their own gateways and want to use their own gateway as the preferred gateway, but also want to have a fallback strategy in case their gateway is not available. This strategy is built using `CompositeRoutingStrategy` internally. It first attempts to ping the preferred gateway (using `PingRoutingStrategy` with `StaticRoutingStrategy`), and if that fails, it falls back to the specified fallback strategy. ```javascript const strategy = new PreferredWithFallbackRoutingStrategy({ preferredGateway: 'https://my-gateway.com', fallbackStrategy: new FastestPingRoutingStrategy({ timeoutMs: 500 }), }); ``` #### CompositeRoutingStrategy The `CompositeRoutingStrategy` allows you to chain multiple routing strategies together, providing maximum resilience by trying each strategy in sequence until one succeeds. This is ideal for complex fallback scenarios where you want to combine different routing approaches. **How it works:** 1. Tries each strategy in the order they're provided 2. If a strategy successfully returns a gateway, that gateway is used (remaining strategies are skipped) 3. If a strategy throws an error, moves to the next strategy 4. If all strategies fail, throws an error **Common use cases:** - **Performance + Resilience**: Try fastest ping first, fallback to random if ping fails - **Preferred + Network**: Use your own gateway first, fallback to AR.IO network selection - **Multi-tier Fallback**: Try premium gateways, then standard gateways, then any available gateway ```javascript import { createWayfinderClient, CompositeRoutingStrategy, FastestPingRoutingStrategy, RandomRoutingStrategy, StaticRoutingStrategy, NetworkGatewaysProvider, } from '@ar.io/wayfinder-core'; const ario = ARIO.init({ rpc: createSolanaRpc('https://api.mainnet-beta.solana.com'), }); // Example 1: Performance-first with resilience fallback const performanceWayfinder = createWayfinderClient({ routingStrategy: new CompositeRoutingStrategy({ strategies: [ // Try fastest ping first (high performance, but may fail if all gateways are slow) new FastestPingRoutingStrategy({ timeoutMs: 500, gatewaysProvider: new NetworkGatewaysProvider({ ario, sortBy: 'operatorStake', limit: 10, }), }), // Fallback to random selection (guaranteed to work if gateways exist) new RandomRoutingStrategy({ gatewaysProvider: new NetworkGatewaysProvider({ ario, sortBy: 'operatorStake', limit: 20, // Use more gateways for fallback }), }), ], }), }); // Example 2: Preferred gateway with multi-tier fallback const preferredWayfinder = createWayfinderClient({ routingStrategy: new CompositeRoutingStrategy({ strategies: [ // First, try your preferred gateway new StaticRoutingStrategy({ gateway: 'https://my-preferred-gateway.com' }), // If that fails, try fastest ping from top-tier gateways new FastestPingRoutingStrategy({ timeoutMs: 1000, gatewaysProvider: new NetworkGatewaysProvider({ ario, sortBy: 'operatorStake', limit: 5, // Only top 5 gateways }), }), // Final fallback: any random gateway from a larger pool new RandomRoutingStrategy({ gatewaysProvider: new NetworkGatewaysProvider({ ario, limit: 50, // Larger pool for maximum availability }), }), ], }), }); ``` # Telemetry (/sdks/wayfinder/wayfinder-core/telemetry) Wayfinder can optionally emit OpenTelemetry spans for every request. **By default, telemetry is disabled**. You can control this behavior with the `telemetry` option. ```typescript const wayfinder = createWayfinderClient({ // other settings... telemetrySettings: { enabled: true, sampleRate: 0.1, // 10% sampling exporterUrl: 'https://your-otel-exporter', clientName: 'my-app', clientVersion: '1.0.0', }, }); ``` # Verification Strategies (/sdks/wayfinder/wayfinder-core/verification-strategies) Wayfinder includes verification mechanisms to ensure the integrity of retrieved data. Verification strategies offer different trade-offs between complexity, performance, and security. | Verifier | Complexity | Performance | Security | Description | | ------------------------------- | ---------- | ----------- | -------- | ------------------------------------------------------------------------------------------------------------ | | `RemoteVerificationStrategy` | Low | Low | Low | Checks the `x-ar-io-verified` header from the gateway that returned the data. If `true`, the data is considered verified and trusted. | | `HashVerificationStrategy` | Low | High | Low | Computes the SHA-256 hash of the returned data and comparing it to the hash of a **trusted gateway** (_**recommended for most users**_). | | `DataRootVerificationStrategy` | Medium | Medium | Low | Computes the data root for the transaction (most useful for L1 transactions) and compares it to the data root provided by a **trusted gateway**. | | `SignatureVerificationStrategy` | Medium | Medium | Medium | - **ANS-104 Data Items**: Fetches signature components (owner, signature type, tags, etc.) from trusted gateways using range requests, then verifies signatures against the data payload using deep hash calculations following the ANS-104 standard.- **L1 Transactions**: Retrieves transaction metadata from gateway /tx/\ endpoints, computes the data root from the provided data stream, and verifies the signature using Arweave's cryptographic verification. | #### RemoteVerificationStrategy This strategy is used to verify data by checking the `x-ar-io-verified` header from the gateway that returned the data. If the header is set to `true`, the data is considered verified and trusted. This strategy is only recommended for users fetching data from their own gateways and want to avoid the overhead of the other verification strategies. ```javascript const wayfinder = new Wayfinder({ verificationSettings: { // no trusted gateways are required for this strategy enabled: true, strategy: new RemoteVerificationStrategy(), }, }); ``` #### HashVerificationStrategy Verifies data integrity using SHA-256 hash comparison. This is the default verification strategy and is recommended for most users looking for a balance between security and performance. ```javascript const wayfinder = new Wayfinder({ verificationSettings: { enabled: true, strategy: new HashVerificationStrategy({ trustedGateways: [new URL('https://turbo-gateway.com')], }), }, }); ``` #### DataRootVerificationStrategy Verifies data integrity using Arweave by computing the data root for the transaction. This is useful for L1 transactions and is recommended for users who want to ensure the integrity of their data. ```javascript const wayfinder = new Wayfinder({ verificationSettings: { enabled: true, strategy: new DataRootVerificationStrategy({ trustedGateways: [new URL('https://turbo-gateway.com')], }), }, }); ``` #### SignatureVerificationStrategy Verifies signatures of Arweave transactions and data items. Headers are retrieved from trusted gateways for use during verification. For a transaction, its data root is computed while streaming its data and then utilized alongside its headers for verification. For data items, the ANS-104 deep hash method of signature verification is used. ```javascript const wayfinder = new Wayfinder({ verificationSettings: { enabled: true, strategy: new SignatureVerificationStrategy({ trustedGateways: [new URL('https://turbo-gateway.com')], }), }, }); ``` # x402 Payments (/sdks/wayfinder/wayfinder-core/x402-payments) Wayfinder can be configured to work with the [x402 payment protocol](https://docs.ar.io/learn/gateways/x402-payments#what-is-x402) for paid gateway services and higher rate limits. This allows you to seamlessly make requests that may require payment without having to manually handle payment flows. To get started, install the `@ar.io/wayfinder-x402-fetch` package. The `@ar.io/wayfinder-x402-fetch` package is a simple wrapper of the [x402-fetch](https://github.com/coinbase/x402/tree/1d4c253aef959b73b46d42e7f0ccf30c7ce64955/typescript/packages/x402-fetch) library, which creates a fetch implementation to automatically handles x402 payment flows. You can use this fetch implementation with Wayfinder to enable x402 payments for your requests. ```javascript // Set up your wallet for x402 payments const privateKey = process.env.X402_PRIVATE_KEY; // Your private key const account = privateKeyToAccount(privateKey); // Create x402-enabled fetch implementation const x402Fetch = createX402Fetch({ walletClient: account, }); // Create Wayfinder client with x402 fetch to handle payments const wayfinder = createWayfinderClient({ fetch: x402Fetch, routingSettings: { // Configure to use x402-enabled gateways strategy: new StaticRoutingStrategy({ gateway: 'https://paid-gateway.example.com', }), }, }); // Requests will now automatically handle x402 payments const response = await wayfinder.request('ar://transaction-id'); ``` **How it works:** 1. When a gateway returns a `402 Payment Required` status 2. The x402 fetch automatically handles the payment flow 3. The request is retried with payment credentials 4. You get access to premium gateway services **Use cases:** - Higher rate limits on data requests - Access to premium gateway features - Supporting gateway operators through payments To learn more about x402 payments, visit the [x402 documentation](https://docs.ar.io/learn/gateways/x402-payments). # useWayfinderRequest (/sdks/wayfinder/wayfinder-react/(hooks)/usewayfinderrequest) Fetch the data via wayfinder, and optionally verify the data. ```tsx function WayfinderData({ txId }: { txId: string }) { const request = useWayfinderRequest(); const [data, setData] = useState\(null); const [dataLoading, setDataLoading] = useState(false); const [dataError, setDataError] = useState\(null); useEffect(() => { (async () => { try { setDataLoading(true); setDataError(null); // fetch the data for the txId using wayfinder const response = await request(`ar://${txId}`, { verificationSettings: { enabled: true, // enable verification on the request strict: true, // don't use the data if it's not verified }, }); const data = await response.arrayBuffer(); // or response.json() if you want to parse the data as JSON setData(data); } catch (error) { setDataError(error as Error); } finally { setDataLoading(false); } })(); }, [request, txId]); if (dataError) { return Error loading data: {dataError.message}; } if (dataLoading) { return Loading data...; } if (!data) { return No data; } return ( {data} ); } ``` # useWayfinderUrl (/sdks/wayfinder/wayfinder-react/(hooks)/usewayfinderurl) Get a dynamic URL for an existing `ar://` URL or legacy `arweave.net`/`arweave.dev` URL. Example: ```tsx function WayfinderImage({ txId }: { txId: string }) { const { resolvedUrl, isLoading, error } = useWayfinderUrl({ txId }); if (error) { return Error resolving URL: {error.message}; } if (isLoading) { return Resolving URL...; } return ( ); } ``` # Wayfinder React (/sdks/wayfinder/wayfinder-react) A set of React hooks and components for integrating Wayfinder, the decentralized data access system for Arweave. Wayfinder-react wraps the functionality of wayfinder-core in user-friendly React components and hooks, making it easy to integrate ar.io network functionality into your React applications with built-in loading states, error handling, and caching. ## Quick Start ### Install Wayfinder React ```npm npm install @ar.io/wayfinder-react @ar.io/wayfinder-core @ar.io/sdk ``` ### Install polyfills (required for web environments) Crypto polyfills are required for web environments due to the use of `crypto`, `buffer` and `process` types in wayfinder-react dependencies (i.e. `arbundles`). ```npm npm install --save-dev vite-plugin-node-polyfills ``` ```js // vite.config.js plugins: [ nodePolyfills({ globals: { Buffer: true, global: true, process: true, }, }), ], }); ``` Configure your bundler (Webpack, Vite, Rollup, etc.) to provide polyfills for `crypto`, `process`, and `buffer`. Refer to your bundler's documentation for polyfill configuration. ### Setup the provider ```jsx // App.tsx function App() { return ( ); } ``` ### Use the available hooks ```typescript function WayfinderImage({ txId }: { txId: string }) { const { resolvedUrl, isLoading, error } = useWayfinderUrl({ txId }); if (error) { return Error resolving URL: {error.message}; } if (isLoading) { return Resolving URL...; } return ( ); } ``` ## Next Steps } title="Hooks Reference" description="Comprehensive guide to all available React hooks" href="/sdks/wayfinder/wayfinder-react/usewayfinderrequest" /> } title="Wayfinder Core" description="Learn about the underlying core library" href="/sdks/wayfinder/wayfinder-core" />