AR.IO Node Filtering System

The AR.IO Node filtering system provides a flexible way to match and filter items based on various criteria. The system is built around JSON-based filter definitions that can be combined to create both simple and complex matching patterns.

Unbundling and Indexing Filters

When processing bundles, the AR.IO Node applies two filters obtained from environment variables:

ANS104_UNBUNDLE_FILTER=""
ANS104_INDEX_FILTER=""

The ANS104_UNBUNDLE_FILTER determines which base layer transactions and data items, in the case of bundles nested in other bundles, are processed, and the ANS104_INDEX_FILTER determines which data items within the processed bundles are indexed for querying.

Webhook Filters

There are also two filters available that are used to trigger webhooks. When a transaction is processed that matches one of the webhook filters, the gateway will send a webhook to the specified WEBHOOK_TARGET_SERVERS urls containing the transaction data.

WEBHOOK_INDEX_FILTER=""
WEBHOOK_BLOCK_FILTER=""

The WEBHOOK_INDEX_FILTER is used to trigger a webhook when a transaction is indexed. The WEBHOOK_BLOCK_FILTER is used to trigger a webhook when a block is processed.

Important Notes

• All tag names and values are base64url-decoded before matching
• Owner addresses are automatically converted from owner public keys
• Empty or undefined filters default to "never match"
• Tag matching requires all specified tags to match
• Attribute matching requires all specified attributes to match
• The filter system supports nested logical operations to any depth, allowing for very precise control over what data gets processed

All these filters can be used in various contexts within the AR.IO Node, such as configuring webhook triggers, controlling ANS-104 bundle processing, or setting up data indexing rules. The filtering system is designed to be intuitive yet powerful, allowing for precise control over which items get processed while maintaining readable and maintainable filter definitions.

Filter Construction

Basic Filters

The simplest filters you can use "always" and "never" filters. The "never" filter is the default behavior and will match nothing, while the "always" filter matches everything.

{
  "never": true //default behavior
}

Tag Filters

Tag filters allow you to match items based on their tags in three different ways. You can match exact tag values, check for the presence of a tag regardless of its value, or match tags whose values start with specific text. All tag values are automatically base64url-decoded before matching.

{
  "tags": [
    {
      "name": "Content-Type",
      "value": "image/jpeg"
    }
  ]
}

Attribute Filters

Attribute filtering allows you to match items based on their metadata properties. The system automatically handles owner public key to address conversion, making it easy to filter by owner address. You can combine multiple attributes in a single filter:

{
  "attributes": {
    "owner_address": "xyz123...",
    "data_size": 1000
  }
}

Nested Bundle Filter

The isNestedBundle filter is a specialized filter that checks whether a data item is part of a nested bundle structure. It's particularly useful when you need to identify or process data items in bundles that are contained within other bundles. The filter checks for the presence of a parent_id field in the item.

{
  "isNestedBundle": true
}

Note: When processing nested bundles, be sure to include filters that match the nested bundles in both ANS104_UNBUNDLE_FILTER and ANS104_INDEX_FILTER. The bundle data items (nested bundles) need to be indexed to be matched by the unbundle filter.

Complex Filters Using Logical Operators

For more complex scenarios, the system provides logical operators (AND, OR, NOT) that can be combined to create sophisticated filtering patterns. These operators can be nested to any depth:

{
  "and": [
    {
      "tags": [
        {
          "name": "App-Name",
          "value": "ArDrive-App"
        }
      ]
    },
    {
      "tags": [
        {
          "name": "Content-Type",
          "valueStartsWith": "image/"
        }
      ]
    }
  ]
}

Advanced Examples

{
  "and": [
    {
      "tags": [
        {
          "name": "App-Name",
          "value": "ArDrive-App"
        },
        {
          "name": "Content-Type",
          "valueStartsWith": "image/"
        }
      ]
    },
    {
      "attributes": {
        "data_size": 1000000
      }
    },
    {
      "not": {
        "isNestedBundle": true
      }
    }
  ]
}