Rate Limits

Anyhunt uses rate limits to ensure fair usage and service reliability. Understanding these limits helps you build robust integrations.

Types of Limits

Request Rate Limit

Maximum requests per minute per API key:

Concurrency Limit

Maximum simultaneous operations being processed:

Monthly Quota

Total API calls per billing cycle (includes all API types):

API-Specific Limits

Different APIs have different resource costs:

Crawl Limits

Batch Scrape Limits

Rate Limit Headers

Every response includes rate limit information:

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 45
X-RateLimit-Reset: 1704067200

Handling Rate Limits

When rate limited, you'll receive:

{
  "type": "https://anyhunt.app/errors/RATE_LIMITED",
  "title": "Too Many Requests",
  "status": 429,
  "detail": "Too many requests. Please try again later.",
  "code": "RATE_LIMITED",
  "details": {
    "retryAfter": 15
  }
}

Best Practices

1. Check headers - Monitor remaining quota proactively
2. Implement backoff - Use exponential backoff on 429 errors
3. Queue requests - Buffer requests during high load
4. Use webhooks - Avoid polling by using webhook callbacks for Crawl and Batch APIs
5. Leverage caching - Cached responses don't count against quota

Example: Exponential Backoff

async function scrapeWithRetry(url, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    const response = await fetch('https://server.anyhunt.app/api/v1/scrape', {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ url }),
    });

    if (response.status !== 429) {
      return response.json();
    }

    const retryAfter = response.headers.get('Retry-After') || Math.pow(2, i);
    await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
  }

  throw new Error('Max retries exceeded');
}

Quota Management

Check Remaining Quota

View your current quota usage in the Anyhunt Console. The dashboard shows:

• Current plan and monthly quota
• Used and remaining credits
• Usage breakdown by API type
• Quota reset date

Cache Hits Don't Count

When content is served from cache, it doesn't count against your quota. Check the fromCache field in responses:

{
  "data": {
    "id": "scrape_abc123",
    "fromCache": true
  }
}

Failed Requests Refunded

If a request fails (timeout, blocked URL, etc.), the quota is automatically refunded.

Optimizing Usage

Use Appropriate APIs

Choose the right API for your use case:

Batch Similar Requests

Instead of multiple Scrape calls:

// Inefficient: 10 separate calls
for (const url of urls) {
  await scrape(url);
}

// Efficient: 1 batch call
await batchScrape(urls);

Leverage Caching

Default cache duration is 1 hour. Identical requests within this window are served from cache at no cost.

Need Higher Limits?

• Upgrade your plan - Higher plans have higher limits
• Contact sales - For custom enterprise limits
• Optimize usage - Batch similar URLs, leverage caching

Error Codes

See Error Handling for retry strategies.