Web Research API

The Web Research API provides web search and page content fetching capabilities, ideal for building AI Agents, RAG applications, data collection, and more. It supports real-time web search, batch page fetching, and content extraction.

Feature Overview

Web Search: Real-time search of internet content with relevant pages and summaries
Page Fetching: Batch fetching of content from specified URLs
Smart Extraction: Automatic extraction of page titles, body text, and core content
Flexible Filtering: Support for time range, result count, and other filtering options

Basic Information

API Endpoints

Search: https://api.routin.ai/v1/web/search
Fetch:  https://api.routin.ai/v1/web/fetch

Authentication Add your API Key in the request header:

Authorization: Bearer YOUR_API_KEY

The Web Research API is powered by professional web search services like Tavily, providing high-quality search results and content extraction capabilities.

Web Search API

API Endpoint

POST /v1/web/search

Request Parameters

Required Parameters

Parameter	Type	Description
`query`	string	Search query keywords

Optional Parameters

Parameter	Type	Default	Description
`maxResults`	integer	5	Number of results to return (1-20)
`startDate`	string	-	Start date (ISO 8601 format)
`endDate`	string	-	End date (ISO 8601 format)
`includeImages`	boolean	false	Whether to include image information

Response Format

{
  "items": [
    {
      "url": "https://example.com/article",
      "title": "Article Title",
      "content": "Article summary or body content...",
      "score": 0.95
    }
  ]
}

Response Field Descriptions:

url: Web page link
title: Page title
content: Page content summary
score: Relevance score (0-1), higher is more relevant

Code Examples

Basic Search

import requests

url = "https://api.routin.ai/v1/web/search"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}
data = {
    "query": "latest AI developments",
    "maxResults": 5
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

for item in result["items"]:
    print(f"Title: {item['title']}")
    print(f"URL: {item['url']}")
    print(f"Score: {item['score']:.2f}")
    print(f"Content: {item['content'][:100]}...")
    print("-" * 80)

interface SearchRequest {
  query: string;
  maxResults?: number;
}

interface SearchResultItem {
  url: string;
  title: string;
  content: string;
  score: number;
}

interface SearchResponse {
  items: SearchResultItem[];
}

async function webSearch(query: string, maxResults: number = 5): Promise<SearchResponse> {
  const response = await fetch('https://api.routin.ai/v1/web/search', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ query, maxResults }),
  });

  return await response.json();
}

// Usage example
const result = await webSearch('latest AI developments', 5);
result.items.forEach(item => {
  console.log(`Title: ${item.title}`);
  console.log(`URL: ${item.url}`);
  console.log(`Score: ${item.score.toFixed(2)}`);
  console.log(`Content: ${item.content.substring(0, 100)}...`);
  console.log('-'.repeat(80));
});

const axios = require('axios');

async function webSearch(query, maxResults = 5) {
  const response = await axios.post(
    'https://api.routin.ai/v1/web/search',
    {
      query: query,
      maxResults: maxResults
    },
    {
      headers: {
        'Authorization': 'Bearer YOUR_API_KEY',
        'Content-Type': 'application/json'
      }
    }
  );

  return response.data;
}

// Usage example
webSearch('latest AI developments', 5)
  .then(result => {
    result.items.forEach(item => {
      console.log(`Title: ${item.title}`);
      console.log(`URL: ${item.url}`);
      console.log(`Score: ${item.score.toFixed(2)}`);
      console.log(`Content: ${item.content.substring(0, 100)}...`);
      console.log('-'.repeat(80));
    });
  })
  .catch(error => console.error('Search failed:', error));

using System;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;

public class WebSearchClient
{
    private readonly HttpClient _httpClient;
    private const string BaseUrl = "https://api.routin.ai/v1";

    public WebSearchClient(string apiKey)
    {
        _httpClient = new HttpClient();
        _httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {apiKey}");
    }

    public async Task<WebSearchResponse> Search(string query, int maxResults = 5)
    {
        var request = new WebSearchRequest
        {
            Query = query,
            MaxResults = maxResults
        };

        var json = JsonSerializer.Serialize(request);
        var content = new StringContent(json, Encoding.UTF8, "application/json");

        var response = await _httpClient.PostAsync($"{BaseUrl}/web/search", content);
        response.EnsureSuccessStatusCode();

        var responseJson = await response.Content.ReadAsStringAsync();
        return JsonSerializer.Deserialize<WebSearchResponse>(responseJson);
    }
}

public class WebSearchRequest
{
    public string Query { get; set; }
    public int MaxResults { get; set; }
}

public class WebSearchResponse
{
    public List<SearchResultItem> Items { get; set; }
}

public class SearchResultItem
{
    public string Url { get; set; }
    public string Title { get; set; }
    public string Content { get; set; }
    public double Score { get; set; }
}

// Usage example
var client = new WebSearchClient("YOUR_API_KEY");
var result = await client.Search("latest AI developments", 5);

foreach (var item in result.Items)
{
    Console.WriteLine($"Title: {item.Title}");
    Console.WriteLine($"URL: {item.Url}");
    Console.WriteLine($"Score: {item.Score:F2}");
    Console.WriteLine($"Content: {item.Content.Substring(0, Math.Min(100, item.Content.Length))}...");
    Console.WriteLine(new string('-', 80));
}

curl -X POST https://api.routin.ai/v1/web/search \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "query": "latest AI developments",
    "maxResults": 5
  }'

Time Range Search

Search for content within a specific time range.

import requests
from datetime import datetime, timedelta

url = "https://api.routin.ai/v1/web/search"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Search content from the last 7 days
end_date = datetime.now()
start_date = end_date - timedelta(days=7)

data = {
    "query": "GPT-5 release",
    "maxResults": 10,
    "startDate": start_date.isoformat(),
    "endDate": end_date.isoformat()
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

print(f"Found {len(result['items'])} results from the past week:")
for item in result["items"]:
    print(f"• {item['title']}")
    print(f"  {item['url']}\n")

async function searchWithDateRange(
  query: string,
  daysBack: number = 7
): Promise<SearchResponse> {
  const endDate = new Date();
  const startDate = new Date();
  startDate.setDate(startDate.getDate() - daysBack);

  const response = await fetch('https://api.routin.ai/v1/web/search', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      query,
      maxResults: 10,
      startDate: startDate.toISOString(),
      endDate: endDate.toISOString(),
    }),
  });

  return await response.json();
}

// Search content from the last 7 days
const result = await searchWithDateRange('GPT-5 release', 7);
console.log(`Found ${result.items.length} results from the past week:`);
result.items.forEach(item => {
  console.log(`• ${item.title}`);
  console.log(`  ${item.url}\n`);
});

public class WebSearchRequest
{
    public string Query { get; set; }
    public int MaxResults { get; set; }
    public DateTime? StartDate { get; set; }
    public DateTime? EndDate { get; set; }
}

// Search content from the last 7 days
var endDate = DateTime.Now;
var startDate = endDate.AddDays(-7);

var request = new WebSearchRequest
{
    Query = "GPT-5 release",
    MaxResults = 10,
    StartDate = startDate,
    EndDate = endDate
};

var result = await client.Search(request);
Console.WriteLine($"Found {result.Items.Count} results from the past week:");
foreach (var item in result.Items)
{
    Console.WriteLine($"• {item.Title}");
    Console.WriteLine($"  {item.Url}\n");
}

Page Fetching API

API Endpoint

POST /v1/web/fetch

Request Parameters

Required Parameters

Parameter	Type	Description
`urls`	string[]	List of URLs to fetch

Response Format

{
  "items": [
    {
      "url": "https://example.com/article",
      "title": "Article Title",
      "content": "Full page content..."
    }
  ]
}

Response Field Descriptions:

url: Original URL
title: Page title
content: Extracted page body content

Code Examples

Single Page Fetch

import requests

url = "https://api.routin.ai/v1/web/fetch"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}
data = {
    "urls": ["https://example.com/article"]
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

for item in result["items"]:
    print(f"Title: {item['title']}")
    print(f"URL: {item['url']}")
    print(f"Content length: {len(item['content'])} characters")
    print(f"Preview: {item['content'][:200]}...")

interface FetchRequest {
  urls: string[];
}

interface FetchResultItem {
  url: string;
  title: string;
  content: string;
}

interface FetchResponse {
  items: FetchResultItem[];
}

async function fetchWebPages(urls: string[]): Promise<FetchResponse> {
  const response = await fetch('https://api.routin.ai/v1/web/fetch', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ urls }),
  });

  return await response.json();
}

// Usage example
const result = await fetchWebPages(['https://example.com/article']);
result.items.forEach(item => {
  console.log(`Title: ${item.title}`);
  console.log(`URL: ${item.url}`);
  console.log(`Content length: ${item.content.length} characters`);
  console.log(`Preview: ${item.content.substring(0, 200)}...`);
});

const axios = require('axios');

async function fetchWebPages(urls) {
  const response = await axios.post(
    'https://api.routin.ai/v1/web/fetch',
    { urls },
    {
      headers: {
        'Authorization': 'Bearer YOUR_API_KEY',
        'Content-Type': 'application/json'
      }
    }
  );

  return response.data;
}

// Usage example
fetchWebPages(['https://example.com/article'])
  .then(result => {
    result.items.forEach(item => {
      console.log(`Title: ${item.title}`);
      console.log(`URL: ${item.url}`);
      console.log(`Content length: ${item.content.length} characters`);
      console.log(`Preview: ${item.content.substring(0, 200)}...`);
    });
  })
  .catch(error => console.error('Fetch failed:', error));

public class WebFetchClient
{
    private readonly HttpClient _httpClient;
    private const string BaseUrl = "https://api.routin.ai/v1";

    public WebFetchClient(string apiKey)
    {
        _httpClient = new HttpClient();
        _httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {apiKey}");
    }

    public async Task<WebFetchResponse> Fetch(params string[] urls)
    {
        var request = new WebFetchRequest { Urls = urls };
        var json = JsonSerializer.Serialize(request);
        var content = new StringContent(json, Encoding.UTF8, "application/json");

        var response = await _httpClient.PostAsync($"{BaseUrl}/web/fetch", content);
        response.EnsureSuccessStatusCode();

        var responseJson = await response.Content.ReadAsStringAsync();
        return JsonSerializer.Deserialize<WebFetchResponse>(responseJson);
    }
}

public class WebFetchRequest
{
    public string[] Urls { get; set; }
}

public class WebFetchResponse
{
    public List<FetchResultItem> Items { get; set; }
}

public class FetchResultItem
{
    public string Url { get; set; }
    public string Title { get; set; }
    public string Content { get; set; }
}

// Usage example
var client = new WebFetchClient("YOUR_API_KEY");
var result = await client.Fetch("https://example.com/article");

foreach (var item in result.Items)
{
    Console.WriteLine($"Title: {item.Title}");
    Console.WriteLine($"URL: {item.Url}");
    Console.WriteLine($"Content length: {item.Content.Length} characters");
    Console.WriteLine($"Preview: {item.Content.Substring(0, Math.Min(200, item.Content.Length))}...");
}

curl -X POST https://api.routin.ai/v1/web/fetch \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "urls": ["https://example.com/article"]
  }'

Batch Page Fetch

Fetch multiple pages in a single request.

import requests

url = "https://api.routin.ai/v1/web/fetch"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Batch fetch multiple URLs
urls = [
    "https://example.com/article1",
    "https://example.com/article2",
    "https://example.com/article3"
]

data = {"urls": urls}
response = requests.post(url, headers=headers, json=data)
result = response.json()

print(f"Successfully fetched {len(result['items'])} pages:")
for i, item in enumerate(result["items"], 1):
    print(f"\n{i}. {item['title']}")
    print(f"   URL: {item['url']}")
    print(f"   Content: {len(item['content'])} characters")

const urls = [
  'https://example.com/article1',
  'https://example.com/article2',
  'https://example.com/article3',
];

const result = await fetchWebPages(urls);

console.log(`Successfully fetched ${result.items.length} pages:`);
result.items.forEach((item, index) => {
  console.log(`\n${index + 1}. ${item.title}`);
  console.log(`   URL: ${item.url}`);
  console.log(`   Content: ${item.content.length} characters`);
});

var urls = new[]
{
    "https://example.com/article1",
    "https://example.com/article2",
    "https://example.com/article3"
};

var result = await client.Fetch(urls);

Console.WriteLine($"Successfully fetched {result.Items.Count} pages:");
for (int i = 0; i < result.Items.Count; i++)
{
    var item = result.Items[i];
    Console.WriteLine($"\n{i + 1}. {item.Title}");
    Console.WriteLine($"   URL: {item.Url}");
    Console.WriteLine($"   Content: {item.Content.Length} characters");
}

Use Cases

1. AI Agent Web Search

Provide AI agents with real-time web search capabilities to retrieve the latest information.

def ai_agent_search(user_query: str):
    """AI Agent using web search to enhance responses"""
    # 1. Execute web search
    search_result = web_search(user_query, max_results=5)

    # 2. Pass search results as context to LLM
    context = "\n\n".join([
        f"Source: {item['title']}\n{item['content']}"
        for item in search_result['items']
    ])

    # 3. Generate response using Chat Completions API
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that answers questions based on provided web search results."},
            {"role": "user", "content": f"Question: {user_query}\n\nSearch Results:\n{context}"}
        ]
    )

    return response.choices[0].message.content

2. RAG Knowledge Base Construction

Build a RAG knowledge base by fetching web content.

def build_knowledge_base(urls: list[str]):
    """Build vector knowledge base from web pages"""
    # 1. Fetch web content
    fetch_result = fetch_web_pages(urls)

    # 2. Chunk the text
    chunks = []
    for item in fetch_result['items']:
        text_chunks = split_text(item['content'], chunk_size=500)
        for chunk in text_chunks:
            chunks.append({
                'text': chunk,
                'source': item['url'],
                'title': item['title']
            })

    # 3. Generate embeddings
    texts = [chunk['text'] for chunk in chunks]
    embeddings = client.embeddings.create(
        model="text-embedding-3-small",
        input=texts
    )

    # 4. Store in vector database
    for chunk, embedding in zip(chunks, embeddings.data):
        vector_db.insert(
            vector=embedding.embedding,
            metadata=chunk
        )

3. Content Aggregation and Monitoring

Monitor the latest content for specific keywords.

def monitor_keywords(keywords: list[str]):
    """Monitor latest content for keywords"""
    from datetime import datetime, timedelta

    results = {}
    for keyword in keywords:
        # Search for content from the last 24 hours
        search_result = web_search(
            query=keyword,
            max_results=10,
            start_date=(datetime.now() - timedelta(days=1)).isoformat(),
            end_date=datetime.now().isoformat()
        )
        results[keyword] = search_result['items']

    return results

4. Competitor Analysis

Batch fetch competitor website content for analysis.

def analyze_competitors(competitor_urls: list[str]):
    """Analyze competitor website content"""
    # Batch fetch competitor pages
    fetch_result = fetch_web_pages(competitor_urls)

    # Use LLM to analyze content
    analysis = []
    for item in fetch_result['items']:
        response = openai.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": f"Analyze the following competitor page's product features, pricing strategy, and target users:\n\n{item['content'][:2000]}"
            }]
        )
        analysis.append({
            'url': item['url'],
            'title': item['title'],
            'analysis': response.choices[0].message.content
        })

    return analysis

Error Handling

Make sure to add comprehensive error handling logic in production environments.

import requests
from requests.exceptions import RequestException, Timeout

def safe_web_search(query: str, max_retries: int = 3):
    """Web search with error handling"""
    url = "https://api.routin.ai/v1/web/search"
    headers = {
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
    }

    for attempt in range(max_retries):
        try:
            response = requests.post(
                url,
                headers=headers,
                json={"query": query, "maxResults": 5},
                timeout=30
            )

            # Check response status code
            if response.status_code == 401:
                raise Exception("Invalid or unauthorized API Key")
            elif response.status_code == 429:
                raise Exception("Rate limit exceeded, please retry later")
            elif response.status_code >= 500:
                raise Exception("Server error, please retry later")

            response.raise_for_status()
            return response.json()

        except Timeout:
            print(f"Request timeout, retrying {attempt + 1}/{max_retries}")
            if attempt == max_retries - 1:
                raise Exception("Request timeout, please check network connection")

        except RequestException as e:
            print(f"Request failed: {e}")
            if attempt == max_retries - 1:
                raise

    return None

async function safeWebSearch(
  query: string,
  maxRetries: number = 3
): Promise<SearchResponse | null> {
  const url = 'https://api.routin.ai/v1/web/search';

  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      const response = await fetch(url, {
        method: 'POST',
        headers: {
          'Authorization': 'Bearer YOUR_API_KEY',
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ query, maxResults: 5 }),
      });

      if (response.status === 401) {
        throw new Error('Invalid or unauthorized API Key');
      } else if (response.status === 429) {
        throw new Error('Rate limit exceeded, please retry later');
      } else if (response.status >= 500) {
        console.log(`Server error, retrying ${attempt + 1}/${maxRetries}`);
        if (attempt === maxRetries - 1) {
          throw new Error('Server error, please retry later');
        }
        continue;
      }

      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`);
      }

      return await response.json();

    } catch (error) {
      console.error(`Request failed (attempt ${attempt + 1}/${maxRetries}):`, error);
      if (attempt === maxRetries - 1) {
        throw error;
      }
    }
  }

  return null;
}

public async Task<WebSearchResponse?> SafeWebSearch(
    string query,
    int maxRetries = 3)
{
    for (int attempt = 0; attempt < maxRetries; attempt++)
    {
        try
        {
            var request = new WebSearchRequest
            {
                Query = query,
                MaxResults = 5
            };

            var json = JsonSerializer.Serialize(request);
            var content = new StringContent(json, Encoding.UTF8, "application/json");

            var response = await _httpClient.PostAsync(
                $"{BaseUrl}/web/search",
                content
            );

            if (response.StatusCode == System.Net.HttpStatusCode.Unauthorized)
            {
                throw new Exception("Invalid or unauthorized API Key");
            }
            else if (response.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
            {
                throw new Exception("Rate limit exceeded, please retry later");
            }
            else if ((int)response.StatusCode >= 500)
            {
                Console.WriteLine($"Server error, retrying {attempt + 1}/{maxRetries}");
                if (attempt == maxRetries - 1)
                {
                    throw new Exception("Server error, please retry later");
                }
                continue;
            }

            response.EnsureSuccessStatusCode();

            var responseJson = await response.Content.ReadAsStringAsync();
            return JsonSerializer.Deserialize<WebSearchResponse>(responseJson);
        }
        catch (HttpRequestException ex)
        {
            Console.WriteLine($"Request failed (attempt {attempt + 1}/{maxRetries}): {ex.Message}");
            if (attempt == maxRetries - 1)
            {
                throw;
            }
        }
        catch (TaskCanceledException ex)
        {
            Console.WriteLine($"Request timeout (attempt {attempt + 1}/{maxRetries})");
            if (attempt == maxRetries - 1)
            {
                throw new Exception("Request timeout, please check network connection", ex);
            }
        }
    }

    return null;
}

Common Error Codes

Error Code	Description	Solution
401	Invalid or missing API Key	Check if Authorization header is correctly set
400	Invalid request parameters	Check if query or urls parameters are correct
429	Rate limit exceeded	Reduce request frequency or upgrade quota
500	Internal server error	Retry later or contact support
503	Service temporarily unavailable	Retry later

Best Practices

Set reasonable result counts: Configure maxResults based on actual needs to avoid wasting resources
Use time range filtering: For time-sensitive content, use startDate and endDate parameters
Optimize batch fetching: When batch fetching multiple URLs, recommend no more than 10 URLs per request
Cache content: For unchanging web content, consider caching fetch results
Error retry: Implement exponential backoff retry mechanism to handle temporary errors
Validate results: Check the returned score field to filter low-relevance results
Monitor usage: Regularly check dashboard statistics to optimize API usage
Respect robots.txt: When fetching pages, respect the target website's crawler protocol

Quota and Billing

The Web Research API is billed per request. Each search or fetch request consumes 0.001 quota, with actual costs based on the configured multiplier.

Billing Rules:

Web Search: Per search request (regardless of result count)
Page Fetching: Per fetch request (billed per request, not per URL count)

Optimization Recommendations:

Use batch fetching wisely, processing multiple URLs in a single request
Cache search and fetch results to avoid duplicate requests
Use maxResults to control return count

More Resources

Chat Completions API - Chat interface for generating responses with search results
Embeddings API - Text embeddings for building knowledge bases
Images API - Image generation interface
Model List - View all supported models

Web Research API

On this page