Integration · Crawl

No API? Your sitemap is enough.

AACSearch reads your sitemap or start pages, fetches each page safely, strips the markup and indexes the text. For documentation, it splits pages by headings so a result links to the exact section.

Two ways to crawl

Sitemap crawl

Every URL in sitemap.xml becomes a document: title, text and URL, refreshed on a schedule.

Documentation crawl

DocSearch-style: start URLs, allowed domains and CSS selectors for heading levels and text, so each section is its own result with an anchor.

Safe by design

Public addresses only

The crawler refuses private and loopback addresses and re-checks every redirect, so it can't be turned against an internal network.

Bounded runs

Page and depth limits keep a crawl from running away on a site with infinite calendars or filters.

What it enables

Cited AI answers

The assistant answers from crawled pages and links to the section it used.

Meaning-based search

Hybrid search finds the right article even when the visitor's words differ from the title.

Specifications

Sources
sitemap.xml, single URLs, documentation start URLs
Pages per docs crawl
50 by default, up to 1,000
Link depth
10 by default
Documents per sitemap run
500 by default, up to 2,000
Redirects followed
up to 5, each re-checked
Schedule
manual, hourly, daily or weekly

Before you start

  • A public sitemap or a list of start URLs
  • For documentation: CSS selectors of your page title, headings and body
  • Pages that render their text in HTML (client-only apps may need the documents API instead)

Questions