How the robot works
It reads public files on a schedule.
One request per second per site, never more. Each file is kept with its date and a fingerprint of exactly what came back.
It shows a change only after seeing it twice.
A second check, at least an hour later, must agree. A reordered file is not a change.
It never calls a failure an absence.
If a site times out or turns the robot away, the report says so. A missing answer is never a missing file.
What each status means
SUCCESS
Read.
ADDED
A line appeared.
REVALIDATED
Read again. No change.
REMOVED
A line is gone.
NOT_FOUND
The site says the file does not exist. From a public-records source, it is a failed check, never an absence.
MODIFIED
A line changed. Both versions are shown.
UNAVAILABLE
The check failed. Not an absence.
MATCHED
The ad system lists this seller.
RESTRICTED
The site turned the robot away.
UNMATCHED
The ad system's sellers.json has no matching seller.
INCOMPLETE
Only partly read. No changes come from it.
CONFIDENTIAL
The ad system lists this seller as confidential. It stays hidden.
EMPTY
Read. Nothing inside. From a public-records source, after a read with records, it is a failed check, never an absence.
A change word shows on a report for 7 days.
What each file is
ads.txt
A site's list of the companies allowed to sell its ad space. Standard: IAB Tech Lab ads.txt.
app-ads.txt
The same list for apps, published on the app developer's website. Standard: IAB Tech Lab app-ads.txt.
sellers.json
An ad system's list of the sellers it pays. The robot checks each ads.txt line against it. Standard: IAB Tech Lab sellers.json.
robots.txt
The rules a site sets for crawlers, including AI crawlers. Standard: RFC 9309.
llms.txt
A short guide a site writes for AI tools. A proposed format, published at llmstxt.org.
DMARC
A DNS record telling mail servers what to do with mail that fails the domain's checks. Standard: RFC 9989.
MTA-STS
A policy that says mail to the domain must travel encrypted. Standard: RFC 8461.
BIMI
A DNS record pointing to the logo inboxes may show beside the domain's mail. Standard: BIMI Group.
security.txt
Where to report a security flaw in the site. Standard: RFC 9116.
What the robot does not check
Anything behind a login, account settings, or private feeds. Whether crawlers obey robots.txt. Business deals: a file line is not a contract.
When a domain changes hands
If a domain changes owner or goes dark, the report shows what it publishes now, with the date.
Where the site lists come from
Site lists come from Tranco, a research ranking of popular sites. The robot uses it only to choose which sites to check. It never republishes the ranks.
List in use: Tranco list K9YYW, downloaded Nov 6, 2026.
How subscribers get the data
- Subscribers download the data behind their feed as files, from their feed page.
- The files are rebuilt each week from confirmed checks only. Latest holds the current state of everything the Watch reads; changes holds every confirmed change in the history the plan covers.
- Each download link is signed for one subscription and works for 24 hours.
- Days the robot did not finish its checks are listed in each file.
For developers
- Every report has a free JSON twin: /wp-json/ribr/v1/record/{domain}.
- Every free Watch has one too: /wp-json/ribr/v1/watch/{path}, with its latest 100 changes.
- Subscribers whose plan includes the API get more read-only endpoints, with a key from their feed page: /wp-json/ribr/v1/changes and, where the plan includes them, /wp-json/ribr/v1/watchlist/changes and /wp-json/ribr/v1/history.
- The JSON carries the same facts as the pages, never more.