About the robot

It is called RIBRbot. It reads the public files websites publish about themselves.

How to spot it

Mozilla/5.0 (compatible; RIBRbot/1.0; +https://runitbyrobot.com/bot/; support@runitbyrobot.com)

How it behaves

One request per second per site, never more. When a site gives a file an ETag, the robot sends it back on the next visit, so an unchanged file is not sent again. It obeys robots.txt rules for RIBRbot. It stops reading at the file's size limit: 1 MB for most files, and up to 4 MB for a few large public datasets. It gives up on a request after 10 seconds.

How to stop it

Block RIBRbot in your robots.txt. Or remove your site: that also covers DNS records, which robots.txt does not.

Who runs it

Tiny Blue Labs. Write to support@runitbyrobot.com. A person reads it.