CareerVertexBot
Our crawler identifies itself as CareerVertexBot/2.0 (+https://careervertex.uk/bot). It reads public job listing pages from a short list of specialist recruitment agencies, so that senior contractors can find roles that fit them and apply to the agency directly.
- It obeys robots.txt. To keep it out, add
User-agent: CareerVertexBotandDisallow: /. - It makes at most one request every 2 seconds to a site, and reads each jobs page about once a day.
- It reads public pages only: no logins, no forms, and never LinkedIn.
- Listings are never republished. Every match links back to your page, and applications come to you from the candidate's own email.
To be added to or removed from the watchlist, email [email protected].