Just curious.
It’s listed as MojeekBot
here: MojeekBot
Yes it’s MojeekBot
So if I set the User-agent
to Mojeek
, this wouldn’t block MojeekBot?
Correct, as on the link you pointed to: MojeekBot
MojeekBot obeys the Robot Exclusion Standard. MojeekBot will obey the first record with a User-Agent containing “MojeekBot”. If there is no such record it will obey the first entry with a User-Agent of “*”.