recaptime.dev
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
Disallow webcrawlers to index Diffusion commits
Summary:
Phorge already sets `Disallow: /diffusion/` and `Disallow: /source/`.
Thus consequently also disallow accessing specific commits via `/r*`.
See https://secure.phabricator.com/T4610 for previous discussions.
Closes T15670
Test Plan:
Go to `/robots.txt` in the web browser.
Cross fingers that more webcrawlers abide by RFC 9309.
Reviewers: O1 Blessed Committers, speck
Reviewed By: O1 Blessed Committers, speck
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15670
Differential Revision: https://we.phorge.it/D25474
+2
+2
src/applications/system/controller/robots/PhabricatorRobotsPlatformController.php
+2
src/applications/system/controller/robots/PhabricatorRobotsPlatformController.php
···
18
18
$out[] = 'User-Agent: *';
19
19
$out[] = 'Disallow: /diffusion/';
20
20
$out[] = 'Disallow: /source/';
21
+
// See T15670. Also prevent directly accessing commits in Diffusion.
22
+
$out[] = 'Disallow: /r*';
21
23
22
24
// See T15662. Prevent indexing line anchor links in Pastes. Per RFC 9309
23
25
// section 2.2.3, percentage-encode "$" to avoid interpretation as end of