Piwiks own robots.txt is empty, but as the reporter from ticket #431 mentioned a global disallowed is the desired behaviour in most cases.
Starting with the file integrity check in Piwik 0.5.5 you get a warning when you have changed the robots.txt what I guess several people will have done.
To prevent this, robots.txt should not be checked.
Vote to simply remove robots.txt from the distro because:
Part 2:
We initially put the robots.txt because otherwise, when Piwik is installed in a subdomain, it shows up as 404 in the error logs, which is a nuisance.
Maybe we can
which would give us the best of both worlds?
I'll modify the installer to simply create an empty robots.txt in the piwik root if it doesn't already exist, rather than creating exceptions in the updater and integrity check.
I think this will be an issue as it would require write permissions on the root folder, which are currently not required..
On review, why not re-open #431 and implement the original recommendation?
(In [1957]) fixes #1215 - protect locally modified 'robots.txt'; skip file integrity check and don't overwrite when updating