Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File Integrity Issues after Upgrade #16938

Closed
eric215 opened this issue Dec 11, 2020 · 13 comments
Closed

File Integrity Issues after Upgrade #16938

eric215 opened this issue Dec 11, 2020 · 13 comments
Labels
Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Milestone

Comments

@eric215
Copy link

eric215 commented Dec 11, 2020

After each of the last few upgrades I receive various File Integrity Errors. After the latest upgrade to 5.0.4 I received the messages below.

I DO delete these files and directories after upgrading, but various files/directories appear on the next upgrade.

Either these are added by Matomo and should be marked as "OK" or they should NOT be added during the upgrade process

`Directories were found in your Matomo, but we didn't expect them.
--> Please delete these directories to prevent errors. <--

Directory to delete: libs/bower_components

To delete all these directories at once, you can run this command:
rm -Rf /home1/behuntne/public_html/pi1226/libs/bower_components

Files were found in your Matomo, but we didn't expect them.
--> Please delete these files to prevent errors. <--

File to delete: misc/others/ExamplePiwikTracker.php
`

@sgiehl
Copy link
Member

sgiehl commented Dec 11, 2020

@eric215 With every release files might be removed as they are not used any longer. Normally the update process should remove the files automatically. In some cases that might not work correctly for various reasons (like permissions). In that case those files will be reported in the integrity report. It is safe to simply remove those files.

@tsteur
Copy link
Member

tsteur commented Dec 13, 2020

@eric215 Matomo should for example not create libs/bower_components during a Matomo 4 release. Have you maybe removed that directly before, then updated and it was showing the same directory to delete again?

@eric215
Copy link
Author

eric215 commented Dec 18, 2020

Sorry for the delay in responding, personal issues kept me offline.

Yes, I am confident that I deleted libs/bower_components in the past, and I do recall when I deleted it this time it had a timestamp of "today". I'm sorry I didn't think at the time to take a screen shot

@tsteur
Copy link
Member

tsteur commented Dec 21, 2020

Thanks @eric215 We're planning to do another release. Be great if you could check if it then complaints about the same files again and let us know if it does.

@tsteur tsteur added Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users. Can't reproduce (yet) For issues that are reported by several people, but can't be reproduced reliably and need more data labels Dec 21, 2020
@eric215
Copy link
Author

eric215 commented Dec 22, 2020

Upgrading from 4.0.5 to 4.1 I did NOT have an issue with any of the above-mentioned files hooray!

I did have a file-integrity failure on vendor/bin/lessc I can confirm that vendor/bin did not exist before the upgrade. I can create a separate issue if you'd prefer.

@tsteur tsteur added Regression Indicates a feature used to work in a certain way but it no longer does even though it should. and removed Can't reproduce (yet) For issues that are reported by several people, but can't be reproduced reliably and need more data Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users. labels Dec 22, 2020
@tsteur tsteur added this to the 4.2.0 milestone Dec 22, 2020
@tsteur
Copy link
Member

tsteur commented Dec 22, 2020

Thanks @eric215 I think this might be actually a new "regression" as we had just changed a library there and probably something needs adjusting. Thanks for this & happy holidays

@dev-101
Copy link

dev-101 commented Dec 29, 2020

I have noticed this issue, too, it is present for at least a year, possibly longer. Several different systems (vps, cloud), permissions and ownerships are in proper order, not a single other script that runs on different server setups complains about this, only Piwik/Matomo has a problem automatically removing obsolete files. I thought it was on purpose, to avoid problems when downgrading or when upgrade fails to complete. Think about this, if rights/owns were not in order, it would also fail in executing upgrades and other things (e.g. installing plugins). So, the issue, unless intentional feature, is with something else.

@tsteur
Copy link
Member

tsteur commented Dec 29, 2020

It's not intentional but we also haven't been able to reproduce any issue here yet except for when new files were added or removed and they weren't excluded yet for example.

@sgiehl
Copy link
Member

sgiehl commented Jan 7, 2021

Can we close this issue or is here anything left we should do?

@dev-101
Copy link

dev-101 commented Jan 7, 2021

Yes, whenever possible, the best way to "solve" the issue is to close it. 🤣

We aren't using some exotic systems that can be hard to reproduce, cloud linux and self-managed vps/dedi are ubiquotus these days, so that is not an excuse. On the other hand, matomo couldn't work properly if file and user systems are faulty, because caching would fail and many other things along the way, e.g. update itself.

@dev-101
Copy link

dev-101 commented Jan 7, 2021

Something just occurred to me, is it possible that shell_exec being disabled could be the cause of it? If yes, then you should add a fallback function to remove files.

@tsteur
Copy link
Member

tsteur commented Jan 10, 2021

@dev-101 shell_exec shouldn't be used during the update AFAIK.

@tsteur tsteur added Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users. and removed Regression Indicates a feature used to work in a certain way but it no longer does even though it should. labels Jan 10, 2021
@tsteur tsteur added Regression Indicates a feature used to work in a certain way but it no longer does even though it should. and removed Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users. labels Jan 10, 2021
@tsteur
Copy link
Member

tsteur commented Jan 10, 2021

@sgiehl @dev-101 I'll close the issue for now as the original file integrity issue should be fixed. We could always keep track of people experiencing file update issues in a separate issue (unless there is already one). Be great to know when someone experiences issues that files weren't correctly updated or deleted what operating system they use, how often this happens (eg on every update or only randomly etc)

@tsteur tsteur closed this as completed Jan 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Development

No branches or pull requests

4 participants