This is what I get when I try to upgrade from 3.6.1 to 3.9.1:

File integrity check failed and reported some errors. You should fix this issue and then refresh this page until it shows no error.
Directories were found in your Matomo but we didn't expect them.
--> Please delete these directories to prevent errors. <--

Directory to delete: vendor/mnapoli
Directory to delete: vendor/myclabs
Directory to delete: vendor/tecnick.com
Directory to delete: vendor/tedivm

Files were found in your Matomo but we didn't expect them.
--> Please delete these files to prevent errors. <--

File to delete: PULL_REQUEST_TEMPLATE
File to delete: config/environment/cli.php
File to delete: config/global.ini.auto-backup-before-update.php
File to delete: misc/How to install Piwik.html
File to delete: misc/internal-docs/content-tracking.md
File to delete: misc/others/api_internal_call.php
File to delete: misc/others/cli-script-bootstrap.php
File to delete: misc/others/iframeWidget.htm
File to delete: misc/others/phpstorm-codestyles/Piwik_codestyle.xml
File to delete: misc/others/phpstorm-codestyles/README.md
File to delete: misc/phpstorm-codestyles/Piwik_codestyle.xml
File to delete: misc/phpstorm-codestyles/README.md
File to delete: misc/proxy-hide-piwik-url/piwik.php
File to delete: libs/PiwikTracker/.git
File to delete: libs/bower_components/jquery-ui/ui/.jshintrc
File to delete: vendor/monolog/monolog/.php_cs
File to delete: vendor/pear/archive_tar/.travis.sh
File to delete: vendor/piwik/device-detector/.php_cs
File to delete: vendor/twig/twig/.editorconfig

This is quite a long list including a .git directory so I'm not sure I can actually safely delete them.