After upgrading the widget AJAX request started returning error 500. The error was:
Unable to write in the cache directory (/var/www/piwik/tmp/templates_c/e9
and it was solved with:
rm -rf /var/www/piwik/tmp/templates_c
Please consider doing it automatically on upgrades to save on troubleshooting time.
I don't think it was necessary to delete it. Most likely there was a permission problem aka wrong user:group.
There wasn't, I checked and just in case did chmod -R 755 and chown -R. Didn't help.
Looks like the cache directory couldn't be cleared or written to which pretty much has to be some kind of permission problem. Clearing cache directories is done during the update.