In case you are referring to the file compression: This is not something Matomo does, but your web server when delivering static files. So you need to set your web server up so that it gzip-compresses (and optionally brotli-compresses if you like) the matomo.js, which reduces its size by quite a large fraction.

I'm mostly referring to the javascript optimization, if it could be minified some more maybe? Google only says "reduce the amount of unused javascript code". The report doesn't mention the gzip compression with regards to matomo.js

This is hard to avoid as Matomo can't know which features you don't want to use and splitting up matomo.js is not trivial (#19144).
And if file size is the highest priority and you don't use any non-default Matomo features or cookies, then one can try recreating the core of matomo.js in less than 1KB like I tried out in https://github.com/Findus23/MatomoLiteTracker

Thank you for suggesting MatomoLiteTracker, it's interesting.

I personally find that 35KiB+ is nothing to worry about but I use Matomo in basically any website and that file always appears in the list of problems with PageSpeed Insights so I thought I would open an issue to just let you know.

It's definitely not going to be a life-changing improvement to any ranking, but still noteworthy for some websites