Segments are sometimes urldecoded and sometimes not
Segments expressions seem to be sometimes URL decoded and sometimes not which results in code like this: https://github.com/piwik/piwik/blob/2.15.0/core/Segment.php#L107-L109 even though there are methods like this to get a segment string: https://github.com/piwik/piwik/blob/2.15.0/core/API/Request.php#L483 if we have such methods (I think there are other ones too), they should always return the segment string in the same version. This looks like a bug in the code

Core uses code from plugins etc
We have many places in core where it accesses API.getSegmentsMetadata from API plugin. Core should however not know anything about the plugins. We should have a class like Piwik\Plugin\Segments that holds a list of all Piwik\Segment\Segment classes defined by plugins and operate on this one. API.getSegmentsMetadata should work with Piwik\Plugin\Segments as well and use it to format the metadata. It should not be formatted to an array in Piwik\Segment\Segment (currently Piwik\Plugin\Segment).

Deprecate 'API.getSegmentDimensionMetadata' event

It should be possible to define segments only via Dimension classes and maybe via Segment classes in a Segments directory or so

Ideally we would as well rename and refactor the underlying classes in core. Eg we have Piwik\Segment but it's confusing what this method does as we also have Piwik\Plugin\Segment. Also Piwik\Segment seems to parse expressions but also handle SQL code etc and seems to just do too much.

This is something we should tackle for Piwik 4.0 at the latest