Here is the offending plugin, by the way: https://github.com/Joey3000/piwik-LoginEncrypted

Which did re-generate /tmp/assets/asset_manager_non_core_js.js, but it still didn't get served

Before looking any further into it, have you checked the "Network" tab in your browser re caching headers / etag? Maybe it was still cached in your browser.

Thanks for looking into this. I've played around a bit more with this, and here are the results.

You are right, if one just navigates to the login page (e.g. signing out of Piwik after generation of new keys) the "Transferred" column in the Network tab says "cached" for the CSS, core and non-core asset files, meaning, I guess that they don't actually get requested.

One can, actually, get stuck using the cached files if one then (after an unsuccessful login) gets presented an exception page due to failed decryption, and then navigates to the login page again using the "Go to Piwik" link on the exception page.

But: A simple login page reload by the user (i.e. just a click on the reload arrow; not a forced reload) is enough to make the browser request and download the updated assets.

Calling

AssetManager::getInstance()->getMergedNonCoreJavaScript();

to re-generate the assets right after having removed them by

AssetManager::getInstance()->removeMergedAssets([my_plugin_name]);

makes no difference - if missing, the assets will be re-generated when the browser requests them anyway.

I guess that the reason for the browser using the local cache is that the cache buster value on the HTML doesn't change, not even when one calls the above re-generation function. It looks like an MD5 hash and is equal for the CSS, the core and non-core JS minified assets, as well as other files. (A hash of all together at some point in the past?)

It's always, e.g. [removed]/index.php?module=Proxy&action=getNonCoreJs&cb=d4b95f8d81cc43b9b783a06d1997c8c2

Is there a way to trigger re-generation of the cache buster value?

I can image the plugin doing, e.g.:

1. Remove non-core JS asset: AssetManager::getInstance()->removeMergedAssets([my_plugin_name]);
2. Re-generate the asset: AssetManager::getInstance()->getMergedNonCoreJavaScript();
3. Re-generate the cache buster value (or: this could be done automatically if asset is detected missing by the above step and therefore re-generated [see note below])

So that an up-to-date cache buster value exists whenever the login HTML page gets navigated to without a page reload request by the user.

[Note] I don't know though, if a possible value change in between page resource requests would have a impact in general Piwik usage, not related to this plugin - e.g., due to asset absence detection after (de-)activation of a plugin. For example:

• The HTML page gets requested, containing an old cache buster value (it's there plenty of times, not just in the minified assets requests)
• The CSS asset gets requested using the old value -> OK
• The non-core JS asset gets requested using the old value, is absent and a new cache buster value gets generated -> OK
• The core JS asset and other page resources get requested (still) using the old value -> would this be OK?
• Any race conditions with resource requests around the time of the new value generation?

==> In case issues may occur on automatic cache buster value generation on asset absence detection, it could be done by the plugin separately on the step 3 above.

Oops, I was wrong: the CSS asset uses a different cache buster value. But all JS page resources use a common one. And it seems to differ in some places in development mode (probably for instant change take-over, without caching):

• In /core/View.php::render():

    if (Development::isEnabled()) {
        $cacheBuster = rand(0, 10000);
    } else {
        $cacheBuster = UIAssetCacheBuster::getInstance()->piwikVersionBasedCacheBuster();
    }

• In /core/View.php::applyFilter_cacheBuster():

    $cacheBuster = UIAssetCacheBuster::getInstance();
    $tagJs       = 'cb=' . $cacheBuster->piwikVersionBasedCacheBuster();
    $tagCss      = 'cb=' . $cacheBuster->md5BasedCacheBuster($content);

• Whereby /core/AssetManager/UIAssetCacheBuster.php::piwikVersionBasedCacheBuster():

    $pluginsInfo = '';
    foreach ($pluginNames as $pluginName) {
        $plugin       = Manager::getInstance()->getLoadedPlugin($pluginName);
        $pluginsInfo .= $plugin->getPluginName() . $plugin->getVersion() . ',';
    }
    $cacheBuster = md5($pluginsInfo . PHP_VERSION . Version::VERSION . trim($currentGitHash));

• And /core/AssetManager/UIAssetCacheBuster.php::md5BasedCacheBuster():

    return md5($content);

So, the CSS minified asset cache buster value seems to be an MD5 checksum of the file content (see /core/AssetManager/UIAsset/OnDiskUIAsset.php::getContent()). While the JS one seems to be an MD5 checksum of a constant string containing names and versions of plugins, PHP and Piwik versions and a "current Git Hash" - so it's tied to the current environment versioning. No wonder it doesn't change. :)

Is it possible to change the core and non-core minified JS asset files to content-based MD5 checksums, like with the CSS one? (Or is there a reason for the constant value?)

I do not really remember the background of this but I think it was for performance. So far the JS files were not supposed to be changed unless a plugin was eg activated / deactivated etc and it was good enough.

I wonder if a workaround for now could be to manually load this JavaScript file from JS? Similar to RequireJS does etc. It would not be ideal for sure especially since the file would be requested and loaded only after all other JS is loaded.

What we could do, is to use the same cache buster for core JS file as we do currently, and for non-core JS files use the same as CSS. This should be still kinda fast as users usually do not use many custom plugins and not many have actually JS files defined. We'd have to test re performance though and I think it currently does not really have a priority unless it's super easy to implement. A PR is always welcome :)

Do you think you could maybe fix it with the workaround for now?

Nice plugin BTW 👍

What we could do, is to use the same cache buster for core JS file as we do currently, and for non-core JS files use the same as CSS. This should be still kinda fast as users usually do not use many custom plugins and not many have actually JS files defined. We'd have to test re performance though and I think it currently does not really have a priority unless it's super easy to implement.

That sounds good to me as a long-term solution. No worries, it's not a "show stopper", because the issue doesn't result a silent fail, but I output clear instructions to the user as to what to do (clear browser cache, reload the page): https://github.com/Joey3000/piwik-LoginEncrypted/blob/1.0.1/lang/en.json#L3. And it doesn't happen often - only after new encryption keys generation (on plugin installation or when a super user manually re-generates them, which is not needed often and is actually not needed ever at all - I'm going to add this point to the FAQ). Although then every user will see the issue, no just the one who triggered the key re-generation.

Do you think you could maybe fix it with the workaround for now?

I'll have a look into it.

Nice plugin BTW

Thanks! I have to return the compliment. Piwik has been very well prepared for plugins - I see that on every turn (architecture, "how-to"s, examples in the code, etc.) - you guys are doing a great job with that. I myself am a noob with web development, although I have some background in C from a while ago, which helps.

Thanks for the hint with loading the key JavaScript file from JS! It works flawlessly. (I used the following: https://api.jquery.com/jQuery.getScript/). The file is fetched using an AJAX GET request and jQuery even appends an always changing cache busting parameter by default.

So, with that setup, Piwik does the minified JS cache buster update on plugin activation/deactivation (when the static plugin code gets added/removed to/from minified assets), which works (because the currently active plugins, with their names and versions, influence the cache buster value).
While the public key file (which changes on key change by a super user) is fetched by the static code with its own always different cache buster value and is therefore always up-to-date.

P.S.: I took the liberty of modifying the issue title, so that it better reflects what is currently known.
P.P.S: That the key file is loaded after everything else doesn't seem to matter much in my case, because the key in it is used for the first time only once the user clicks the "login" button (or the "submit" button on a password change request). So that there should be enough time for the file to be loaded.

Nice to hear 👍 and thx for changing the title 👍

As far as I understand, this shouldn't occur in a standard flow when a final user installs the plugin. When developing a plugin one should use the development mode (./console development:enable where Piwik makes sure to load the original source javascript files (not the merged one). So closing as wontfix. If i'm wrong feel free to comment/re-open