Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split piwik.js into multiple files and provide a merged one #6106

Closed
tsteur opened this issue Sep 2, 2014 · 26 comments
Closed

Split piwik.js into multiple files and provide a merged one #6106

tsteur opened this issue Sep 2, 2014 · 26 comments
Labels
c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@tsteur
Copy link
Member

tsteur commented Sep 2, 2014

Now that js/piwik.js is getting larger and larger - for instance content tracking added already more than 1000 new lines of code - it would be really nice to split js/piwik.js into multiple files or modules. Would probably also make it easier to write tests for it. I'd have to write and create many tests that rely on DOM but are not related to tracking. Would also make it maybe possible to release some components separately, at least it would give us the possibility.

We could use grunt or something else to merge those files together and to minify/shrink it for a single file piwik.js.

Alternatively, we could also use something based on PHP instead of grunt if we then offer third party plugins to extend the tracker as well.

@tsteur tsteur added this to the Mid term milestone Sep 2, 2014
@mattab
Copy link
Member

mattab commented Sep 2, 2014

Alternatively, we could also use something based on PHP instead of grunt if we then offer third party plugins to extend the tracker as well.

+1 - Plugins extending piwik.js in a modular way makes a lot of sense, as users can enjoy features and benefits without having to change anything in the tagged code in their websites. It makes the Piwik platform much more powerful for web analytics!

@mattab mattab added the c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. label Sep 9, 2014
@wikiloops
Copy link

Happy to see someone adressed the issue, I have been thinking about the increasing size of the piwik.js myself.
Ideally, the provided script should either
a) be reduced to the needed components on updating (maybe a little like its done when customizing a jQuery-UI version, in case you are familiar with that),
or
b) the selection of script to serve might be generated on the fly by analyzing the enabled functions of PIWIK (I guess thats the PHP implementation you adressed).

I imagine the file to serve would be a lot smaller in my particular use case, since I neither use Ecommerce nor Content Tracking at this point...
Thanks guys, you are doing a great job, just felt like underlining this issues importance :)

@mattab
Copy link
Member

mattab commented Oct 28, 2014

Plugins extending piwik.js in a modular way makes a lot of sense, as users can enjoy features and benefits without having to change anything in the tagged code in their websites. It makes the Piwik platform much more powerful for web analytics!

Btw we released a new plugin for this awesome purpose: CustomTrackerJs

@wikiloops
Copy link

I noticed that, but that only allows to ADD more stuff as far as I have understood, what I'm looking for is a skinny piwik.js core file... or am I mistaken?

@mattab
Copy link
Member

mattab commented Oct 28, 2014

@wikiloops The main goal of this issue is not to make the piwik.js smaller but to make the code easier to maintain and understand by splitting it in multiple files. If you want a smaller piwik.js code, please create a new issue for this suggestion 👍

@wikiloops
Copy link

will do, thanks matt :)

@dhoko
Copy link

dhoko commented Aug 26, 2015

As suggested via #8642,you can split the source using browserify. You do not need to have gulp nor grunt, there is npm for that.

And if you don't have the time to do it, we (the community) can do it for you.

@tsteur
Copy link
Member Author

tsteur commented Aug 26, 2015

👍 if someone would work on it. It would be great to have as mentioned in the issue

@dhoko
Copy link

dhoko commented Aug 26, 2015

Building it with PHP... :finnadie:

I'm ok to split the code but with a front-end tooling.

PS: Using a modular code will be a great option to provide:

  • Default piwik for all
  • Piwik for mobile (without shitty polyfills for IE < 9) so a smaller lib
  • Custom build

@tsteur
Copy link
Member Author

tsteur commented Aug 27, 2015

👍 sounds great!

@dhoko
Copy link

dhoko commented Aug 28, 2015

You can follow the progress refacto browserify.

Do you have a page to test each functionalities provided by piwik ?

@tsteur
Copy link
Member Author

tsteur commented Aug 28, 2015

👍 We have a test suite for piwik.js under $PiwikUrl/tests/javascript

@dhoko
Copy link

dhoko commented Aug 31, 2015

I don't understand how to run them :/ I tried via ./console it doesn't work cf #8678 then I start a php server into tests/javascript/ it doesn't work too because of:

<script src="../../js/piwik.js?rand=2c1ba87dbd1d350d81ba2612f945b3fc" type="text/javascript"></script>

So I run a php server from root...

screenshot from 2015-08-31 16 30 53

Or:

$ ../../node_modules/.bin/phantomjs testrunner.js 
Failure opening http://localhost/tests/javascript/

😢

@mattab
Copy link
Member

mattab commented Sep 1, 2015

@dhoko
Copy link

dhoko commented Sep 1, 2015

Yup that's why i tried to run PhantomJS too ;)
Le 1 sept. 2015 4:52 AM, "Matthieu Aubry" notifications@github.com a
écrit :

Hi @dhoko https://github.com/dhoko did you see
https://github.com/piwik/piwik/tree/master/tests/javascript#javascript-tests
?


Reply to this email directly or view it on GitHub
#6106 (comment).

@tsteur
Copy link
Member Author

tsteur commented Sep 1, 2015

It looks like static files are not served. Is your web server not serving them maybe? What I find weird is that we usually do not have some of the requested files like jash/Jash.css or tests/piwiktest.js

@dhoko
Copy link

dhoko commented Sep 1, 2015

It was: $ php -S localhost:8000

@tsteur
Copy link
Member Author

tsteur commented Sep 1, 2015

I'm not familiar with the php builtin webserver but think that should work. Maybe someone else knows more about it? Maybe you're running this from within the $PiwikDIR/tests/javascript directory and not within $PiwikDIR

@dhoko
Copy link

dhoko commented Sep 1, 2015

  • I start a php server into tests/javascript/ it doesn't work too because of* ;)

@tsteur
Copy link
Member Author

tsteur commented Sep 1, 2015

not sure what you meant? :) The PHP webserver should be started into /path/to/piwik and then the URL should be /tests/javascript :)

@dhoko
Copy link

dhoko commented Sep 1, 2015

Fist I started the webserver from tests/javascript/ it didn't work, then from the root /path/to/piwik:

So I run a php server from root...

:/

From test/javascripts :

screenshot from 2015-09-01 11 53 35

From root:
screenshot from 2015-08-31 16 30 53

@tsteur
Copy link
Member Author

tsteur commented Sep 1, 2015

I just tried to reproduce via the command you mentioned (built in PHP webserver). Getting the same error when opening http://localhost:8000/tests/javascript but works when opening http://localhost:8000/tests/javascript/

@dhoko
Copy link

dhoko commented Sep 1, 2015

Mouhahaha same here, it's ok with tests/javascript/. 🎉

merci !

2015-09-01 12:32 GMT+02:00 Thomas Steur notifications@github.com:

I just tried to reproduce via the command you mentioned (built in PHP
webserver). Getting the same error when opening
http://localhost:8000/tests/javascript but works when opening
http://localhost:8000/tests/javascript/


Reply to this email directly or view it on GitHub
#6106 (comment).

@dhoko
Copy link

dhoko commented Sep 1, 2015

On master :)

screenshot from 2015-09-01 14 00 39

@tsteur
Copy link
Member Author

tsteur commented Sep 1, 2015

It's kinda possible that they fail randomly if you run them in a browser and eg scroll while tests are running etc :( They should always run via phantomjs

Yay for working tests :)

@mattab mattab modified the milestones: Mid term, Long term Aug 28, 2016
tsteur added a commit that referenced this issue Aug 30, 2016
* Add link to Product changelog in the Developer changelog (#10046)

Here is a recent comment from a community member ` another colleague found and pointed me to your changelog for 2.16.1, which was admittedly hard to find, ` which triggered me to add a clear link in our visible changelog root file.

* Update FrontController documentation (#10049)

* Update link to the overview

* Remove deprecated method from example

* Add TweakStyle browser icon

* Add possibility to execute plugin test suite

* 2.16.2-b1

* Fix pagination in datatables when displaying all entries (#10063)

* fix #4410 The string to escape is not a valid UTF-8 string (#10050)

*  	Trigger a Goal conversion for "Event matching goals" even when an event is tracked without a URL (#10018)

* use idGoal parameter in system tests

* Trigger a Goal conversion for "Event matching goals" even when an event is tracked without a URL

* Test files for green build

* Add new expected test file

* do not start a session during tracker mode or if session is not started yet (#10071)

Eg could start a session during tracking if a tracker plugin is not compatible with the current Piwik and it would try to trigger a notification because of this.

* Implements #9389 - UserId module, that adds a new Users report (#9883)

* Implement the UserId module, that adds a new Users report showing all unique user IDs and some aggregated data. Includes reindexing of raw visitors log into aggregated user ids table

* Update UserId module: add total_actions, total_events and total_searches columns; format dates

* Use an injected model instead of creating a new object

* Rework the UserId plugin to use core archiver instead of custom indexer

* Users report small changes:
- remove logger injection
- change archive record name
- add low population filter
- add all columns visualization
- add datatable_archiving_maximum_rows_userid_users configuration, default value is 50000

* Remove unused method parameter

* Users report: remove custom visualizations and add data-row-metadata attribute for every row

* minor UserId tweaks, enable row evolution for userId etc (#10072)

* addding userId plugin description and enable plugin during update (#10073)

* Fix processed metrics are not shown in visitor map when viewing cities

Eg nb_actions_per_visit
bounce_rate
avg time per visit

Bounce rate seems to always show 100% now but not sure why

* some more fixes

* fix tests after adding new userid feature

* Fix tests

* Ignore idvisitor in system tests as it always changes in each test
* Skip email tests in case intl extension is not installed
* Update ui tests

* update submodules

* Explain a user should call setSessionCookieTimeout when changing visit length

* value should be specified in seconds, not milliseconds

* fix example was wrong

* language update

* allow + in database names during setup (#10091)

* only fail when at least 5 domains extensions are failing, so it does not fail every time IANA adds a new domain extension...

* Remove travis `group: legacy` and use their latest infrastructure (#10094)

* Removed group:legacy from travis.yml template

* use latest php 5.3 instead of php 5.3.3

* Run MySQL off the RAM in our CI server to have fast builds

* Use travis files from latesst master branch commit

* updates travis.yml

* language update

* refs matomo-org/searchengine-and-social-list#3 improves detection for google searches without keyword

* refs matomo-org/searchengine-and-social-list#3 improves detection for yahoo searches without keyword

* Fix some typos in comments (found by codespell) (#10120)

Signed-off-by: Stefan Weil <sw@weilnetz.de>

* refs matomo-org/searchengine-and-social-list#3 improves detection for yahoo japan searches without keyword

* fixes #10130 7 days in Evolution over the period when Period is Day

* misc: Fix typos (found by codespell) (#10136)

Signed-off-by: Stefan Weil <sw@weilnetz.de>

* plugins: Fix typos (found by codespell)

Signed-off-by: Stefan Weil <sw@weilnetz.de>

* submodule update

* language update

* improve referrer type detection for direct entries

ignore fallback detection (same host for referrer and action) if ignore unknown urls is activated for current site

* adds some more test cases

* update translation command to use ProgressBar instead of deprecated ProgressHelper

* Manage Website: Links to website (#10087)

* Manage Website: Links to website

Add clickable urls for each website as requested in #10066.

* Remove extra = and whitespace

Updated style

* New tab, rel noreferrer, ng-href, commas

Changed the urls to open in new tabs, rel=noreferrer, ng-href, and add commas back into the list as requested

* Fix pasting error

Edit out extra </a> and {{ urls}}

* Change div to span

Change div to span to have url on the correct line

* When only a icon is rendered, the title is not enough to provide an accessibility label, so it an aria-label to the icon so screen reader element lists and such have something to display.

* Update urlnormalizer.js (#10171)

* Update piwik.js (#10170)

* JavaScript SDK: add support to send tracking requests to multiple Piwik servers and/or store in one or  multiple websites ID (#10085)

* Add support to configure multiple trackers easily

* added tests

* fix tests

* 2.16.2-b2

* Use Common::sendHeader for consistency

* Up to date minified piwik.js #10085

* Screenshot UI tests pixel alignment minor change

* Escape the value as suggested in the Pr comments.

* language update

* refs plugin-CustomDimensions#31 fix doc block for custom dimension was wrong

* add callback ability to trackPageView (#10172)

* Update piwik.js

* Update piwik.js

* submodule updates

* language update

* update languages

* rebuilt piwik.js

* split all tests (#10192)

* submodule update

* fix some tests

* device detector update

* Add mention to BrowserStack.com in our readme (#10205)

* Check if UserCountry plugin is activated before showing location data

* ensure to reset translated categories for each system test (#10210)

* link to useful CI build result dashboard and tweaks

* in Evolution over the period when Period is Day the limit should be actually 8 instead of 7

See #10130 (comment)

* Fix the flags of Switzerland and Nepal

Adding a transparent padding ensures the flags will not be stretched to
the 16x11 format of all the other flags.

* Update the flag of Tokelau

* Update the flag of Myanmar

* Update the flag of Lesotho

* Update the flag of Libya

* Update the flag of French Guiana

* fixed some typos [ci skip]

* fix report order of browser reports

* update tests with new order

* Update JQuery to the latest 1.12.4 version (#10260)

* Update JQuery to the latest 1.* version

* UI tests: Updated Jquery to latest version

* submodule update

* Update jquery to latest 1.*, new release checklist test

* language update

* Remove uneeded HTML files from a JS library

* Put idSite into single quote for support custom string id (#10247)

* Put idSite into single quote for support custom id

This is small change, but is necessary for #9975 and #4920. On my tests, even with single quote, numeric idSite works fine.

* Change unit test expected to support test with strings

* Change expected data to support test with strings

* submodule update

* submodule update

* Tweak core:archive output logging to better explain where to find error log depending on server configuration

* Adding hyphen for consistency

* Composer: remove the disable tls flag, refs #10023 (#10277)

* Do not throw an exception when debugging tracker (#10267)

This method is only called when Tracker debugging is enabled. It may cause problems if an unknown type is used and then the tracking completely fails which also stops tracking. Debugging info should not really throw exceptions, at least not in this case.

* Fix CSV export cannot be used with dates like previousX (#10273)

* fixes #9838 - show description bubbles below column if they would be out of viewport if shown above

* submodule update

* Let plugins provide custom piwik.js tracker tests (#10270)

Plugins could provide custom tracker tests by creating a file 'tests/javascript/index.php' within the plugin.

Won't be a documented feature for now but could add it to dev changelog if needed.

Content within the test file could be eg 
```
<script type="text/javascript">
test("MyPiwikTest", function() {
    expect(1);

    var tracker = Piwik.getTracker();

    equal(typeof tracker.trackGoal, 'function', 'trackGoal' );
});
</script>
```

* Fix segment data disappears from UI after changing auto_archive setting  (#10078)

* refs #10036 fallback to global done flag if plugin done flag has no archive/data

* global done flag might be the same as the plugin done flag if no segment and no range is used

* installing touch-punch lib to fix dragging issue on mobiles (#10045)

*  Reset the list of items in the cart, after tracking an ecommerce conversion (#10279)

* Clarify that token_auth must be  an admin or super user token.

* Reset the list of items in the cart, after tracking an ecommerce conversion

But do not reset the list of itms in the cart after tracking an ecommerce cart update.
fixes #10252

* Adding changelog note
`piwik.js`: after an ecommerce order is tracked using `trackEcommerceOrder`, the items in the cart will now be removed from the JavaScript object. Calling `trackEcommerceCartUpdate` will not remove the items in the cart.

* Add possibility to specify a custom row aggregation operation (#10288)

* Add possibility to specify a custom row aggregation operation

* added test for row metadata aggregations

* When JSLint validation failed, explain where to find the error messages refs #10271 (comment)

* trackLink does not invoke callback if request is done via GET (#10294)

* trackLink does not invoke callback if request is done via GET

* Update piwik.js

* Make JSlint errors output actually useful to developers

* JSLint: better error reporting, correct line numbers #7232

* Improved join generation for segments (#10264)

* better join generation for segments
* do not update visit if there are no values to be updated

* Calling an undefined JS API method should not trigger an error in the console (#10176)

* When calling a non existing method, do not fail JS execution but print a useful error message in browser console

* Fix

* Throw an error so developers can see in the browser console

* Up to date minified piwik.js

* Improve detection for empty keywords by allowing to define that in search engine definitions

* updates piwik/searchengine-and-social-list dependency

* refs #9984 - adds confirmation when changing segments definition

* refs #9984 - adjust UI test to automatically confirm confirmation

* better naming

* record actual screen resolution when using JS tracker (#10296)

* Add current domain to list of domains automatically only if no path is set (#10051)

* Do not add current domain automatically if setDomains is called, might also fix a bug in IE11

* this might fix jslint test

* added tests and fixed a bug

* fix tests

* Up to date piwik.js

* Minor changes

* Document new internal piwik.js change https://github.com/piwik/piwik/pull/10051/files

* Add link to multiple tracker docs in changelog. #10174

* make sure we prefer forwarded proto header over regular header (#10081)

* 2.16.2-b3

* Fixes #10117

* Fix typo

* Refs #10115

* use better message depending of the availability of archiving triggered by browser

* add option to hide segment definition change confirm

* Add a test for isInsideAnIFrame method in piwik.js  (#10300)

 #10271 #10105

* Searching recursively in sub-tables should work as expected (#10303)

* When a datatable is manipulated in memory, the correct id subtable is stored in metadata and must be fetched from there.

* piwik.js: Prepend a slash to all paths such as when users don't specify leading slash for URL path (#10302)

* Load the JS file in the test where it is used, to prevent errors running tests on IE8
jslint not working on IE8 it seems

* Fix JS error when running tests in edge browser

* iframe waits for document ready

* Fix the JS tests on Chrome

* Fix issue IE

* Fix for IE9: wait for window.onload

* Fixes #10105 - copied from #10271 by @Kijewski (#10306)

* submodule updates

* Added API method to get a single goal (#10305)

* added method to get a goal

* always fetch goal directly to not having to iterate over all goals

* added new changelog entry for devs

* updates phpunit dependency / submodule updates

* submodule updates

* Use log tables to decide which tables can be purged (#10304)

* use log tables to decide which tables can be purged

* make sure to return the number of deleted visits

* fix tests

* UI tests: Log tables purge

* added app manifest (#10321)

* Fixes #10276

* Update piwik/network component to 0.2.0 release for IPv6 ranges support #9411 (#10324)

* language update

* Segment editor, updating segments, logic when to display messages  (#10322)

* Display the right message when segment is set to real time

* Correct logic

* UI tests: minor messaging change

* Submodules including matomo-org/travis-scripts#28 #10323

* Update integration tests for #10276

* Remove comment

* submodule update

* submodule updates

* Php7 compatibility, Pear archive_tar package #10058

* Update submodules

* fixes #10310 - use indexOf polyfill function to avoid errors in older IEs (#10327)

* pass instance of both row objects to aggregate callback when aggregating them (#10330)

* Update composer libraries, while enforcing the minimum php in composer.json  (#10334)

* Enforce the Minimum PHP version in composer.json using "platform" feature
Added in composer/composer@a57c51e

It was found that this flag is very important to use to make sure the package we build and release will be compatible with this PHP version.
Otherwise, the minimum PHP version enforced would be the version where our build script runs, which could be PHP 5.5 or PHP 7 which would have caused serious issues.

* Update doctrine/cache, monolog,piwik/network, piwik/cache, piwik/php-tracker  etc.
#10058

* Release checklist to check that composer.json is correctly defined

* Use latest travis scripts for matomo-org/travis-scripts@30d7b16

* .travis.yml file is out of date, auto-updating .travis.yml file.

* Use latest travis scripts for matomo-org/travis-scripts@63445cf

* .travis.yml file is out of date, auto-updating .travis.yml file.

* Fetch latest referrer spam blacklist refs  #10058

* When Piwik config file becomes un-readable, make it clear that the file still exists but is not readable (#10331)

* * In messages, make it more clear whether config file exists and/or is (not) readable
* When the config file exists but is not readable, provide users the choice between making the file readable, or starting a fresh install
* As long as Piwik is not installed, piwik.php should not return 500 when config file is not readable
* Fixes #10283
* UI tests: updated message when config file is not found

* Update submodules to latest - refs #10058 (#10335)

* 2.16.2-b4

* Refs #10331 #10283

* 2.16.2-b5

* Add require_once to prevent build failure

* Add missing file

* 2.16.2-b6

* Professional Services for Piwik, neutral branding within Piwik app (#10339)

Professional Services for Piwik - neutral branding within Piwik app

* 2.16.2-rc1

* Update submodule, and fix to update file

* 2.16.2-rc2

* UI tests: fixed typo

* language update

* add possibility to create ui tests for users with view access only

* Added Recommended Drivers During Installation (#10346)

* Fixed

* Made Recommended Changes

* Add support section to plugin json template (#10358)

* Add support section to plugin json template

As suggested by a developer. The Marketplace has been supporting this for quite a while already.

* Added support section to example theme plugin json

* added wiki section

* added wiki section

* Adding couple issues to the changelog  #10058

* Do not document until finalised #10270

* Update to Device detector 3.7.1 (#10362)

* Device detector 3.7.1

* Fix build

* 2.16.2 KABOOM

* fix #10361 (#10361)

Now only the english string is changed, as translations are handled via https://www.transifex.com/piwik/piwik/

* Add pull request template mentioning PRs should be made against 3.x-dev (#10030)

* Do not limit password length (#10349)

* Disabled Max Password Length Check

* Disabled Max Password Length Check

* Reset config files

* Translation & other fix

* Fixed Test Case for Very Long Passwords

* UI tests:  #10357

* fixes #10374 generate:visualizationplugin was not working

* support section is no longer needed

* language update

* Update global.ini.php (#10408)

* submodule updates

* Remove again the Hiring file  #9814

* Minor Improvement to description

* Fix the integration test and patch Zend Validate

* on Travis use PHPUnit from composer

* Create index.htm files in all tmp/ sub-folder as an additional safety net (#10414)

* Create index.htm files in tmp/ folder as safe net

*  UI test

* silent fail

* fix unit test

* Minor Improvement to description

* Fix release checklist

* Fix release checklist

* UI test

* UI test logic

* Actually make methods public to keep BC

* Fix fatal error when requesting RSS format  (#10407)

* Fixes #10399

* add test reproducing issue #10399

* another health check

* Fix no required Piwik version was defined (#10413)

* Fix no required Piwik version was defined

* plugins should specify an upper version constraint to the next major version

* PHP 7: when there is a fatal error in API request processing, display the original error in the api response (#10406)

* PHP 7: when there is a fatal error in API request processing, display  the original error

* PHP 7 compatibility

* Use Piwik 40.0.0 so that test does not need to be updated soon

* UI test

* UI tests: plugins not compatible with next version are shown

* language update

* submodule update

* Split piwik.js into multiple files and provide a merged one (#10441)

refs #6106

* Fixes #10143 (#10424)

* Fix issue when running archiving via the old archive.php path
Notice: Use of undefined constant PIWIK_DOCUMENT_ROOT - assumed 'PIWIK_DOCUMENT_ROOT' in plugins/CustomPiwikJs/Commands/UpdateTracker.php

* fix tests
@mattab
Copy link
Member

mattab commented Dec 14, 2023

feel free to re-open or re-create if needed 👌

@mattab mattab closed this as not planned Won't fix, can't repro, duplicate, stale Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

4 participants