@hpvd opened this Issue on February 4th 2016

Log analytics seems to be really interesting.

To make usage of log analytics easier / acessible by more users
there should be an easy possibility to set it up and make it start working
at "manage website"/ Website edit

Which give some simple kind of dialog / form to make it work.

A very first approche may look like this:

A) Show hint:
"Phyton needs to be installed and working, Log file has to be accesible."

B) Where to find log file [field for path]
+ example for needed format

C) Import period [field for date range]
+ give hint: "use short one for test"
+ example for needed format

D) What to include
- include bots (sets --enable-bots)
- include static files (JS, CSS, images, etc.) (sets --enable-static)
- include http errors (sets --enable-http-errors)
- include http-redirects (sets --enable-http-redirects)

E) How to process:
E1) How many threads to use [field for number] (sets --recorders=N)
+ give hint: "use no more than CPU-cores available"

E2) How many log lines at once [field for number] (sets --recorder-max-payload-size=N)
+ give hint: "default: 300"

F) Regular import:
F1) Should there be a regular import of logs [drop down yes/no] (setup cron automatically)
+ give hint: "should be disabled on testing"

F2) Import frequency of logs [drop down of typical values] (setup cron automatically)

G) Save & Start import [Button]

details extracted from https://piwik.org/docs/log-analytics-tool-how-to/

@masteranalyze commented on March 29th 2016

The approach writed more up by hpvd,for start is very good.

Maybe we can setup this as an milestone for piwik 3.

It should be an option when adding the website,to chose practically witch method to use for that website:Javascript or Server Side.

If server side,to provide the details writed by hpvd,in an simple step by step process,so everything can go smooth.And on end as @hpvd writed an button :Start Import Save .


@masteranalyze commented on July 22nd 2020

I think the log analytics tool with python is very good,but should be inside Matomo,and once it`s inside from there we can start.

With the javascript tracking : Matomo uses cookies to store a unique visitor ID, used to recognize visitors from previous visits. When cookies are disabled, Matomo will still be able to determine unique visitors based on IP address and other footprints, but this will be slightly less accurate.

So with javascript we have an unique visitor ID,but that is not so important,what is important for MERGING data is what do Server side Vs Javascript cointains in as unique data?

Either if we use Server side or Javascript,the only thing in common for both methods at least the one that i could identify maybe they are others as well is :THE IP OF THE USER .

Either server either java,the ip of the user is the same.

Example : In Website 1,we activate Server tracking and javascript tracking,so both methods basically.

In Matomo we should be able to import the Server side tracking from the log file.
In the same time javascript is tracking details about the users.

In order to merger data,from the Java to Server side,we must have : Unique visitor Ip .

As the Ip is unique,same ip witch the user haves in server log will have in javascript,and that ip can be identified on javascript,by doing that,all details tracked on same IP with Javascript,should be merged in the new report.

The same way in javascript we have via coockies Unique Visitor Id,the same way for accurate merging data we should have:Unique Visitator Ip .

By identifying the same IP on server and javascript tracking,and giving each Ip an unique identifier,data could be merged accurate into an new nice report with the good of both worlds server side tracking and javascript tracking.

If an user ip is dynamic and changing,that is not an problem,becaue the new ip will be tracked and be once again unique in server side + javascript side,except that if the user log-in in the website and doing actions,we will know that the same user just changed the ip adress.

Anyhow the final report by merging data will be more accurate that anything that is on the market,but that needs to happen via the Matomo web interface in order to be fully usable.

I think the Unique IP Identifier could be an good start for merging accurate the data into an Final Report.

Powered by GitHub Issue Mirror