Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Piwik is unable to load the logs, its struck after processing 707521 records #8010

Closed
bsmurali opened this issue May 27, 2015 · 7 comments
Closed
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@bsmurali
Copy link

Its struck at the following steps, unable to move forward.

Please help me ASAP

707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)

@mattab
Copy link
Member

mattab commented May 28, 2015

Hi @bsmurali

Which version of Piwik do you use?

can you maybe see any error in your web server error log?

Btw: in the future feel free to report bug for Log Analytics script in the other issue tracker here: https://github.com/piwik/piwik-log-analytics/issues/

@bsmurali
Copy link
Author

We are on piwik version 2.1.0

No I haven't seen any errors in our error log except the following

Wed May 27 02:04:49.129127 2015] [authz_core:debug] [pid 4656:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:52502] AH01628: authorization result: granted (no directives), referer: http://localhost/piwik/index.php?module=CoreHome&action=index&idSite=1&period=day&date=2015-03-17
[Wed May 27 02:09:49.571452 2015] [authz_core:debug] [pid 4656:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:52540] AH01628: authorization result: granted (no directives), referer: http://localhost/piwik/index.php?module=CoreHome&action=index&idSite=1&period=day&date=2015-03-17
[Wed May 27 02:14:50.092779 2015] [authz_core:debug] [pid 4656:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:52574] AH01628: authorization result: granted (no directives), referer: http://localhost/piwik/index.php?module=CoreHome&action=index&idSite=1&period=day&date=2015-03-17
[Wed May 27 02:19:50.597505 2015] [authz_core:debug] [pid 4656:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:52635] AH01628: authorization result: granted (no directives), referer: http://localhost/piwik/index.php?module=CoreHome&action=index&idSite=1&period=day&date=2015-03-17
[Wed May 27 02:24:51.117832 2015] [authz_core:debug] [pid 4656:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:52674] AH01628: authorization result: granted (no directives), referer: http://localhost/piwik/index.php?module=CoreHome&action=index&idSite=1&period=day&date=2015-03-17
[Wed May 27 02:29:51.638158 2015] [authz_core:debug] [pid 4656:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:52709] AH01628: authorization result: granted

@bsmurali
Copy link
Author

and the following info on error logs.

[Fri May 15 15:33:18.741558 2015] [authz_core:debug] [pid 4940:tid 1480] mod_authz_core.c(827): [client 127.0.0.1:60431] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:19.365562 2015] [authz_core:debug] [pid 4940:tid 1416] mod_authz_core.c(827): [client 127.0.0.1:60434] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:19.802365 2015] [authz_core:debug] [pid 4940:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:60436] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:20.036366 2015] [authz_core:debug] [pid 4940:tid 1420] mod_authz_core.c(827): [client 127.0.0.1:60437] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:32.469646 2015] [authz_core:debug] [pid 4940:tid 1484] mod_authz_core.c(827): [client 127.0.0.1:60440] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:32.469646 2015] [authz_core:debug] [pid 4940:tid 1448] mod_authz_core.c(827): [client 127.0.0.1:60441] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:32.547646 2015] [authz_core:debug] [pid 4940:tid 1432] mod_authz_core.c(827): [client 127.0.0.1:60443] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:32.719248 2015] [authz_core:debug] [pid 4940:tid 1480] mod_authz_core.c(827): [client 127.0.0.1:60446] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:33.499253 2015] [authz_core:debug] [pid 4940:tid 1484] mod_authz_core.c(827): [client 127.0.0.1:60448] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:33.561653 2015] [authz_core:debug] [pid 4940:tid 1492] mod_authz_core.c(827): [client 127.0.0.1:60449] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:33.624053 2015] [authz_core:debug] [pid 4940:tid 1420] mod_authz_core.c(827): [client 127.0.0.1:60451] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:33.702054 2015] [authz_core:debug] [pid 4940:tid 1440] mod_authz_core.c(827): [client 127.0.0.1:60453] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:41.346103 2015] [authz_core:debug] [pid 4940:tid 1480] mod_authz_core.c(827): [client 127.0.0.1:60456] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:41.346103 2015] [authz_core:debug] [pid 4940:tid 1420] mod_authz_core.c(827): [client 127.0.0.1:60457] AH01628: authorization result: granted (no directives)
[Fri May 15 15:33:41.580104 2015] [authz_core:debug] [pid 4940:tid 1432] mod_authz_core.c(827): [client 127.0.0.1:60460] AH01628: authorization result: granted (no directives)

@mattab
Copy link
Member

mattab commented May 29, 2015

Hi @bsmurali Please upgrade to latest PIwik where this bug may be fixed. if you have issues with Log analytics feel free to create bug report in the log analytics tracker: https://github.com/piwik/piwik-log-analytics

@mattab mattab closed this as completed May 29, 2015
@mattab mattab added the answered For when a question was asked and we referred to forum or answered it. label May 29, 2015
@bsmurali
Copy link
Author

bsmurali commented Jun 4, 2015

Hi mattab,

Unfortunately, piwik upgrade failed with the following error page

( ! ) Fatal error: Call to undefined method Piwik\Menu\MenuTop::addEntry() in C:\wamp\www\piwik\plugins\ExampleUI\ExampleUI.php on line 48
Call Stack

Time Memory Function Location

1 0.0008 253944 {main}( ) ..\index.php:0
2 0.0158 1286800 require_once( 'C:\wamp\www\piwik\core\dispatch.php' ) ..\index.php:27
3 0.9089 13623424 Piwik\FrontController->dispatch( ) ..\dispatch.php:31
4 0.9095 13629880 Piwik\FrontController->doDispatch( ) ..\FrontController.php:97
5 0.9386 15212544 call_user_func_array ( ) ..\FrontController.php:531
6 0.9386 15212704 Piwik\Plugins\CoreHome\Controller->index( ) ..\FrontController.php:531
7 0.9389 15213152 Piwik\Plugins\CoreHome\Controller->getDefaultIndexView( ) ..\Controller.php:170
8 1.0004 16331400 Piwik\Plugin\Controller->setGeneralVariablesView( ) ..\Controller.php:132
9 1.0209 16787224 Piwik\Menu\MenuTop->getMenu( ) ..\Controller.php:639
10 1.0209 16787536 Piwik\Piwik::postEvent( ) ..\MenuTop.php:70
11 1.0209 16787632 Piwik\EventDispatcher->postEvent( ) ..\Piwik.php:699
12 1.0212 16788912 call_user_func_array ( ) ..\EventDispatcher.php:130
13 1.0212 16789072 Piwik\Plugins\ExampleUI\ExampleUI->addTopMenuItems( ) ..\EventDispatcher.php:130
A fatal error occurred

Please contact the system administrator, or login to Piwik to learn more.

@bsmurali
Copy link
Author

I was able to upgrade the piwik successfully. Unfortunately the problem didn't fix. Its still in same state as i informed originally i.e similar to this. It process only 200 to 500 visits and exits out of the process.

707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)
707521 lines parsed, 700371 lines recorded, 1 records/sec (avg), 0 records/sec (current)

in http error logs I only see the following error message

[Tue Jun 09 14:44:58.463817 2015] [:error] [pid 2668:tid 1480] [client 127.0.0.1:60884] Error in Piwik (tracker): Error query: SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction In query: UPDATE piwik_log_visit SET idvisitor = ?, visit_total_time = ?, visit_last_action_time = ?, visit_exit_idaction_url = ?, visit_total_actions = visit_total_actions + 1 , custom_var_k1 = ?, custom_var_v1 = ? WHERE idsite = ? AND idvisit = ? Parameters: array ( 0 => '\xf4\x06\tk%\x9dB\x16', 1 => 287, 2 => '2015-05-21 13:39:15', 3 => 1335714, 4 => 'Not-Bot', 5 => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; MALC; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)', 6 => 2, 7 => 9492073, )

@mattab
Copy link
Member

mattab commented Jun 11, 2015

Hi @bsmurali

If you're not using 2.13.1 please upgrade to this latest version

if you are already using 2.13.1 and have this issue, please create a new bug report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Development

No branches or pull requests

2 participants