New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Plugin: BotTracker to track bot-actions #2391
Comments
The plugins comes with a widget to show the data from the bot_db-table. ToDos:
Oh, and there is one big limitation: |
Nice I exactly look for this module,but after copying this module I get this error. Unable to load plugin 'BotTracker' because '/home/******/public_html/plugins/BotTracker/BotTracker.php' couldn't be found. You can manually uninstall the plugin by removing the line Plugins[] = BotTracker from the Piwik config file. The file is there! ?? Any Ideas |
Hmmm.... sounds strange. I installed the plugin about 30 times during development and test. First check, if the folder looks exact as shown in the message (e.g. "BotTracker" or "bottracker") and then check the folder and file permissions. Is the read-access restricted? |
We are using piwik 1.4 and getting getting the following error when activating the plugin Fatal error: Call to undefined method Piwik_BotTracker::LogToFile() in example.com/analytics/piwik/plugins/BotTracker/BotTracker.php on line 41 |
Replying to jekko:
|
Hi jekko, try the new version (v0.12). btw: I tested the plugin with Piwik 1.3 and 1.4 |
thomas its working, ty |
Changelog
|
Changelog
|
Hello, I'm running piwik 1.4 with BotTracker 0.16. Plugin has been installed and activated. Widged has been added. Sites contain the Java Script code. Question: Any hint very much appreciated. Best regards, sun |
did you read this? Oh, and there is one big limitation: Most bots don't use JavaScript. So if you use only the Piwik-JS-API (default), you don't get any results. I use the PHP-API, so every bot hits my visitor-log. That's why I wrote this plugin. |
Hi sun, first of all: the plugin is not able to track multi-sites. But I will put that on my todo-list. To test the plugin I use Firefox with a plugin called "User Agend R G". But remeber: The plugin will only catch non-JS-Vots if you use the PHP-Tracking-API! |
Dev-Status:
In addition to that I will change the database so the plugin can track multiple sites seperately. I'm a professional programmer for more than 16 years now, but mainly on the mainframe. In the last years I also code some java-applications but PHP is only a hobby. So don't expect a fast solution here. I try to learn the API by looking into the examples, the source and doing some try-and-error-debugging. If someone will jump in and offers help... the door is wide open! Just leave me a message. And the last point: |
Changelog
I had add a column to the database, so if you install the new version, it will drop the old database and create a new one. Then the script will insert the bot-list for all sites where the user is admin.
|
Hi Thomas, I looked further into my issue not getting any count or timestamp. Having that said, I modified BotTracker.php providing me some more log data at function checkBot.
I can see, that the bot access gets catched and that the proper row gets detected (43). GoogleBot matches in my database ID 43. Also I can see, that the code execution enters the if statement where I print the value of $row['botId'] which by above example is 4. So the query updating the database gets performed. But, rather than updating row 43, he is updating row 4, which is the wrong site as well as the wrong bot. Shouldn't the if statement at the checkBot statement more be like:
I replaced $row['botId'] with just $row. At least on my installation he updates now the correct row. Could you please verify and let me know your comment. Best regards, sun |
Hi sun, I get the variable $row from the function Piwik_FetchOne:
This function returns an labled array with all selected columns. In your case it should be Because of this I'm surprised, that you get an positive result when you only use $row. How do you print the log-data? Best regards, |
Hi Thomas, it's PHP 5.2.12 and MYSQL 5.1 The 2 lines
just before the if statement generate
which results into using your code that row 4 gets updated and using my code, row 43 gets updated. sun |
One more.
So with my code it counts for Row 41, which is MSN in my database. sun |
Hi sun, I found the error. The function Piwik_FetchOne did not return an array, it returns the value. I will fix an test it. Thank you. |
Yep. That works for me as well. Thanks! |
Thomas: I'm glad to see you're continuing to develop this oft-requested feature. A couple of comments:
|
Hi vipsoft, thanks for the tips. As you can see I already updated the plugin. Will the update-scripts run automaticly when you update the plugin? Will all scripts run in the right order if I have skipped a version? And I have some questions concerning the usage of Trac:
|
Oh, and by the way: |
Changelog 0.21
I have written and tested 2 update-scripts: v0.18 and v0.21 |
In checkBot():
|
The UPDATE uses only the botId because it's the unique index of the table. So if I found a qualified row while using the idSite, I can update the table with just using the botId. Do you have a description of the tracker cache and how to use it? The table is very small and my sites doesn't get very much hits, so performance tuning is not my top priority. But I will compare both ways if I can implement the cache. |
The tracker cache are files in tmp/cache/tracker that are automatically loaded with each tracker request. You can see this being used by SitesManager.php in recordWebSiteDataInCache(). Please also take a look at Matt's ideas in #653 |
There are some points to think about:
I wish to make some of these features flexible (e.g. how much user agents should be stored or how much time between 2 hits for logging). |
Changelog 0.22
I've tested a lot, but I'm sure there are bugs left. Please report anything, that doesn't work as designed. |
To solve some of Matt's "aditional features" I have to generate a new table that logs every hit of a bot.
Is there more to think about? |
The update-script is only important for those, who migrate Piwik from 1.x to 2.x But I think, I found your problem. GitHub names the download-file "BotTracker-master" because it's the master-branch of the source. The plugin is names BotTracker and the folder must have exact the same name. |
Oh yeah, I noticed that "master" part. At first I thought I had downloaded the wrong file. Anyway, I renamed it like you said, and it worked! The new widget is all up and running now. Thank you very much :-) |
a quick note; if you "Upload the plugin via ZIP" (you can find this link in Settings > Marketplace > Upload a ZIP plugin") then the directory would have been put with the right name automatically. |
Oh ok, that's good to know. I have another question now. 2 or 3 spiders have visited while I've been watching the past 30-45 minutes, but they don't show up in the widget. Such as google and baidu, which I guess are both pre-configured. Does this plugin only show the bots/spiders under certain circumstances? |
Yeah, over a 2 hour period, Google visited twice and Baidu 3 times. But the new BotTracker widget (installed 2 hours before the 1st search engine spider visit) doesn't show any visits. Maybe I'm misunderstanding something? There's also another discrepancy, which you probably can't help with. But maybe worth a mention, in case I'm misunderstanding, or it represents a larger problem. SMF could be reporting 8 guests, 1 user, 2 spiders. But the Real Time Visitor Count stays at zero. And I realize that SMF reports "in the last 15 minutes" and Piwik Real Time reports every 3 minutes. But I refreshed SMF to get it's latest, and then immediately switched to Piwik tab and refreshed it. So I would think that's pretty close -- enough to catch at least a couple of visitors. I went through that routine several times over the last 4 hours, but the SMF visitors is always higher than Piwik. And Piwik most of the time is on zero, while SMF almost always is reporting visitors. So maybe I just don't understand something? Thanks again :-) |
Woohoo - this ticket is definitely growing too big g Hi Brynn, Many webcrawler, spider and bots don't load the images in a page and most of them don't execute JacaScript. So you cannot track them if you don't use the PHP-API. |
From now on, the current version can be downloaded in GitHub or the |
Oh, wait a minute....either more controls have become available, or I did not see them initially.... Now there's an update, that I'm pretty sure wasn't there before. And also, there's Settings > BotTracker Configuration, which I don't think was there before, but maybe I just missed it. However, it appears to be already set up anyway. I use both js and image tracking. (Although that seems to essentially double my visit count, so I may think about changing that. However, I'm not even clear what php-api is. I guess I can find something in the user manual about it. Although it will probably be too complicated for me to figure out. But I'll read up on it anyway. Maybe it would be a good idea to include a note with the BotTracker plugin, that it doesn't work with js/image tracking? Or I don't know, maybe most people who use Piwik would already know that much. But like I said, I don't even know what api-php is yet. Ok, well thanks for your help anyway. If I can figure out the api-php, and it's something I can use, maybe I will. Otherwise, I guess I just won't have info about the spiders from Piwik. |
update function has some problems at the moment |
there seem to be a problem wit bottracker and piwik 2.2rc1 |
Just install latest version of BotTracker v.0.49 from 22. Jun. 2014.
|
This looks weird. I don't think the plugin is responible for this unwanted update. As you suggest, there could be a problem in the backup-routine durng install or update. |
Plugin is not responsible for this bug, see: #5409 |
@dandv see https://github.com/Thomas--F/BotTracker |
BotTracker Plugin
How to install?
Author
Any help is welcome
Changelog
Feedback
Please leave a comment if you have any feedback, suggestion, or bug report.
Keywords: bot, spider, search, engine, agent, third-party-plugin
The text was updated successfully, but these errors were encountered: