Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve SEO analytics in Piwik #1459

Closed
mattab opened this issue Jul 3, 2010 · 32 comments
Closed

Improve SEO analytics in Piwik #1459

mattab opened this issue Jul 3, 2010 · 32 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@mattab
Copy link
Member

mattab commented Jul 3, 2010

This ticket is a place-holder for all SEO Analytics feature requests.

We've narrowed down scope of this ticket and here are the specifications for Piwik 1.X:

  • SEO data points (Pagerank, page indexed,etc.) should be each recorded as "archive_numeric" fields (all of them being "integer" values).
  • The "seo archiving" code should not hook on the standard daily archiving, but instead be a scheduled task running once a day and pre-archiving all websites SEO (ie. running with "HIGH" priority so it's pre-processed before scheduled reports (which use SEO data))
  • These values vary over time. The evolution and changes are critical to the health of the site. The data will be updated and automatically archived, in a scheduled task running once a day.
  • One data point is an exception: "Domain age" should be stored in the Piwik_Option table, once for each website (store, time of domain creation in unix timestamp)
  • The API will then query these numeric values using existing functions eg $archive->getDataTableFromNumeric(array('pagerank','pages_google','pages_bings','alexa'))
  • Currently the widget(https://demo.piwik.org/index.php?module=Widgetize&action=iframe&widget=1&moduleToWidgetize=SEO&actionToWidgetize=getRank&idSite=1&period=day&date=2013-03-09&disableLink=1&widget=1) loads super slow. After this change, if cron is setup, the widget will render instantly. This is awesome.
  • Bonus: with the new code, it should be possible to include the "Row Evolution View" for this report! a new "View Evolution [icon]" link, will open the awesome existing multi row evolution report, in a popover, showing history of seo metrics!
  • Add metadata support for SEO plugin, so that the SEO report shows up in Piwik Mobile & Scheduled reports.
  • Periods display the stats for last day of period. Show last day in light grey.
@halfdan
Copy link
Member

halfdan commented Jul 3, 2010

I would very much like to see a widget displaying how a website is ranked on Alexa over time (using a graph maybe). This could be done by hooking on the ArchiveProcessing_Day.compute event.

@mattab
Copy link
Member Author

mattab commented Jul 3, 2010

Social widget (twitter, FB)

Ideas for improvement

  • Add a new widget for Social metrics: starting with Facebook and Twitter
  • Admin user can edit the IDs.
  • UI has a top link 'Edit settings', when clicked, shows fields for Facebook and Twitter IDs
  • Social IDs are recorded in the piwik_option table (simpler than having to modify the piwik_site table). Also, there could be several Social IDs per website (several FB fan pages eg.)
  • Admin can save FB or Twitter or both, clicks save. The data is saved using Piwik_SetOption function (for the given idSite).
  • List of metrics to show
    • Twitter followers
    • Tweets

    • Facebook fans
    • (FB interactions from their api?)

This would be a useful tool for webmasters to build a dashboard of all their social presence. Good to have next to the SEO widget.

Maybe other networks could be integrated later. See for example postrank API (digg, reddit, delicious, etc.).

@robocoder
Copy link
Contributor

aboutus.org recently added some "site metrics" and "online visibility audit" tools to their site. These might be nice as Piwik widgets and/or reports within this plugin.

@robocoder
Copy link
Contributor

WordTracker is one of the better known SEO tools, but I ran across some references to tools (later developing into web services) by Michael Marshall, i.e., "link-reputation-tool", "Theme Master", "OptiRanker", and "SEM Logic".

Theme Link Reputation gives you insight into the quality of your referring sites. You enter a URL and target keywords. The tool uses Google's API ti return a survey of the top 50 backlinks (backlink URL, # inbound links, # outbound links, a "reputation score", and the link text).

ThemeMaster/OptiRanker is a SEO tool. Instead of keyword density analysis, it centers on the idea of "themes", related keywords and word combinations, using a lot of math (see "Latent Semantic Indexing"). Meanwhile, SEM Logic is more about competitive intelligence.

@robocoder
Copy link
Contributor

seopanel.in is gpl v2 but might be an interesting integration play.

@anonymous-matomo-user
Copy link

See is what position in the serps (in the search engine results page) the website was when the user clicked (in addition to what the query was).

To easily have the ability to click on a referring page URL, without having to deal with truncation.

Ability to view traffic and statistics by day, for instance, show me traffic on Tuesday, or Friday, without the requirement to specify dates. So you can match days of the week, not dates.

Ability to easily visualize user paths throughout the site.

@anonymous-matomo-user
Copy link

To improve the optimization of Piwik I would recommend that we attempt to use the new logo (icon) created and attempt to get people currently using Piwik to embed this into the footer of their home page. This should help with the link popularity of Piwik.

@peterbo
Copy link
Contributor

peterbo commented Mar 3, 2011

Some ideas for the SEO-Plugin:

  • Request the own website for measuring Page Loading Speed (images, (javascript), etc.), warn if an image/etc. exceeds recommended size
  • check robots.txt for errors / wrong restrictions
  • check google "site:website.tld"-query for domain visibility (number of results) (also in evolution-graph) (very important)
  • check SE ranking for 2 or 3 keywords (with evolution graph)
  • check for Domain/IP popularity (how many distinct domains / IPs link on your website)
  • How many other Domains are hosted on the IP of my Website (for example: http://www.bing.com/search?q=ip%3A91.121.8.168)
  • Server Ping / Traceroute (does this make sense)
  • apart from the backlinks from yahoo, get backlink data from the piwik archives (site referer-data), to compare both or sum them or whatever
  • analyze quality of backlinks (PR, visibility (site:website.tld query), MOZ rank of a site (seomoz.org, semrush.com))

@anonymous-matomo-user
Copy link

The key to this tool would 100% be the tracking over time, much of SEO is about today, and now. This tool would become viral if it kept a log of a website's overall internet presence over time. If an SEO was to set this up whenever a new client started, they would be able to show a solid history of improvements over time.

@robocoder
Copy link
Contributor

I expect we would use scheduledTasks or a cron job to fetch (and record) 3rd-party metrics daily.

@robocoder
Copy link
Contributor

(In [4025]) refs #1459 - domain age parser improvement

@anonymous-matomo-user
Copy link

Some additional idea's for SEO plugin:

  • Ranking data, this can be queried from various suppliers if for example API connections could be made to f.e. Raventools or others. Also ranking data in many occassions is available in the referrer URL from google (&cd=). Note, universal search results may cause different numbers to appear, apart from regular text links. universal search results are identified by sa=X parameter in referrer.
    Of course this data should be available over time (as @john pointed out..)
  • There are numerous formulas out there to calculate 'potential extra traffic' . would be nice to indicate what that extra potential might be if traffic is X and current position = Y. This could be calculated using CTR data and to be considered raw estimate, yet useful.
  • Server uptime / performance integration (downtime correlatates not only with traffic, but also search engine rankings). Can be done using various monitoring tools, most of them provide API's.
  • Connection with Google Webmaster Tools data API to provide various insights on current website. For example crawl issues, Messages could also be displayed. Many marketers are daily in their analytics program but not in GWT.
  • what percentage of pages listed in sitemap is receiving traffic (match with URL's receiving traffic).
  • Duplicate content detection , a tool to calculate or show URL's that are most likely to be considered duplicate content.
  • integrate CopyScape Premium API to detect per URL if there are copycats for your content.
  • visualize Link neighborhood (unsure if possible, depends on data availability)

Q: - What SEO's use alexa rank? (outside US?)

Also a summary or action pane would be cool; algorithmically show suggestions based on 'what has been found'. Considerations might be taken if you were to be able to supply the type or niche of the websites as 'actions' might vary accordingly. f.e. Content websites differ heavily in terms of goals and KPI's opposed to ecommerce websites.

@anonymous-matomo-user
Copy link

I think most SEO's use Alexa as a very quick reference to the overall traffic/authority and value of a website or domain. I think a way to help Piwik offer basic SEO information and tools might be to partner with companies, offer some basic versions of the data set in many tools like majesticseo, seomoz or others. I can open the door to discussions. This would help Piwik get in the door with these guys, as they would then support Piwik and help drive traffic to Piwik. In return Piwik would market the paid versions of these tools. Piwik could even get a commission on any sales of these guys premium tools that are made from the basic use within Piwik.

@mattab
Copy link
Member Author

mattab commented Apr 22, 2011

martijn,
we now show the search engine ranking (via &cd) in the real time Live! widget and in the Visitor Log.
I like the idea of integrating with other APIs for sure, but if they only offer a premium paid for version, it is difficult for us to integrate it since it wouldn't provide value for most users (not willing to pay). John, maybe you have more ideas on how we could work on this? Do majesticseo or seomoz also provide interesting things for non paid users?

@anonymous-matomo-user
Copy link

@peterb: You say: check google "site:website.tld"-query for domain visibility

Please use site:website.tld/folder to get statistics for single websites hosted on the same server.

@anonymous-matomo-user
Copy link

what a about bot metrics? i dont exactly know if its possible but maybe someone else.

  • how often are bots visiting my page
  • which bots
  • which sites do they visit

@robocoder
Copy link
Contributor

webdevoase: see BotTracker in #2391

@mattab
Copy link
Member Author

mattab commented Jun 1, 2011

A simple improvement: cache the result of the API calls for 1 day, so that loading the widget again and again is fast. Currently, it does the Requests to all services on every widget load.

@robocoder
Copy link
Contributor

In #2869, the Yahoo metrics are no longer available and have been removed from the plugin's API and Widget.

@robocoder
Copy link
Contributor

(In [5683]) refs #1459 - add SEO unit test for http://piwik.org

also:

  • update Google backlinks scraper (not currently used)
  • remove unused Mock in Database.test.php
  • added a daily CI build in case we go a few days without a commit

@mattab
Copy link
Member Author

mattab commented Jan 25, 2012

  • update Google backlinks scraper (not currently used)

Could we use google BL scraper or is the number really meaningless? maybe Bing?

@anonymous-matomo-user
Copy link

Possibly what you can do is to use something like www.opensiteexplorer.org/ or www.majesticseo.com for backlink information. If you interested I can possibly work out a deal for use of their software?

@mattab
Copy link
Member Author

mattab commented Jan 26, 2012

john, these look good but opensiteexplorer is limited to 5 queries per day. Maybe we could cache the results and call once a day (and rotate if more than 5 URLs to call).

majesticseo seems to not have daily limits to just grab the number of backlinks which is nice? is it true?

Also, what about data quality? Maybe we could pick one or call both? thoughts?

@anonymous-matomo-user
Copy link

Well I know both of these companies, can may be able to sort out a deal with them if you want me to contact them and see if we could sort out a deal?

I think they both have API options.

Just let me know.

@mattab
Copy link
Member Author

mattab commented Feb 3, 2012

Thanks for proposing!! :)

I don't know what kind of deals we could sort out though because if we get access to a special API token allowing more requests per day, this token would be visible to everyone (if bundled in Piwik source code) so could be reused by non piwik users.

it would be interesting to ask them their thoughts, for sure.

Let us know if you had another idea or if you manage to contact them! cheers

@mattab
Copy link
Member Author

mattab commented Dec 14, 2012

See cutroni.com/blog/2012/08/13/seo-customizations-for-google-analytics/

Decreasing priority as scope is too broad.

@anonymous-matomo-user
Copy link

Hi. I am one of the directors of MajesticSEO and we are long time users of Piwik on our site as well.

We would like to help to give Majestic data directly into Piwik and will try to work with Piwik to make this happen.

majesticseo seems to not have daily limits to just grab the number of backlinks which is nice? is it true?<<

That's not EXACTLY true - but near as makes no difference. What I think would be great here is two or three three ways we can give FREE information to Piwik users and then more ways in which users taht have accounts with Majestic can get a closer integration.

Free ideas:

Yes - we could easily give PIWIK users link counts for any URL. In addition, we can give Flow Metric scores (values between 0-100) showing the influence of a URL. We can also show how many links to are URL are from .EDUs, how many are nofollowed, etc etc.

In addition, Piwik could add value to this data by maintaining in particular the Flow Metrics scores and the link counts over time. OUr numbers update every day, but we obviously cannot keep complete historical data of all 4 trillion URLs in our system. We do have backlink history charts (which could also be given for free within Piwik) but if PIWIK maintained Flow Metrics history scores for the site's URLS, this would show whether a page is increasing or decreasing in influence over time.

Premium ideas:

If a user wanted, then the actual link LISTS could be brought back by users connecting their PIWIK accounts to their MajesticSEO accounts. This would obviously cost the user to have an account on Majestic, but they would only need an entry level subscription. This uses our "OpenApps" API and from there, we would be able to show users a list of links won and lost on a day-by-day basis, for example... not necessarily just for their own site, although as Piwik is predominantly an analytics package and not a competitor analysis package, starting with your own sites seems to make sense.

If this all sounds of interest, we'll try and work with PIWIK to get it done.

Dixon Jones.

@mattab
Copy link
Member Author

mattab commented Feb 4, 2013

@MajesticSEO Thanks for your message. Let's discuss if and how we could integrate the reports in the default SEO widget http://dev.piwik.org/trac/raw-attachment/ticket/3525/piwik-seo-plugin-update.png

@mattab
Copy link
Member Author

mattab commented Feb 11, 2013

Created ticket in: #3734 for majestic SEO integration

SEO Panel at: seopanel.in looks interesting as well!

@mattab
Copy link
Member Author

mattab commented Mar 25, 2013

After brainstorming and discussion we have narrowed down the vision for Piwik 1.X SEO reporting following @john's awesome advice. See the ticket description for the specs. It will be awesome!

@mattab
Copy link
Member Author

mattab commented Feb 1, 2014

SEO: See cutroni.com/blog/2012/08/13/seo-customizations-for-google-analytics/

myseotool.com/

@mattab mattab added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab removed the P: normal label Aug 3, 2014
@mattab mattab modified the milestones: Long term, Mid term Dec 5, 2016
@mattab mattab closed this as not planned Won't fix, can't repro, duplicate, stale Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

5 participants