VisitsSummary - Empty results in weeks where a month change ocurrs #11799

I seem to be getting wrong results when I get user count and "week" period.

https://piwikinstance/?module=API&language=es&token_auth=xxxxxxxx&method=VisitsSummary.getUsers&idSite=xx&date=2017-05-16,2017-06-16&period=day returns:

<?xml version="1.0" encoding="utf-8" ?>
<results>
    <result date="2017-05-16">428</result>
    <result date="2017-05-17">417</result>
    <result date="2017-05-18">395</result>
    <result date="2017-05-19">349</result>
    <result date="2017-05-20">284</result>
    <result date="2017-05-21">239</result>
    <result date="2017-05-22">388</result>
    <result date="2017-05-23">357</result>
    <result date="2017-05-24">351</result>
    <result date="2017-05-25">289</result>
    <result date="2017-05-26">372</result>
    <result date="2017-05-27">283</result>
    <result date="2017-05-28">256</result>
    <result date="2017-05-29">365</result>
    <result date="2017-05-30">386</result>
    <result date="2017-05-31">369</result>
    <result date="2017-06-01">410</result>
    <result date="2017-06-02">389</result>
    <result date="2017-06-03">315</result>
    <result date="2017-06-04">238</result>
    <result date="2017-06-05">422</result>
    <result date="2017-06-06">419</result>
    <result date="2017-06-07">394</result>
    <result date="2017-06-08">389</result>
    <result date="2017-06-09">343</result>
    <result date="2017-06-10">256</result>
    <result date="2017-06-11">267</result>
    <result date="2017-06-12">392</result>
    <result date="2017-06-13">392</result>
    <result date="2017-06-14">362</result>
    <result date="2017-06-15" />
    <result date="2017-06-16" />
</results>

But https://piwikinstance/?module=API&language=es&token_auth=xxxxxxxx&method=VisitsSummary.getUsers&idSite=xx&date=2017-05-16,2017-06-16&period=week returns:

<?xml version="1.0" encoding="utf-8" ?>
<results>
    <result date="2017-05-15,2017-05-21">1723</result>
    <result date="2017-05-22,2017-05-28">1652</result>
    <result date="2017-05-29,2017-06-04" />
    <result date="2017-06-05,2017-06-11">1754</result>
    <result date="2017-06-12,2017-06-18">986</result>
</results>

Is this a bug or am I doing something wrong?

Larger dataset. I can't seem to find a pattern.

By "day":

<?xml version="1.0" encoding="utf-8" ?>
<results>
    <result date="2016-10-16" />
    <result date="2016-10-17" />
    <result date="2016-10-18" />
    <result date="2016-10-19">1</result>
    <result date="2016-10-20">29</result>
    <result date="2016-10-21">61</result>
    <result date="2016-10-22">55</result>
    <result date="2016-10-23">76</result>
    <result date="2016-10-24">111</result>
    <result date="2016-10-25">166</result>
    <result date="2016-10-26">170</result>
    <result date="2016-10-27">143</result>
    <result date="2016-10-28">124</result>
    <result date="2016-10-29">114</result>
    <result date="2016-10-30">94</result>
    <result date="2016-10-31">137</result>
    <result date="2016-11-01">153</result>
    <result date="2016-11-02">179</result>
    <result date="2016-11-03">155</result>
    <result date="2016-11-04">125</result>
    <result date="2016-11-05">100</result>
    <result date="2016-11-06">81</result>
    <result date="2016-11-07">146</result>
    <result date="2016-11-08">157</result>
    <result date="2016-11-09">164</result>
    <result date="2016-11-10">153</result>
    <result date="2016-11-11">130</result>
    <result date="2016-11-12">114</result>
    <result date="2016-11-13">107</result>
    <result date="2016-11-14">178</result>
    <result date="2016-11-15">144</result>
    <result date="2016-11-16">168</result>
    <result date="2016-11-17">130</result>
    <result date="2016-11-18">140</result>
    <result date="2016-11-19">99</result>
    <result date="2016-11-20">88</result>
    <result date="2016-11-21">157</result>
    <result date="2016-11-22">132</result>
    <result date="2016-11-23">129</result>
    <result date="2016-11-24">155</result>
    <result date="2016-11-25">148</result>
    <result date="2016-11-26">110</result>
    <result date="2016-11-27">108</result>
    <result date="2016-11-28">112</result>
    <result date="2016-11-29">159</result>
    <result date="2016-11-30">163</result>
    <result date="2016-12-01">196</result>
    <result date="2016-12-02">198</result>
    <result date="2016-12-03">147</result>
    <result date="2016-12-04">131</result>
    <result date="2016-12-05">187</result>
    <result date="2016-12-06">226</result>
    <result date="2016-12-07">192</result>
    <result date="2016-12-08">178</result>
    <result date="2016-12-09">169</result>
    <result date="2016-12-10">119</result>
    <result date="2016-12-11">116</result>
    <result date="2016-12-12">185</result>
    <result date="2016-12-13">8</result>
    <result date="2016-12-14">9</result>
    <result date="2016-12-15">7</result>
    <result date="2016-12-16">4</result>
    <result date="2016-12-17">5</result>
    <result date="2016-12-18">7</result>
    <result date="2016-12-19">7</result>
    <result date="2016-12-20">12</result>
    <result date="2016-12-21">9</result>
    <result date="2016-12-22">16</result>
    <result date="2016-12-23">20</result>
    <result date="2016-12-24">12</result>
    <result date="2016-12-25">22</result>
    <result date="2016-12-26">21</result>
    <result date="2016-12-27">19</result>
    <result date="2016-12-28">27</result>
    <result date="2016-12-29">34</result>
    <result date="2016-12-30">38</result>
    <result date="2016-12-31">55</result>
    <result date="2017-01-01">176</result>
    <result date="2017-01-02">275</result>
    <result date="2017-01-03">263</result>
    <result date="2017-01-04">283</result>
    <result date="2017-01-05">250</result>
    <result date="2017-01-06">287</result>
    <result date="2017-01-07">215</result>
    <result date="2017-01-08">171</result>
    <result date="2017-01-09">264</result>
    <result date="2017-01-10">267</result>
    <result date="2017-01-11">275</result>
    <result date="2017-01-12">265</result>
    <result date="2017-01-13">277</result>
    <result date="2017-01-14">197</result>
    <result date="2017-01-15">173</result>
    <result date="2017-01-16">273</result>
    <result date="2017-01-17">271</result>
    <result date="2017-01-18">266</result>
    <result date="2017-01-19">254</result>
    <result date="2017-01-20">289</result>
    <result date="2017-01-21">191</result>
    <result date="2017-01-22">170</result>
    <result date="2017-01-23">278</result>
    <result date="2017-01-24">307</result>
    <result date="2017-01-25">258</result>
    <result date="2017-01-26">280</result>
    <result date="2017-01-27">355</result>
    <result date="2017-01-28">230</result>
    <result date="2017-01-29">175</result>
    <result date="2017-01-30">275</result>
    <result date="2017-01-31">303</result>
    <result date="2017-02-01">297</result>
    <result date="2017-02-02">301</result>
    <result date="2017-02-03">280</result>
    <result date="2017-02-04">221</result>
    <result date="2017-02-05">206</result>
    <result date="2017-02-06">363</result>
    <result date="2017-02-07">350</result>
    <result date="2017-02-08">336</result>
    <result date="2017-02-09">324</result>
    <result date="2017-02-10">295</result>
    <result date="2017-02-11">235</result>
    <result date="2017-02-12">226</result>
    <result date="2017-02-13">315</result>
    <result date="2017-02-14">310</result>
    <result date="2017-02-15">292</result>
    <result date="2017-02-16">272</result>
    <result date="2017-02-17">296</result>
    <result date="2017-02-18">230</result>
    <result date="2017-02-19">210</result>
    <result date="2017-02-20">329</result>
    <result date="2017-02-21">319</result>
    <result date="2017-02-22">299</result>
    <result date="2017-02-23">300</result>
    <result date="2017-02-24">314</result>
    <result date="2017-02-25">264</result>
    <result date="2017-02-26">204</result>
    <result date="2017-02-27">253</result>
    <result date="2017-02-28">246</result>
    <result date="2017-03-01">349</result>
    <result date="2017-03-02">337</result>
    <result date="2017-03-03">358</result>
    <result date="2017-03-04">249</result>
    <result date="2017-03-05">192</result>
    <result date="2017-03-06">368</result>
    <result date="2017-03-07">373</result>
    <result date="2017-03-08">332</result>
    <result date="2017-03-09">322</result>
    <result date="2017-03-10">347</result>
    <result date="2017-03-11">263</result>
    <result date="2017-03-12">233</result>
    <result date="2017-03-13">332</result>
    <result date="2017-03-14">325</result>
    <result date="2017-03-15">316</result>
    <result date="2017-03-16">319</result>
    <result date="2017-03-17">271</result>
    <result date="2017-03-18">251</result>
    <result date="2017-03-19">213</result>
    <result date="2017-03-20">328</result>
    <result date="2017-03-21">316</result>
    <result date="2017-03-22">306</result>
    <result date="2017-03-23">305</result>
    <result date="2017-03-24">251</result>
    <result date="2017-03-25">245</result>
    <result date="2017-03-26">215</result>
    <result date="2017-03-27">337</result>
    <result date="2017-03-28">322</result>
    <result date="2017-03-29">314</result>
    <result date="2017-03-30">303</result>
    <result date="2017-03-31">324</result>
    <result date="2017-04-01">243</result>
    <result date="2017-04-02">237</result>
    <result date="2017-04-03">328</result>
    <result date="2017-04-04">356</result>
    <result date="2017-04-05">319</result>
    <result date="2017-04-06">337</result>
    <result date="2017-04-07">354</result>
    <result date="2017-04-08">241</result>
    <result date="2017-04-09">264</result>
    <result date="2017-04-10">403</result>
    <result date="2017-04-11">392</result>
    <result date="2017-04-12">396</result>
    <result date="2017-04-13">322</result>
    <result date="2017-04-14">220</result>
    <result date="2017-04-15">240</result>
    <result date="2017-04-16">231</result>
    <result date="2017-04-17">345</result>
    <result date="2017-04-18">378</result>
    <result date="2017-04-19">358</result>
    <result date="2017-04-20">390</result>
    <result date="2017-04-21">351</result>
    <result date="2017-04-22">249</result>
    <result date="2017-04-23">214</result>
    <result date="2017-04-24">340</result>
    <result date="2017-04-25">364</result>
    <result date="2017-04-26">350</result>
    <result date="2017-04-27">345</result>
    <result date="2017-04-28">372</result>
    <result date="2017-04-29">290</result>
    <result date="2017-04-30">278</result>
    <result date="2017-05-01">263</result>
    <result date="2017-05-02">411</result>
    <result date="2017-05-03">428</result>
    <result date="2017-05-04">413</result>
    <result date="2017-05-05">368</result>
    <result date="2017-05-06">284</result>
    <result date="2017-05-07">252</result>
    <result date="2017-05-08">414</result>
    <result date="2017-05-09">358</result>
    <result date="2017-05-10">441</result>
    <result date="2017-05-11">396</result>
    <result date="2017-05-12">349</result>
    <result date="2017-05-13">265</result>
    <result date="2017-05-14">230</result>
    <result date="2017-05-15">360</result>
    <result date="2017-05-16">428</result>
    <result date="2017-05-17">417</result>
    <result date="2017-05-18">395</result>
    <result date="2017-05-19">349</result>
    <result date="2017-05-20">284</result>
    <result date="2017-05-21">239</result>
    <result date="2017-05-22">388</result>
    <result date="2017-05-23">357</result>
    <result date="2017-05-24">351</result>
    <result date="2017-05-25">289</result>
    <result date="2017-05-26">372</result>
    <result date="2017-05-27">283</result>
    <result date="2017-05-28">256</result>
    <result date="2017-05-29">365</result>
    <result date="2017-05-30">386</result>
    <result date="2017-05-31">369</result>
    <result date="2017-06-01">410</result>
    <result date="2017-06-02">389</result>
    <result date="2017-06-03">315</result>
    <result date="2017-06-04">238</result>
    <result date="2017-06-05">422</result>
    <result date="2017-06-06">419</result>
    <result date="2017-06-07">394</result>
    <result date="2017-06-08">389</result>
    <result date="2017-06-09">343</result>
    <result date="2017-06-10">256</result>
    <result date="2017-06-11">267</result>
    <result date="2017-06-12">392</result>
    <result date="2017-06-13">392</result>
    <result date="2017-06-14">362</result>
    <result date="2017-06-15" />
    <result date="2017-06-16" />
</results>

By week:

<results>
<result date="2016-10-10,2016-10-16"/>
<result date="2016-10-17,2016-10-23">173</result>
<result date="2016-10-24,2016-10-30">564</result>
<result date="2016-10-31,2016-11-06">583</result>
<result date="2016-11-07,2016-11-13">604</result>
<result date="2016-11-14,2016-11-20">644</result>
<result date="2016-11-21,2016-11-27">625</result>
<result date="2016-11-28,2016-12-04"/>
<result date="2016-12-05,2016-12-11">814</result>
<result date="2016-12-12,2016-12-18">216</result>
<result date="2016-12-19,2016-12-25">91</result>
<result date="2016-12-26,2017-01-01"/>
<result date="2017-01-02,2017-01-08">1167</result>
<result date="2017-01-09,2017-01-15">1170</result>
<result date="2017-01-16,2017-01-22">1183</result>
<result date="2017-01-23,2017-01-29">1250</result>
<result date="2017-01-30,2017-02-05"/>
<result date="2017-02-06,2017-02-12">1437</result>
<result date="2017-02-13,2017-02-19">1357</result>
<result date="2017-02-20,2017-02-26">1439</result>
<result date="2017-02-27,2017-03-05"/>
<result date="2017-03-06,2017-03-12">1526</result>
<result date="2017-03-13,2017-03-19">1425</result>
<result date="2017-03-20,2017-03-26">1382</result>
<result date="2017-03-27,2017-04-02"/>
<result date="2017-04-03,2017-04-09">1528</result>
<result date="2017-04-10,2017-04-16">1571</result>
<result date="2017-04-17,2017-04-23">1585</result>
<result date="2017-04-24,2017-04-30">1622</result>
<result date="2017-05-01,2017-05-07">1704</result>
<result date="2017-05-08,2017-05-14">1728</result>
<result date="2017-05-15,2017-05-21">1723</result>
<result date="2017-05-22,2017-05-28">1652</result>
<result date="2017-05-29,2017-06-04"/>
<result date="2017-06-05,2017-06-11">1754</result>
<result date="2017-06-12,2017-06-18">986</result>
</results>

Anyone?

Looking through the data again, it seems to be happening when the results range contains a month change:

2016-11-28,2016-12-04
2016-12-26,2017-01-01
2017-01-30,2017-02-05

Could you at least point me to the files responsible for this functionality?

Hi @elpendor

Thank you for the report! If this issue is confirmed, we will definitely address it.

Could you at least point me to the files responsible for this functionality?
The main code is in core/Archive* and it's spread over many files.

Do you maybe manage to reproduce this issue on https://demo.piwik.org , If so please post the URLs & steps to reproduce? that would be very helpful

I think I know what's the problem is (or at least the source). I've been importing old logs and running archive.php after each import.

I've removed the archive tables and reprocessed the logs via:

./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=YOUR_PIWIK_URL_HERE

And the problem seems to be gone now.

I'm guessing archive.php is not supposed to be used for old logs, just the cron jobs?

As long as your logs are imported in chronological order then it should work? becausewhen importing log data in the past, we automatically invalidate the archives for those days and periods and the core:archive re-processes them at the next run.