due notes count when opening the app for first time each day
Just curious, usually when I open the app, I have studied all the due notes from the previous day, so initially the "Due for Study" field on the home screen will be blank. Then, after a minute or two, it will be populated by a number. Usually this number doesn't change. The process of generating due notes is complete. But occasionally the field will be populated first with one number, and then a minute or so later, without touching or navigating the app, it is replaced by a larger number. I just find it odd that the app is apparently sometimes populating that field with a number before the process of determining all notes Due for Study is complete.
This happened again today.
I've seen this many times and wondered if there could be bug like an additional file being created and run to generating due notes when there should just be one file, and also that this could be contributing to sync issues. Or a function to generate dues notes is somehow aborted causing the first number to appear, and then is resumed to generate the final number, but also causing a problem with long sync times as the app tries to sync after the first aborted attempt at the same time as the the function is resumed and generating new due notes, and then a second sync attempt is attempted while the first is still going on, etc etc.
Anyway, I've wondered if this is the sign of a small problem that might be the indication of a the cause of much bigger ones.
Showing page 2 out of 2. View the first page
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
31 Posted by zeppo on 25 Feb, 2020 03:49 PM
PS: My notes are strictly text-- no images or audio.
Support Staff 32 Posted by Vlad Borovtsov on 26 Feb, 2020 05:23 AM
Well, actually images or audio doesnt add any complexity in this case. The key thing is that you have a lot. I'll look what i can do, maybe there is a way to make logic run smoothly.
Vlad Borovtsov closed this discussion on 26 Feb, 2020 05:23 AM.
zeppo re-opened this discussion on 26 Feb, 2020 04:45 PM
33 Posted by zeppo on 26 Feb, 2020 04:45 PM
Thanks. If you have any questions let me know. When you look into the unexpected cause of these displayed "counts" (perhaps because an assumption was made that was too small about how large a library would be or how many would come due in a day?), you might see if for a similar reason this is triggering multiple requests to sync and perhaps even queuing them all so that where most people get one group of data that needs to be synced, I'm having to have multiple updates synced, or some kind of log jam is occuring that is causing start and stops of syncs, or multiple sets of data needing to be resolved, etc etc. I'm sure it is probably not intended that this would happen with sync, but it also seems to me that this count up was not intended-- so even though it might not be expected, worth taking a closer look at whatever unintended consequences my library is triggering.
34 Posted by zeppo on 26 Feb, 2020 04:45 PM
ps : for what it is worth, here is today's:
clicked to open app (11:17:50)
appears onscreen at 11:17:52
39 seconds later (11:18:31 ) due count shows at 76
10 seconds later count is 134 (11:18:41)
9 seconds later (11:18:50) count is 205, flickers back to 134, then up to 228 (10:18:51), then back to 134, then 228, (10:18:52), which is the final number due
Support Staff 35 Posted by Vlad Borovtsov on 26 Feb, 2020 04:48 PM
Hi,
it seems i accidentally caught your counting issue today.
36 Posted by zeppo on 26 Feb, 2020 05:44 PM
Using my database?
Support Staff 37 Posted by Vlad Borovtsov on 26 Feb, 2020 05:48 PM
No, that counting is happening with any huge DB. My current testing one contains about 20k notes.
38 Posted by zeppo on 10 Mar, 2020 06:00 PM
Hi, Vlad
Just curious if you've looked at the code and learned anything more about this.
thanks
On Wednesday, February 26, 2020, 12:48:08 PM EST, Vlad Borovtsov <[email blocked]> wrote:
|
Support Staff 39 Posted by Vlad Borovtsov on 10 Mar, 2020 06:06 PM
Hey,
yes, i think i've found it.
But the problem is that i didn't come to solution i like.
The most obvious fix solves the counting problem, but then counts starts flickering during updates. Due to complex app architecture, nice solution will take time :)
40 Posted by zeppo on 10 Mar, 2020 08:25 PM
Why does it sometimes flash back to a lower count more than once?
And when the program requests a count to display is it at the same time also triggering some instructions that will affect how long sync will take, or, for instance, multiple saved states that each one of which now need to be merged and/or synced? I don't know how it works. I'm just wondering since this is an unintended event that happens with larger libraries, if maybe there are other unintended events triggered at the same time that explain why my sync times grow to be so ridiculously long (like triggering syncs that are interrupted, or multiple states that are requested to be synced instead of just one.)
Support Staff 41 Posted by Vlad Borovtsov on 11 Mar, 2020 06:01 AM
It flashes back to a lower count bcs of cache.
If i remove the cache, it will start showing a correct value, but there will be short moments when count is just missing.
this is happening bcs stacks list and "due notes update" routine running in a separate contexts for better performance. Stacks list notices a change in DB and trying to reload, but counting of exact count due notes is kind of heavy operation, to it gets placed into queue, while stack list takes cached value immediately, just to keep displaying something.
As for the sync, of course, due states update doing some modification in DB and will trigger sync. But solving that counting UI issue won't affect sync performance at all.
42 Posted by zeppo on 11 Mar, 2020 01:59 PM
Yeah, my thought wasn't that solving the *display* of "count up" would *solve* some sync issue as well, but, rather, would *alert* you to some *corresponding* sync issue that would have to be solved independently. For instance, if an unintended command to display the count is taking place the where the assumption in the code was this would be taking place at the *end* the completion of calculation of total due notes. In that scenerio, I wondered if maybe such an unintended command to display was also accompanied by another order to save a state that would then be requested to be synced (again with operating under the false assumption that final determination of due notes was complete, when in fact it was not yet complete). In that case, just as there were multiple commands to display the due notes counts where only one was intended to be displayed at the end, there might also be multiple commands regarding sync where only one was intended at the end -- so that you have interrupted start and stops of requests to sync, or you have much larger amount of information requested to be synced (multiple states with increasingly updated information) rather than just a single state with the final due count at the end of the process. Again, I am thinking of possibilities without knowing the code, just possible logic that I could imagine happening in a program.
At the end, even if this were true, one could say that this is not important if you don't care about time and how much data is synced. But for me it sync time has always been a big problem (30 minutes to over an hour), and is sometimes accompanied by more than just a couple of "sync progress" bars that have made me wonder in the past just how many times the app is trying to sync, or if the sync is stopping and starting again, etc.
Again, obviously this would be much less of a blind process for me if I knew the structure and the code, so I apologize for that, but as it stands, I'm curious if something some *corresponding* (not direct) connection to sync might be going on here. I figure if the odds are against it its still a stone worth turning over given these sync times. Anyway, I appreciate that you are taking a look at this, Vlad. Thanks.
Support Staff 43 Posted by Vlad Borovtsov on 11 Mar, 2020 05:38 PM
Yeah, i'll check it. Honestly, there are a lot of sync triggers to make UX seamless as much as possible between the desktop and mobile version.
44 Posted by zeppo on 07 Apr, 2020 03:47 PM
Odd event regarding this happened Mar 28. For some reason the mac app took a much longer time than usual scheduling due notes:
Mar 28 - LONG TIME SCHEDULING FOLLOWED LATER BY CRASH
11:08:50 click to open
11:08:54 app appears onscreen
11:10:11 30 due
11:11:34 68 due
11:14:47 124 due
11:15:45 68 due
11:15:47 129 due
11:15:47 68 due
11:15:49 129 final due count
The normal amount of time it takes is about one and a half minutes. Pretty consistently it will show the first due count at about 45 seconds followed by the next count up about 10 seconds later. After those first two counts, it is more variable on when and how many more updates to the count appears. But as you see on Mar 28, the whole process took about seven minutes. Because it took so long, I didn't really trust that it was complete at 129 due and just left the app open. When I remembered and went back to it, it crashed around 11:21:00 A crash report was sent to Apple and I believe one was sent to you because of having previously selected to automatically send them to you. But I wasn't prompted with the chance to add any note, so I'm not sure. At any rate, I saved the report
I have just been using my iphone to take slow-mo video of the count and time, and was almost that very day going to quit doing it, figuring I pretty much had any useful data I was going to get from it, with the app pretty much having the same routine. For some reason I decided to do one more, and lo and behold, it turned out this odd event.
So you might check for my crash report.
But also, is there a way you could at least change the font of the final due number so I would have some indication of when the processing of the count is complete? Perhaps in bold like the "All Notes" count. Or maybe have all these intermediate due note counts in black and the final in blue. Or maybe the final due in red? (Different colors is easier to distinguish than bold vs regular) . It would help me avoid shutting down the app in the middle of the process, thinking it was complete when it wasn't. It would also let me know whether the app is trying to sync before the due notes count is complete. I don't have sync enabled, or even have my iphone app installed right now. But when I was using the iphone and tried to sync, this was always a concern in regards to the cause of the ridiculously long sync times. I would even try not having my phone connected to the internet when opening for the first time each day and wait until the scheduling was complete to connect. But in cases like this, after a couple of minutes, I am likely to assume the process is complete after a couple of minutes, when in reality it has 5 more minutes to go.
Support Staff 45 Posted by Vlad Borovtsov on 08 Apr, 2020 06:32 AM
Hello,
the scheduling usually should not take long, but its strongly depends on the size of your library. The study metadata is also lightweight, so it should not affect sync speed so much, but it also depends on size of your library and study rules assigned to stacks.
If you want us to investigate your particular case, can you share your library with us?
As for background schedule indication - i'll think about it.
46 Posted by zeppo on 08 Apr, 2020 02:42 PM
If you can make a change so we know when scheduling is complete, then the use can relate some more info to you as to when in the process a problem is occuring.
You already have my library. As for that particular day where it went from a "normal" (for me) scheduling time of 1.5 minutes to 7 minutes, I don't have a saved copy of the library for that day. But you might glean something from the crash report that I believe was sent to you, or I can email it to you.
the summary was:
Crashed Thread: 11 Dispatch queue: SQLQueue 0x7f94074124b0 for SyncedData.sqlite
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Application Specific Information:
objc[756]: __NSArrayI object 0x60400058ad10 overreleased while already deallocating; break on objc_overrelease_during_dealloc_error to debug
objc[756]: __NSArrayI object 0x6000003bd960 overreleased while already deallocating; break on objc_overrelease_during_dealloc_error to debug
abort() called
*** error for object 0x6040006a64e0: Invalid pointer dequeued from free list
Support Staff 47 Posted by Vlad Borovtsov on 08 Apr, 2020 02:50 PM
Hmm, yes, the that part of log says something. But such type of issues usually very hard to debug.
as for indication - i'll try to do something in the next release if possible.
48 Posted by zeppo on 08 Apr, 2020 03:05 PM
Thanks. Given that a fix to problems with lengthy scheduling and sync times may not be available to those of us with large and old libraries, it might at least help us to use the app in a way that reduces complications and also provide a little more useful info to you when problems occur.
drewmccormack closed this discussion on 30 Apr, 2020 07:38 AM.
zeppo re-opened this discussion on 03 May, 2020 03:50 PM
49 Posted by zeppo on 03 May, 2020 03:50 PM
Interesting that this count up behavior always happens on the mac app, but on the iphone app, sometimes it doesn't happen. I was reminded of this when I reinstalled the iphone app and switched back to using it. This is true with whether sync is turned on or not. I initially synced the data from the mac app to the iphone and used sync daily for a about a week, and sometimes it occurred and sometimes not. Since then I reset the cloud sync on both devices and haven't turned it back on, and still, sometimes the count up occurs and occasionally does not. This was true late last year when I was using the iphone app exclusively before switching to the mac app. So though the programming is supposed to be the same with both apps, there is this difference.
Support Staff 50 Posted by drewmccormack on 03 May, 2020 04:50 PM
The code is exactly the same on each device. Most likely either you are seeing a case where sync has already done the scheduling, so it doesn’t happen again, or it just happens to go faster for some reason, with less refreshing.
Note that the iOS app can open in the background, so may do the scheduling before you open the app.
Kind regards,
Drew
51 Posted by zeppo on 03 May, 2020 10:29 PM
I turned off sync, and I've always turned off background app refresh for the app.
drewmccormack closed this discussion on 05 May, 2020 08:06 AM.
zeppo re-opened this discussion on 11 May, 2020 02:45 PM
52 Posted by zeppo on 11 May, 2020 02:45 PM
What does the 33kb Data.sqlite-shm do?
When you open the iOS app for the first time on a given day, if this file updates, the "count up" issue does not occur. Otherwise, you get the "count up".
Support Staff 53 Posted by drewmccormack on 11 May, 2020 03:02 PM
It’s part of the database architecture. You shouldn’t mess with it. It is not specific to Studies, just to the database itself.
Drew
54 Posted by zeppo on 11 May, 2020 10:38 PM
Do you think sync attempts would be smoother when this file updates when the app opens for the first time of a given day-- that is, would the update of this file facilitate a less problematic sync? The fact that its update results in scheduling that doesn't have these odd "count ups" side effects makes me wonder if there are other side effects that might be avoided when it updates vs not updating.
Support Staff 55 Posted by drewmccormack on 12 May, 2020 07:05 AM
That file is simply used for any change to the database at all. It is not specific to scheduling, though it will change with scheduling because that is an update to the database. The race that causes the odd jumps is in the user interface layer, and is not related to the database.
Drew
drewmccormack closed this discussion on 12 May, 2020 07:05 AM.
zeppo re-opened this discussion on 12 May, 2020 03:48 PM
56 Posted by zeppo on 12 May, 2020 03:48 PM
On the occasion where the "count up" occurs, the file doesn't change, even though the due notes number for the day has been generated (a process that typically takes about a minute fifteen seconds to two minutes. I would presume that during this time it is updating which notes are due for the day and would be considered scheduling.
Also, today I notice that even after reopening the app and beginning a new study session of "All Due Notes" for the day, after studying around 12 notes and closing the app, the file still hasn't updated. I guess a change in study "history" is not something that triggers an update in this file.
Its interesting that some days this file updates upon first open of the app for the day, and sometimes it does not. Maybe has to do how long it has been since I completed the previous day's study session?
Support Staff 57 Posted by drewmccormack on 12 May, 2020 04:15 PM
I’m not sure. I could actually be confusing that file with the WAL file. In any case, it is well outside our expertise as app developers: it is a database level detail. We have no control over it.