due notes count when opening the app for first time each day

zeppo's Avatar

zeppo

13 Jun, 2019 05:20 PM

Just curious, usually when I open the app, I have studied all the due notes from the previous day, so initially the "Due for Study" field on the home screen will be blank. Then, after a minute or two, it will be populated by a number. Usually this number doesn't change. The process of generating due notes is complete. But occasionally the field will be populated first with one number, and then a minute or so later, without touching or navigating the app, it is replaced by a larger number. I just find it odd that the app is apparently sometimes populating that field with a number before the process of determining all notes Due for Study is complete.

This happened again today.

I've seen this many times and wondered if there could be bug like an additional file being created and run to generating due notes when there should just be one file, and also that this could be contributing to sync issues. Or a function to generate dues notes is somehow aborted causing the first number to appear, and then is resumed to generate the final number, but also causing a problem with long sync times as the app tries to sync after the first aborted attempt at the same time as the the function is resumed and generating new due notes, and then a second sync attempt is attempted while the first is still going on, etc etc.

Anyway, I've wondered if this is the sign of a small problem that might be the indication of a the cause of much bigger ones.

  1. Support Staff 1 Posted by drewmccormack on 14 Jun, 2019 07:28 AM

    drewmccormack's Avatar

    I don't think it is a problem. The scheduling can occur in the background when the app gets woken, so it's possible it already has a number.

    Drew

  2. drewmccormack closed this discussion on 14 Jun, 2019 07:28 AM.

  3. zeppo re-opened this discussion on 14 Jun, 2019 01:13 PM

  4. 2 Posted by zeppo on 14 Jun, 2019 01:13 PM

    zeppo's Avatar

    I don't have background refresh activated for the app.

    This was the first time the app had been opened that day (going back to the previous afternoon). and the Due for Study field was empty when the app was first opened. So it doesn't not appear that the app already had a number. About 30 seconds after opening the number 5 appears in the Due for Study field. Then about three minutes after that it is replaced with the number 79. It would appear that either scheduling was initiated and then aborted/disrupted, or there were two separate instances of scheduling going on, after which the Due for Study field was populated with the number five from the first attempt.

  5. drewmccormack closed this discussion on 17 Jun, 2019 08:53 AM.

  6. zeppo re-opened this discussion on 17 Jun, 2019 02:15 PM

  7. 3 Posted by zeppo on 17 Jun, 2019 02:15 PM

    zeppo's Avatar

    It also happened the following day in the same manner, but first showing 5 Due for Study. The one thing in common was they were from the same parent folder or "case" as you call them, but from different stacks in that folder. (That "case" is itself within another "case" of a group of cases). I don't know if this is because the process of searching for due notes always begins with the same "case", or if there is some reason this particular case has become a part of a second due notes process that is being run, and which perhaps is not always evident because of the smaller grouping of notes within it not always having anything due.

  8. 4 Posted by zeppo on 17 Jun, 2019 11:17 PM

    zeppo's Avatar

    I checked back on all the previous times I noted this problem and see that the cards initially scheduled notes do come from all different folders/cases. And while the first number of Due notes is usually small, there have been a couple of times where it was initially a rather large number (e.g., once it was 56 which changed to 84).

    Obviously there's no way I'll be able to figure out the cause, but I would consider what occurs per code once the Due for Study number posts-- if a sync is triggered, then this could be a problem where the app is trying to sync the first number which is shortly after changed and a second sync attempted while the first is still in progress, etc. Could this be the cause of a hanging sync attempt that is interminably long, the first attempt aborted by the second, but then the first (first number due) attempts to resync, aborting the second, etc etc. ? I mean, I have no idea how it works, I'm just imagining ways it could be a problem that is linked to the sync issues this app has.

    By the way, the "updating library" text in the pop up that used to appear seems to be gone since the last update. Pop up still occurs, but with just a blank white space now where the text used to be. Not sure if this was intentional.

  9. Support Staff 5 Posted by drewmccormack on 18 Jun, 2019 07:17 AM

    drewmccormack's Avatar

    To me it does sound like something that could arise due to sync, but I thought the sync was turned off now (?)

    Drew

  10. 6 Posted by zeppo on 18 Jun, 2019 02:23 PM

    zeppo's Avatar

    I turned sync off in February because of the unsolvable problems, mainly, but also to see what issues occur with each app when they are the only app loaded and used. Unless this is a relic that persists from when I was syncing, it isn't caused by sync. It seems likely that it occurs just as a product of the coding, but what I am wondering is if it might be the _cause_ of sync issues (ie, long sync times and/or crashes) that I had when sync was active prior to February. If you were to look at whatever code would cause the Due for Study field to populate this way, and then see if that same code would trigger a sync, then it is possible that there are conflicts causes when the two apps simultaneously trying to sync both values, an initial sync of the first Due for Study value and a second sync of the second Due for Study value. If that were the case, then one way to clean up the issue is to find out what is causing the field to populate twice. Is something being aborted? Is some file/function triggered to be run twice when it should only run once? Are there two instances/files etc that have been created that are triggering this to happen twice when it should only happen once? Then it would seem to me cleaning that up would help.

  11. drewmccormack closed this discussion on 19 Jun, 2019 06:08 AM.

  12. zeppo re-opened this discussion on 15 Jul, 2019 02:54 AM

  13. 7 Posted by zeppo on 15 Jul, 2019 02:54 AM

    zeppo's Avatar

    I believe it may be that taking a screenshot when I first open the app for the day, before it has scheduled (none showing due) may be what sometimes causes the due notes field to populate with a smaller number before settling on the larger final number. I stopped taking screenshots during this time and so far it has stopped doing this.

  14. drewmccormack closed this discussion on 16 Jul, 2019 03:13 AM.

  15. zeppo re-opened this discussion on 19 Jul, 2019 02:36 PM

  16. 8 Posted by zeppo on 19 Jul, 2019 02:36 PM

    zeppo's Avatar

    Spoke too soon. I stopped taking screenshots when opening, but on Wednesday when I turned on long-term scheduling for a stack (that had been loaded many days before) the issue returned yesterday morning when I opened the app for the first time that day. The length of time to complete the scheduling with the final number due also doubled from around 2 minutes to four. Happened again today, including lengthier scheduling time. So screenshots not involved.

  17. drewmccormack closed this discussion on 20 Jul, 2019 11:33 PM.

  18. zeppo re-opened this discussion on 20 Jan, 2020 06:26 PM

  19. 9 Posted by zeppo on 20 Jan, 2020 06:26 PM

    zeppo's Avatar

    By the way, having moved my data to the mac app, I can confirm this behavior occurs on the mac app, too (as with today, with no data in the cloud, sync off (and both apps having used the Reset feature), and the iOS app installed on a factory reset phone but not yet synced (sync off).

    For instance, opening the app for the first time today, starting with no due notes, after fourty-five seconds, the number 30 appeared in Due for Study. After ten more seconds it changed to 66. After another ten seconds, it changed to 83. After twenty-five more it changed to 86. After thirty more seconds, it changed to 94. I gave it another minute but it remained at 94, which was hopefully enough time for it to be the final number. It would be helpful if there was some pop up or other indicator that would let me know the scheduling task is complete so I don't interrupt it.

    Have you ever noticed this "count up" behavior in due notes being scheduled, or does the final number due when scheduling a new batch each day always appear at once, (without a series of numbers appearing as it counts up to the final amount?)

  20. Support Staff 10 Posted by drewmccormack on 20 Jan, 2020 06:32 PM

    drewmccormack's Avatar

    No, that sounds really odd. Scheduling should generally only take 10 seconds or so at the most.

    Maybe the issue is that you have a lot of scheduling history. If you want, we can do some profiling with your data to see if we can make it faster. If you want to do this, zip up your mac library in Containers > com.mentalfaculty.studies.mac and share it with me.

    Kind regards,
    Drew

  21. 11 Posted by zeppo on 20 Jan, 2020 06:38 PM

    zeppo's Avatar

    I'll get it over to you at some point. I have always wondered if whatever is causing this also manifests in my extremely long sync times (as in possibly essentially running mulitiple syncs corresponding to the multiple scheduling events, or syncs being triggered by one scheduling event (ie, in this case the first being 30 notes) but being interrupted by the next ("unexpected" ) scheduling event (in this case being 66) etc. I mean, obviously it could also affect in some other way I haven't thought of, but this has always been a behavior that I have seen.

  22. 12 Posted by zeppo on 20 Jan, 2020 08:53 PM

    zeppo's Avatar

    I'm having trouble uploading to dropbox, getting an error on some files, even when I zip the library using the mac compression feature. Switching from Safari to Google. Seems to make a difference, but is taking a while to upload.

    is [email blocked] the email address to use?

  23. 13 Posted by zeppo on 20 Jan, 2020 09:40 PM

    zeppo's Avatar

    I believe I have the mac app library zip file emailed to you or available for you on dropbox.

    But I may follow up with another one later that is after studying all due notes today, so that if you happen to test it tomorrow you may see the same scheduling I see.

    Zeppo

  24. 14 Posted by zeppo on 21 Jan, 2020 12:18 AM

    zeppo's Avatar

    I had a few confusing and botched attempts to get the more recent file to you that is from after studying all due notes from today. But I believe it is on dropbox for you now in slightly compressed form. So when you open on Jan 21 it should schedule the same number for you as it does for me. The only question is whether it will exhibit this odd behavior. It usually does it more often than not.

  25. Support Staff 15 Posted by drewmccormack on 21 Jan, 2020 12:11 PM

    drewmccormack's Avatar

    We did a test of scheduling. On our admittedly powerful Macs, it took about 15 seconds. That's probably to be expected for a large library, because it has to effectively load and process every single note schedule.

    What we did notice is that you have a few smart stacks. Smart stacks can slow things down quite a lot, because they are effectively searches, and whenever something changes, they have to be reprocessed. You might find the app gets quite a bit more snappy if you remove those.

    Kind regards,
    Drew

  26. 16 Posted by zeppo on 21 Jan, 2020 02:42 PM

    zeppo's Avatar

    I don't mind the amount of time it takes to schedule, though I can eliminate some smart stacks. It is the amount of time it takes to sync that is a problem for me. So I was wondering if you see a "count up" to the final due notes number, or does the final number just pop in without any other numbers appearing before it? Not an actual count up of 1,2,3,4..., etc , but of a few interim numbers, today's being 33, 52, 79, then the final count of 80, in my case about 35 seconds apart, but with your perhaps a couple of seconds.

  27. 17 Posted by zeppo on 21 Jan, 2020 04:16 PM

    zeppo's Avatar

    PS: Actually I was thinking I had more smart stacks than I do. I have used others in the past but must have deleted them at some point. The two I do have now I just created in the past couple of days and have had no bearing on the apps behavior that I have described. Nothing has changed in how the app behaves for me. Maybe they will add to the time needed to sync, but I haven't turned that on yet since creating these smart stacks. I only need the smart stack in order to sort and tag certain cards that, though graded wrong on last study, were nevertheless given a next due date of over a year because of the bug in your app that you've finally corrected. Now that I have moved the data to the mac app, I want to sort and restudy these notes to correct their study scheduling. Then I can delete the smart stack.

    But let me know if you see the "count up" behavior in my stack, and in your own test stacks that have a large number of cards like mine.

  28. Support Staff 18 Posted by Vlad Borovtsov on 21 Jan, 2020 05:23 PM

    Vlad Borovtsov's Avatar

    We did another test with your library and we can see the issue, it is counting, exactly as you described it. We will look more into the issue to see if its possible to get a better performance with heavy smart stacks.

  29. 19 Posted by zeppo on 21 Jan, 2020 05:32 PM

    zeppo's Avatar

    Thanks.

    To be clear though, this "counting" behavior or any of the problems I've been having has nothing to do with smart stacks (unless there remains some unseen relics of every smart stack a user has created in the past even after deleting them.) The two I have were only recently created.

    EDIT : "... has _nothing_ to do with..."

  30. Support Staff 20 Posted by drewmccormack on 22 Jan, 2020 07:13 AM

    drewmccormack's Avatar

    OK, that is more clear then. Vlad is investigating more now.

  31. 21 Posted by zeppo on 23 Jan, 2020 07:18 PM

    zeppo's Avatar

    drew and vlad:

    using the library I sent you, sort All Notes for "medias"

    You will see a card with first facet "medias de red"

    Click on card and click on info.

    You will see
    Last Studied: August 22, 2018
    Next Scheduled: April 6 2020
    Recent Run: 1 incorrect
    year and month retentions both 50%

    Now turn off facet 2 as a prompt and click done.
    Click on card and click on info.

    You will now see
    Last Studied: August 22, 2018
    Next Scheduled: April 6 2020
    Recent Run: 1 incorrect
    year and month retentions both 90%

    Now turn back on facet 2 as a prompt.
    Turn off facet 1 as a prompt and click done.

    Click on card and click on info.

    You will now see
    Last Studied: August 22, 2018
    Next Scheduled: August 23, 2018
    Recent Run: 1 incorrect
    year and month retentions both 0%

    Now turn back on facet 1 as a prompt and click done.

    Click on card and click on info.
    Last Studied: August 22, 2018
    Next Scheduled: August 23, 2018
    Recent Run: 1 incorrect
    year and month retentions both 50%

    This note is an example of what I'm trying to clean up now among my notes. It was caused by a bug I pointed out to you that you have since corrected. ( https://mentalfaculty.tenderapp.com/discussions/suggestions/10614-gradingscheduling-bug )
    But the problem is that when it comes to notes where both facets can be a prompt, I cannot simple sort to find these notes by due date. They like hidden easter eggs I can only find by using the steps I just gave you. That is why I created the "ALL" smart stack so that I can view and sort notes without being sorted by stacks as the All Notes.

    Now sort All Notes for "lejos".
    You will see a card with first facet "lejos (adv)"

    Click on card and click on info.

    You will see
    Last Studied: July 5, 2019
    Next Scheduled: September 10, 2029
    Recent Run: 1 Correct
    Year Retention 0%
    Month Retention 20%

    I'm going to have to sort out cards like this one that are being scheduled for over a decade out, despite the fact that my one month retention expectation is 20%. That's why I'm asking about parameters, as I figure out what I'm up against in terms of where this app is falling short and my strategies to correct it.

    I'm also pointing this out because maybe there is something here that is involved with my data that is involved with the sync issues I have had. I'll also point out that I have notes going back to 2012 with the original Mental Case 1, so maybe there is something in they way notes were migrated into Mental Case 2 and Studies along the way that are involved. I'm only just starting to take on the task of "catching up" on notes like these. That's why I've switched over to the mac app for now (I don't use sync between mac app and iOS app because I've given up on that, except in this case where I had to move data over to mac.)

  32. Support Staff 22 Posted by drewmccormack on 24 Jan, 2020 07:21 AM

    drewmccormack's Avatar

    We'll see if we can look into this at some point. Busy with disappearing images at the moment.

  33. 23 Posted by zeppo on 23 Feb, 2020 06:12 PM

    zeppo's Avatar

    I don't know if you solved the disappearing images problem in the last update and have had the chance to look at this yet. But its particularly interesting to me how in the "count up", the last update in the count, and sometimes the last two, will after increasing the displayed count for a split second flicker back to the previous count that was displayed . One day this week it was at its most peculiar:

    8:26:58 am clicked to open the mac app --
    8:27:00 app displays on onscreen
    8:27:40 -- 40 seconds later 29 notes show due
    8:27:49 -- 9 seconds later 74 due
    8:27:59 -- 10 seconds 99 show due, the immediately flicker back to show 74 again...
    8:28:00 -- ... but returns 1 sec later with 117 due ( not 99 ), and flickers back again not to 99, but to 74 again
    8:28:01 -- shows 117 again, which remains the final count due

    I don't have sync activated and should have all cloud data removed from resets, and internet not connected when I open app.

    Seems odd to me, like there are multiple scheduling events being resolved.

  34. Support Staff 24 Posted by Vlad Borovtsov on 24 Feb, 2020 02:01 AM

    Vlad Borovtsov's Avatar

    Is this counting thing happening all the time?

  35. Support Staff 25 Posted by drewmccormack on 24 Feb, 2020 09:15 AM

    drewmccormack's Avatar

    It is not necessarily unexpected that you would see the count update over time for a large library. The scheduling is done per stack, because it can take a little while. So as new stacks get scheduled, you may see the count rise, and eventually settle. This is as designed.

    It is a bit odd that the number goes down, but that may just be due to some caching artefact in memory.

  36. 26 Posted by zeppo on 24 Feb, 2020 04:24 PM

    zeppo's Avatar

    Yes, Vlad, every time.

    Today:
    clicked to open app , appears onscreen at 10:36:52
    45 (10:37:37 ) seconds due count shows at 71
    10 seconds later count is 121
    7 seconds later count is 146, flickers back to 121, then to 146 (10:37:52)
    27 seconds later (10:38:19) count changes to 178 ,flickers back 1 sec to 146, then up to 192, flickers back to 146, then back to 192 (10:38:21 ) which is the final number due

  37. 27 Posted by zeppo on 24 Feb, 2020 04:27 PM

    zeppo's Avatar

    Seems odd that a cache would hold on to 146, but not 178.

  38. 28 Posted by zeppo on 24 Feb, 2020 05:10 PM

    zeppo's Avatar

    Actually, now that I think of it, Vlad, it hasn't always done it every time. There was a stretch where sometimes it would do it and sometimes it wouldn't. Maybe perhaps a factor of how many end up being due, I don't know. I've added activated long term scheduling for some more notes recently, so I'm back to getting due note counts over 100 more regularly.

  39. Support Staff 29 Posted by Vlad Borovtsov on 25 Feb, 2020 04:19 AM

    Vlad Borovtsov's Avatar

    Hm, ok.
    Do you have a lot of notes in DB? Does it takes long for Studies to process all due?

  40. 30 Posted by zeppo on 25 Feb, 2020 03:48 PM

    zeppo's Avatar

    If you look back in this thread to Jan 21, you will recall that I actually sent you my database and you saw the same "counting up" behavior. But you had to put it aside to work on the image deletion problem. I have about 9500 notes with long-term scheduling. It typically takes about a minute and a half. I included time stamps for a couple of days in my last few posts. Here is today's:

    clicked to open app 10:20:59 , appears onscreen at 10:21:01
    44 (10:21:45 ) seconds due count shows at 50
    10 seconds later count is 123
    17 seconds later (10:22:12 ) count is 155
    20 seconds later (10:22:29) count changes to 163 ,flickers back 1 sec to 155, then up to 186 (10:22:31), which is the final number due

    The flickering back and forth of the due count is fast and sometimes tricky to catch. I was manually advancing the frames of video I took. As of today, I started using slow motion to make it easier. The initial 40-45 seconds, followed by a 9-10 second gap to the next "count up" seems to be consistent. After that its more random as to when the due count changes.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac