Performance: Improve how MB handles local cache

[pikurasa]

Description

Every time a user creates a new project, it -- a JSON string -- is stored into the browser's cache.

We've had performance issues upon loading Music Blocks (MB) for a while, and finally determined that loading takes longer the more projects a user has stored to their local browser cache.

In order to solve this problem, we need to look into how we handle local data, so that loading time can be improved.

Expected Behavior

We want MB to load as fast as possible. Preferably as fast as it does from a private tab.

How to Reproduce

One way you can test this is:

• Create a bunch of projects (or load them, either from the Planet or from the Examples folder), and reload MB
• Compare loading time against opening MB in a private tab (e.g. Incognito Mode); the loading time when in a private tab will be much quicker than from a tab where local cache is being read.

Console log Errors:

I'm not really sure what logs may be related to this issue. It would be helpful if someone with experience in performance could look into this.

Environment:

• Operating System: All
• Browser (if applicable): All
• Version of Software/Project: All

Note: Faster computers, especially ones with decent GPUs, in my testing, are less noticeably effected by this issue because the overall loading time is faster in either case. If you test on a faster computer, you should also try on a slower machine as well. And, in any case, always make sure to compare against a session that has many projects loaded (see above).

Checklist

• I have read and followed the project's code of conduct.
• I have searched for similar issues before creating this one.
• I have provided all the necessary information to understand and reproduce the issue.
• I am willing to contribute to the resolution of this issue.

---

Thank you for contributing to our project! We appreciate your help in improving it.

📚 See contributing instructions.

🙋🏾🙋🏼 Questions: Community Matrix Server.

[Commanderk3]

@pikurasa We can use APIs like IndexedDB or Dexie. js

[pikurasa]

@pikurasa We can use APIs like IndexedDB orDexie. js

Please tell us more about those options.

[Commanderk3]

I am researching on this. Till now what I have learned is that localStorage can't handle large amounts of data. This could be the reason for being slow.
IndexedDB on other hand is relatively faster when it comes to handle large data.
I will let you know once I find a optimal solution.

[manasbhalerao23]

@pikurasa we can also use origin private file system (OPFS) which is a highly optimal solution to this

[FirePheonix]

Hey @Commanderk3 , I think we already ARE using indexedDB.
You can view it your Applications tab by pressing CTRL + SHIFT + i on windows OS.

Image

I think the issue here is that the website is currently loading every single one of my projects fully from the database into memory, even if it only shows one project on the screen.
So when we have multiple projects, it LOADS EVERY SINGLE of them into the memory, causing this performance issue.

AND as mentioned by @pikurasa sir:
"Faster computers, especially ones with decent GPUs, in my testing, are less noticeably effected by this issue because the overall loading time is faster in either case."
In case of faster computers, we already have that much memory so it doesn't cause the issue.

A solution to this is implementing lazy loading into ProjectStorage.js . By doing this we ONLY allow meta data, like Name, data, image of project saved and NOT the entire DATA OF PROJECT itself.

I might actually be wrong with this, can you review this comment? I'm looking into it's implementation.

[Commanderk3]

@FirePheonix, what you said is correct. It loads more than one project. All these projects are shown in "My Projects" on the planet page.

[apsinghdev]

@pikurasa @walterbender I guess combining project suppression for larger projects with lazy loading and archiving older projects in localStorage would make it comparatively faster. Additionally, moving from localStorage to IndexedDB would require us to estimate how much storage is typically needed to run Music Blocks in the browser. I am working on it. will share more details on this.

[Commanderk3]

@apsinghdev We are not using IndexedDB right? But dev tools show MB is using IndexedDB.

[apsinghdev]

@Commanderk3 At some places we use indexedDB but in most instances, we use local storage.

[Aryan4884]

@apsinghdev why cant we totally switch to IndexedDB in place of local storage?

[apsinghdev]

@apsinghdev why cant we totally switch to IndexedDB in place of local storage?

First we have to know how much storage we need for faster experience in MB. If localstorage provides enough of that, we don't need to switch. If localstorage is not enough, we should.

[pikurasa]

One observation I have is that it seems to take (considerably) longer to run MB from the web than it does to run it locally.

Of course, it takes some time to download all the files from the web. However, I suspect that something else is compounding the issue. It may be worth also looking into the order that things are loaded to see if there is some sort of race condition or other sort of inefficiency.

[apsinghdev]

@pikurasa, I ran a bunch of tests using some tools to check the performance and learned that the real culprit is not the project storage but the way we make HTTP requests and fetch initial CSS and JavaScript.

The current fully loaded time of Music Blocks is 15.9 sec. It's not in the optimal range. It should be cut by at least 50%. Probably, I'll do that by next week.

[Commanderk3]

@apsinghdev That might be true. Because when I consoled out the time taken for retrieving data from local storage it only showed a few seconds (less than 3 s). I didn't give much thought to this because my device is powerful.

[apsinghdev]

@apsinghdev That might be true. Because when I consoled out the time taken for retrieving data from local storage it only showed a few seconds (less than 3 s). I didn't give much thought to this because my device is powerful.

IMO, console statements aren't very effective when calculating the time. I tried it too but didn't get accurate results.