Folder navigation is still slow

[turboderp]

This has been an issue before, apparently, but navigating folders is still really slow for me in 2.8.7.

I have a folder with 92k images. Opening this folder in Nemo takes around 13 seconds. CPU usage is high while it's opening.

For reference, ls > /dev/null on the folder takes around 100 ms. ls -l /dev/null takes roughly 2 seconds.

What's more, once the folder is showing, a single CPU core jumps to 100% for about 20 seconds. Don't know what's going on here but during this time Nemo can't navigate away from the folder. It's more or less frozen except you can scroll up and down the folder pane. I have thumbnails disabled, for what it's worth.

If I try to open one of the images while Nemo is taking this little break, it's unresponsive at first, but clicking many times eventually brings up several windows saying it's opening the image and I can click a button to cancel. The button does nothing, and after this Nemo seems to become unstable.

It doesn't seem to be related to images, either. I have another folder with about 11k documents (PDF and MSWord). This takes about 1.5 seconds to load, and afterwards there's about a 2.5 second period of 100% CPU usage during which I can't navigate away from the folder.

Navigating smaller folders is also slow. Folders with a few dozen files takes maybe a quarter of a second to load, which is perhaps not much, but it feels very sluggish compared to other file managers that load up small folders instantly.

[howdev]

Yes it is very slow on 2.8.7 and no one else has noticed this? This is horrible. Needs urgent fix.
Very slow even the folder has one or two files. When you open or switch folder you can see blank page before icons appears. You do not see this kind of lag happening with Thunar

It is not the plugins. I disable all the nemo plugins and makes no improvement

[kmkom]

I can confirm the low speed in Nemo. :-(
My workaround: Install and use Dolphin.

[turboderp]

My workaround is Thunar. I would keep fighting with Nemo (or even contribute code, though I'm still getting familiar with Linux development), but the project seems to have been abandoned. So I don't really see the point.

Or am I missing something? Are there actually developers working on Nemo?

[clefebvre]

The latest commit is more recent than the comment you replied to.

Please stick to technical aspects.

[clefebvre]

Try and benchmark Caja and Nautilus against Nemo on a set of sample directories. They all rely on gvfs so we need to see what's at play here, where's the bottleneck and if it's recent.

[turboderp]

To be fair the latest commit is really minor, and other than that it's been sparse in the past few months. And feedback on bug reports and feature requests is just as sparse. Again I could be missing something obvious, like maybe all the active development goes on somewhere else or whatever. I'm new, what do I know.

Soon as I get home tonight I'll run some tests vs Caja and Nautilus. Maybe some older versions of Nemo, too.

I am wondering, though. I've seen it mentioned here a number of times and on various forums, and there's never anyone replying that it works fine for them. Sort of goes to whether this is a bug, a compatibility issue or an optimization issue. Can I take it this problem isn't showing up on your setup?

[mtwebster]

For what it's worth, nemo extensions can have a substantial impact on speed versus nemo running with no extensions enabled - some will have more effect than others. This was part of the reason we added a plugin manager, to make it easier to enable/disable these at need, rather than always having them running (and consuming resources.) Am I pleased with the speed of nemo loading folders? I'll admit it - no - at least not with heavily populated folders - but I don't find it poor enough that it becomes a high priority to 'fix' right now.. but it is kept in the back of my mind when poring through the source - what can be optimized, re-jiggered, that might improve things, how big an impact will this change I'm making have, etc... there are a lot of things at play here. I also think it's a bit unfair to compare Nemo to Nautilus - nautilus as it exists now is a heck of a lot less useful than nemo, a lot has been ripped out of it over the past few years, naturally this will impact speed positively for them.

[turboderp]

Well, don't have a lot of time to experiment today as it turns out, but a quick test shows Caja and Nautilus perform roughly (or exactly) the same as Nemo. The behaviour is the same as described above, on the same folders.

If it's worth mentioning, this is on a 4GHz i7-6700K with 32 GB of RAM and a fast SSD.

Disabling all plugins and extensions didn't change anything.

Idunno if I'd ever call the performance acceptable. I personally don't think 92k files is an edge case, let alone the 11k folder which still takes 4 seconds to fully load up (vs slightly less than a second in Thunar, about 300 ms with ls -l ). I could be alone in thinking that, of course.

But there must be something else going on when loading a folder with a single file still takes that quarter of a second. I.e. there's some constant overhead which must be quite significant to tie up a 4 GHz CPU for any noticeable amount of time. I'm also wondering what goes on after the file list is shown. It only takes 1.5 seconds to get the list of 11k files (filename, size and date) in list view, and after that I can scroll through the list but I can't do anything else (navigate away from the folder, open a file, etc.) until this mysterious background task completes.

Oh. I'll leave that there, but I actually just figured out what that "background task" is. Because I can't actually scroll to the very bottom of the list because it's still populating. Hm. If I knew anything about GTK I might have something constructive to say about that. :) Like, why not populate the whole list in one go, but I honestly don't know if it's as simple as that.

[howdev]

Caja is not much different to Nemo. Caja is only very slightly faster navigating folders. Still takes significant amount of time loading a few contents. You will see a real difference when using Thunar. Thunar is still much faster than caja when both uses gtk2

[howdev]

It is not the extensions and previews. Nemo is the same when all extensions disabled. Thunar also has files previews. Should check the gtk3 icon view or content loading code

[howdev]

No one knows the technical aspects here, since none knows the cause as yet.

[kmkom]

I get this results on my PC (i5, Samsung-SSD):
I created a folder with about 20k of small jpg-files on the SSD. When i open this folder in Nemo 2.8.7 Nemo need about 65 seconds CPU time until it is ready (of course the files in the view appears after few seconds in Nemo). See screen shot below.
Opening the same folder in Dolphin 4.14.2, Dolphin needs about 1.5 - 2 seconds.

[clefebvre]

That would indicate the bottleneck is in either something all 3 of them do (which is possible since Nautilus, Caja and Nemo all share the same family tree), or in something all 3 of them use (the first dependency I'd suspect would be gvfs).

If it's down to gfvs it's outside of our control... but... we don't necessarily have to use gvfs for everything. We might be able to list directories without it for instance.. or we could be smart and list both with and without (list quickly without first to get the main data we need to start rendering the view), and list with (takes longer and we use the data to refine the view)... so for instance, say we get the number and types of items quickly but we rely on gvfs for emblems, custom icons, date info... we're able to create a view with standard icons very fast, and we update icons and add date info as we gather it from gvfs.

If it's not in gvfs and the bottleneck is internal, same idea. We separate data into fast-to-get and slow-to-get (I know the last modified info for instance has a retrieval cost) and we don't wait to get the slow data to start rendering the view.

Note: In case this sounds easy to some people, it shouldn't. First we need to positively identify what takes time, then we need to think of alternatives and how to implement them.

[turboderp]

I tried this on the folder full of images:

$ time ls -l > /dev/null

real    0m2.038s
user    0m0.232s
sys 0m0.628s

$ time gvfs-ls -a time::modified > /dev/null

real    0m1.998s
user    0m0.612s
sys 0m0.416s

$ time gvfs-ls -a standard,etag,id,access,time,unix,owner > /dev/null

real    0m8.336s
user    0m5.144s
sys 0m1.500s

Dunno if that helps.

I might add that if I dump the output of the last command to a file, it's about 90 MB of data, so maybe the extra 6.3 is spent just formatting it all. Still far from the 30+ seconds it takes Nemo to show the same folder (with only name, size and date-modified columns enabled), so I'm thinking gvfs can't be the only bottleneck, if it's a bottleneck at all.

[clefebvre]

Thanks that helps.

[howdev]

If gvfs is suspected to be slow then Thunar does not use gvfs?
can someone explain why not just check the code of Thunar of loading folders? That is what I would do but C or C++ is not my language. Why not do something simple when an example already exist.

[turboderp]

I don't think the problem is gvfs. gvfs-ls uses gvfs to list the folder, and it's as fast as ls as far as I can tell.

I personally suspect it's a GTK thing, and that it simply takes too long to populate the list. If you open a large folder and scroll a bit down the list right after it first appears, you'll be able to see the scrollbar moving as Nemo is apparently adding files one by one. I could say something more constructive if I knew much about GTK, but it's not an uncommon problem for list views in various other frameworks. That is, whenever you add elements to any sort of view, a little bit of layouting will be done so the view can look correct with its current number of elements. Geometry of the scrollbar will be recalculated, system will work out what stuff needs to be redrawn, and so on. It's normally not a big deal, but when you add tens of thousands of elements one by one it can add up. In the worst case it might be performing some redundant graphical operation over and over again, maybe just redrawing the scroillbar but waiting for vsync every time to prevent flicker, and also guaranteeing that it takes at least 1/60 s.

What's needed in any case is either to suspend the view until the list is complete, or to use a method that's designed to populate the whole thing in one go.

[howdev]

that is what I mentioned before check the icon view in gtk something is wrong there that it takes so long. I am talking about just a few folders not a big list of folder like you said.
Thunar's performance is matched to what the developer designed it to do. It specifically says this on the Thunar web page. Nemo has a design problem and that may come from Nautilus.

[efournival]

I can confirm this on Arch Linux (Cinnamon 3.0.1): with Nemo, changing directory takes 0.25 - 0.5 second while Thunar takes < 0.1 s (difficult to see).
My system is installed on an entry level SSD.
Before blaming GVFS or GTK, I think it would be interesting to make some sort of benchmark on the directory changing logic. I will try to do it as I'm fine with C and C++ but unfortunately not with GTK nor Nemo architecture. :(

EDIT: I just checked with Nautilus and it is nearly as fast as Thunar (while still being slower than it, but it's very acceptable) so GVFS is not the cause of the issue in my opinion.

[bigbruno]

Nemo 3.0.6 is slow too.

[howdev]

yes nemo 3.06 is still slow, there is no improvement from previous version. Takes especially long time to load folders into view, take /usr/bin for example. What is taking up the processing to display the files into view?

[ghost]

I was using the 3rd-party icon theme called 'Paper'. Upon switching icon theme to Mint-Y (sadly still only in green at the moment), or a Mint-X theme, I found that Nemo sped up considerably.

[UgoYak]

I noticed a clear improvement in navigation between folders by disabling the extension "ChangeColorFolder+NemoPython" (Nemo 3.2.2 on Cinnamon 3.2.7). I don't know if it's the same problem but, to me make a upturn.

[ghost]

@UgoYak : Hmm. I can't remember what that extension even does. I too might disable it.

[deutrino]

This is still a problem for me in 3.2.2, and not always with folders with a lot of contents at the folder level. I think it also affects folders with a lot of data in subdirectories.

[ghost]

For my part, I note that, now I can use the Paper theme with seemingly no ill affects (contrary to a statement I made in a post above). So it seems that the problem on my system was elsewhere - or at least that the ultimate problem was. That ultimate or underlying problem nay have had to do with apparmor, which might have been causing problems on my machine, or with mount points in fstab.

[Hangarter35]

Had this problem too : Nemo VEEEEERY slow to display a folder with many files, espacially my photos files.

For my part, disabling most of Nemo plugins solved this problem (I keep only 2 "reduce pdf file size" plugins).

LMDE 2 Cinnamon 3.2

[howdev]

nemo 3.2.2 is still the same, just slow, there is no improvement

[alvaro-canepa]

I have the 3.2.2 and disabling the plugin ChangeColorFolder+NemoPython, the navigation was normal now.
(Sorry my English)

[howdev]

I don't have ChangeColorFolder plugin, still slow.

[alvaro-canepa]

try to disable all plugins. If this work, enable one by one and test.

[tradesouthwest]

@alvaro-canepa dude, you rock. This was the fix for me. Plugins must not be memory tweaked.

[c0rru]

@alvaro-canepa thank you, got my nemo fixed xD didnt thought about the plugins at all.

[neocogent]

Disabling plugins and extensions made Nemo much, much faster for me as well. Both opening and closing. It used to take so long to close that it always popped up a "Force Quit or Wait" dialog. Now just boom, gone. Not sure which plugin is the culprit but it's probably worth enabling them one by one to find out.

[deutrino]

It's not just plugins. This happens for me on a fresh install.

[hieplq]

I use mint 18.1 with latest update, ssd drive
if i delete a sub big folder (about > 8G) then after that super folder can't open any more
i have to use tree mode or command line to navigate.

[sagg629]

I've made some rudimentary tests and I found out that Nemo seems to load thumbnails of ALL files (images dir). I remember that Explorer in Windows 7 loads thumbnails of images seen in the moment, the more I scroll the more thumbnails it loads.
I think Nemo should follow this approach.
I don't have the ChangeColorFolder extension BTW.
Using tree mode or console directly to perform navigation in large dirs with large files.

[tradesouthwest]

Nemo 2.8.7---guess I need to update---BUT, I reloaded Mint Cinnamon 17.3 just because everytime I update Mint, Nemo runs slower and slower and slower. when I had the brand new install of Mint, Nemo ran like a charm. I did one update (python, libs, and minor binaries) I could tell the slowing as soon as I opened Nemo.

Maybe Mint should not include Nemo as a default FM. Too many bugs.

And yes, it does run "better" without plugins, but what fun is that.

[guilhermefeitosa66]

I have installed recently linux mint 18.2 and Nemo (3.4.6) is slow.
I tried to disable some plugins, but without success.
(Sorry my English)

[asmith15]

I've also just upgraded to Mint 18.2 and am now replacing Nemo with Thunar (which I loved back in my Xfce days). I have a folder on an NFS share that contains 32 (not thousand) subdirectories. Some of these contain in their depths hundreds of thousands of files.

Whatever number of seconds it used to take before (which didn't bother me enough to find the bug tracker) is gone. It now takes forever. As soon as I click into that very important directory: the window shows me an empty directory listing (nearly gave me a heart attack) and no indication that it's doing anything. But "top" tells me a nemo is using 100% of one core. I left it on for a half an hour and it doesn't seem to get any further ahead.

This is on top of the fact that Nemo has been crashing on me periodically for years (from the start really). Dissapointing that it's not getting fixed. But given that it's the default file manager in Linux Mint I guess the code is so horrible that noone is willing to take it over and fix it up. I'm almost tempted to try it myself but I suspect I'd not be able to contain my frustration.

[mtwebster]

Some performance improvements have been implemented - see:

#2291 (comment)

[xenopeek]

To anybody still affected please retest on Linux Mint 20 Cinnamon or with Nemo 4.6.4.