data as single domain in VisIt, no subsets

[keithbennett]

In GitLab by Piotr Raczka on 02 Apr 2019, 09:09 (GitLab issue GL#5)

Hello,

When I open the 'Controls/Subset' window in VisIt, I find that the whole SDF output is marked as a single domain, despite being quite large (100's of GB) and coming from a computation on 100's of cluster nodes. This is quite surprising, because with a single domain one cannot use parallel upload and processing capabilities of VisIt, as outlined in the VisIt manual (https://visit-sphinx-user-manual.readthedocs.io/en/latest/Subsetting/Using_the_Subset_Window.html). As a result it is impossible to process large (~TB size) SDF files in VisIt, and for smaller ones there is little improvement in performance when using multiple cores for the task. There are various tricks to restrict the processed domain, but even then it would be useful to have the possiblity of manually turning on/off some subdomains and species, not to mention the speedup due to parallelism. Why then these subdomains and species are not explicitly visible in the SDF file from the beginning?

Cheers

Piotr Raczka

[keithbennett]

In GitLab by Keith Bennett (@keithbennett) on 02 Apr 2019, 11:55

Due to the architecture of VisIt, it is not possible to have a the file automatically divided amongst the requested number of processors and also have this information show up in the subset domain view. (Or at least, I have never been able to find a way of doing this. I would happily accept any suggestions on this.) I think that many of the file formats have an explicit domain split written into them. The disadvantage of this is that you cannot subdivide the domains any further. For SDF, we have chosen to split the domain amongst processors dynamically at runtime. The downside of this approach is that the domain split information does not get recognised by VisIt's subset list. You can see the domain split that has been used by plotting "CPUs/Current rank". You will see that this changes according to the number of CPUs being used by VisIt. Admittedly, the ability to turn on/off domains would be nice, but the reader does already do a perfectly good job of parallelising.

I am not sure what you mean about species, though. The particles are already written on a per-species basis.