Use AssetBundles in procedural generation

[yamins81]

Is there some way to create a URL-accessible library of prefabs somewhere that allows them to be loaded? Let's say we create 10,000 prefabs on someone's desktop. Is that the only place we can create binaries from for real production? Is there some way to create a binary that will load prefabs from a storage location? Eg. so that we can add to the library of prefabs without having to rebuild a HUGE binary and then transfer that huge binary? It's really annoying.

@jazzguitarist What do you think? Is this even possible?

[yamins81]

This is basically the same issue as using AssetBundles.

[yamins81]

@jazzguitarist wrote to me in an email:

assetbundles stay loaded until unloaded. To avoid memory problems, its therefore necessary to unload assetbundles at some point. There are two easy solutions I thought of. One is to have a method in prefab database which unloads all assetbundles. Inserting a call to this in procedural generations init function will ensure that no more than the prefabs actively used in the scene are loaded. If this causes memory issues as well, prefabs can also be unloaded after instantiated. This may make the room generation very slow though, unsure. Next option is to determine a good number of assetbundles to be loaded at any given time. Then implement a cache structure which unloads the least recently used assetbundle whenever you load a new bundle and are out of space in the cache. Unsure how loaded assetbundles effect memory, but either of these solutions should work fine.

[pbashivan]

From what I have been reading so far, it seems like the only way to create new bundles is through the editor. If that's true then we can't really make a new bundle for each object.

[yamins81]

Can't you script it using this method or similar? https://docs.unity3d.com/460/Documentation/ScriptReference/BuildPipeline.BuildAssetBundle.html

[pbashivan]

I used this function to build the bundles, but you can't use this function to assign assets to bundles using this method. Assigning assets to bundles and creating new bundle names is the issue.

[yamins81]

Hm I don't understand the problem. Isn't that function's arguments like a list of assets and a pathname? Why can you start with a list of prefabs, loop through the list, and create new assetbundles for each one with a distinct path name?

[pbashivan]

I'll check on this but I think path is where to put the bundle files. Not sure if it's actually filename, or at least that's how I used this function before.
I'll update when I get to test it.

[yamins81]

But I'm still having trouble understanding your problem ... What makes the suggestion I made above not possible to implement?

[pbashivan]

What I mean to say is, the path is a path to a directory (for putting the bundle files in it) not path to a specific filename for a bundle file. Maybe we can discuss it this evening.

[yamins81]

you're saying that BuildAssetBundle function is deprecated in unity 5.x.

what about this http://answers.unity3d.com/questions/1002779/creating-assetbundles-entirely-in-code-in-5x.html

[yamins81]

look at the second function signature here:

https://docs.unity3d.com/ScriptReference/BuildPipeline.BuildAssetBundles.html

[yamins81]

Notes from @jazzguitarist about using assed bundles:

1. Extend the Setup Prefabs process in PrefabDatabase.cs to load the prefabs into Assets/PrefabDatabase/ as AssetBundles.

--> This also would mean you need to move the actual prefabs to the PrefabDatabase folder as well, so they are not contained in the resources folder.

--> You will also need to refactor some of the Generate and Setup prefab code to operate in the PrefabDatabase directory instead of the Resources directory.

(As a note, I recommend making a directory structure like this to keep things clean and clear:
/Assets/PrefabDatabase/GeneratedPrefabs/... and /Assets/PrefabDatabase/AssetBundles/..., this
way you can just duplicate the tree of GeneratedPrefabs/ in AssetBundles but contain the
bundles instead. If you prefer to just have the bundle and prefab in the same place, that's
probably fine too, but might cause some confusion)

2. After all this is done, the next task would be to implement a Load method (i already inserted an unimplemented one in PrefabDatabase.cs) this will be used to replace the call of Resources.Load() in ProceduralGeneration.cs. I put a comment in there about switching once that's done.

As for storing which prefabs are being used (for loading and unloading AssetBundles), that data can be recorded whenever you call that load method to be implemented in PrefabDatabase.cs.

You should also probably implement an unload method in PrefabDatabase.cs and make a call to it in Init() in ProceduralGeneration.cs.

Later down the road you can try to store the ones which will be used again, but sadly, the things being used are not pre-determined (re: this, I wrote a comment early on in ProceduralGeneration.cs that said something regarding where prefabs are predetermined, but that was incorrect, so ignore that comment, I just forgot to delete it).

Once all this stuff is implemented, you can change the call from Resources.Load to prefabDatabase.Load in ProceduralGeneration, run the new Prefab Generation and Prefab setup code. And hopefully things will run.

[yamins81]

Still have to test whether it's better have all the assets in one big bundle or many small asset bundles

[chengxuz]

The right thing todo is to put all prefabs in mongo database (including all the information like scales, etc) and then use LoadFromCacheOrDownload to load all the things, rather than loading all the prefabs just from directory.

Currently the loading is finished by loading all the resources under the folder.(line 291 in PrefabDatabase.cs)

[pbashivan]

@chengxuz
I'm changing the code to use the asset bundles instead of resources. I'm using "LoadFromFile" to load the bundles locally. I have looked at the "LoadFromCacheOrDownload" function, it takes a url and load the asset from there. Are you suggesting to put the bundle files on some server and put the link to it on MongoDB?

[chengxuz]

Yes, that would be the ideal state that we would like to get to. The comment is actually responding to your question about Prefabdatabase on Slack. So currently the prefabdatabase is built by loading all the local resources under the folder (see line 291 in PrefabDatabase.cs). I think PrefabDatabase.cs actually describes how this struct is built and what is in there.

[pbashivan]

Oh I see. PrefabDatabase keeps a list of all available prefabs as a property, right? I changed the code such that it goes over all bundle files in /PrefabDatabase/AssetBundles and adds those entries to the prefabs list property.
I'm going to stick with local bundle files for now as we need to wrap it up very soon and begin the tests on the environment.
Another relevant question, do you know why we have duplicate code in ProceduralGeneration.cs for setting up prefabs?

[qbilius]

Hey guys, why don't we move chatting of this sort to Slack?

[chengxuz]

I think there are currently two different Slack channels for 3Dworld, and one of them (in yaminslab) actually has been linked with the Github repo which means every issue and responses would be posted in that channel too. So @qbilius , do you think the channel at dicarlolab also needs this link? Or we should simply discuss those things in the dicarlolab (as you and Pouya are not in yaminslab)?

[chengxuz]

I would continue Pouya's work for this issue now. And there are 3 things to do:

1. TEST: Prefabs cross-platform generating and using.
2. TEST: Assetbundles creating and loading through binary files with 5000 objects.
3. IMPLEMENT: Replacing loading from files under some folder to Loadfromcacheordownload (https://docs.unity3d.com/ScriptReference/WWW.LoadFromCacheOrDownload.html), including modification to SetupBundles in PrefabDatabase and CompileAssetBundles in PrefabDatabase.

@pbashivan , please feel free to add other things you think need to be done. Thanks for talking about and explaining your implementations to me!

Best,
Chengxu

[chengxuz]

Hi, @pbashivan , I met a bug about objects falling through floor. It seems that this should be related to the objects and models imported. I don't have this bug before merging. And your branch without merging already has this bug (you can test by setting permitted_items to be ["bed", "sofa"]). Some of the objects will fall, while some won't.

From google, I learned that I should change something about the object or model. But should I do it before Create Prefabs? Or during dynamically loading?

Any idea about how to fix this?

Thanks,
Chengxu

[pbashivan]

@chengxuz I'm seeing this happening only for the objects in "Props" and not for the ones I try from shapenet (Can you also confirm?). Apparently it's not because of bundling cause when I manually insert prefabs of those objects into the scene the same thing happens. Probably something related to generating the prefabs.

[chengxuz]

HI, @pbashivan , thanks for responding. Then maybe we can try to solve this falling through floor later. Currently I met a problem during building the binary file. It turns out that AssetDatabase.LoadAssetAtPath in this sentence:

        PrefabDatabase database =  AssetDatabase.LoadAssetAtPath<PrefabDatabase> 
                ("Assets/ScenePrefabs/PrefabDatabase.prefab");

is only available in Editor mode. So I tried to replace it with some other functions such as AssetBundle.LoadFromFile, but I failed. I wonder whether you have done something similar and I would be grateful if you could provide some suggestions.

Best,
Chengxu

[qbilius]

No idea. I didn't have any issues importing; though, admittedly, building prefabs crashed, as you know, and the build was also not working, but I never looked into how the objects looked like.

Maybe it would make sense for you to try running the same procedure on a Windows machine (say, kubelik) and see if that works.

[chengxuz]

@pbashivan @qbilius , thanks for responding! May I ask which version of shapenet you guys are using? I used v2, but it seems that Pouya used v1, is that right? What's the difference of this two version? Which one should we use?

[qbilius]

I'm using v2

On Tue, Oct 11, 2016, 13:19 Chengxu Zhuang [email protected] wrote:

@pbashivan https://github.com/pbashivan @qbilius
https://github.com/qbilius , thanks for responding! May I ask which
version of shapenet you guys are using? I used v2, but it seems that Pouya
used v1, is that right? What's the difference of this two version? Which
one should we use?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABMaMd9ZM2RzENdN_ZG880VS7Q72BbwUks5qy8UugaJpZM4JLp4M
.

[pbashivan]

@chengxuz I also used v2. v1 had some problems with importing the textures if I remember correctly.

[chengxuz]

OK, I think I might have found the reason of my problems. I downloaded the ShapeNetCore.v2.zip directly. But it seems that the contents there is different from that in the folder "ShapeNetCore.v2". This might cause some problems.

Anyway, after copying files directly from the folder, I could now import, create, and setup the assetbundles successfully.

Thanks for helping!

[chengxuz]

Hi, @pbashivan @qbilius , is it possible to delete something in sandbox to free some space?

I would like to zip the shapenet.v2 folder (as the zip file already existing seems not having the same contents), but it seems that there is no enough space left. Could you for example delete the Shapenet.V2.zip file and create it again? Or where do you think I should download the data from?

A better way is of course expanding the disk space of sand-box.

[pbashivan]

Can't really comment on that cause I haven't really used it.

On Tue, Oct 11, 2016 at 4:19 PM Chengxu Zhuang [email protected]
wrote:

Hi, @pbashivan https://github.com/pbashivan @qbilius
https://github.com/qbilius , is it possible to delete something in
sandbox to free some space?

I would like to zip the shapenet.v2 folder (as the zip file already
existing seems not having the same contents), but it seems that there is no
enough space left. Could you for example delete the Shapenet.V2.zip file
and create it again? Or where do you think I should download the data from?

A better way is of course expanding the disk space of sand-box.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL1xV8qvp_cr-vuG9QZUwhMsHQwCmUtqks5qy-8-gaJpZM4JLp4M
.

[qbilius]

Could you get the zip from kubelik instead?

On Tue, Oct 11, 2016, 20:15 Pouya [email protected] wrote:

Can't really comment on that cause I haven't really used it.

On Tue, Oct 11, 2016 at 4:19 PM Chengxu Zhuang [email protected]
wrote:

Hi, @pbashivan https://github.com/pbashivan @qbilius
https://github.com/qbilius , is it possible to delete something in
sandbox to free some space?

I would like to zip the shapenet.v2 folder (as the zip file already
existing seems not having the same contents), but it seems that there is
no
enough space left. Could you for example delete the Shapenet.V2.zip file
and create it again? Or where do you think I should download the data
from?

A better way is of course expanding the disk space of sand-box.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
#37 (comment)
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AL1xV8qvp_cr-vuG9QZUwhMsHQwCmUtqks5qy-8-gaJpZM4JLp4M

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABMaMWssghPNje-0nlkdhlUUOHJQ4IkAks5qzCaSgaJpZM4JLp4M
.

[chengxuz]

Thanks for responding!

@qbilius Is the kubelik the windows machine with user threed? I am actually doing that now, but the contents there (in C:\Users\threed\ThreeDWorld\Assets\Models\sel_objs) seem to be different from that in sandbox, for example, there are only 53 subfolders there, instead of 55, and the total space occupied is also different. I guess it should be OK, right?

Best,
Chengxu

[qbilius]

Sorry, I can't check it now. I think you're looking at the imported objects
and not all were imported, I think. There is another place where the
original objects are.

On Tue, Oct 11, 2016, 20:22 Chengxu Zhuang [email protected] wrote:

Thanks for responding!

@qbilius https://github.com/qbilius Is the kubelik the windows machine
with user threed? I am actually doing that now, but the contents there (in
C:\Users\threed\ThreeDWorld\Assets\Models\sel_objs) seem to be different
from that in sandbox, for example, there are only 53 subfolders there,
instead of 55, and the total space occupied is also different. I guess it
should be OK, right?

Best,
Chengxu

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABMaMaIYudTPdnpE7BQpesF7q96GCtXnks5qzChggaJpZM4JLp4M
.

[chengxuz]

Hi, @qbilius , thanks very much for responding at later night. I think I have found the original objects in the windows machine!

[chengxuz]

Hi, @pbashivan , how fast is the creating prefabs on Mac or Windows? It takes me more than 10 hours to only create prefabs for 545 objects on Linux, is it normal?

[qbilius]

Sounds normal to me -- see this plot I made:
https://dicarlolab.slack.com/messages/3d_environment/search/prefab/

Also, let me reiterate again: can we please switch back to Slack for
chatting? We have the 3d_environment channel there, we've been chatting
there all the time, so I don't really see why we're using GitHub issues for
this.

2016-10-12, tr, 12:48 Chengxu Zhuang [email protected] rašė:

Hi, @pbashivan https://github.com/pbashivan , how fast is the creating
prefabs on Mac or Windows? It takes me more than 10 hours to only create
prefabs for 545 objects on Linux, is it normal?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABMaMTZxLE514Uto2HxYKuGs4j3GJiN4ks5qzQ9mgaJpZM4JLp4M
.

[pbashivan]

I tried generating ~200 prefabs on my mac, it took almost an hour or so.
Actually, it was very memory demanding so I tried generating them in small
batches to speed it up.
On Wed, Oct 12, 2016 at 12:54 PM Jonas Kubilius [email protected]
wrote:

Sounds normal to me -- see this plot I made:
https://dicarlolab.slack.com/messages/3d_environment/search/prefab/

Also, let me reiterate again: can we please switch back to Slack for
chatting? We have the 3d_environment channel there, we've been chatting
there all the time, so I don't really see why we're using GitHub issues for
this.

2016-10-12, tr, 12:48 Chengxu Zhuang [email protected] rašė:

Hi, @pbashivan https://github.com/pbashivan , how fast is the creating
prefabs on Mac or Windows? It takes me more than 10 hours to only create
prefabs for 545 objects on Linux, is it normal?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
#37 (comment)
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ABMaMTZxLE514Uto2HxYKuGs4j3GJiN4ks5qzQ9mgaJpZM4JLp4M

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL1xVwJdTnunn9RK1DzCUvz3zDClLsg4ks5qzRDSgaJpZM4JLp4M
.

[yamins81]

@chengxuz @pbashivan @qbilius

Are you making sure to hit "play" in the editor when creating prefabs?

See the little section on "Generating prefabs” on the readme in the ThreedWorld repo. That may clear up some of the memory problems being seen here.

[qbilius]

Well, I wrote that section so my benchmarking plot was done using those
guidelines.

2016-10-12, tr, 15:08 Dan Yamins [email protected] rašė:

@chengxuz https://github.com/chengxuz @pbashivan
https://github.com/pbashivan @qbilius https://github.com/qbilius

See the little section on "Generating prefabs” on the readme in the
ThreedWorld repo That may clear up some of the memory problems being seen
here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#37 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABMaMeRROIi3rdVUqiJfUoEpk-uA1N30ks5qzTAegaJpZM4JLp4M
.

[yamins81]

Ah, ok. I think @pbashivan might have not been doing that if he had problems with memory. @chengxuz might also might not have been, so just wanted to make sure they knew it.

[chengxuz]

@yamins81 @pbashivan @qbilius thank you all for commenting. I would try to generate the prefabs during "Play" mode.

BTW, about whether we should discuss things here or on slack, I was originally thinking that discussion here would be more organized and specific for only one issue, which means once this issue is solved, we could close it and don't need to read those discussions in future. But discussions on slack seem to be always there, which means that they would show up when you look for older discussions about other things or different issues would be discussed in the channel together. But now it seems that some discussions do have general use for the whole project. So I don't know where we should discuss it here or on slack. What about other people?

[pbashivan]

@yamins81 I did hit the play button, but it would still consume many GB of memory nonetheless.

[chengxuz]

The current status of this issue is:

1. TEST: Prefabs cross-platform generating and using.
2. TEST: Assetbundles creating and loading through binary files with 5000 objects. The speed is too slow to generate so many prefabs.
3. IMPLEMENT: Replacing loading from files under some folder to Loadfromcacheordownload, including modification to SetupBundles in PrefabDatabase and CompileAssetBundles in PrefabDatabase. There is still one thing we can do: Make it possible for using private assetbundles, this might require to use C# SDK for AWS in Unity.

[yamins81]

@chengxuz we should replace using LoadFromCacheOrDownload with our own downloading and caching and use of LoadFromFile, to get around the hard 4GB cache size limit. This is very time sensitive.

[pbashivan]

@chengxuz @yamins81 I think it would make more sense to use the object name in scale_relat_dictrather than the full bundle URL path.
e.g.:
46e777a46aa76681f4fb4dee5181bee
instead of
"http://threedworld.s3.amazonaws.com/46e777a46aa76681f4fb4dee5181bee.bundle"

Let me know if you agree.

[chengxuz]

@pbashivan in current case where we may need to replace the Loadfromcacheordownload methods, I agree...

[yamins81]

Agreed.

[pbashivan]

I noticed that the object names returned in the SceneInfo also follows the same pattern for reporting the object names. I think it would be best to use a uniform way of referring to objects (preferably without the "http:..." part.

[yamins81]

Agreed

[pbashivan]

While running TDW_image_generator on the 500 object build, the script hangs at some point and looking at the output_log.txt on the server I found this error occurring. It seems like this happens after I do the scene_switch. The complete log can be found on Richard's machine at:
/home/richard/Documents/ThreeDWorld_Server/ThreeDWorld/ServerTools/output_log_error.txt

Error: Using memoryadresses from more that 16GB of memory

STACK: at System.Environment.get_StackTrace()
at SimulationManager.HandleLog(System.String logString, System.String stackTrace, LogType type)
at UnityEngine.Application.CallLogCallback(System.String logString, System.String stackTrace, LogType type, Boolean invokedOnMainThread)
at UnityEngine.AssetBundle.LoadAsset_Internal(System.String , System.Type )
at UnityEngine.AssetBundle.LoadAsset(System.String name, System.Type type)
at UnityEngine.AssetBundle.LoadAsset(System.String name)
at PrefabDatabase.LoadAssetFromBundleWWW(System.String fileName)
at ProceduralGeneration.AddObjects(System.Collections.Generic.List1 prefabList)
at ProceduralGeneration.Init()
at ProceduralGeneration.Start()

Does the unity script load all the objects again when switching scenes? Maybe we are not clearing the previously loaded assets before scene switch.

[yamins81]

1. re-implementing cache of files because the LoadFromCacheOrDownload has a max 4gb limit which is too low -- in future versions of unity apparently this limit will be removed but for now we need to implement our cache to avoid this limit.

2. @chengxuz has a bunch of commits in the mongodb_inter branch that clean up some of the memory and loading/reloading of asset bundle issues

3. @pbashivan can you look into whether prefabs need to be unloaded as well as asset bundles and maybe that's causing a memory problem on scene_switch