Pages: 1 2 [3] 4   Go Down
  Print  
Author Topic: Easy multi-threading in Quest3d  (Read 16547 times)
June 28, 2011, 12:10:44 am
Very cool.  Nice to see this.  The texture loading demo seems to have some issues for me.  The textures will flash on the screen during loading, and sometimes mess up the editor.  I believe the problem is calling "Copy Image" in the background.  According to this: http://www.gamedev.net/topic/374948-multithreaded-direct3d/ , D3D has to be inited with D3DCREATE_MULTITHREADED, which also degrades performance.  My computer is quad core, so that might make it worse.

I made it a lot more stable by using a Surface instead of CopyImage to decompress the image (I am assuming decompression is the bulk of the texture loading time, when the Buffer is already in memory).  Quest still flickers a little bit if I keep loading the textures, but nothing drastic.

(btw, I've programmed a lot of multi-threaded texture loading code in c++/OpenGL)
June 28, 2011, 08:37:49 am
Take in mind to get benefit in your application from multithread you don't need D3DCREATE_MULTITHREADED since there are many tasks to do which aren't D3D related. Ideally you may leave the rendering task exclusive for the q3d main thread where using another processes to prepare it so quest3d focus on rendering. Simplifying a lot the story, you could leave the quest3d main thread just for rendering.

What the texture example try to show is the transparent-background-loading where doing the same through usual channels would stall your application.
I didn't find any flick when sending to video-memory the texture, if you call a copyimage into a multithread process which is into a start3dscene will flick since q3d thread and your created thread are not same speed.

PD1. *I am thinking on a way to implement critical-sections for the channel to help doing more complex tasks.

PD2. Next tool in mind is a (real) multithreaded resource manager where I am waiting for Quest3d5 to see its new importer. How much speed and fluency would your application earn doing the resource-management in multithread?
June 28, 2011, 09:27:56 am
I think since Q3d4 directx is in multithread mode.
July 13, 2011, 12:49:10 pm
Hi, Parrot. Seeing your example, seems you are going faster than the engine  Evil

For what you want to do you shall load each cgr in a iterative way (1 by 1) where you shall check the channel you called to load is already loaded before going for next one.

(I will upload an example)
July 13, 2011, 06:24:19 pm
Example loading 800 files (>1mb each) attached. (Put it in the test folder of the Parrot attached example)

Edit: Replaced previous example Startgroup4.cgr with easier/simplified one: Startgroup5.cgr

* StartGroup5.cgr (184.97 KB - downloaded 270 times.)
July 13, 2011, 06:29:01 pm
I made it a lot more stable by using a Surface instead of CopyImage to decompress the image (I am assuming decompression is the bulk of the texture loading time, when the Buffer is already in memory).  Quest still flickers a little bit if I keep loading the textures, but nothing drastic.
Nelson, it will flicker inside q3d editor, if you put the surface/copyImages or any other method you use to send texture to videomemory outside of the "Start3dScene" you may not have any flick.
July 15, 2011, 08:25:57 pm
Wiki page started for the channel: http://wiki.q3dtools.com/doku.php?id=channels:q3dtool_mtcaller
August 19, 2011, 11:45:39 am
Hello all, new updates on the multithreading aspect:

Added three new channels to help on synchronization.
-Q3dTool_MTCriticalSection
-Q3dTool_MTEnterCriticalSection
-Q3dTool_MTLeaveCriticalSection
*There is a sample included at package about thread synchronization.

Added a template folder to make easy the use of the multithread aspect.
[99 - Q3dTools]
     [Multithread]

---
Those users interested in the update (without cost) just drop e-mail.
October 25, 2011, 04:52:03 pm
For test purposes, there has been added a new version of the channel for the beta6 x64:
http://quest3d5.quest3d.com/forum/index.php?action=dlattach;topic=70317.0;attach=133677
October 26, 2011, 12:48:20 pm
Just out of curiosity.... what do people actually do with these channels. I'm pretty sure many things will fail because of synchronization issues between threads. So what can you do with it?
October 26, 2011, 01:09:52 pm
Quote
Just out of curiosity.... what do people actually do with these channels. I'm pretty sure many things will fail because of synchronization issues between threads. So what can you do with it?

The one and only reason I am interested in the MT_Caller channels is for the possibility that it will enable background loading of models without stalling the primary thread.  

I recall Tom Beddell mentioning that it seems to be working so far (using the new Scene Importer) which was encouraging. However, I don't think Jorge is 100% convinced about the stability ..yet.

There seems to be a higher potential for crashes if you attempt to use the MT channels with any Q3d-OO functionality. I would be ok with that limitation, but even then - the MT channel still requires more testing to determine whether it is truly stable for all other purposes,
or if it will always be problematic for one reason or another.
October 26, 2011, 01:20:33 pm
I noticed one problem on exit with scene importer. Maybe Artur can fix it.
October 26, 2011, 01:38:59 pm
Just out of curiosity.... what do people actually do with these channels. I'm pretty sure many things will fail because of synchronization issues between threads. So what can you do with it?

I use for things like file uploading, start a third program (where you need the output result in q3d), internet connections (like a login system, the application doesn't hang-on waiting for server response), DB queries. It would be ideal if possible to use with the scene importer to load/unload models in real background, generating RTTs and other producer/consumer situations; however there are limitations like you can't use it with OO since the "THIS" channel is a global static, generating an RTT may produce flicks, etc.
About synchroniztion you can use critical sections: http://www.q3dtools.com/wiki/doku.php?id=channels:q3dtool_mtcaller:synchronization:mtcriticalsection
(you can make enter also the Quest3d main thread itself, so all threads work in friendship)
February 27, 2012, 09:21:09 am
nice.
Pages: 1 2 [3] 4   Go Down
  Print  
 
Jump to: