Discussion Mouse support in graphics client

[dumbo2007]

I have almost reached the stage with the collision detection stuff I am working on, where I can define a collision mesh for any vessel type in Orbiter and it will be used when the vessel is close to a base. This will be extended to support collisions anywhere in the solar system gradually

So defining the collision mesh for a vessel currently involves using various primitive shapes or a convex hull of points positioned in the vessel's co-ordinate system to approximate the vessel's geometry. This involves entering/editing the translation and rotation angles for each shape independently and immediately seeing the effect as in Maya/3DS Max etc, while the simulation is running - 'insim' henceforward !

So I need to provide a GUI that supports all this. Then the values can be directly typed in and be applied on the vessel object insim. I was wondering if something similar to mouse hold and drag is possible to detect in the graphics client window. Then the collision meshes can be simply dragged around and dropped into place. I dont think Orbiter provides callbacks for the mouse when outside a virtual cockpit, but can we hook into the windows messaging pipeline ?

A 3D Max like editing ability in the sim window would be great and fun to use too.

Also I am currently trying out Qt 4.8.1 for the GUI as I think it has the most capabilities and it can be themed with the typical dark look (that Autodesk products seem to have for some reason)

---------- Post added at 02:38 PM ---------- Previous post was at 12:26 PM ----------

I guess I need to use DirectInput in some way :

http://www.two-kings.de/tutorials/dinput/dinput03.html

 

[jedidia]

I was actually wondering about this possibility myself, in view of a future editor for IMS.

I don't know how feasible it is to use directInput, as I assume orbiter already uses it. I don't think it's possible to attach two different eventhandlers to the same device, as usually whichever gets the event first will process it and not pass it on. It's no problem to use them in parallel for two separate windows, in the same way Orbiter Galaxy uses another D3D instance to render its window, and the irrlicht event managment to process mouse inputs, but this only works because the two devices don't have any real connection with each other.

Plus, even if that were possible, you don't get the necessary information from Orbiter to process the input (it's not that much use to know that the screen has been clicked at XY if you don't have access to the 3-d architecture that would allow you to make sense of that information). If the event handlers can be run in parallel for the same device, a graphics client could probably evaluate the click. If not, there's nothing even a graphics client can do about it...

As for your plans to add an actual mesh editor to your add-on, I'm not sure how much this is a necessity and how much you are suffering from feature creep. I would suggest to get the collision detection working and released first (we're all pining for it, after all ) before bothering with stuff that can be done by already existing tools (after all, what problem is it for an add-on developer to make a collision mesh with the same program he made the mesh for his vessel?).

 

[dumbo2007]

Well, you do know the 3d architecture. See if the user clicks a point x,y on the screen, then you store it.

The you get the camera position and orientation. The user always clicks at the camera plane. Then you draw a ray from the camera plane, in the direction in which the camera is looking. So perpendicular to the camera plane and through the clicked point. Now the ray has to constantly check if it has struck an object. For which you need the 3d objects and their positions and for speed eliminate all far away objects. This Bullet will do for me - it has ray casting support. All the vessels are present in the Bullet dynamics world too through their collision meshes

The Orbiter window only serves to gather user input.

Also graphics clients must be receiving user input of some sort, since MFDs depend on that. And we do drag the camera around, so the client must be receiving the mouse input

--------------

No no I am not going for a full fledged editor. Just 6 text boxes to enter the tx,ty,tz, rx,ry,rz and some icons to choose the collision shape. Without this gui its very difficult to make the collision shapes approximate the vessel geometry. I need to be looking at the shapes and the vessel simultaneously and for that I use the 3d viewer that Orbiter provides. Also there are a large number of vessels so I cant edit the colllision definition file, start orbiter, see that something is off by a few degrees, then close orbiter, edit it again, etc etc

-----------

On the other hand, if there was just 1 big convex hull - basically built out of a bunch of points in a 3d mesh - then it wouldnt be so much of an issue. I do support that now, the only thing is that if we use primitive shapes instead of a convex hull , then its much faster. Convex-Convex collisions arent very fast unless the number of points are low - about 50

 

[jedidia]

All the vessels are present in the Bullet dynamics world too through their collision meshes

Ah... I didn't think of that. Yes, that could work.

Also graphics clients must be receiving user input of some sort, since MFDs depend on that. And we do drag the camera around, so the client must be receiving the mouse input

Yes, but I assume it receives it from orbiter. If orbiter doesn't pass on an event, the client won't get it.

 

[dumbo2007]

Yes, but I assume it receives it from orbiter. If orbiter doesn't pass on an event, the client won't get it.

Oh darn, Orbiter has got too much control man ! The input should have passed through the client to the core. Sometimes I wish Orbiter was just a library like Bullet or Glut which I can use as I need and register call backs for the events in the simulation pipeline that I am interested in, rather than allow Orbiter control of the main thread. I mean its not like its a GUI library or something requiring to run an internal event polling loop [/rant]

 

[jedidia]

Sometimes I wish Orbiter was just a library like Bullet or Glut which I can use as I need and register call backs for the events in the simulation pipeline that I am interested in, rather than allow Orbiter control of the main thread.

Probably martin will start to wish that too when we continue to demand more and more control... :lol:

It would make the development of "conventional" add-ons a lot more difficult, though...

 

[dumbo2007]

ok, continuing on with speculation,

The only reason I asked is because, well, lets say Orbiter compiled into a dll. A dll can load other dlls, so all the other addons would still work. In fact they would not need to be changed at all.

For purposes of editing only, the main application can be a poweful GUI application like say Qt which takes over the main thread and that will use the dll to maintain a simulation world and get the vessel objects and camera information, celestial body states etc to draw them from the point of view of the camera. No need for callbacks too then. The client simply gets the entire state of the simuation from the core and renders whats necessary, culling other objects out of the viewport.

For MFDs , the client determines what was clicked on the screen and conveys it to Orbiter through a function call. So Orbiter needn't be bothered with user input/display at all. That would have been a cleaner separation of graphics & core in my opinion(for whatever its worth )

Currently thats only possible by running the core in a separate thread without any client attached. Then getting information out of Orbiter from that thread and sending it off for display in the rendering thread through a queue or something. Come to think if it.....I should actually try that

---------- Post added at 04:41 PM ---------- Previous post was at 04:29 PM ----------

I wonder if the Vessel selection window and other windows that we can open from inside Orbiter run in a separate thread or the same rendering thread. If they do, then how do changes get communicated to the rendering thread in a thread-safe way. Like say vessel focus changes when I select another vessel in the vessel selection dialog.

I am thinking of running Qt in a separate thread and pushing the changes to the main thread in which the Bump plugin is running, through a queue or something.

 

[jedidia]

I wonder if the Vessel selection window and other windows that we can open from inside Orbiter run in a separate thread or the same rendering thread. If they do, then how do changes get communicated to the rendering thread in a thread-safe way.

It's pure windows GUI, actually, so windows handles the threading for them. I don't know if the orbiter core does any threading by itself, but those dialog boxes are certainly not part of the rendering thread. They get drawn by windows, or, in the case of Orbiter Galaxy, by irrlicht via an additional d3d9 instance. Mindblast used XNA to render into a dialog box for the navigator.

So those windows are all seperate threads with their own drawing loops. They don't interact with the main render thread in any way. You can pretty much run whatever in an orbiter dialog box as rendering as well as eventhandling are completely unrelated to the main thread.

 

[dumbo2007]

hmm ok very nice,

But the selection in the dialog box does get conveyed to the rendering thread. I am interested in that part now. How does that happen ? Is a particular variable tied to the selection dialog and Windows takes care of updating this. But what if the rendering thread is checking this variable for a change at the same time ?

 

[jedidia]

Is a particular variable tied to the selection dialog and Windows takes care of updating this.

No, much simpler. As windows GUI elements, the dialog box will receive the mouse event from windows (or keyboard event, if it has focus), instead that the event is passed to the orbiter main process. You then tell orbiter what to do by calling API functions (for example, oapiSetFocusOject to change the active vessel). Orbiter never gets to know about the user input, it will merely carry out the instructions. No synchronisation required.

 

[asmi]

Ohh guys - so much for the speculations... As far as I know Orbiter does not use DirectInput, but uses windows messages like most other applications do. Most modern games do not use DI, but use HID interfaces instead.

 

[orb]

As far as I know Orbiter does not use DirectInput

Of course it uses DirectInput, but not for mouse, only for joystick.

 

[asmi]

Of course it uses DirectInput, but not for mouse, only for joystick.

I thought we're talking about the mouse here...

 

[dumbo2007]

ok, so can a graphics client be notified when a point is clicked on the window and the co-ordinates of the point ?

 

[asmi]

ok, so can a graphics client be notified when a point is clicked on the window and the co-ordinates of the point ?

GC is able to receive/intercept any windows message that goes to the render window.

 

[dumbo2007]

Awesome ! thats what I wanted to hear

 

[Hielor]

Sorry, but: what's the point? There already exist apps to convert .msh files into other things for import into "real" 3d editing programs--at least, I distinctly remember importing the Shuttle-A into Wings3d at some point, and the process was relatively easy.

Once you've done that, make the collision mesh in your 3d app of choice using the original model as a reference, and export the collision mesh. You'll still be manually making the collision mesh for everything, but you'll be using a toolset that's well-known and proven, and more importantly that you didn't have to write yourself. I'm assuming that there are a finite number of meshes you're planning on converting, in which case importing them to another graphics app will be far easier than coding your own 3d editor. Any addon developer wanting a collision mesh can make it in their 3d app of choice, so your 3d editor in orbiter would be useless to them.

In fact, if you insist on coding something, why not write something to automatically convert a mesh to a collision hull? Doesn't have to run in real time, and could follow a simple set of rules to get close to the final product, then you could clean it up by hand. A good starting point might be "if the vertex has concave angles at its polygons, delete it. If a vertex has only shallow (less than 5 degrees or so) angles, delete it." Not perfect, but it could be a start.

The only vessel I know of for which an in-game editor would be useful is the XR2 due to its mesh encoding, and your plan seems like an awful lot of work for just one ship.

 

[dumbo2007]

Yeah, good point, I can use the concave decomposer that Bullet provides to decompose a concave mesh into one or more convex ones.

My interest in asking about mouse drag and drop support was to check if in the far(or near!) future I could allow users to pick higher level objects and place them around a base, like say a bunch of vehicles and other objects.

Also currently what I do is try to approximate the geometry of a vessel using basic shapes which are placed around the vessel. Its faster to carry out collisions using basic shapes. These need to be placed correctly for which a dragging functionality is useful. Of course I could also let the user just enter the values in a text box and see the shape moved immediately.

My intention is not to create a 3d editor for meshes at all.

 

[Hielor]

If you're giving the user the ability to move (and, presumably, add) objects in 3d space inside Orbiter, that's exactly a 3d editor. Might not have vertex operations, but it's still a simple 3d editor.

Placing objects on the ground is actually a simpler problem than selecting/moving arbitrary objects, because the ground in Orbiter is a regular surface, so finding the intersection of the click and the ground is relatively easy.

On that note, earlier you mentioned a ray perpendicular to the camera plane, which is not correct--because the Orbiter viewing window is perspective transformed rather than an orthogonal viewport, the rays for all the pixels create a frustum of a rectangular pyramid.

 

[dumbo2007]

Yes right I ll have to account for the perspective projection. So does the field of view allow me to somehow get the angle in which I should shoot the ray. I havent looked at the detailed math yet.

---------- Post added at 01:06 PM ---------- Previous post was at 07:38 AM ----------

Yeah thats right, what I have in mind is a simple 3d editor only to be used for placing objects and maybe drag some shapes around when approximating geometry(see previous post).
The Bullet physics world viewer for example supports dragging and dropping objects around while the simulation is running. It does this through a point to point constraint, so the mouse pointer becomes a constraint thats also solved along with all the other forces to finally calculate the position and orientation of the body. It does this on each iteration.

The great thing about this is one can pick up objects and drop them and they would bounce about and roll on the floor like in the real world. Also one can see all the objects in the scene at the same time. So I was hoping to bring in something like that eventually for constructing detailed vessels for example. This has nothing to do with hard core space travel stuff though.