External MFDs for Orbiter

[yagni01]

MFDs call oapiOpenInputBox() to read user data, and oapiOpenInputBox() always renders the dialog box into the main window.

Is there a way to hook it?

I look forward to the next evolution :thumbup:

 

[kamaz]

Actually, I think this add-on is complete as far as I am concerned.

The basic functionality (i.e. viewing the MFD on another device) works. Touch screen device can be used. The panel layout is user configurable. As an icing on the cake, the Cougar frames can be used to control the MFDs. So, this completes support for my setup. Also, given that MFDs are the most problematic thing in the simpit, I think I have made a small breakthrough by demonstrating how to make 2 MFDs for a mere 180 EUR

Hooks in the DLL for integration with Orb::Connect etc. are provided. It's now left to add VNCMFD support in Orb::Connect, so Orb::Connect can be used for controllling the MFD buttons in a simpit environment, if you want to use something other then Cougars.

So, basically, I do not see any missing features

I now have some ideas about what to do with unused buttons on the Cougars, but this will likely take form of a separate add-on.

 

[Zatnikitelman]

I've been lurking in this topic, following your progress since you first posted this and I must say, I'm impressed. I tried this out on my phone and have no problems with it. This looks great for simpit builders!

 

[yagni01]

Actually, I think this add-on is complete as far as I am concerned.

The basic functionality (i.e. viewing the MFD on another device) works. Touch screen device can be used. The panel layout is user configurable. As an icing on the cake, the Cougar frames can be used to control the MFDs. So, this completes support for my setup. Also, given that MFDs are the most problematic thing in the simpit, I think I have made a small breakthrough by demonstrating how to make 2 MFDs for a mere 180 EUR

Hooks in the DLL for integration with Orb::Connect etc. are provided. It's now left to add VNCMFD support in Orb::Connect, so Orb::Connect can be used for controllling the MFD buttons in a simpit environment, if you want to use something other then Cougars.

So, basically, I do not see any missing features

I now have some ideas about what to do with unused buttons on the Cougars, but this will likely take form of a separate add-on.

For 'readonly' MFDs you pretty much have it. I would like a config value to set the default frame rate (I would like to start with it high).

As SolarLiner suggests, for MFDs that require target input (Orbit, IMFD, etc.), a way to provide that input box remotely would be extremely valuable. Please give it some thought.

 

[yagni01]

I just had a thought that could take this tablet idea to the next level. . . replicating the multifunction display in the XR-series vessels. It's baked into the XR panels, but the autopilots are all available via Orb:Connect subscriptions, as would be the ability to update them. . . Now that would be cool.

 

[kamaz]

All hail the Probe!



I have solved the dialog box problem. Here is an addon which allows you to enter the needed values through a network socket instead.

The reason I used a socket, is that (1) you cannot easily tell which MFD triggered the dialog box, so you don't know which client you should render the dialog box to; (2) I did not feel like writing a code which would render the dialog boxes to VNC framebuffer, (3) although VNC can take keyboard input, keyboard input on tablets is not very convenient and (4) by separating this functionality from VNCMFD, you can use it with another display solution (such as WebMFD).

But you can now write a "Mission Control" application which uses a VNC client component for displaying the MFD screen and pops up dialog boxes when it receives data from this add-on.

Usage description (short): install & enable the addon, use e.g. PuTTY to connect to port 31337 (TCP) and have fun

Thanks to:

- Martin Schweiger for Orbiter and for having no objections to this add-on

- cjp, who first reverse engineered the dialog code in Orbiter 2006 ( http://www.orbiter-forum.com/showthread.php?p=57223&postcount=55 ) and the published the FreeOrbit MFD, which made my disassembly efforts much easier

- Hex-Rays SA for the excellent IDA Pro disassembler / debugger, and providing the freeware version

- Microsoft Research for the excellent Detours Express library which I used for patching the orbiter core

- Microsoft for the excellent Visual Studio Express compiler

Readme file is below; the add-on (source & DLL) is atatched.

NetDialog
=========

The NetDialog Orbiter plugin intercepts function calls in Orbiter core
responsinble for drawing MFD dialog boxes and redirects these to a network socket.

The plugin is intended for use with solutions allowing remote display of
MFD screens, such as WebMFD or VNCMFD.

Since the plugin patches the Orbiter binary at run-time, it will only work with
Orbiter version 2010-P1 (100830). Both orbiter.exe and orbiter_ng.exe are supported.

Operation
=========

After the plugin is loaded, a TCP server is started on port 31337.
No function interception is done (yet), and Orbiter operates normally.

After the client connects to the server, the plugin intercepts two core functions,
responsible for drawing the object selector and the input box.

When the MFD code wants to open the body selector box or a dialog box (for example,
if user presses "REF" or "TGT" button in OrbitMFD), the plugin will send the
prompt to the connected network client. It will then wait for the response from the
client (terminated with CRLF) and forward this to the MFD code. It will then send
the status message to the client.

A typical network exchange look like this:

SERVER>QUERY Orbit MFD: Target\r\n
CLIENT>iss\r\n
SERVER>STATUS 1\r\n

(\r\n represents the newline characters. It is sent automatically by a terminal
software when the user presses the "Enter" key.)

The "QUERY" message contains the dialog box title.
The "STATUS" message informs the client if MFD accepted the input.
"1" means that the input was accepted, "0" means that it was not.

If the client response is empty (i.e. contains nothing but the \r\n characters)
the server replies with "ABORT" message.

NOTE: The plugin will NOT accept a new data request before the previous one
is satisfied (i.e. between the QUERY and STATUS/ABORT messages). In other words,
pressing the MFD button which prompts for user input for the second time will
have no effect, until the user response is sent.

For the MapMFD, please enter "[no orbit]" and "[no base]" (without quotes)
to remove the orbit / base selection.

Only one network connection at the time is supported.

After the client disconnects, the function interceptions are removed, and Orbiter
returns to normal operation (i.e. the dialog boxes are drawn in the main window).

Technical details
=================

Runtime function interception is done using the Detours 3.0 library from Microsoft.
Please see Hook.cpp for more information.

License
=======

This add-on uses Detours Express 3.0 library from Microsoft. This library is made
available under the license agreement which includes the following:

<<<<

You may use, copy, reproduce, and distribute this Software for any non-commercial purpose,
subject to the restrictions in this MSR-SSLA. Some purposes which can be non-commercial
are teaching, academic research, public demonstrations and personal experimentation.
You may also distribute this Software with books or other teaching materials,
or publish the Software on websites, that are intended to teach the use of the Software
for academic or other non-commercial purposes.

You may not use or distribute this Software or any derivative works in any form for
commercial purposes. Examples of commercial purposes would be running business operations,
licensing, leasing, or selling the Software, distributing the Software for use with
commercial products, using the Software in the creation or use of commercial products
or any other activity which purpose is to procure a commercial gain to you or others.

>>>>

Please see orbitersdk\samples\netdialog\detours\license.rtf for more information.

Compilation
===========

The add-on was developed using Microsoft Visual C++ 2010 Express.

First, you must compile the Detours library. To do so, run the Visual Studio Command prompt.
cd to orbitersdk\samples\netdialog\detours\license.rtf and run "nmake" to build the library.

Then, open orbitersdk\samples\netdialog\netdialog.sln in Visual Studio and build the project.

 

[kamaz]

I have also noticed that libvncserver has an interesting feature.

Multiple clients can connect to the same server (i.e. the same port) and they will get a shared display.

I believe that someone out there will find that useful One obvious application is an instructor / student setup.

 

[Thanatox]

This is great. I have been waiting for this kind of functionality for some time. Your solution is elegant and I applaud you for it. It literally made me smack my forehead and say why didn't I think of that.

I tried it with my asus transformer this morning and it worked great.

For anyone trying to use this add-on:
It's worth noting that once you enable the add-on the VNC server starts automatically when orbiter is running. this confused me for a moment I assumed that I would have to activate them like extmfd

I haven't yet had a chance to look under the hood but I would be interested to see if you could bind keyboard keys to the MFD options client side. it seems like this would make it more compatible with different devices via joy 2 key as well as home brew keyboard hacks.

 

[yagni01]

I haven't yet had a chance to look under the hood but I would be interested to see if you could bind keyboard keys to the MFD options client side. it seems like this would make it more compatible with different devices via joy 2 key as well as home brew keyboard hacks.

For a non-touchpad, you could use AutoHotKey (AHK) to open and position your client windows, then map key presses to a mouse click action at each specific button location. If the keys are really game controller buttons, though, I don't know if his Cougar functionality would be directly usable.

Also, I have had no luck dis-abling the plugin without removing the dll. Trying to uncheck the box immediately shuts down Orbiter, and it's still enabled when re-started.

But I agree - this is very cool.

 

[Thanatox]

I can reproduce the disabling bug found by yagni01.
When I un-check VNCMFD in the modules window orbiter crashes.

Here is the relevant section of Orbiter.log: (I think)

Finished initialising world
Module DeltaGliderIV.dll ..... [Build ******, API 060425]
Finished initialising status
Finished initialising camera
Finished initialising panels
OrbConnect::Initializing solar system with 32 bodies
OrbConnect::Found Config File
OrbConnect::Module Initialized
Finished setting up render state
OrbConnect::Module Shutdown
ERROR: DDraw object is still referenced: 22
---------------------------------------------------------------
>>> ERROR: Destroy framework objects failed
>>> [OrbiterGraphics::Exit3DEnvironment | .\OGraphics.cpp | 1034]
---------------------------------------------------------------
**** Closing simulation session

It seemed like it might have somthing to do with orbConnect so I tried disabling that addon but it still crashes. Orbiter.log below:

Finished initialising world
Module DeltaGliderIV.dll ..... [Build ******, API 060425]
Finished initialising status
Finished initialising camera
Finished initialising panels
Finished setting up render state
ERROR: DDraw object is still referenced: 18
---------------------------------------------------------------
>>> ERROR: Destroy framework objects failed
>>> [OrbiterGraphics::Exit3DEnvironment | .\OGraphics.cpp | 1034]
---------------------------------------------------------------
**** Closing simulation session

 

[kamaz]

Thanks for the kind words, bug reports and feature requests, here is the new release.

One very important thing -- config file format has changed. Your old file will not work. Please consult the supplied VNCMFD.ini for more information.

Changes since the last release (rev 25):

- Panel update frequency is now configurable via the INI file.
- Reworked joystick support. Now, any joystick device can be used, not only Cougar.
- Limit joystick polling frequency to 20Hz. Just in case.
- Eliminate excessive writes to logfile.
- Clear screen on simulation exit.
- Make module unloading work.

 

[kamaz]

If the keys are really game controller buttons, though, I don't know if his Cougar functionality would be directly usable.

Cougar is a joystick with 28 buttons and 0 axes.

So, I have generalized the code. You can now use buttons on any joystick device. See the INI file for examples.

Also, I have had no luck dis-abling the plugin without removing the dll.

I introduced this bug when I made VNC servers persistent between simulations. It should be now fixed.

For a non-touchpad, you could use AutoHotKey (AHK) to open and position your client windows, then map key presses to a mouse click action at each specific button location.

What about writing a small command line program which would send button presses to MFDs via Orb::Connect. So you'd invoke it like this:

PressButton.exe 192.168.1.4 2 5

and it would use Orb::Connect to send a button event to the 5th button of 2nd MFD running on host 192.168.1.4. Then use AutoHotKey or something to invoke that program.

 

[Ripley]

Cougar is a joystick with 28 buttons and 0 axes...

You're talking about this Cougar, aren't you?

 

[yagni01]

I was doing some basic math on kamaz's approach, and find it scales pretty well. My baseline was $600 (USD) for a cheap computer and a 21" touch screen monitor (~$300 ea). That system should support up to 6 MFDs.

6 cheap tablets @ $70 each == $420
If touchscreens are flaky, add 3 sets of Cougar displays @ $70 each == $210
Price: anywhere from $70 - $630 vs. a fixed $600 :thumbup:
Pro: can buy incrementally, custom arrangements (and portable w/o Cougar)
Con: Limited individual MFD size

YMMV

---------- Post added at 05:58 PM ---------- Previous post was at 05:57 PM ----------

You're talking about this Cougar, aren't you?

Yes, that's the one.

 

[kamaz]

You're talking about this Cougar, aren't you?

Yes.

---------- Post added at 10:23 PM ---------- Previous post was at 10:07 PM ----------

I was doing some basic math on kamaz's approach, and find it scales pretty well.

That system should support up to 6 MFDs.

Before jumping in, I'd suggest checking if your computer can handle 6 instances of VNC'd MFDs. The framerate drops with the number of MFDs, resolution and refresh rate.

Good thing is that performance can be improved. csanders has already made the fast MFD grabbing code for the inline client available here, I didn't get around to integrating it yet

For the D3D9Client (although it's already quite fast) improving the performace requires hacking the renderer -- although I have tried it, and it looks like it can be done

Con: Limited individual MFD size

See the point above about resolution negatively impacting performance.

One way around it is using a VNC client which can do scaling -- here is one (for .NET): http://cdot.senecac.on.ca/projects/vncsharp/

---------- Post added at 10:37 PM ---------- Previous post was at 10:23 PM ----------

a 21" touch screen monitor (~$300 ea)

...why do you need the touchscreen?

---------- Post added at 11:05 PM ---------- Previous post was at 10:37 PM ----------

The performance hit due to running VNCMFD should be the same as due to running ExtMFD, because in both cases the MFD screen is ripped using BitBlt() (And then, we have csanders' code which should make us faster. At least with the inline client.)

The VNC server(s) in my code run in separate thread(s), so they don't affect the framerate (multicore machines are a blessing ).

One more thing you should look at is performance of the client device -- i.e. what framerate can it render at a given resolution. Which VNC encoding is used will also affect performance.

 

[yagni01]

I was using a "theoretical" max for instances. Both scenarios depend on a main CPU that could feed that # of MFDs with a decent framerate.

...why do you need the touchscreen?

to maintain the same functionality as the tablets.

For myself, I've designed my flight deck around larger displays, i.e. replicate Honeywell 1310s, which are 11.3"x8.5", or 10" - 12" 4:3 ones, which would 'blend' multiple MFDs. As there are no commercial gamer bezels in those sizes, its either a) custom create a bezel, b) build an 'off-to-the-side' button cluster, c) use a separate keypad to drive button pushes, or d) have a touchscreen. Pros and cons to each, but I'm looking to use a touchscreen anyway right now. But the portait style displays were why I was hoping for a client that could display two MFDs with ability to move both sets of SEL/MNU buttons to the bottom.

 

[kamaz]

You mean something like that?

I just put that together in C#; took me about 10 minutes. The buttons are non-working at the moment, because I still didn't have time to code client-side support for Orb::Connect.

 

[yagni01]

You mean something like that?

I just put that together in C#; took me about 10 minutes. The buttons are non-working at the moment, because I still didn't have time to code client-side support for Orb::Connect.

Yes, although you could use the buttons from the VNC, no? Except to map the upper sel/mnu buttons down below?. Aesthetically, you could give this a nice 3D bezel look and feel, 'cause I haven't seen a bezel from Thrustmaster for something like this.

 

[kamaz]

We have a problem Trying to use Orb::Connect to send key events to VNC-exported MFDs results in a crash.

ORB:VirtualMFDs
ORB:VirtualMFDs=83066280,370489960
ORB:SendMFDKey:83066280:59

...goes KABOOM! After some investigation, it looks like a bug in the Orbiter core: http://orbiter-forum.com/project.php?issueid=976

So, it looks like VNCMFD must implement its own socket interface for controlling buttons. Just what I wanted to avoid. Oh well.

 

[kamaz]

Yes, although you could use the buttons from the VNC, no? Except to map the upper sel/mnu buttons down below?.

Sure. Like the attached screenshot?

I actually got that client to talk to Orb::Connect... and immediately hit that oapiSendMFDKey() bug. Oh well.

Aesthetically, you could give this a nice 3D bezel look and feel, 'cause I haven't seen a bezel from Thrustmaster for something like this.

Me? Decorations? Forget it, I almost failed 3rd grade because of drawing classes! I'll post the project files when I get this client to work, you will be able to add furry dicez and the like yourself...