Advanced Question exchanging data between an MFD and an external aplication

[jedidia]

Ok, the Galaxy Generator for orbiter galaxy is finished for the moment, Major priority now will be to link the whole thing with orbiter.

For that purpose, I would very much like to be able to send data directly between an MFD inside Orbiter and the galaxy map, that otherwise works as a completely separate aplication.

For example, I choose a destination in the map program, then go back to orbiter and the MFD should show me a vector so I know where I have to point my ship, tell me when the export has finished, maybe even display a map of the targeted system and stuff like that. Once I arrive at the destination, the Map program should update accordingly. For this I need to send some data (not much) between the two. I could simply do this by exporting it to a file and make the programs check for it every few seconds, but I'm pretty sure that there are ways to pass the data directly through the memory. I just have no Idea what they are, so I would be quite thankfull for a few suggestions.

 

[MeDiCS]

I have little experience with this, but I think could either use a mutex or a named pipe on both ends.

EDIT:
There are multiple ways of doing that: http://en.wikipedia.org/wiki/Inter-process_communication.

 

[Moach]

well... from what i've read, DLL's can be shared by two separate processes and act as a bridge between them, this way, you don't need to write and read files and should be much faster:hmm:

Here's a brief explanation of attaching dll's and all...
hope it helps!

 

[Face]

For example, I choose a destination in the map program, then go back to orbiter and the MFD should show me a vector so I know where I have to point my ship, tell me when the export has finished, maybe even display a map of the targeted system and stuff like that. Once I arrive at the destination, the Map program should update accordingly. For this I need to send some data (not much) between the two. I could simply do this by exporting it to a file and make the programs check for it every few seconds, but I'm pretty sure that there are ways to pass the data directly through the memory. I just have no Idea what they are, so I would be quite thankfull for a few suggestions.

Since orbiter_ng is multi-instancable, I'd go for socket communication. Using the TCP/IP stack for simple client/server protocols is rather easy and comes with the added value of immediate internet "features". And if you use a ASCII-based protocol, you can even debug it easily with something like puTTY.

regards,
Face

 

[jedidia]

Thanks to everyone for the suggestions. I'd like to avoid sending protocolls around, because I don't know jack about them. I'm willing to learn if I don't find any other means, though. But getting a pointer from one app to the other is in the end all I really need.

I thought about dlls, of course, but there's one problem I can't wrap my head around: If I make a dll with a class, I'll have to instanciate the class in both apps, and of course those instances again would be unable to communicate with each other. It seems there is a workaround in the tutorial Moach posted, but I'll have to take a very close look at it, with me being inexperienced and the tutorial written in rather bad english. Thanks to all, I hope I'll come up with something.

 

[dbeachy1]

Here is an example of how to use a named pipe: http://msdn.microsoft.com/en-us/library/aa365592(v=VS.85).aspx

 

[Face]

Thanks to everyone for the suggestions. I'd like to avoid sending protocolls around, because I don't know jack about them. I'm willing to learn if I don't find any other means, though. But getting a pointer from one app to the other is in the end all I really need.

I think you will not get away with sending a pointer around. You will have to serialize all data you want to exchange, as long as you are not instancing your objects in shared memory (and this is a different beast).

IMHO, using sockets or using e.g. named pipes in Windows is an almost equal amount of coding to do. In both cases you'd have to design a protocol.

If your project is open-sourced, I could help you out a bit with the socket approach...

regards,
Face

 

[Moach]

hmm, see THIS

it explains how to create a shared static section in a DLL using #pragma...
it might be windows-specific, but hey... so are DLL's:hmm:

 

[computerex]

I would recommend against doing that. Shared data sections are a pain in the ass in my opinion cause of all the restrictions. I am with face on this one, text based protocols are epically easy to debug, and are very simple. But with sockets it doesn't even have to be text based if you don't want

 

[Face]

hmm, see THIS

it explains how to create a shared static section in a DLL using #pragma...
it might be windows-specific, but hey... so are DLL's:hmm:

I know, this is what I meant above with objects in shared memory. You can't simple put a pointer there and pass it around, as the memory the pointer may point to is in the other address-space. So you'd have to put everything in that shared memory, being restricted by certain rules:

• only statically initialized
• no handles (file-handle, device-context, window-handle, etc.)
• no big data-structures, as everything is in the DLL itself

So in the end, most shared-memory approaches boil down to having a string and running a data-exchange protocol. This is only my experience, of course, YMMV.

regards,
Face

 

[jedidia]

You can't simple put a pointer there and pass it around

most inconvenient. I was hoping to be able to do just that. Protocoll it is, then... *sigh*

If your project is open-sourced, I could help you out a bit with the socket approach...

Thanks a lot for the offer. The project is not officially open sourced, because I don't feel comfortable with all the legalities that come with it. However, neither do I want it to be restricted. Everyone that wants to use it can use it, as far as I am concerned (which is about the attitude of the third-party code I used in it).

I think it would already help me a lot to have a code example for BOTH SIDES, i.e. the code I have to insert into one app and the code I have to insert into the other, just to get the connection between the two.

 

[Hielor]

most inconvenient. I was hoping to be able to do just that. Protocoll it is, then... *sigh*

You still can't pass a pointer through any kind of protocol without the corresponding data. The pointer is only valid in the original process's address space.

 

[Face]

Thanks a lot for the offer. The project is not officially open sourced, because I don't feel comfortable with all the legalities that come with it. However, neither do I want it to be restricted. Everyone that wants to use it can use it, as far as I am concerned (which is about the attitude of the third-party code I used in it).

I think it would already help me a lot to have a code example for BOTH SIDES, i.e. the code I have to insert into one app and the code I have to insert into the other, just to get the connection between the two.

There are many, many tutorials on the web for the keyphrase "TCP client server example vc++", most prominently the MSDN examples themselfs:

• client: http://msdn.microsoft.com/en-us/library/ms737591(v=VS.85).aspx
• server: http://msdn.microsoft.com/en-us/library/ms737593(v=VS.85).aspx

But I think you will not be happy with this, that's why I offered my help. Of course, as it is not possible to see the code, that would be a problem .
Anyway, my advice would be to do the following:

1. Summarize what you want to be transfered. E.g. direction vector for target system, system model (names, planets, moons, etc.), status message from Orbiter client,... you name it.
2. Imagine you are the Orbiter client and you are talking to the galaxy generator. What would a typical conversation look like?
3. Tokenize this conversation. I.e., convert the flow of the conversation into well-edged tokens, e.g. initial setup, direction command, system model request, status message.
4. Imagine you have a command-line interface to the galaxy generator. Run a typical session using your tokens, think how the returned output will look like.
5. Now that you have your protocol, implement it as state machine in the server/client code. I.e., first state is initial setup - the "greeting", second state is sending a direction command from server to client, third state is client reporting proper direction alignment, etc...

If you do this all ASCII-based, you could start with the galaxy generator without any Orbiter implementation at all. Just use a telnet client (or e.g. a puTTY with raw connections) to hack in commands and check the output. If this is stable enough for you, go on with the Orbiter implementation itself.

Hope this helps,
Face

 

[jedidia]

But I think you will not be happy with this, that's why I offered my help. Of course, as it is not possible to see the code, that would be a problem

I could send you the code, no problem, but I think it would be a lot of trouble for you to make sense of it and implement the stuff (this is actually my first serious C++ app, so the code is probably a bit unstructured... plus it's undocumented).

Basically, what I have to pass from the generator to the MFD is a rather large data type containing the whole solar system, and three additional coordinates. The only thing I have to pass back is a boolean, indicating wheather the destination has been reached or not. So, what you're saying is that first I should write an "export function" for the data type that exports all of its data into a text protocol, which then can be sent to the mfd. Did I get that about right? If yes, what datatype should that protocoll be in? filestream? std::string? char*?

Maybe you could also shortly explain the difference between ASCII-based and Text based. I mean, a string is basically an array of chars, which are ASCII-codes, or did I get that completely wrong?

 

[Face]

I could send you the code, no problem, but I think it would be a lot of trouble for you to make sense of it and implement the stuff (this is actually my first serious C++ app, so the code is probably a bit unstructured... plus it's undocumented).

You see, showing programming ideas from one programmer to another is best done in code . I could only help you to setup such a thing properly if I can code it right into your system, otherwise I can only fallback to telling you what I think you could do in natural language... and this language is rather indeterministic.
So if you are willed to share your work with me, please send it (via email or hidden download or whathaveyou) and I'll put a prototype framework in. If it is your concern that someone else is seeing or even "stealing" your code, I can only promise you here to not give it away or show it to someone else.
If is is your concern that I might not understand what you did... trust me, I've seen many bad code, and I always "got it" sooner or later. And I think you can only get better at programming if you study as much code (of other programmers) as possible.

Basically, what I have to pass from the generator to the MFD is a rather large data type containing the whole solar system, and three additional coordinates. The only thing I have to pass back is a boolean, indicating wheather the destination has been reached or not. So, what you're saying is that first I should write an "export function" for the data type that exports all of its data into a text protocol, which then can be sent to the mfd. Did I get that about right? If yes, what datatype should that protocoll be in? filestream? std::string? char*?

The "export function" is what is called serialization in literature. So you got that right, yes. Today most serialization is either binary or XML. But you really don't have to go that route. Start with a basic protocol with session initiation and as few information exchange as possible.

Maybe you could also shortly explain the difference between ASCII-based and Text based. I mean, a string is basically an array of chars, which are ASCII-codes, or did I get that completely wrong?

I don't have a strict definition for ASCII-based vs. text-based, in fact I use both for the same thing sometimes. What I basically mean with it is a protocol that is human-readable.

It would go like this:

Client: <Connecting>
Server: <Accepting>"SELECTED Barnard's Star\n"
Client: <MFD displays "Barnard's Star selected, downloading data...">"REQUEST INFO\n"
Server:"NAME Barnard's Star\nCLASS Red dwarf\nPLANETS 3\nPlanet 1\nPlanet 2\nPlanet 3\n"
Client: "REQUEST PLANET 1\n"
Server:"NAME Planet 1\nTYPE Gaseous\n...."
Client: "REQUEST MAP\n" (to display your system diagram)
Server:"START BINARY 1024\n<1024 Bytes binary data for picture>"
Client: "REQUEST BEARING\n"
Server:"BEARING 0.577350269 0.577350269 0.577350269\n"
Client: <displays map, data and bearing in MFD>"REQUEST EXPORT\n"
Server:<exports system configuration files>"REQUEST READY\n"
Client: <Disconnecting><loads Orbiter with system><Connecting>
Server:<Accepting>"SELECTED Barnard's Star\n"
Client: <is at Barnard's Star and knows it>"REACHED SELECTION\n"
Server:<Updates map to show new position>
Client: ...

Is this example helpful to get the idea through?

regards,
Face

 

[Moach]

i believe this is pretty much it... for char* types, that is...


another possibility, perhaps still unexplored, is to adapt your generator code to compile as a DLL... with a handful of cleverly placed #ifdefs, you can even have it go both ways at the turn of a switch...

if you develop with that i mind, the only major code difference should be the entry point...


this approach, however clumsy at first, will get you the best possible integration to Orbiter, since once attached, your generator will be running in the same process as the simulator, allowing pointers and whatnot to be freely handed around


i think it's worth considering...:hmm:

 

[Enjo]

At my job I use Boost Interprocess for such things. The synchronisation section will interest you, if you want to use it.

 

[jedidia]

another possibility, perhaps still unexplored, is to adapt your generator code to compile as a DLL...

Since the generator has its own graphic window and uses the irrlicht 3d engine, I don't think that's possible...

If it is your concern that someone else is seeing or even "stealing" your code

Nah, not at all. I would feel kind of honoured if someone would consider it good enough to find it worth stealing :lol:

If is is your concern that I might not understand what you did... trust me, I've seen many bad code, and I always "got it" sooner or later.

Ok, I'll wrap it up for you this evening, then. Thanks a thousand lots!

Is this example helpful to get the idea through?

it is very helpfull in that I finally realise that it's actualy a conversation, with bits sent after each other, and not everything sent in one Block!

 

[computerex]

Since the generator has its own graphic window and uses the irrlicht 3d engine, I don't think that's possible...

Well if the application has to run at the same time as Orbiter, I think you should be able to simply wrap the thing up as an Orbiter plug-in, then you wouldn't need sockets. Communicating with one dll to another one in the same process is much easier

 

[jedidia]

Well if the application has to run at the same time as Orbiter, I think you should be able to simply wrap the thing up as an Orbiter plug-in, then you wouldn't need sockets. Communicating with one dll to another one in the same process is much easier

No doubt, but I'll have the same program using two different display devices, eventhandlers aso. I don't think that's possible, or is it?