Project Orbiter economic simulation

MeeexC

Member
Joined
Nov 19, 2011
Messages
37
Reaction score
21
Points
8
Location
Ulm
Hello!


Here I would like to present an idea and a project that will accompany me in the near future.

But briefly something in advance:

I played Orbiter very actively until the release of Orbiter 2016. Many of my favourite add-ons didn't work any more and in addition I had less or no free time. I always took a look at the forum to see where things were going. Also, Orbiter is my playground when I want to try some things that don't fit into my work. So I had a very messy dev folder and many Orbiter versions.... However, I never finished the add-ons, everything stayed in experimental state.

What is different this time?

Not much, I still have a time-consuming job and my family has grown. However, I really want to learn and experiment with what is to be implemented here even if it doesn't fit into any of my current projects. The chance that it will take a very long time is high though, since I can only clear a few hours per week at a time.


But what's the point now?

I want to play around a bit with AI in Orbiter. I'm a scientist by training and have always worked with data analysis and physical simulations. In the last few years, however, this has evolved into a new direction, and it now ends up with control systems based on physical models. In doing so, I incorporate AI, such as models represented by artificial neural networks. Now I would like to try one here. But for that it needs a concrete development goal:


A very simple economic simulation with artificial counter-players. Here AI is used to simulate them.


Features:


1. there are resources, which can be mined on the planets/moons with mines

2. transport of the resources to earth brings credits

3. the credits can be used to spawn bases, mines, ships or anything similar

4. integration of an AI

4.1 Control of ships in the fleet that are not operated by the player

4.2 Control of one or more computer players


My interest is mainly in the last feature. However, I found 1. - 3. needs the add-on to be more rounded; and I to get back into the API of Orbiter.


So far there are only fragments to all this. And I don't even like them, that's why I want to start over from the beginning.


Why do I post the project now?

Because I want to post the detailed developments to the listed points. I want to discuss as much as possible here (to motivate me to continue) and since I can imagine making the project open-source someday it would be not bad to document a lot from the beginning.

So I would be very happy if you give me feedback on the ideas or my approaches, I am very sure that many people here have better suggestions than me when it comes to implementing something in Orbiter. I now have to read back into the Orbiter SDK and see what was possible or what I remember.

Now to my first approaches:


To 1. it would be nice if resources on the moons and planets are heterogeneously distributed and one would have to scan the surface with satellites to find worthwhile spots. But here I would not know how to implement this. So my first version would be that the resources are a property of the mine. The easiest way I can think of is to declare a fuel as a resource. This is then continuously replenished up to the capacity limit.


To 2. there is a station in the earth orbit, which serves as storage for the mined resources. In a base on earth the exchange in credits takes place. I think the resources can be as with the mine also simply as "Fuel" bound to a container, which is then transported by the ship (I bet here a little on the fact that the UCGO update comes).

I want everything to run in game, so I thought the fuel system is suitable, I seem to remember that with VESSELSTATUS2 multiple fuels were also possible (LOX for example).

As for the credits, I don't have an idea yet. Either they are like the resources and then just accessible at the earth base (where you can then buy new stuff). Or you can get them from a MFD. Generally I think about letting the whole interaction run over a MFD. Maybe you have a good idea how this can be programmed best. Somehow I like the idea to implement the resources and the credits as fuel, because I think everything is already available in Orbiter, you just have to call it differently. But I'm not sure yet that it works that way.


To 3. that depends now strongly on the first two points. It would be cool if there is a dry dock in Earth orbit where you can buy ships and base containers, which then become a mine when landing on a planet.


The first three points should be very easy to implement in the first version. I have to try out my suggestions here first. If you already have input for me or know directly how to implement something I would be very grateful.


To 4. this is somehow my core business. I want to build two separate AI modules here. The first one is an autopilot+. Here it should be possible to control every ship in the sim (that you own) remotely. It should be possible to fly from base to base automatically. That means you are either landed/docked or somewhere in orbit and now a target is set, no matter where and the ship flies there and lands by itself. For aerodynamic ships an atmospheric landing should also be possible. So this is the most complex task (but this is where I want to play). In principle, all the necessary tools already exist: IMFD to do the routing and plan the approach, landing autopilots for atmosphere-less celestial bodies, and tools to help you land on Earth. Right now I'm thinking about how to connect all this without having to reinvent the wheel. In the end the player stands with a ship of his choice on the runway and enters a base on Mars as a destination, then the autopilot takes over the planning of the route (time window and dV optimization) and performs the entire flight to landing. Once arrived, a trade task can be executed and a new flight is already saved (e.g. back to the runway on Earth).

However, I must say I imagine just the automatic approach to Earth with Reentry still difficult. But I don't see any point in all this that should not work. Do you?

The second module takes over the decisions for the computer player. So it programs the autopilots of the ships, builds mines and buys new ships etc… So the space should be fuller and give the player a possibility to compete.


That is so far my rough plan.


But I am very limited in time, so this project will be a generation project I am sure. If someone among you should have desire to participate I am pleased of course very much about it!


Now I am curious about your feedback!


Best regards

Max
 

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
36,860
Reaction score
1,526
Points
203
Location
Langendernbach
Sounds interesting, I am sketching out something similar right now, but at a lower pace, while following the discussions in this thread: https://www.orbiter-forum.com/threa...ar-space-actually-look-like.39594/post-575918

But my focus is rather on the middleware of things, between OrbiterAPI and the actual scenario. I want to have some set of vessels, modules and tools, to create scenarios, that are simply a long-running economic simulation from that entry point.
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,357
Reaction score
512
Points
153
Location
Vienna
2 thoughts:
  1. If you primarily want to play with AI, why do you blow your project's scope up that much? The task of coding a universal AI autopilot is a huge project on its own already, I'd say.
  2. If you want to test your AI economic simulation, why not start in a secured environment first before diving into the Orbiter specifics?
 

N_Molson

Addon Developer
Addon Developer
Donator
Joined
Mar 5, 2010
Messages
8,855
Reaction score
2,584
Points
203
Location
Toulouse
Interesting, I had such an idea more than once, while not knowing how it could be implemented.

The advice above seems to be good : coding a fully functional autopilot/AI might well be the greatest challenge and should be a work on its own. In its simplest form, it could be used to add "random ATC" traffic to Orbiter, like in Flight Simulator. Of course it is a bit pointless in historical scenarios (you don't have spacecraft jams wait in line to dock to the ISS, things are planned), but in the sci-fi context of the DeltaGlider, it could really add a lot of immersion. I think DanSteph did something like basic ATC for an asteroid base, you might want to check that.

For the economic aspect of thing after a decade thinking about it, my feelings are mixed. As you wrote, it would have a lot to do with UGCO. It means working closely with a quite complex add-on written by someone else. In a perfect world, DanSteph would develop such an addon and I'm sure he had thoughts about it. Maybe you could try to contact him on his forum (there is an English section) and share your thoughts ?

Now with my very limited knowledge of computer science, I feel that time acceleration is going to ruin a lot of things. A full-AP atmospheric reentry and landing sure can be done, but unlikely with time warp on. You'll probably have to move most of the traffic to an "abstract" level, where you say "such vessel should take 15 minutes to reenter and land", without fully simulating things.
 

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
36,860
Reaction score
1,526
Points
203
Location
Langendernbach
Hmmm.... what about some contest to develop AI spacecraft for Orbiter?

  1. We start out with the Basic Shuttle-A example code.
  2. We modify this minimally to provide some basic framework for simulating any kind of AI with it.
  3. This framework should also provide means for adapting to time acceleration.
  4. We also install diagnostic functions into it to verify, that allows testing the AI during flight.
  5. This is then the base for implementing different AIs.
  6. These have to run some different scenarios for verifying its capability:
    1. A general flight dynamics test to ensure that the flight model was not altered.
    2. A flight from Earth to moon and back, minimum time
    3. A flight from Earth to moon and back, minimum fuel
    4. etc...
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,357
Reaction score
512
Points
153
Location
Vienna
Well, I may be mistaken, but aren't most of the flight simulator "AI" implementations simple flight path calculators, that replay the plane's path while not controlling the plane according to flight model at all?
Certainly one thing is to have a proper controlling autopilot while another is to provide an immersive and performant AI traffic.
 

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
36,860
Reaction score
1,526
Points
203
Location
Langendernbach
Thats true - maybe the middleware could then be made much simpler and restricted to task planning alone.
 

N_Molson

Addon Developer
Addon Developer
Donator
Joined
Mar 5, 2010
Messages
8,855
Reaction score
2,584
Points
203
Location
Toulouse
Well, I may be mistaken, but aren't most of the flight simulator "AI" implementations simple flight path calculators, that replay the plane's path while not controlling the plane according to flight model at all?
Certainly one thing is to have a proper controlling autopilot while another is to provide an immersive and performant AI traffic.

I think you have a very good point. Video games are like movies, its a lot about illusion. Modern action movies are very immersive yet in reality you would see at best a couple of actors playing in front of a green screen. In flight simulators yes AI planes typically do crazy things, especially when it comes to taxiing and taking off, and in FSX you see airliners taking amount of Gs that would probably gooify all their passengers. ?

So no it doesn't have to be accurate, especially not if the main focus is an economic simulation. We sure can tolerate that the AI vessels sometimes act a bit weird, as long as it is immersive enough. A fully accurate simulation sounds like a dead end to me, again time warp, and also it will become unbearable if you have a few dozens AI vessels loaded. Because the Solar System is really huge, we need an ungodly LOT of AI vessels if we want to encounter one.

From there, I can imagine something which is probably somewhat KSP-ish :

1) You simulate flight plans with departure and arrival times. At the most basic level the vessel plot from A to B is a linear path (highly unrealistic, but with maths/physics skills you can refine it). The vessel itself isn't loaded at all in the simulation ; the AI module (dll) tracks its flight path and stores the data in a config file.
2) When the AI plot says it should be in visual range - it doesn't even has to be super-accurate - (in a bubble a few kilometers wide around the player's vessel), you spawn an actual instance of the AI vessel. There you can load a flight model. Flight Simulator uses very simplified flight models for AI planes, and I think its an acceptable tradeoff, because that way writing the actual autopilots will be much less a hassle. Also you save some computing power : after all you might want to simulate a Coruscant-like planet with dozens of flying cars buzzing around. It would be so cool. If time warp is an issue just disable the "bubble" when time warp is on (yes hitting time warp would instantly clean the skies around you which is a bit weird but I think you could probably use x10 - see how KSP limits the warp to x4 when you hit the atmosphere and even warns you with a popup when calculations are lagging behind).

From there you can even implement a very basic collision detection system : a simple check that compares your vessel radius (say the wingspan for a DG) with the AI vessels. Not super accurate but that's not important, the idea is that to enforce the simple fact that you should not fly straight into another vessel path or park in its spot. The tricky part might be if you want the player to be able to dock with the AI vessel, you might want to de-activate the check (or make sure the docking port is out of the "collision bubble").

So yes, I'm pretty sure something fun can come out of this. Maybe not something accurate, but that could still give O2016 like a second breath :cool:
 

Marijn

Active member
Joined
Mar 5, 2008
Messages
744
Reaction score
144
Points
43
Location
Amsterdam
I have an unfinished project which allows the user to browse for a high valued asteroid using the API of Asterank browser (https://www.asterank.com/). Then it checks if NASA has already calculated a trajectory for it (https://trajbrowser.arc.nasa.gov/traj_browser.php). If so, you can create a mission for the XR series to that asteroid. The user can change many settings and on each change the rocket equation is ran to reflect the new status. When done, a download is triggered which has the normal Orbiter add-on structure, so you'll unzip it in Orbiter's root folder.

The NASA trajectory browser has all the state vectors and timestamps hidden in the page source. The UI doesn't show it, but with this information you can start the mission at multiple stages. For example, you can download the landing part an all consumables will have the amounts which you would have when starting from earth. Also, I was able to automatically set up the instruments like IMFD for each stage. It's pretty neat. Hopefully I am able to finish it. Here's an impression:
 

Owenmck

Read the documentation!
Joined
Apr 18, 2020
Messages
125
Reaction score
115
Points
58
Location
Scotland
From there you can even implement a very basic collision detection system : a simple check that compares your vessel radius (say the wingspan for a DG) with the AI vessels. Not super accurate but that's not important, the idea is that to enforce the simple fact that you should not fly straight into another vessel path or park in its spot. The tricky part might be if you want the player to be able to dock with the AI vessel, you might want to de-activate the check (or make sure the docking port is out of the "collision bubble").

Perhaps a kind of docking request? That would be cool, having to request docking permissions from an AI vessel...
 

N_Molson

Addon Developer
Addon Developer
Donator
Joined
Mar 5, 2010
Messages
8,855
Reaction score
2,584
Points
203
Location
Toulouse
Might work though I can imagine a couple of sci-fi scenarii where you have to deliver a shipment of angry space-marines to an alien-infested station ?
 

MeeexC

Member
Joined
Nov 19, 2011
Messages
37
Reaction score
21
Points
8
Location
Ulm
Okay mates I'm a bit surprised how much feedback this thread is getting. Somehow it is also quite nice that Orbiter has shrunk to the "inner circle".

Thanks a lot first. I wanted to reply to all the posts directly, but I think that would be too chaotic. So I'll try to summarize my thoughts:


First, I like the idea of Urwumpe very much, such a simulation would just also tease me. It's like in Europa Universalis let the AI do everything and time warp; who knows that game....
Face is right of course! The autopilot is the center of attention for me. It would also be enough to just take care of it. I had only two fears: On the one hand, I thought programming this mining base would bring me back up to date in terms of API. On the other hand, I wondered what someone would do with the autopilot alone if they weren't an add-on developer. Doesn't it take all the fun out of just typing in a destination and the autopilot does it already?


But okay, I'll take the economic simulation to the back (also because of UCGO), the autopilot is necessary here too, so I'll start here. Later, the whole economy can be integrated.

A few things about the autopilot:

- Normally I work in Matlab, because I often have to include hardware, and I'm just very slow in C. First of all I will try to get a communication between Matlab and Orbiter with Orb:Connect or Orb::Connect Web Edition. Martin Schweiger once answered to me that he has implemented a solution for this too (Direct Data Exchange) Let's see what I can realize in the short term.

- The time warp thing is really a point. I already had this on the list, but I haven't found a good solution yet. First of all I have to ask here where the bottleneck is. I think the physics in Orbiter can be simulated accurately at any rate. So it depends only on the reaction time of the autopilot? Or am I missing something, like a timer adapted to the frame rate or something similar.

I didn't come up with the solution of using ready-made trajectories and adapting the vessel status to them, that's a good idea and Marijn's cutout of the planning tool looks very good! Especially that you can update the MFDs I find super cool!

The approach I had in mind is based on how I want to design the autopilot. A model of the ship is represented by an artificial neural network (ANN). So this ANN behaves like the ship itself depending on its environment. By comparing it with "reality", this ANN becomes better and better.The controller now tries to output the correct impulses with the feedback of this model. The controller itself can be an ANN, which has been trained with the help of a reference (trajectory in our case) and becomes better and better with the help of "punishment criteria" (or does more and more what the reference specifies). Why so complicated? Because I want to play exactly with that AND I wanted to use the model of the ship for the time warp. Here, the actual laws of physics in Orbiter and the control commands of the autopilot are suspended and the ship follows a trajectory that the model specifies. To me this just makes sense, I don't know how far I can show it here. The charm of this variant would be that the ships don't follow a path like "on rails", but make "corrections". ANNs are very cheap in computing cost once trained. So time warp should not be a problem here. What do you think?

The cool thing is, the controller ANN represents a pilot. There is an initial training and from then on the "pilot" learns something with every flight: min of flight duration and dV. But I have to keep in mind the cost of computing performance. I work normally on CPU/GPU clusters and never have to worry about that.


The questions that need to be addressed now:

- Communication between Orbiter and Matlab

- Integration of existing tools (Ascent autopilots, IMFD for routing, PursuitMFD for landing and docking, Aerobreak for reentry) or rather setting up everything from one piece?

- An ANN learning environment to integrate new add-on ships?
 

N_Molson

Addon Developer
Addon Developer
Donator
Joined
Mar 5, 2010
Messages
8,855
Reaction score
2,584
Points
203
Location
Toulouse
Somehow it is also quite nice that Orbiter has shrunk to the "inner circle".

That's scares me a little bit, will it go nova or white dwarf ? ?

I think the physics in Orbiter can be simulated accurately at any rate.

Well back in the days one told me that using any time warp actually degraded the quality of the simulation (the sim "skips" steps, say if you take a trajectory "drawn" from 100 points at x1, it will only use 10 at x10 I'm exagerating but you get the idea). That's why Martins hardcoded the behavior of the major planets and moons, because those have to behave correctly at any rate (else they would fly out the Solar system or something). For that he used data from observatories. Well that's what I remember I could be wrong. Also the good thing is that today computers can do much more than their early 2000's ancestors, which means that brute computing force can be an option.
 

MeeexC

Member
Joined
Nov 19, 2011
Messages
37
Reaction score
21
Points
8
Location
Ulm
That's scares me a little bit, will it go nova or white dwarf ? ?
Me, after one year of pandemic and home-office ... I gained definitely critical mass ☀️

I think during this week I will play a bit with Orb:Connect and see what I can find out about time warping. Furthermore, I will report my findings and then lets see how to move on.
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,357
Reaction score
512
Points
153
Location
Vienna
Hm. I understand the goal of creating a universal autopilot with the help of machine learning. This is certainly a very interesting topic and a challenging task. Seen on its own, I'd say go ahead and have fun. However, in the context of what you first proposed, mentioned as to-be-integrated-later (economics), and highlighted as low value (no fun to type in and let it fly), I'd suggest to drop the very technical autopilot idea and instead concentrate on the overall traffic idea.
I.e.: don't use your machine learning approach to create the ultimate autopilot, instead use it to create a traffic environment. Don't let the AI control the single vessel, instead let it act as route planer. Don't control the airfoils to make it turn perfect, instead present the way-points that are necessary to go from A to B.

Your neural network would then be the space agency administration, not one of the dozen pilots flying around.

With this, you would have an equally interesting and challenging task on your hands, with the added value of immersion-enhancing AI traffic for the community. If the actual execution of your way-point is then simply replayed without flight model at all, properly flown by a traditional control theory autopilot, or run by a pilot AI, is secondary to the value generated.
In addition, the traffic AI also is a stepping stone for your economics goal, because once you have the machinery, you can learn a second instance that gives focus to the economic aspect of its paths (being a strong economic competitor) instead of the immersion aspect (do as many flights as necessary to make it feel alive).

On the technical side, this would also lessen the burden of making it overly performant, because the path modeling can be done in a much slower pace than the vessel controlling. In addition, you can do it cloud based like e.g. MSFS does with the world model population AI to guess buildings from ground textures, so the actual integration into Orbiter via various interfaces is not so important at first. You could concentrate on what your personal interest is at the moment: building a machine learning system, then train it. E.g. by simply reading the solar system configuration files to build up an internal object database as input, and generating scenario files as output, you already have a rudimentary interface to Orbiter to train it on.
 

MeeexC

Member
Joined
Nov 19, 2011
Messages
37
Reaction score
21
Points
8
Location
Ulm
Hm. I understand the goal of creating a universal autopilot with the help of machine learning. This is certainly a very interesting topic and a challenging task. Seen on its own, I'd say go ahead and have fun. However, in the context of what you first proposed, mentioned as to-be-integrated-later (economics), and highlighted as low value (no fun to type in and let it fly), I'd suggest to drop the very technical autopilot idea and instead concentrate on the overall traffic idea.
I.e.: don't use your machine learning approach to create the ultimate autopilot, instead use it to create a traffic environment. Don't let the AI control the single vessel, instead let it act as route planer. Don't control the airfoils to make it turn perfect, instead present the way-points that are necessary to go from A to B. [...]
I follow. You are of course correct when you say computer generated traffic has more added value to the community than an autopilot. I also see that I need the routing for the ships and the trajectories anyway, regardless of whether the airfoils and RCS are used in the end to follow the flight path or not. Which doesn't preclude integrating that later, if I still have the time and mood for it then :D.

I'll have to think about how to implement this. With my autopilot approach, the trajectories would have resulted dynamically from the simulation. With your suggestion, I'll have to figure out how to properly map the landing approaches with reentry.

You would then precalculate this and insert it into the scenario file? Then, however, nothing can develop further in the simulation, at some point all flights have arrived or one builds in loops (I do not remember at all what is possible there, I have always treated the Scenarien stepmotherly). Or what would be the point here?

I will sketch something tonight and see where I can start then, to keep as many options open as possible. As with any project, I feel that the phase leading up to the first line of code is the most important.


Cheers
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,357
Reaction score
512
Points
153
Location
Vienna
Obviously the idea of using the scenario file as quick interface to Orbiter would mean to augment it with a way-point format for the vessel-specific executor (be it simple path follow, control theory based circuits or AI autopilot). I.e. your output is not only the vessel instance and where it departs from, but also the flight-path your AI created for it.

I do not have too much experience with modern machine learning systems, but I recon they work best if the feedback cycle is short, i.e. there is less chaotic system between their commanding and their learning (just as with real people). So if we see it from the perspective of a space agency director, the cycle could look like so:
  • Start with the presumption of the goal being a certain percentage of traffic on available simulated spaceports. I.e. your input is the goal of having Nx vessels in the action vicinity of every airport X at any given time, your measurement is the real number of said vessels, giving a percentage how far this goal is reached, leading to a good/bad metric for your machine learning. How you distribute this numbers initially and how "action vicinity" is defined, is subject to your world modeling. I'd suggest weighted RNGesus and simple radius distance, perhaps also weighting the later.
  • With this, your machine spits out a scenario with vessels and their appropriate flight plan. You feed this into Orbiter with vessels that can execute this plan, then let it run.
  • As soon as a vessel is at a way-point, save the scenario and recalculate the input for your machine.
  • The learning mechanism can now adjust depending on how good it distributed the traffic, or perhaps even how close the simulated vessel with the given executor followed the flight plan to the way-point. It recalculates the remaining way-points and produces another scenario.
  • Rinse, repeat.
So at first, your neural network would learn to distribute vessels across the solar system to make it feel as alive as possible, even if these flights don't make sense in an economic way. But given the fewer parameters to take into account, I think it is a plausible first attempt.
Once such a cycle is in operation, you can later on tweak the goal and simulation object set, i.e. you can add economic value and cost to each flight, and measure the simulated wealth of the vessel fleet.

But I guess calculating a flight-plan is already hefty, if it takes into accout:
  • dV budget
  • Duration
  • Slingshots
  • Atmospheric effects
  • Thermal and acceleration limits
  • Human endurance
If you manage to train your network for this, it would already be quite a feat.
 

N_Molson

Addon Developer
Addon Developer
Donator
Joined
Mar 5, 2010
Messages
8,855
Reaction score
2,584
Points
203
Location
Toulouse
Can't agree more. It seems quite a huge task so don't be afraid to start small.
 

MeeexC

Member
Joined
Nov 19, 2011
Messages
37
Reaction score
21
Points
8
Location
Ulm
That's a good suggestion. I think I have everything together now to start. I will try something during the next few days because I wonder if it works. I let you know my final thoughts for the beginning later. Maybe I can already show something.
 
Top