The Future of Lua with Orbiter? Is it at a Dead-End?

[kuddel]

This seems to be a Linux issue, which I am not able to help much as I am a Windows guy
Is this maybe related / the same issue ?

 

[johnnymanly]

You're on Linux then?
The latest Orbiter won't run on Linux. https://www.orbiter-forum.com/threads/problem-with-latest-orbiter-build-on-linux.41564/

 

[Thunder Chicken]

You're on Linux then?
The latest Orbiter won't run on Linux. https://www.orbiter-forum.com/threads/problem-with-latest-orbiter-build-on-linux.41564/

It should run under Wine. Orbiter 2016 does fine.

 

[johnnymanly]

Sorry my earlier reply was too brief.
It runs with DX7, not with DX9.

EDIT
It works using DXVK for wine.

 

[Thunder Chicken]

@Thunder Chicken : The PR has been merged. So you can now enjoy the new Lua functions by downloading the latest build.

Changes:​

• added following file i/ofunctions to Lua interface:
  • openfile(fname,mode,root)
  • closefile(fname,mode)
  • savescenario(name,descr)
  • writeline(f,line)
  • writescenario_string(scn,item,str)
  • writescenario_int(scn,item,i)
  • writescenario_float(scn,item,d)
  • writescenario_vec(scn,item,vec)
  • readscenario_nextline(scn)
  • readitem_string(f,item)
  • readitem_float(f,item)
  • readitem_int(f,item)
  • readitem_bool(f,item)
  • readitem_vec(f,item)
  • writeitem_string(f,item,str)
  • writeitem_float(f,item,d)
  • writeitem_int(f,item,i)
  • writeitem_bool(f,item,b)
  • writeitem_vec(f,item,vec)
• added following utilityfunctions to Lua interface:
  • formatvalue(val,prec)

All new functions are "oapi" module functions, so to be called with oapi. prefix.
The documentation-sources are available, but currently no new documentation artifacts are generated.
If you like to get some detailed information on these new functions, I would recommend looking at the source code (mainly somewhere around here).

Have fun

Just a quick question - will the readscenario and writescenario functions work in a vessel script as well as a MFD script? I am trying to sort out how to read and write vessel states to/from the scenario file. I am not sure if equivalents to clbkSaveState and clbkLoadStateEx exist for Lua yet.

EDIT: I looked through Interpreter.cpp in the code base and it seems Lua still only has four callbacks available: prestep, poststep, setclasscaps, and postcreation.

 

[kuddel]

I have no definitive answer here, but for the readscenario_nextline(scn) function all you need is a valid "scn" file handle.
Same goes for the writescenario_xxx(scn,item,value) functions.
So once you got that they work!
I do not know if the callbacks, that provide that handle, are implemented to be called for Vessel scripts.
They might not.

 

[Thunder Chicken]

I have no definitive answer here, but for the readscenario_nextline(scn) function all you need is a valid "scn" file handle.
Same goes for the writescenario_xxx(scn,item,value) functions.
So once you got that they work!
I do not know if the callbacks, that provide that handle, are implemented to be called for Vessel scripts.
They might not.

It does not seem that any such vessel callbacks exist. It's rather the reverse problem that we had with ScriptMFD trying to write MFD states - we had the callbacks, but not the necessary read/write functions. For vessel scripts, we have the read/write functions, but can't get the scn handle through callbacks to read at the start of the session and to automatically write at the end of the session.

 

[Gondos]

So, I've been working on a port to lua of the original Atlantis Shuttle to see how far we can go with this.
The first thing to note is that it's really annoying to debug, the errors are sometimes logged in orbiter.log, sometimes in the console (when starting with no GC), sometimes they are nowhere to be seen... You often have to revert back to "printf" style debugging and launch the process from a command line to see the output.
That being said, I could get the VC working and the autopilot is able to start the gravity turn and drop the SRBs and the ET.
Performance-wise, I could not see much of a difference with the C++ version (both range from 400 to 1000fps during an ascent in x64-debug mode).
The WIP is here if you're interested, there's a LuAtlantis scenario in the Atlantis folder on this branch ; you can start the autopilot from a LuaMFD with a "V:launch()" command.
The next major roadblock is the AscentMfd because in the C++ version it's embedded in the same module as the vessel so it has direct access to its functionnalities. I'll have to figure something out with the ScriptMFD and see how they can play together.
We could promote the ScriptMFD from a plugin to a module but I don't like having every MFDs showing even when their vessel is not loaded so a dynamic registration is still a must have IMHO.
Also some APIs are using references (e.g. CreateVariableDragElement) so we'll have to find a solution for that as well (maybe doing some kind of explicit boxing).
All in all, my main concern is with the debuggability issue, it's too easy to misspell a variable somewhere and having a nil silently inserted that will bite you in the butt on the other side your codebase...

 

[Thunder Chicken]

Performance-wise, I could not see much of a difference with the C++ version (both range from 400 to 1000fps during an ascent in x64-debug mode).

This has been my experience as well. My converted add-ons seem to run as well in Lua form as they did as C++ modules. I haven't clocked them, but I haven't noticed any particular lagginess.

Also some APIs are using references (e.g. CreateVariableDragElement) so we'll have to find a solution for that as well (maybe doing some kind of explicit boxing).

The folks managing the Orbiter code-base have been pretty helpful in getting the remaining C++ API methods mapped into Lua. I've used add_force in Lua to work around this particular missing method but it probably should be raised as an issue in Github. https://github.com/orbitersim/orbiter.

The first thing to note is that it's really annoying to debug, the errors are sometimes logged in orbiter.log, sometimes in the console (when starting with no GC), sometimes they are nowhere to be seen... You often have to revert back to "printf" style debugging and launch the process from a command line to see the output.

I'm an old fart engineer who learned to diagnose code in FORTRAN 77 with print statements so I am not so horrified by this, but folks used to modern full-featured IDE capabilities and debugging might be. There is the oapi.dbg_out(msg) method that can display output to the screen in Orbiter.

But, on the flip side, Lua can be implemented with nothing more than a text editor, which is a win for those who don't program professionally or frequently and struggle to set up compilers, especially if they are on Linux like me. I've found that VS Code with a Lua plugin that annotates and catches basic syntax errors is a good setup that lets me get coding with a minimum of fuss.

All in all, my main concern is with the debuggability issue, it's too easy to misspell a variable somewhere and having a nil silently inserted that will bite you in the butt on the other side your codebase...

That's an intrinsic bug (or feature) of Lua - rather loose data typing and assignment. I don't know if there are script languages that are better in this regard. I am aware Lua is used for add-on development in other simulator and games so it might be the most suitable ecosystem for those who do a lot of gaming.

 

[Gondos]

There's been some progress on the Lua front. Here is a quick preview :

As you can see, performance is ok even on a debug build.
Testers are welcome

 

[Thunder Chicken]

I get Github notifications for all the commits @Gondos has been putting in...getting exciting! Lua is becoming usable! Yay!

 

[Gondos]

To improve the robustness in case of errors in Lua code, I'm trying to use luaL_error inside the C functions handling the Lua interface.
Currently there are lots of different macros/functions to check the validity of arguments but none is using this API.
Does anybody know if there is a reason for that ?

 

[kuddel]

No, I just looked at it and when I needed some new, I did a similar thing Martin had done.
Whether this is good / best practice, I have absolutely no idea.

 

[Gondos]

No, I just looked at it and when I needed some new, I did a similar thing Martin had done.
Whether this is good / best practice, I have absolutely no idea.

lua_error is different from the error macros currently existing in that it does a longjump so care must be taken to free eventual resources that have been allocated since the stack will be unwinded quite brutally. That's why I'm wondering if there may be a reason it's not been used. As far as I can tell if we're careful there should be no problem but I wanted to be sure...

 

[Gondos]

Progress report :
2D panels, virtual cockpit and mesh generation/patching has been added to the Lua branch, it opens a whole new set of possibilities, getting us closer to what's available only in C++ right now.
Here's a quick preview of a DeltaGlider clone in Lua (WIP, lots of things still missing) :

 

[Thunder Chicken]

@Gondos I've updated to the March 21, 2024 version of Orbiter and it looks good, except I am having problems with Orbiter loading the Lua modules associated with my Lua vessel add-on.


If you recall this was the issue that I had during testing. You offered to look at my code to see if there was a way to organize it to avoid these errors. I PM'd you my Lua vessel addon so you can take a look at it. This vessel did work fine previously, but this error checking seems to be preventing it from loading properly. Can this error checking be toggled or disabled?

EDIT: Opened as issue #449 in GitHub https://github.com/orbitersim/orbiter/issues/449

EDIT2: Closed issue #449. There apparently was an undefined animation in my setclasscap, and all vessel capabilities after that were ignored which lead to the errors. Lua just calls the undefined animation nil, but it is never used in code so it didn't matter, but this error checking code doesn't like it.

I'm not quite sure if I like this error checking code. It really didn't steer me to the problem, and actually broke the script that otherwise would have worked fine (and did work fine). Perhaps this error code could be run by setting a debug flag in the Orbiter start menus?

 

[Gondos]

@Gondos I've updated to the March 21, 2024 version of Orbiter and it looks good, except I am having problems with Orbiter loading the Lua modules associated with my Lua vessel add-on.

View attachment 37447
If you recall this was the issue that I had during testing. You offered to look at my code to see if there was a way to organize it to avoid these errors. I PM'd you my Lua vessel addon so you can take a look at it. This vessel did work fine previously, but this error checking seems to be preventing it from loading properly. Can this error checking be toggled or disabled?

EDIT: Opened as issue #449 in GitHub https://github.com/orbitersim/orbiter/issues/449

EDIT2: Closed issue #449. There apparently was an undefined animation in my setclasscap, and all vessel capabilities after that were ignored which lead to the errors. Lua just calls the undefined animation nil, but it is never used in code so it didn't matter, but this error checking code doesn't like it.

I'm not quite sure if I like this error checking code. It really didn't steer me to the problem, and actually broke the script that otherwise would have worked fine (and did work fine). Perhaps this error code could be run by setting a debug flag in the Orbiter start menus?

The error checking is done by Lua itself and cannot be disabled, it has always worked that way. I just made it so that in case of error it shows as an overlay instead of crashing to the desktop. I can make a flag to disable the overlay but since the error handling just aborts a function in the middle of its execution, I'm afraid that silently ignoring it will just cause more headaches...
Edit: I'll have a look to only show the first error because it almost always explodes into a cascade of unrelated errors and the last one showing is often the least useful

 

[Thunder Chicken]

The error checking is done by Lua itself and cannot be disabled, it has always worked that way. I just made it so that in case of error it shows as an overlay instead of crashing to the desktop. I can make a flag to disable the overlay but since the error handling just aborts a function in the middle of its execution, I'm afraid that silently ignoring it will just cause more headaches...
Edit: I'll have a look to only show the first error because it almost always explodes into a cascade of unrelated errors and the last one showing is often the least useful

I just replied to your PM before I saw this. The problem is is that this error checking seems to halt script execution for completely functional scripts for things relatively trivial as undefined variables (which Lua just evaluates as nil and moves on). And for certain errors the script still crashes anyway before one can even see the errors in Orbiter.

I've found that setting the Orbiter logging to verbose puts most of these errors into the log file, which shows all the errors and makes it a bit easier to see trends and diagnose the underlying issue. I rather needed this to diagnose the undefined rudder animation in my code. The first error really didn't tell me anything useful.

Unsure what to suggest, but my general concern is that unless the errors are more clear they are going to be very frustrating to new add-on developers. The hope was that Lua scripting would provide a less frustrating avenue of entry to folks less familiar with programming and intimidated with setting up and using a C++ environment.

 

[Gondos]

I just replied to your PM before I saw this. The problem is is that this error checking seems to halt script execution for completely functional scripts for things relatively trivial as undefined variables (which Lua just evaluates as nil and moves on). And for certain errors the script still crashes anyway before one can even see the errors in Orbiter.

I've found that setting the Orbiter logging to verbose puts most of these errors into the log file, which shows all the errors and makes it a bit easier to see trends and diagnose the underlying issue. I rather needed this to diagnose the undefined rudder animation in my code. The first error really didn't tell me anything useful.

Unsure what to suggest, but my general concern is that unless the errors are more clear they are going to be very frustrating to new add-on developers. The hope was that Lua scripting would provide a less frustrating avenue of entry to folks less familiar with programming and intimidated with setting up and using a C++ environment.

That may depend on the complexity of the addon but I'm not sure ignoring errors will help beginners debug their stuff. Let's face it, nobody looks at the logs, and if there isn't a big red alert message telling there's a problem right now on that very line, it will take some time to track back to that line when something else behaves poorly later.
Usually you want it to fiercely explode as soon as possible and not when you finally use a value that failed to be initialized when the module was loaded.
Anyway, I'll see how to make this behavior configurable.