The Future of Lua with Orbiter? Is it at a Dead-End?

[Thunder Chicken]

Regarding the truncation of floats: This is something I've already noticed. This also has nothing to do with the changes in Interpreter.dll, this happens with the direct C++ calls as well.
This bug might have been introduced by me lately but I have to investigate in that. It seems like the default precision has somehow dropped to 3 instead of the expected 6.
This looks like a unexpected change in the stream-based I/O library. I will fix that whenever I find the time.

I'm just doing testing to see the behavior of the methods when various inputs are supplied.

Inputs as formatted in the script, all types verified:

VAL_STR = "foo" --string
VAL_INT = 42 --number
VAL_FLT = 3.141592 --number
VAL_VEC = {x=1.20, y=-3.40, z=5.60} --table (Orbiter vector)

oapi.writescenario_string

Proper use works:

• oapi.writescenario_string(scn, "VAL_STR", VAL_STR) writes VAL_STR foo

It seems that writescenario_string will write Lua numbers without any modification:

• oapi.writescenario_string(scn, "VAL_FLT", VAL_FLT) writes VAL_FLT 3.141592
• oapi.writescenario_string(scn, "VAL_INT", VAL_INT) writes VAL_FLT 42

If supplied a table, method fails, but simply doesn't write anything to the output scenario file:

• oapi.writescenario_string(scn, "VAL_VEC", VAL_VEC) writes nothing.

oapi.writescenario_int

Proper use works:

• oapi.writescenario_int(scn, "VAL_INT", VAL_INT) writes VAL_INT 42

If supplied a float, it truncates it to an integer:

• oapi.writescenario_int(scn, "VAL_FLT", VAL_FLT) writes VAL_FLT 3

If supplied a string, method fails, but simply doesn't write anything to the output scenario file:

• oapi.writescenario_int (scn, "VAL_STR", VAL_STR) writes nothing

If supplied a table, method fails, but simply doesn't write anything to the output scenario file:

• oapi.writescenario_int (scn, "VAL_VEC", VAL_VEC) writes nothing

oapi.writescenario_float

Proper use works, but truncates output:

• oapi.writescenario_float (scn, "VAL_FLT", VAL_FLT) writes VAL_FLT 3.14

If supplied a integer, it writes it as a float:

• oapi.writescenario_float (scn, "VAL_INT", VAL_INT) writes VAL_INT 42.00

If supplied a string, method fails, but simply doesn't write anything to the output scenario file:

• oapi.writescenario_float (scn, "VAL_STR", VAL_STR) writes nothing

If supplied a table, method fails, but simply doesn't write anything to the output scenario file:

• oapi.writescenario_float (scn, "VAL_VEC", VAL_VEC) writes nothing

oapi.writescenario_vec

Proper use works...:

• oapi.writescenario_vec (scn, "VAL_VEC", VAL_VEC) writes VAL_VEC 1.20 -3.40 5.60

But if the table is modified to have higher precision it truncates the output:

​

VAL_VEC = {x=1.205730, y=-3.406839, z=5.600375}​

• oapi.writescenario_vec (scn, "VAL_VEC", VAL_VEC) writes VAL_VEC 1.21 -3.41 5.60

It also doesn't deal with very small or large numbers very well:

VAL_VEC = {x=1.205730e-5, y=-3.406839e4, z=5.600375e-2}​

• oapi.writescenario_vec (scn, "VAL_VEC", VAL_VEC) VAL_VEC 0.00 -34068.39 0.06

Attempting to write any of the other variable types fails and are simply not written out:

• oapi.writescenario_vec (scn, "VAL_INT", VAL_INT) writes nothing
• oapi.writescenario_vec (scn, "VAL_FLT", VAL_FLT) writes nothing
• oapi.writescenario_vec (scn, "VAL_STR", VAL_STR) writes nothing

I'd say that the truncation of the floats and loss of precision with large and small numbers, particularly in the vector tables, is probably the biggest issue I see right now. This can temporarily be side-stepped by writing out everything as strings, but that's probably not desired long term.

 

[kuddel]

Hi @Thunder Chicken ,

1. thanks for the testing
   
2. The issue with truncating of floating-point values (C++ double types to be precise) is something that has been present in Orbiter since ages
   That's why most of the add-ons that needed a specific precision internally build a string and use writeScenario_string(...).
   That is something that is very addon-specific and is still the best way if "default behaviour" is not enough for accuracy/precision.
   I am currently working on a fix that makes this a bit more predictable!
3. Very large or very small values are always a problem when serializing ("to string") them. But the string-work-around (see 2.) is available, so there's nothing we should really do about this.
   By the way, the Lua value of -3.406839e4 to be written as -34068.39 is 100% accurate
4. My approach for improving the serialization is something like this:
   a) precision is always 6,
   b) trailing zeros will be cut ("1.200000" => "1.2")
   c) writeScenario_float(...) will always serialize with a decimal point ("1.0" will not become "1", but "1.0").
   This is not absolutely neccessary, but I think it's nice to see in the scenario file that this value was/is a floating point value...
   d) the same rules also apply to vectors serialized by writeScenario_vec(...) - as they are just 3 doubles.

Thats just to inform you. I now have to create an issue & provide a PR to fix it. (...and hope it gets merged )

 

[Thunder Chicken]

Hi @Thunder Chicken ,

1. thanks for the testing

Well, I was the one clamoring for the fix, so I tried to do thorough testing in compensation

1. The issue with truncating of floating-point values (C++ double types to be precise) is something that has been present in Orbiter since ages That's why most of the add-ons that needed a specific precision internally build a string and use writeScenario_string(...).That is something that is very addon-specific and is still the best way if "default behaviour" is not enough for accuracy/precision.I am currently working on a fix that makes this a bit more predictable!
2. Very large or very small values are always a problem when serializing ("to string") them. But the string-work-around (see 2.) is available, so there's nothing we should really do about this.

I wasn't aware it was a long term issue in Orbiter. As long as the behavior is known. Just manipulating things as strings through Lua really isn't a problem.

By the way, the Lua value of -3.406839e4 to be written as -34068.39 is 100% accurate

Well, yes, that's true. It was a quick but poor example, but you get the point.

I now have to create an issue & provide a PR to fix it. (...and hope it gets merged )

Thanks for your efforts. Looking forward to being able to read and write to scenario files finally.

 

[kuddel]

PR for the "predictable precision" has been requested

Fix for #410 (unpredictable precision) by schnepe2 · Pull Request #411 · orbitersim/orbiter

Fixes issue #410 Floating point output stream formatter added to serialize double/float values nice & tidy.

github.com

After that is merged, the ScriptMFD changes can be merged.

 

[kuddel]

Update: PR for "the rest" has been requested to merge.
...after a very very very undelightful fight between me and the CI tests ...One could argue, that without tests it would have been ready to merg a week earlier

Lua extension by schnepe2 · Pull Request #414 · orbitersim/orbiter

PR to resolve #404 & to resolve #405. This PR has currently only one unresolved issue: The automatic (headless) tests currently can not run MFD tests. Still I like to have at least some kind of Tes...

github.com

 

[n72.75]

Update: PR for "the rest" has been requested to merge.
...after a very very very undelightful fight between me and the CI tests ...One could argue, that without tests it would have been ready to merg a week earlier

Lua extension by schnepe2 · Pull Request #414 · orbitersim/orbiter

PR to resolve #404 & to resolve #405. This PR has currently only one unresolved issue: The automatic (headless) tests currently can not run MFD tests. Still I like to have at least some kind of Tes...

github.com

For some reason, I am only just now connecting these two usernames.

I see we have tested that the Github CI builds work correctly, but has anyone tested that each of these work correctly. @Thunder Chicken is rapidly becoming our resident Lua enthusiast

Sorry if I've missed a discussion elsewhere.

 

[kuddel]

@Thunder Chicken : The PR has been merged. So you can now enjoy the new Lua functions by downloading the latest build.

Changes:​

• added following file i/ofunctions to Lua interface:
  • openfile(fname,mode,root)
  • closefile(fname,mode)
  • savescenario(name,descr)
  • writeline(f,line)
  • writescenario_string(scn,item,str)
  • writescenario_int(scn,item,i)
  • writescenario_float(scn,item,d)
  • writescenario_vec(scn,item,vec)
  • readscenario_nextline(scn)
  • readitem_string(f,item)
  • readitem_float(f,item)
  • readitem_int(f,item)
  • readitem_bool(f,item)
  • readitem_vec(f,item)
  • writeitem_string(f,item,str)
  • writeitem_float(f,item,d)
  • writeitem_int(f,item,i)
  • writeitem_bool(f,item,b)
  • writeitem_vec(f,item,vec)
• added following utilityfunctions to Lua interface:
  • formatvalue(val,prec)

All new functions are "oapi" module functions, so to be called with oapi. prefix.
The documentation-sources are available, but currently no new documentation artifacts are generated.
If you like to get some detailed information on these new functions, I would recommend looking at the source code (mainly somewhere around here).

Have fun

 

[Thunder Chicken]

I see we have tested that the Github CI builds work correctly, but has anyone tested that each of these work correctly. @Thunder Chicken is rapidly becoming our resident Lua enthusiast

I coded many of those functions into my autopilot MFD readstatus() writestatus() functions already, but commented them out until these methods would become available. I also need the vessel read/write functions for my Mirage2000 Lua add-on for the animation states and such. I have much on my plate at the moment but I hope to do some live testing of these very soon.

Progress! Now if I could just get OpenOrbiter to function faster than 1 frame per minute on my laptop. Not sure what that problem is. Orbiter 2016 runs like a champ under Wine, but OpenOrbiter is slow, and terrain graphics don't show up at all.

 

[kuddel]

[...]Not sure what that problem is. Orbiter 2016 runs like a champ under Wine, but OpenOrbiter is slow, and terrain graphics don't show up at all.

Are you sure you've selected "the same" D3D9Client? Which is included in OpenOrbiter

 

[Thunder Chicken]

Maybe a dumb question as I have done it before, but where can I download the updated Orbiter release? I keep flipping through Github but I can't seem to find it. I really can't build it myself.

EDIT: Nevermind, found them.

Release Orbiter development build · orbitersim/orbiter

Commits e5ddee7: Reuse memory instead constant release and reallocation. (jarmonik) #446 3954644: Added resource cache for all tile related resources. Cleaned up tile loading implementation. (jarm...

github.com

 

[kuddel]

On github page (https://github.com/orbitersim/orbiter) select "Releases" (at the right side), then under "Assets".

For your 32-bit needs this direct link should always work: https://github.com/orbitersim/orbiter/releases/download/latest/Orbiter-x86.zip

 

[Thunder Chicken]

Are you sure you've selected "the same" D3D9Client? Which is included in OpenOrbiter

I'm not sure what you mean by "the same"? On a fresh installation, I open Orbiter, under Video I set it to full screen and set the screen to 1366 x 768 (my screen resolution), select DG at KSC scenario, and this is what I get:


Just tapping the main engine thruster sends me into orbit:

I'm literally getting about 1 frame every 10 seconds or so. Completely unusable.

 

[Thunder Chicken]

This is all I get for video options:

 

[kuddel]

One select-box further up! Called "Graphics engine:"

 

[Thunder Chicken]

One select-box further up! Called "Graphics engine:"

There are no other options under that pull-down:

 

[Thunder Chicken]

Here's what I get when I run a fresh install of Orbiter 2016:


Frame rate is fine, everything works as it should.

 

[kuddel]

Hmmm, might be that D3D9Client is not available for the 32-bit builds

 

[Thunder Chicken]

Hmmm, might be that D3D9Client is not available for the 32-bit builds

I have downloaded the 64-bit build, but I have no idea what to do with it as it only has the orbiter_ng.exe executable.

 

[kuddel]

That's the one you should run (even in the 32-bit version).
Only Orbiter_NG (Next Generation ) will allow you to select Graphic Clients.

 

[Thunder Chicken]

That's the one you should run (even in the 32-bit version).
Only Orbiter_NG (Next Generation ) will allow you to select Graphic Clients.

I get this pull-down in the 64-bit orbiter_ng.exe:

But it crashes when I select D3D9Client:

In the 32-bit Orbiter-x86 orbiter_ng.exe I get a pull-down with both D3D7Client and D3D9Client. But selecting D3D9Client also crashes Orbiter with the same error as above. Restarting and selecting the D3D7Client works, but it gives me the exact same graphics and lag issues as the 64-bit Orbiter.exe.

I've attached the Orbiter-x86.log that shows a bunch of errors for when it runs but has graphics/lag issues. I've also included the log file for Orbiter-x64 when it crashed. Not sure if any of this helps but it seems nothing works for me.