Project Orbiter::Tools::Mesh::Lint

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
(put on Project, as it is becoming a new baby itself)

Hi.

Due some issues while hacking and animating meshes, I ended up writing a Mesh Lint tool in order to easily detect some common mistakes that happens while development that impairs performance.

Features:
  • Tools for handling meshes, textures, scenarios (WIP) and configuration files (WIP).
    • Lint : checking for errors and inconsistencies
    • copy : convenient copying of artifacts between different Orbiter installations
    • normalize : same than copy, but normalize (see below) the artifact on the process.
    • clean : convenient clean out of artifacts. Nice for test beds.
    • orphan : detects orphaned artifacts
    • rc : Resource Compiler with multiple output formats.
  • Powerful command line interface for automating tasks.
  • Comprehensive reports in text (file and console) and XML (file only) formats
  • Simple but effective Graphical User Interface.

DOWNLOAD: here. The current version is 0.4.1.2 Alpha.

SCREENSHOTS: Google Photos

mesh copy:
It is capable of saving meshes from memory, so it's now possible to fix problems, and not just detect them. A command to test the parser and the save feature is provided (mesh copy) - this will help me to detect any incompatible mesh (and write code to read it). The written meshes are formatted for easy human reading (for double checking), not to optimize size (future versions will handle this better).

mesh normalize:
The tool has also a quick and dirty mesh normalizing (mesh normalize) that, before copying the mesh, do the following actions:
  • Rebuild each Group's geometry:
    • Eliminate duplicated and orphaned vertices
    • Eliminate duplicated and invalid Surfaces
    • Reorder each surface vertex indices ascending or descending (depending of the clockwiseness). This turn surpreendly easy to identify the clockwiseness of the surface, as the indices will be ordered all ascending or all descending. And no, this doesn't changes the clockwiseness of the surface. ;)
    • Fix the GEOM vertices and surfaces counts
  • Fix the Materials count
  • Fix the Materials names section
  • Fix the Textures count
  • Normalize the Texture pathname

Unused Materials and Textures are kept to avoid messing with modules that address them. For the same reason, the Groups are declared in the same order from the source mesh, as also Materials and Textures.

mesh rc:
A (still work in progress) Resource Compiler is currently available. The RC will compile a C++ header file on the OrbiterSDK subdir in the OUTOUT_DIR (or in the directory specified by the target option).

The RC can spit headers (and futurelly C++ source) in the following formats:
  • Orbiter Tool's header format : mesh rc format=header <mask>
    • using #define (default) : mesh rc format=header type=define <mask>
    • using const : mesh rc format=header type=const <mask>
    • using constexpr : mesh rc format=header type=constexpr <mask>
  • Orbiter standard meshc format.
    • using #define (only and default option) : mesh rc format=header [type=define] <mask>
  • SSU enhanced ssumeshc format
    • using const (default) : mesh rc format=header type=const <mask>
    • using constexpr : mesh rc format=header type=constexpr <mask>

The OT header format has the option to generate headers using "#define" as in meshc, or by using "const" as in ssumeshc. Additionally, it's possible to use constexpr (still in tests) for who is willing to limit themselves to C++11 in exchange of somewhat better object code.

Also, a "split file format" is being cooked for the ssumeshc and OT code generation "const" styles, with the header holding only the externs declarations and a counterpart C++ source file with the constants definitions made external. The ssumeshc style file pair aims to be a drop-in replacement for the headers generated by the ssumeshc tool.

The OT header format generate symbols for all named groups, textures and materials using a prefixed format:

GRP_<meshname>_<grouplabel>
MAT_<meshname>_<materialname>
TEX_<meshname>_<texfilename>

For all file formats, the names are normalized to allow only alphanumeric characters and the "_" - all the rest is stripped out, with the exception of the " " that is converted to "_". Repeated symbols are appended by "_" following an incremental counter to avoid name collisions.

Also, if the LABEL tag is not found, the remark on the GEOM tag is used (i.e., everything after the ";", space trimmed, if present). Groups without labels and without remarks on the GEOM tag will be ignored.

The ssumeshc and the meshc formats follows their respective naming conventions, but adds name collision handling. Only groups are named.

Options for customizing the file format and content are currently being planned, but not sure if will be that useful.

As usual, the tool don't overwrite files. If a file already exists, a ".new" appended one is created. If it exists too, the tool aborts the current mesh in error.

mesh clean:
A extra command to delete the meshes from the --output dir is provided (mesh clean). Useful to safely substitute meshes processed by the tool in Test Beds. The clean command only works on the --output dir.

For example, the command sequence:

Code:
orbitertools --root c:\Orbiter\2010p1 --output c:\Orbiter\2010p1.test mesh clean SR71* SR71*.new
orbitertools --root c:\Orbiter\2010p1 --output c:\Orbiter\2010p1.test mesh normalize SR71*
orbitertools --root c:\Orbiter\2010p1 --output c:\Orbiter\2010p1.test mesh copy SR71*

will delete all SR71*.msh and SR71*.msh.new files on the --output dir (and only these files), then will make normalized copies from these meshes from the root Orbiter into the --output one, and then will make a plain copy from the same files into .msh.new files so you can pinpoint failures if (or better, when) the normalizing breaks something.

mesh lint:
Here, a report summary for the full lint scan of my current Orbiter setup:
Code:
MESHES SUMMARY
Counted 23 issues for TEX_HEADER_ABSENT.
Counted 1,509 issues for MAT_NOT_USED.
Counted 545 issues for TEX_NOT_USED.
Counted 996 issues for GRP_VTX_MATH_ERR.
Counted 820 issues for GRP_DUP_LABEL.
Counted 1 issues for MAT_EXTRA_DEF_FOUND.
Counted 5,994 issues for GRP_MAT_OUT_ORDER.
Counted 1,220,362 issues for GRP_DUP_VTX.
Counted 288 issues for GRP_DEPRECATED.
Counted 5 issues for GRP_EXTRA_FOUND.
Counted 12,589 issues for GRP_DUP_SRF.
Counted 6,527 issues for GRP_VTX_ORPHAN.
Counted 4 issues for MAT_LESS_NAMES.
Counted 4 issues for MSH_DEPRECATED.
Counted 5,214 issues for GRP_TEX_OUT_ORDER.
Counted 21 issues for GRP_NO_VTX.
Counted 6 issues for MSH_NO_MATERIALS.
Counted 6 issues for FATAL_PARSE.
Counted 3 issues for GRP_MAT_NDX_ERR.
Counted 2,402 issues for TEX_FN_NONORMAL.
Counted 12 issues for MSH_1STGRP_NO_TEX.
Counted 1,485 issues for TEX_NOT_FOUND.
Counted 8 issues for FATAL.
Counted 1 issues for MSH_NO_GROUPS.
Counted 21 issues for GRP_NO_SRF.
	2,231 meshes processed, 1,993 of them with 1,258,846 issues.

MESH STATISTICS
	Total meshes : 2,223
	Total groups : 74,227
	Total vertices : 17,340,724
	Total surfaces : 14,520,440
	Total materials : 14,613
	Total textures : 7,101
	Time elapsed : 1:17:03.426993 H:mm:ss.mmm


The full report is human friendly - but is also 50 Megabyte size on my machine, so I'm posting just the summary. Follows a sample from the full report:

Code:
Mesh:XRPayload/TankD2 has 296 issues:
	Material name mismatch! Material Mat:#7(issItalian)@L:2,479, is named "issusaf" but "issItalian" is found at 6th position of the names structure.
	Found 1 duplicated vertices on Group Grp:#1
		 1 duplicated vertices for Vtx#31@Line:37 [V#33@L:39]
	Found 200 duplicated vertices on Group Grp:#2
		 2 duplicated vertices for Vtx#527@Line:600 [V#532@L:605, V#612@L:685]
		 1 duplicated vertices for Vtx#825@Line:898 [V#826@L:899]
		 1 duplicated vertices for Vtx#108@Line:181 [V#114@L:187]
		 2 duplicated vertices for Vtx#528@Line:601 [V#533@L:606, V#538@L:611]
	<..........>
<..........>

Mesh:Mir2_Builder has 2 issues:
	Group Grp:#11(Cylinder142) texture definition is out of optimum order. Regroup by Texture for better performance.
	Texture Tex:#1(mir2_airhatch.dds)@Line:3,071 not found!

Mesh:UniversalCars/CargExploSat44Box has 2 issues:
	Texture filename Tex:#1(UniversalCars\explosat44_Box.dds)@Line:199 is not normalized!
	Texture Tex:#1(UniversalCars\explosat44_Box.dds)@Line:199 not found!

And now also in XML format:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<REPORT source="Orbiter Tools" version="0.3.1b alpha">
	<ARTIFACT type="Mesh" name="Velcro/GEM60">
		<ISSUE type="GRP_DUP_HDR" item="Grp:#1" index="1" >Found 20 duplicated vertices on Group Grp:#1</ISSUE>
		<ISSUE type="GRP_DUP_VTX" item="Vtx#21@Line:26" index="21" line="26">	 1 duplicated vertices for Vtx#21@Line:26 [V#24@L:29]</ISSUE>
		<ISSUE type="GRP_DUP_VTX" item="Vtx#9@Line:14" index="9" line="14">	 1 duplicated vertices for Vtx#9@Line:14 [V#12@L:17]</ISSUE>
		<ISSUE type="GRP_DUP_VTX" item="Vtx#45@Line:50" index="45" line="50">	 1 duplicated vertices for Vtx#45@Line:50 [V#49@L:54]</ISSUE>
<.......>
	</ARTIFACT>
</REPORT>

<...> was used to signal where I abbreviate the report.

The issues my lint currently catches are:

  • Runtime problems
    • FATAL_PARSER : Something serious that prevents the Parser to even try to read the mesh
    • FATAL : I screwed up. Please report so I can fix the mess.
  • Syntactically invalid meshes
    • MAT_NAME_MISMATCH : The name of the Material on the name list does not match the name on the Material definition
    • MAT_HEADER_ABSENT : MATERIALS tag not found - perhaps there are too much or too many data on the last group definition?
    • MAT_EXTRA_NAME_FOUND : More Material names found than specified on the MATERIALS header
    • MAT_EXTRA_DEF_FOUND : More Material definitions than specified on the MATERIALS header
    • MAT_LESS_NAMES : There are fewer material names than specified on the MATERIALS header
    • TAG_ALREADY_PARSED : Repeating structure on the Group definition (more than one LABEL, etc)
    • VTX_EXTRA_IGNORED : More numbers on a vertex definition than expected
    • CLR_EXTRA_IGNORED : More numbers on a color definition than expected
    • TEX_HEADER_ABSENT : TEXTURES tag no found - perhaps there are too much materials?
    • EMPTY_LINE_IGNORED : Nuff said. Empty lines as ignored, but since the specification don't mention them, a warning is issued.
    • GRP_EXTRA_FOUND : More groups found than specified on the GROUPS header.
    • GRP_LESS_FOUND : Less groups were found than specified on the GROUPS header.
    • FILE_EXTRA_COMMENTS_IGNORED : Comments on the end of the file are discarded (as there's no syntactic structure to hook them).
  • Mesh issues:
    • MSH_DEPRECATED : Mesh uses a deprecated tag. It is preserved, but a warning is issued.
    • MSH_NO_GROUPS : There're no groups on the mesh!
    • MSH_NO_MATERIALS : There're no materials on the mesh!
    • MSH_1STGRP_NO_MAT : The first group of the mesh don't define a Material.
    • MSH_1STGRP_NO_TEX : The first group of the mesh don't define a Texture.
  • Group issues:
    • GRP_DEPRECATED : Group use a deprecated tag. It is preserved, but a warning is issued.
    • GRP_NO_VTX : No vertices found for this group!
    • GRP_NO_SRF : No surfaces found for this group!
    • GRP_DUP_HDR : There're duplicated data on the group:
      • GRP_DUP_VTX : Duplicated vertices
      • GRP_DUP_SRF : Duplicated surfaces
      • GRP_DUP_EQUIV_SRF : Equivalent surface are surfaces that are defined differently, but draws the same surface as a previously defined one, i.e. 1 2 3 and 2 3 1.
    • GRP_DUP_LABEL : Group reuses a label from another Group.
    • GRP_INVALID_SRF : A surface is invalid (two or more Vertices are equal - such surfaces degenerate to a line or point and are not visible)
    • GRP_VTX_MATH_ERR : Presence of 1.#QNAN and +/- 1.#INF invalid iEEE float numbers
      • QNANs and INFINITies are detected and a Issue emitted.
      • Orbiter handles them, but values being divided by zero (positive and negative INF), and zero being divided by zero (NaN) is something that should not happens on a modelling tool, I'm right?
    • GRP_VTX_ORPHAN : Group has orphans vertices, i.e., vertices are defined but not used by any surface.
    • GRP_MAT_NDX_ERR : Group uses a material that is not defined on the Materiais section.
    • GRP_MAT_OUT_ORDER : The group is not ordered by Material (inside a Texture series) in the mesh, leading to performance issues.
    • GRP_TEX_NDX_ERR : Group uses a texture that is not defined on the Texture section.
    • GRP_TEX_OUT_ORDER : The group is not ordered by Texture in the mesh, leading to performance issues.
  • Material issues:
    • MAT_MATH_ERR : Materials with invalid float values (NAN or INF).
    • MAT_NOT_USED : Material defined but not used.
  • Texture issues:
    • TEX_FN_NONORMAL : Texture pathname cam be normalized (see below)
    • TEX_FN_ABSOLUTE : A texture pathname is absolute, and not relative to the Orbiter's texture dir
    • TEX_NOT_USED : Texture defined but not used
    • TEX_NOT_FOUND : Texture file not found on the file system
    • TEX_PATH_INVALID : Texture filename is invalid under the current OS. It's mutually exclusive with TEX_NOT_FOUND, but if a filename is invalid, it's obvious it is impossible to be on the file system.

At least for me, "normalizing" the Texture filename is interesting because I'm "living" on a UNIX style box, running Orbiter inside WINE, and case matters here. People willing to run Orbiter on Linux machines would not need to run Orbiter from a loopback block device using a non-case sensitive filesystem. I'm pretty sure that a good portion of that TEX_NOT_FOUND issues is related to this on my box.

Since most of the issues I'm detecting don't necessarily bother Orbiter, I'm calling such issues... "Issues" and not errors. :)

Duplicated vertices, however, impairs performance and orphaned vertices wastes memory. I don't know if Orbiters sanitizes the laters, but if don't, they also waste CPU/GPU time as unused vertices gets animated for nothing.

In some cases, I manually found "double sided" surfaces on boxes or something like that, and half of that surfaces are never rendered no matter what (as they are inside the box). Such surfaces are also a waste of CPU/GPU time. I don't have a clue how to detect such cases, but it would be interesting to do so.

About surfaces, in some meshes I found surfaces where not all three vertices are different. Surfaces with 2 equal vertices degenerates to a line, and surfaces with 3 equal vertices degenerates to a point. Such surfaces are deleted when normalizing.

The lint also detects when the Texture/Material grouping is not optimal, as explained on 4.4 Performance Optimisation section, page 9 of 3dModel.pdf manual.

Final Considerations
But I still need some help here. If you can, please:

  • Suggest about further static analysis (or useful statistics) to be performed (calculated) by the tool.
  • Suggest about faster algorithms to do the analysis (I'm going brute force)
  • Suggest about further mesh optimizations that can be automated
  • Large and/or potentially problematic meshes to be used on the tool as a test case
  • Teach me how to embedded these tools into Blender, complementing the Orbiter Exporter.
  • Suggest new features

Both CLI and GUI interfaces are provided on the same package/executable.

A extensive help (but work in progress) is available by using the --help command line option. You also need to provide the Orbiter's root dir by using the command --root C:\Orbiter (or whatever your orbiter installation is). The help explains how to setup environment variables to make easier using the tool.

A good way for testing this tool is using a clone installation of Orbiter, setting the source one using the --root option and setting the target one using the --output option (the output is not checked). It's not advisable to use the same Orbiter installation for output, as this tool is in Alpha phase and will probably corrupt a mesh sooner or later. Anyway, the tool refuses to overwrite a existing mesh for the sake of safety - if a mesh with the same name already exists, it appends ".new" to the pathname. If a .new file already exists, the tool raises an exception for that mesh and process the next one.

Useful commands:
Code:
orbitertools gui
orbitertools cli --root c:\orbiter --report .\lint_report.txt --verbose 1 mesh lint *.msh
orbitertools cli --root c:\orbiter --report .\lint_report.txt --verbose 2 mesh lint DG/*
orbitertools cli --root c:\orbiter --verbose 1 mesh copy *
orbitertools cli --root c:\orbiter --verbose 1 mesh copy DG/*
orbitertools cli --root c:\orbiter --verbose 1 mesh normalize *
orbitertools cli --root c:\orbiter --verbose 1 mesh clean DG/*
orbitertools cli --root c:\orbiter mesh rc format=header type=define output=c:/my/project/source 2001/*
orbitertools cli --root c:\orbiter mesh rc format=ssumeshc 2001/*
orbitertools cli --root c:\orbiter mesh rc format=meshc 2001/*
orbitertools cli --version
orbitertools cli --help
orbitertools cli --license

The "cli" specifier can be omitted. Currently its absence prints a message before processing in stdout, but other than that, the previous version's behaviourship is preserved. The "gui" specifier ignores the rest of the command line.

The --verbose <n> option can be omitted for a silent run.

One can use --verbose 3 for a more detailed debugging log, and --verbose 4 for a suicidal one (even I/O is logged). :)

Currently, the tool don't travel subdirectories recursively on Windows. I'll fix this soon (not sure when, I need to update my Windows box first).

KNOWN ISSUES:
  • Resource Compiler for meshes:
    • Some formats are not implemented yet.
    • The End-Of-Line format is locked for Windows (\r\n). Not sure if this will be ever changed
    • The header content and format cannot be customized.
    • Not sure if the #define label naming approach on the OT's header format will be useful (or even usable) for all the cases. You can always use the meshc or the ssumeshc file formats however.
    • This tool is wat slower than ssumeshc and meshc. The reason is not exactly the interpreted nature of the language used, Python, but because the meshes are effectively parsed one by one before processing instead of looking for the "LABEL" keywords on the file. It's slower but more flexible, as it allows me to implement new approaches (as using the remarks as source of meta-data).
  • copy command for meshes:
    • The copied meshes are formatted for easy reading for humans, not space saving. Other than that, the mesh contents are 100% preserved on the copy (except by comments that could not be semantically attached to an mesh artifact).
  • normalize command for meshes:
    • Some meshes are encrypted. The normalize will corrupt these files. XR2 Ravenstar's vessel mesh is a confirmed hit, the ANSO ones are reported to be crypted and will probably fail too.
    • For reasons currently being investigated, some Virtual Cockpit meshes get some textures corrupted after normalized. It's confirmed that the problem is on the UV coordinates for texture painting in runtime. It appears to be not exactly a problem with the tool, as the issue can be replicated manually by duplicating or deleting vertices from a group. This issue don't affects 3D modellers.
  • GUI:
    • While multi threading is supported, there's no GUI provisioning for that. Please don't (yet) start one command without finishing the previous first. Both commands will be executed allright, but the Message text widget will be cluttered by two process outputs at the same time (at best - concurrency is not being dealt yet)
    • The Scroolbars on the readonly text areas appears to be disabled, but they are working. Ignore the visual clue and use them normally. It's some kludge on the TK, working on it.
    • The text widget crashes the application when large enough reports are emitted. It's a Tk Text widget issue, still looking for a work around. In the mean time, avoid firing large tasks on the GUI.
    • The Orbiter PATH and the OUTPUT PATH are not persisted - a Preferences dialog is WIP. But the environment variables, when existent, are parsed.
    • The verbosity is currently hardcoded to level 3 for debugging purposes.

HISTORY:
  • 0.4.1.2 alpha - 2016-03-15 :
    • Sorting the symbols alphabetically on Resource Compiler.
    • Fixing bug on LINT while checking for materials out of order (with groups using default material from last group).
    • Ditto for textures.
    • Fixing a really dumb mistake on indexing groups (note to myself: groups and vertices starts at 0, textures and materials at 1). Sorry. :facepalm:
  • 0.4.1.1 alpha - 2016-02-20 :
    • More file formats for the Resource Compiler.
    • GUI for the RC.
  • 0.4.1 alpha - 2016-02-18 : Resource Compiler for Meshes.
  • 0.4 alpha - 2015-12-28 : Graphical User Interface using Tk.
  • 0.3.2 alpha - 2015-12-24 :
    • Line numbers on Reports (when applicable).
    • mesh normalize.
    • mesh clean.
    • Fixing environment variable names (sorry).
  • 0.3.1b alpha - 2015-12-21 :
    • GRP_DUP_LABEL
    • TEX_PATH_INVALID
    • Memory saving reports (when needed)
    • XML output reports that works
  • 0.3.1 alpha - 2015-12-20 : I screwed up again. =P
  • 0.3 alpha - 2015-12-20 : Fixes on saving meshes, more linting.
  • 0.2.1 alpha - 2015-11-18 : Better parsing, saving meshes (preserving commentaries), more linting
  • 0.2 alpha - 2015-12-17 : I screwed up. Forget about this. =P
  • 0.1 alpha - 2015-11-18 : First version going through the door.
 
Last edited:

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
37,656
Reaction score
2,377
Points
203
Location
Wolfsburg
Preferred Pronouns
Sire
Could you also check for duplicate group labels? Or maybe illegal file names for the textures?

About the duplicates, I would not be too hard there, since most cases I remember are for allowing using the same mesh for inside and outside camera view.

Also, would it be possible to provide a XML report that could be parsed by a Jenkins plugin?
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
EDIT: I need some sleep! I overwrote this post by accident!!! :facepalm:

I remember saying "OK, I'll to it", I recognized that duplicated surfaces may not be a problem (except when inside closed objects) and also that duplicated surfaces with the same "spin" is effectively a problem.

And asked something about XML....
 
Last edited:

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
37,656
Reaction score
2,377
Points
203
Location
Wolfsburg
Preferred Pronouns
Sire
For sure. Added to the todo list. Jenkins uses some DTD or I can just spit an arbitrarily formatted XML?

Not at all... Jenkins itself isn't really smart, it is just a container for plugin modules. Making a plugin that parses any kind of XML output is not that hard, AFAIR there is even already a plugin for just looking if a unspecified XML file contains an attribute.

If there is a XML output, it is much easier to include the tool into Jenkins build chains or execute it by git.
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
Yet about Jenkins - I didn't did any profiling on the tool, but I know for sure that the tool wastes memory as hell, as the lint component was planned to be a feeder to the fixer component, and then the offending meshes are keep in memory until the time it would be consumed by the fixer, or be consumed by the Report component.

In a synthetic test scenario (a few thousands of broke heavy meshes), the process got up to 8G ram. Not that bad if you are planning a GUI to do the job with a thread producing the errors and the GUI handling them at the same time on a Desktop machine.

But this can be a problem on your Continuous Integration machine if you are running the jobs on a cheap VPS.


No more. The objects are now discarted for XML, File and Console reports. About 200mega of RAM was needed on my 2057 Meshes Test Scenario.



---------- Post added at 11:40 PM ---------- Previous post was at 07:26 PM ----------

If there is a XML output, it is much easier to include the tool into Jenkins build chains or execute it by git.

How about this:

Code:
<?xml version="1.0" encoding="UTF-8"?>
<REPORT source="Orbiter Tools" version="0.3.1 alpha">
	<ARTIFACT type="Mesh" name="astp.msh">
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#423 [V#425]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#474 [V#475]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#522 [V#523]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#429 [V#430]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#407 [V#408]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#421 [V#526]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#377 [V#378]</ISSUE>
	</ARTIFACT>
	<ARTIFACT type="Mesh" name="alt747.msh">
		<ISSUE type="GRP_TEX_OUT_ORDER">Group Grp#12 texture definition is out of optimum order. Regroup by Texture for better performance.</ISSUE>
		<ISSUE type="GRP_TEX_OUT_ORDER">Group Grp#95 texture definition is out of optimum order. Regroup by Texture for better performance.</ISSUE>
		<ISSUE type="GRP_DUP_HDR">Found 14 duplicated vertices on Group Grp#2</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#589 [V#658]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#612 [V#681]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#596 [V#666]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#599 [V#669]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#616 [V#685]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#622 [V#691]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#587 [V#656]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#595 [V#664]</ISSUE>
		<ISSUE type="GRP_DUP_VTX">	 1 duplicated vertices for Vtx#593 [V#663]</ISSUE>
	</ARTIFACT>
</REPORT>
 
Last edited:

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
37,656
Reaction score
2,377
Points
203
Location
Wolfsburg
Preferred Pronouns
Sire
Looks good. I would maybe make the group id an XML attribute of the ISSUE as well... like ISSUE ... groupId="9" ...

Analog for textures and vertices... all attributes optional, if given can be used for further localizing the problem automatically. A line number attribute in the mesh file would also be great.
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
Version 0.3.1 alpha on the Google Drive.

---------- Post added at 12:53 AM ---------- Previous post was at 12:39 AM ----------

Looks good. I would maybe make the group id an XML attribute of the ISSUE as well... like ISSUE ... groupId="9"

Users are never happy. :rofl:

That format was what I could cranked up from the current TXT format. :stirpot: I didn't considered any other format other than plain text, and now I'm paying the price. :blackeye:

I like your suggestion, it would make identifying the problematic artifacts easier, but I need to overhaul the reporting classes to allow such granularity on formatting memory saving reports (that promptly forgets the data as soon as it's reported).

It's not hard - I just never thought about it, and now I have do deal with already written code.

Except by the line number, I just can't reliably keep this data as the linting is done with parsed, consolidated, in memory data structures and all bindings are semantical. I need to think in a reusable, OOP solution - the parser has about 1700 lines of code, I don't intent to rewrite that thing. Again. =D

On a near future version it will be done :coffee:

---------- Post added at 01:06 AM ---------- Previous post was at 12:53 AM ----------

OF COURSE. I had to withdraw the 0.3.1. ONE, ONE FIVE =P DAMN Meshes on that 2k pool triggered an error on the XML generator. =P

In a couple hours it will be fixed.
 
Last edited:

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
0.3.1b Alpha on Google Drive.

That stupid bug is fixed. Version 0.3.1b works as expected.
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
Sorry. :) Just ignore me if I get too nasty there.

The only nastyness is from my own dumbness. :rofl:

I'm saving the comments attached semantically to the data, so I can rebuild the mesh with (almost) all comments intact. This appears to be important, as a lot of meshes I had look appears to use comments as metadata for some kind of resource compiler.

I can save the line number on the same structure I'm using for saving the comments. Simple like that. :rolleyes:

Similarly, i can overload the Report class to save the XML attributes prebuilt together the message. There's no simpler solution than that - mainly because I already overloaded the class to spit XML data. :facepalm:

Thinking is a very useful habit, makes living easier. I should do that more times. :p

In a couple of days a new version with your suggestions will be published, together the Mesh "normalize" that will solve the issues that will not demand a module recompile.
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
Version 0.3.2 alpha: Better XML, Line Numbers, mesh normalize, mesh clean

People,

Version 0.3.2 alpha on the Google Drive.

I made the XML report more useful (I think), added the line numbers on the reports (all of them, not only XML). Each artifact, when associated to a line number, has "@L:#nnn" in his name.

Thanks for the advice, Urwumpe, this made the troubleshooting a lot easier!! :)

Also, my first attempt to fix meshes. I added a command (mesh normalize) that will read meshes from the source and write normalized versions on the output. As normalize, I mean:

  • Fix the Gem vertices and surfaces counts
  • Fix the materials count, rebuild the material names
  • Normalize the textures pathnames to something that works fine in UNIX and Windows
  • Order the surfaces definition to make easier to recognize visually the surface "spin": all the vertex indices are ordered ascending or descending by rotating the indices (the "spin" is not changed).
  • Duplicated and orphaned vertices are deleted
  • Duplicated surfaces (with the same "spin") are deleted
  • Redundant MATERIAL and TEXTURE declarations on consecutive groups are deleted.

Of course, I recommend doing that only on a Orbiter testing installation. :)

Until the moment, I had 3 problems with the normalized meshes (more is expected, I tested only circa 800 meshes to the moment):

1) The XR2 Ravenstar Hull Mesh appears "exploded" after normalized. That was nasty, it took me hours trying to figuring out where in hell I made such a mess until having the idea of trying to load the original mesh into Blender and in the Mesh Viewer. It appears to be some kind of obfuscation on the mesh - not my fault. :)


2) In Delta Glider and SR-71r, some lightened buttons and avionics got screwed up.


I don't understand why it happens (yet), by I know how it happens: by some reason, changing the order of the vertex definition on the mesh screws up the code that does the texturing.

For example, if I take this group from SR71r:
Code:
LABEL FuelCellAvailableLight
MATERIAL 8
TEXTURE 1
FLAG 0
GEOM 4 2
0.3233 0.4721 15.2364 0.0000 0.0000 -1.0000 0.9610 0.7112
0.3400 0.4721 15.2302 0.0000 0.0000 -1.0000 0.9953 0.7112
0.3233 0.4543 15.2364 0.0000 0.0000 -1.0000 0.9610 0.7456
0.3400 0.4543 15.2302 0.0000 0.0000 -1.0000 0.9953 0.7456
0 3 2
0 1 3

And manually change it to:
Code:
LABEL FuelCellAvailableLight
MATERIAL 8
TEXTURE 1
FLAG 0
GEOM 5 2
0.3233 0.4721 15.2364 0.0000 0.0000 -1.0000 0.9610 0.7112 ; COPY
0.3400 0.4721 15.2302 0.0000 0.0000 -1.0000 0.9953 0.7112
0.3233 0.4543 15.2364 0.0000 0.0000 -1.0000 0.9610 0.7456
0.3400 0.4543 15.2302 0.0000 0.0000 -1.0000 0.9953 0.7456
0.3233 0.4721 15.2364 0.0000 0.0000 -1.0000 0.9610 0.7112 ; PASTE
4 3 2 ; Changed Here!
0 1 3

That problems happens too, so it appears to be not a problem on the normalizing code (as the meshes can be loaded into blender and can be visualized on the Mesh Viewer without problems).

Since Delta Glider is open sourced, I can pursue the matter, what wil be done soon later.

And since the testing loop is tedious, i created a command to safely "clean" the test setup and made the test bed setup a little more automated - some modules saves data files on the meshes subdirs (as UMmu), so the clean deletes only files that ends in .msh or .msh.new preserving everything else.

This is the bat file I use here:
Code:
@echo off
set ORBITER_PATH=c:/Games//Orbiter/2010P1
set ORBITER_TOOLS_OUTPUT_PATH=C:/Games/Orbiter/2010P1.tests
set ORBITER_TOOLS_CLI="c:/Games/Orbiter/orbitertools.exe"

"%ORBITER_TOOLS_CLI%" mesh clean "my_vessels/*" "my_vessels/*.new"
"%ORBITER_TOOLS_CLI%" mesh normalize "my_vessels/*"
"%ORBITER_TOOLS_CLI%" mesh copy "my_vessels/*"

pause

Merry Xmas!
 
Last edited:

SolarLiner

It's necessary, TARS.
Addon Developer
Joined
Jun 14, 2010
Messages
1,847
Reaction score
2
Points
0
Location
404 ROAD NOT FOUND
The XR2 Ravenstar mesh (and also the meshes from the AMSO addons IIRC) are encrypted - and so look like a mess when imported without prior decryption.
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
0.4 Alpha - GUI

People,

In the Google Drive, version 0.4 Alpha.

The only novelty is a practical but simple GUI using TK. I hope the thing is self documented (please note the Status Bar on the bottom of the window).

See the main post for Known Issues (end) and Features (start) - there's some idiosyncrasies and a nasty bug to be aware.

Unless some serious bug arises, I will rest from this thing for a few weeks. There're some nice vessels waiting to be coded. :)

Sample screenshot:
 
Last edited:

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
People,

In the Google Drive, version 0.4.1 Alpha.

A (kind of scratch) Resource Compiler was added (see the original post ahead). Very basic features, it's more a proof of concept than a tool. More (and better) is to come.

If any of you has your own ideas about how a RC should spit headers for Orbiter, now is the time. :)
 

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
37,656
Reaction score
2,377
Points
203
Location
Wolfsburg
Preferred Pronouns
Sire

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
Yes, I remember someone (perhaps you) mentioning typed constants in the resource compiler once.

I agree, it makes things safer. But at a cost, unfortunately: you can't include the header more than once, or each object file will have a copy of the constants - a waste of space on projects like mine, where I spread code in multiple source files. And can't be the other way, as in C++ by default all consts are static - i.e., the symbol's scope is limited to the current object file.
 
Last edited:

Urwumpe

Not funny anymore
Addon Developer
Donator
Joined
Feb 6, 2008
Messages
37,656
Reaction score
2,377
Points
203
Location
Wolfsburg
Preferred Pronouns
Sire
Yes, I remember someone (perhaps you) mentioning typed constants in the resource compiler once.

I agree, it makes things safer. But at a cost, unfortunately: you can't include the header more than once, or each object file will have a copy of the constants - a waste of space on projects like mine, where I spread code in multiple source files. And can't be the other way, as in C++ by default all consts are static - i.e., the symbol's scope is limited to the current object file.

Yes, but that is only a minor problem in SSU right now, multiple object files using the same constant only fails for const char pointers without also making them explicitly static.

The problem is, that #defines can simply be redefined when loading two files with the same #define and that type checking is circumvented that way.

Would only newer compilers with C++11 compatibility be used, we could replace const by constexpr, which represents pure compile-time constants and which is by definition always inline.

https://msdn.microsoft.com/en-us/library/dn956974.aspx

Sadly, only VC++ 2013 and VC++ 2015.
 

hutchison66

Donator
Donator
Joined
Apr 10, 2010
Messages
204
Reaction score
5
Points
18
Location
Spain
Hi there how can I down load this Tool can't seem to find it any where
I work with AC3D and every so often when I alter Verticles the Mesh will develop Spikes flying out of it when loaded into Orbiter I would just like a tool to tell me what the problem is.
I am currently working on the Russian MLM Nauka Module for ISS A2Z and every so often these mesh abnormalities appear normally I can track down the fault by trial and error but this time its become really frustrating if your not in a postion to upload this tool or let me have it could I sebd you my mesh to run through the tool so I can fix the problem The Addon is nearly ready to release and would appreciate the help
Thanks
David
 

Lisias

Space Traveller Wanna-be
Joined
May 31, 2015
Messages
346
Reaction score
3
Points
18
Website
www.youtube.com
Hi there how can I down load this Tool can't seem to find it any where

There's a link to my Google Drive at the end of the original post. I will edit it to make it easier to find. In the meantime, click here.

I work with AC3D and every so often when I alter Verticles the Mesh will develop Spikes flying out of it when loaded into Orbiter I would just like a tool to tell me what the problem is.

I'm not sure if this tool can help on this - unless the AC3D is making some serious mistake while exporting the mesh (as issuing NANs and INFs, that are detected by the tool).

However, the mesh normalize command rarely messes up. By reducing the vertexes and surfaces to the minimum and writing files in a somewhat human friendlier format, perhaps it can be useful to you.
 

hutchison66

Donator
Donator
Joined
Apr 10, 2010
Messages
204
Reaction score
5
Points
18
Location
Spain
Thanks for the promtt reply I was looking in OH and on the web for the tool.
I think its a verticle problem because sometimes when I've altered a perfectly functioning mesh verticle it suddenly appears. I think its how it exports it to orbiter. If I could find out which object its in I can work around it. Other than that AC3D is an excellent program.
Never down loaded anything with Google drive before will give it a go
Thanks for your help
 
Top