Find the Mesh Data in a file

Alright, put your hacking/detective gloves on.

I have a file which I know contains a triangle mesh (or multiple triangle meshes). I would like to figure out how to extract this triangle mesh data so that I could write an importer for this file type. This is a file which is associated with a closed optical scanner.

In my initial experiments I have done following



MyFile = open('C:/myfile.cdt','rb')
Lines = MyFile.readlines()


this has allowed me to browse randomly through the file (which is 21349 lines long). Mostly what I see are things like this.

Lines[0] yields:

b’U\xaaU\xaaStatus\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00j\x14\r\x00\x00\x00\x0c\x00\x00\x00\x04\x00\x00\x00

Lines[100] yields:

b’\xf1\xfa!\x11R!!\x87\xd0\xe3\xe9oC\x9e\x10R\xc5\xd4t\x0c9\x9ae\xe9\xf4\x10\x84\xe6*\x17\xec\x13\x9c\x9cjb\x06\xf1\xc4Q<\xbam\r\x98\x83\xa6B\xcf(\xef6\x9d\xbdu\xe4\x91\xe5\xae\xaf\xc8\xdcy\xc6\xe2\xc9\xd4\xc3\xe8\xe9{\xeb\xfc\x8cO\x89\r\x85\x1c@\x0f\xd5!\xe3\x1cd\xb6\xc2\xa1\x9f\xea7\xa62J\x04Ri]\x89\xdbu?\x01\xaa\xef\xc2\x98\x82\x100\x00\xaamU\xbe\x9b\xcb\x1cwkj\xa1!\xff[\x81\x83\x89g,y\xf0\x972{@\xeb\x05N{\xd5x4\xfc\xf7w\xc0\x10\x1e\xf0_RG^A\x1f<"\x07LS\xf1}\xaco\x8a\xc3\x1e\xbc\x12\x16\x12b\xc8\xfanT\x9ae\xb0\xd4<5\x13\x82

After a lot of googling, I assume this is some form of encoding. Occasionally there are actual words nestled in there, and they are always preceded by “\xaa” in the string

\xaaTriangle
\xaaCutTriangle
\xaaEAreaOcclusoin
\xaaEAreaAntagonist
\xaaEAreaField
\xaaEAreaPreparation
\xaaMillBottomPreview

These seem to correspond to pieces of information which i know should be in this file.

Any ideas on how to determine what this encoding is? PM me if you would like me to send you an example file.

google came up with this
http://smd.stanford.edu/help/formats.shtml#cdt

Is it gene related?.. hmmm no think it may be teeth.
Maybe converting to stl could be the go http://www.sharewareconnection.com/software.php?list=Converter+Stl+Cdt

@ batFINGER

I should have put some more info into my post but it was already growing into quite the novel ha ha.

  1. The .cdt file extension in my example is coincidentally related to a Corell Draw file format, so, it muddles any google searches
  2. You are right indeed, it is teeth/dental related
  3. I didn’t want to publish the company name so as not to violate any intellectual property
  4. When I try to search some of the random strings along with phrases like “encoding”, I came up with gene sequencing too…more search problems!!

What I’m aiming for is to be able to figure out the structure of the file. For example, if you look at the STL file importer, it defines a binary read function as follows.


def _binary_read(data):
    # an stl binary file is
    # - 80 bytes of description
    # - 4 bytes of size (unsigned int)
    # - size triangles :
    #
    #   - 12 bytes of normal
    #   - 9 * 4 bytes of coordinate (3*3 floats)
    #   - 2 bytes of garbage (usually 0)

    # OFFSET for the first byte of coordinate (headers + first normal bytes)
    # STRIDE between each triangle (first normal + coordinates + garbage)
    OFFSET = BINARY_HEADER + 4 + 12

    # read header size, ignore description
    size = struct.unpack_from('&lt;I', data, BINARY_HEADER)[0]
    unpack = struct.Struct('&lt;9f').unpack_from

    for i in range(size):
        # read the points coordinates of each triangle
        pt = unpack(data, OFFSET + BINARY_STRIDE * i)
        yield pt[:3], pt[3:6], pt[6:]

First, I would like to try my same technique of manually looking on an STL file to see what that might look like.

MyFile = open('C:/myfile.stl','rb')
Lines = MyFile.readlines()

Lines[0]

b’Exported from blender\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xee]\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x8f\xfcy\xc0\x917b\xc0\xa6\x13\xd1\xbf\xc8\xd2|\xc0\xf9@d\xc0\xea\xd1\xd5\xbf^e\x81\xc0H\x1c\\xc0…a lot more of this

Excellent! Same kind of stuff. This makes since beacuse the python open function in ‘rb’ (read binary) mode simply returns a “bytes object” without any decoding.

This leads me to believe I need to determine what lines are junk, what the layout is…eg normal, V1, V2, V3, if there are any junk lines etc.

How do I determine the size in “Bytes” of a line? I think this would be my first step, simply looking for patterns in line size.

There are typically no “lines” in a binary data stream as there typically isn’t a character to define where one line begins and another ends. For example, in a text file there are "
" characters at the end of each line which is how the readline( ) knows when to stop reading. Looking at your very first example that exporter appears to include "
" characters, but this is typically not the case.

For example, in your .stl exporter there are no "
" characters so there are no lines so you have to read it as one looooong sequence of bytes packed together side-by-side and its up to the importer to know how to parse that gobbily-gook sequence of bytes into something meaningful.

If you are really determined to reverse-engineer something (you weren’t able to find specs on how this file is formated like your .stl example above?) then you should get yourself a hexeditor and forget trying to write a python script to read in a line at a time (as it will do you no good) :evilgrin:.

To be honest I’m kinda surprised once you saw the stream of chaos that those two exporters pooped out in your examples you didn’t give up. You sound quite determined here :stuck_out_tongue:

That said… if you reallllyy want to keep going and want a brief primer on how to decipher that .stl output above in a hexeditor then drop another reply and I suppose I might perhaps possibly write up an explanation for you :stuck_out_tongue:

So I should consider myself lucky? :slight_smile:

For example, in your .stl exporter there are no "
" characters so there are no lines so you have to read it as one looooong sequence of bytes packed together side-by-side and its up to the importer to know how to parse that gobbily-gook sequence of bytes into something meaningful.

Ok, that makes sense.

(you weren’t able to find specs on how this file is formated like your .stl example above?)

Nope, that is precisely the problem. Theses scanners are used in conjunction with a manufacturing process. By controlling the data, the company ensure that they receive the manufacturing business. I just want to be able to scan models for educational/research things. Another cool thing is that these scanners are in something like 15% of dental offices so there is a vast untapped potential for these machines as a general purpose scanners as they retire from clinical use (new models come out).

then you should get yourself a hexeditor and forget trying to write a python script to read in a line at a time (as it will do you no good) :evilgrin:.

googles hexeditor

That said… if you reallllyy want to keep going and want a brief primer on how to decipher that .stl output above in a hexeditor then drop another reply and I suppose I might perhaps possibly write up an explanation for you :stuck_out_tongue:

Replied :slight_smile: Thanks for taking the time to consider this. Even if you don’t have time to write up a bunch, you could give me 10-15 concepts to start reading about. I am a fairly decent independent learner.

Alright… I’ll try to post a few concepts at a time and then let you decide if I should keep going or if “reverse-engineering” just isn’t your thing and you give up :stuck_out_tongue:

As an example, I took the default blender scene of a lone cube, converted its faces from quad->tri, and exported it using the .stl exporter.

Opening this exported file in a hexeditor gives you the following:


4578 706f 7274 6564 2066 726f 6d20 626c  Exported from bl
656e 6465 7200 0000 0000 0000 0000 0000  ender...........
0000 0000 0000 0000 0000 0000 0000 0000  ................
0000 0000 0000 0000 0000 0000 0000 0000  ................
0000 0000 0000 0000 0000 0000 0000 0000  ................
0c00 0000 0000 0000 0000 0000 0000 0000  ................
0400 803f f7ff 7f3f 0000 803f 0000 803f  ...?...?...?...?
ffff 7f3f 0000 80bf ffff 7fbf 0000 803f  ...?...........?
0000 803f 0000 0000 0000 0000 0000 0000  ...?............
0000 0000 803f ffff 7f3f 0000 80bf faff  .....?...?......
7fbf 0300 803f 0000 80bf ffff 7fbf 0000  .....?..........
803f 0000 803f 
etc...

Eeek! Whats the **** is this? :spin:

Its actually not too scary :p. The way you read this is just like you would read a book. Left-to-right one line at a time. The numbers on the left are the actual contents of the file (in hex) while the readable text, series of dots, and question marks on the right is the hex editor’s attempt to decode the data for you (explained below).

Guess the best way to get started is walk you through the first few bytes. The important thing to remember is each pair of values you see represents 1-byte of data. So the first byte in the file is the hex value 45 aka 0x45 or as you would see in your python stream \x45. What in the world does 0x45 mean? :confused:

Well if you google “Ascii Table” and click the first link you can look it up. It will tell you the hex number 0x45 refers to the upper case letter E.

How about the 2nd-byte in the file 78 aka 0x78? You will see it refers to the lower case letter x.

If you continue this process you will notice the first 21 bytes in this file simply spells out Exported from blender. The hex editor has decoded this data for you and displayed it on the right to make things a little easier on the reader. By default hex editors decode the files contents one byte at a time. You can use this knowledge to your advantage to quickly scroll through a file in a hexeditor looking for strings of interest like you were trying to do earlier.

If you look past the data “Exported from blender” on the right and were wondering why you just see a bunch of dots and a few question marks. This is because that data does not describe a string but rather a value of some kind (integer, float, etc [the actual data of the mesh]).

Anywho :p, if you go to your .stl importer binary read function you posted earlier you will see that it says the first thing in the file is a header of 80-bytes in length. Below is the first 80-bytes cropped out:


4578 706f 7274 6564 2066 726f 6d20 626c  Exported from bl
656e 6465 7200 0000 0000 0000 0000 0000  ender...........
0000 0000 0000 0000 0000 0000 0000 0000  ................
0000 0000 0000 0000 0000 0000 0000 0000  ................
0000 0000 0000 0000 0000 0000 0000 0000  ................

Thats your header right there ^^^. As you can see its just a 21-byte character string describing the file and the remaining 59-bytes are nothing. This is just reserved space in case they wanted to add stuff to the description later.

Next comes the decoding of the actual mesh data of the file… should I continue or do you surrender? :evilgrin:

Excellent. I understand that very well.

Now, if we know how he information is “packed,” we skip the header, and any other lines we don’t need (as described in the stl binary read funcion), and then just bounce along, grabbing the data we want, and organizing it into Blender mesh format. This is where the “OFFSET” and “BINARY SRIDE” variables come into play.

Very cool. So, if all i need to do is pass all these “hex” values through a lookup table (ASCII), how come some some things in he file print out nicely eg… “Exported from blender” and others remain as binary?

Inbetween lecture slides for my exam tomorrow I have done the following.

  1. downloaded a hex editor
  2. played around
  3. opened two .cdt files and compared them. Aha, I see that they do not different until the 06 0B location. If I compare to yet another file, i get the same spot. Now i might have an idea of the header length is.

http://dl.dropbox.com/u/2586482/hex_edit.png

So, I know here is going to be other info in this file, identifying the mesh verts and connectivity is going to be the challenge. Also, if anyone wants to follow along…

http://dl.dropbox.com/u/2586482/test.cdt
http://dl.dropbox.com/u/2586482/test2.cdt

More playing…it seems that any of the words which are “visible” are always preceded by “55 AA 55 AA”. Doing a search for that sequence pointed me to all of the following (many of which I had seen by “eye” before)

Status,
CStatus,
Thumbnails,
Field0_1, Field0_2, Field0_4, Field0_8
Field1_1…Field1_8
Field2_1…Field2_8
Field3_1…Field3_8
EAreaPreparation
EAreaOcclusion
EAreaAntagonist
EAreaField
Vds
Triangle
CutTriangle
Model
CLines
Gdm
MillBottomPreview
MillPreview
VDSTriangle
SinterSupport
ConnnectOrders
MCXLModel
BridgeMultiLayer

Now, here is where we get progress.

  1. I know these files contain up to 4 models. I see 4 “Fields”
  2. This demo file only has one model, that’s why I’m using it, hoping it would stand out. Field0_1-8 Comprise about %90 of this file. Field 1,2,3 are more or less empty. Further confirming my suspicion that the mesh data is in the “Fields”
  3. I opened a more complex file with more models…indeed, Fields0,1,2,3 all have data in them.

yay! now, how do I make the crazy characters and symbols turn into numbers!!??

Looks like you are making some good progress narrowing down what’cha need.

So, if all i need to do is pass all these “hex” values through a lookup table (ASCII), how come some some things in he file print out nicely eg… “Exported from blender” and others remain as binary?

You my friend have discovered the big difference between a file stored entirely as a series of ASCII characters (say a .txt file) and one stored in binary.

For example if you were to save the following text file:

HelloWorld 132.123141341 3456456.1 23.4

and open it in a hexeditor you would see:

4865 6c6c 6f57 6f72 6c64 2031 3332 2e31  <b>HelloWorld 132.1</b>
3233 3134 3133 3431 2033 3435 3634 3536  <b>23141341 3456456</b>
2e31 2032 332e 340a                      <b>.1 23.4.</b>

You see your numbers on the right! This is what you are hoping for, but the thing is those numbers aren’t being stored as numbers, but rather strings. That is not how a computer internally stores nor processes numbers. Sure our eyes see numbers but the computer sees a series of characters.

Asking it to subtract or add two strings would make it very sad. In order to make the computer happy you would have to, in your Python script, convert those strings to numbers using a function like StringToInt. This is how things formats like xml or html are stored… very human readable, but not the most efficient nor secure (any ole’ weiner can peek at your stuff)

The contents of binary files, however, are stored exactly like they are in memory. You can read them directly into memory (no conversion required) and the computer will be quite happy with you.

Reasons why people go binary over plain ascii characters:

ASCII is space inefficient

  • The number 132.123141341543563456345634 requires 28 bytes of storage (1-byte per character) in ASCII. But in binary, it only requires 4-bytes.

A number in ASCII isn’t a predictable size

  • In ASCII the number 132.123141341 = 13-bytes
  • In ASCII the number 23.4 = 4-bytes
  • In BINARY both these numbers have the same length of 4-bytes.

So rather then being able to read in these numbers quickly you have to read one character at a time to try and figure out where one ends and another begins. Add on the StringToInt conversion and <insert sad face>.

Imagine a game company having to read in a text file describing their model, possibly containing a million numbers, one character at a time, then having to convert each number before it is usable as opposed to directly feeding the graphics card the binary machine ready data in one big quick read.

Yikes! Getting sidetracked. Train OFF the tracks. I’ll make a new post in a few mins to answer your question: “How do I convert hex editor confusion into purttty numbers”

I swear (after I eat my tasty ham samwitch :wink: )

Excellent. This is fun! Enjoy your ham sandwich and thanks again for playing along.

I will take a pre-sandwich stab. Based on my readings I am going to need to find a starting point and follow along in some step size, each chunk representing a float (or groups of floats)…which hopefully represents something interesting in my future mesh.

I’ve come across some concepts which I’m still sorting out. Take UINT32 vs INT32 vs REAL32. All are 32 bits long, but UINT32 will go from 0 to 2^32-1 whereas INT32 will go from -2^31 to 2^31-1 or something like that and REAL32 has a range which I’m too lazy to think about right now but it’s affected by the where the decimal is and what not.

Summary, different types of numbers use their bits and bytes differently for different purposes/advantage. So, if I think I’m looking for REAL32 but I come across a UINT32…I’m going to get bad results?

Take for example the STL specification (from Wikipedia). Let’s say hypothetically I’m not sure what the file format is, but I know to skip over the first 84 bytes because I’ve compared different files to decipher how long the header is. Now, as a guess, I say “lets cruise through this 4 byte chunks”

I skip over the 8 bytes in the header and size…
UINT8[80] – Header
UINT32 – Number of triangles

I get to the first triangle…I’m reading 4byte floating point numbers…
REAL32[3] – Normal vector (12 bytes)
REAL32[3] – Vertex 1 (12 bytes)
REAL32[3] – Vertex 2 (12 bytes)
REAL32[3] – Vertex 3 (12 bytes)

and then I get to this
UINT16 – Attribute byte count (2 bytes…S#!7!)

Which begs the question…Is some portion of the 4 bytes in UINT32 or the 2 bytes in UINT16 used to indicate what kind of number representation is coming up? Or will I end up taking the 2 bytes from UINT16 and then the first two bytes of the next REAL32 when I reach the end of one triangle and the beginning of another.

I see your reply… looks like you are 100% headed in the right direction…some of what I write here is a rehash of stuff you discovered… for some of your questions I don’t answer here I’ll plop in a new post;)

So far you have read one byte at a time from a hex editor and either decoded it in the trusty ASCII table or simple looked at the readable text the hexeditor decoded for you.

The reason why you do not see any numbers in the hex editor, however, is for the following reason:

1-byte is capable of holding 256 different values… no more. Take a look at your ASCII table designed for decoding a single byte… it only has 256 different entries in it :wink:

Therefore, representing a number in a single byte limits the programmer to using a integer value between -128 and 127 or 0-255. Pretty restrictive demands… what if a mesh has 300 faces? Uh oh.

This is where additional bytes come in:

  • Tack on a second byte and suddenly you can store up to 65,535 different values
  • Tack on two more for a total of four bytes and now you can represent a number up to 4,294,967,295 :eek:

4-byte integers are what the average programmer tends to use these days for storing basic positive and negative integers. So this is likely what you’ll find in your .cdt file for things like # of faces, # of meshes, # of vertices, etc.

4-bytes are also the most commonly used size for storing decimal numbers (i.e 5.432). This is how normals, vertices coordinates, etc will likely be stored in your .cdt.

So summary:
1-byte = storage size for characters in a string
4-byte = storage size for integer or decimal numbers

For a full list of data types and their sizes you can always look here: http://msdn.microsoft.com/en-us/library/s3f49ktz.aspx

Now applying that knowledge to Mr. Hexeditor. Back to my original lone cube + .stl export here is the file contents immediately following the 80-byte header:


0c00 0000 0000 0000 0000 0000 0000 0000  ................
0400 803f f7ff 7f3f 0000 803f 0000 803f  ...?...?...?...?
ffff 7f3f 0000 80bf ffff 7fbf 0000 803f  ...?...........?
0000 803f 0000 0000 0000 0000 0000 0000  ...?............
0000 0000 803f ffff 7f3f 0000 80bf faff  .....?...?......

According to the .stl binary read function the next item in the file is a 4-byte integer (representing the # of faces in the mesh). These four bytes are: 0c00 0000. Believe it or not that is the number of faces in the mesh (in hex), with one minor twist.

Most computers these days store multi-byte data types (i.e. a 4-byte integer) in reverse. This is known as little endian storage :rolleyes:. So before using 0c00 0000 you have to reverse it. See image: http://www.mediafire.com/i/?n4f46dmaarsybad

So the actual value is 0000000c. If you google “hex to integer converter” and plug in that hex number you will get an output of 12. This is supposed to be the # of triangle faces in a cube and indeed it is.

To give a quick overview of the rest… here is an image outlining the header, # of faces, the 3 normals of the first face, and the 3 vertices of the first face in the .stl file format (see below for color key). http://www.mediafire.com/i/?tx45117m4kl58cl

  • Brown = 80-byte header

  • Green = 4-byte integer (# of mesh faces)

  • Blue = 3 four-byte decimal numbers (describing the normal’s x,y,z components for this face) Note: Yes they are zero. The .stl exporter looks like it doesn’t actually write out the normal values, but rather expects the application importing this mesh to auto recalculate the normals for each face. Lazy exporter :cool:

  • Yellow = 9 four-byte decimal numbers (each of the vertices for this face [made up of x,y,z components])

  • Purple = 2 bytes of garbage

For the green you would reverse the bytes and pass it into a hex to integer converter for the blue and yellow since they are floating point numbers you would reverse the bytes for each as described earlier and pass into a hex to floating point converter.

P.S. I’ll give you a few tips in my next post so that you do not have to manually find, reverse the bytes, and pass the resulting 4-byte hex value into a converter just to see whats in a file. So don’t feel bummed about my above tooth-pulling explanation… its just important you know how it works before I give you some shortcuts.

I won’t lie. It was mighty tasty :evilgrin:

Based on my readings I am going to need to find a starting point and follow along in some step size, each chunk representing a float (or groups of floats)…which hopefully represents something interesting in my future mesh.

Bingo!

I’ve come across some concepts which I’m still sorting out. Take UINT32 vs INT32 vs REAL32.

1-byte = 8-bits… a bit of course being the 0’s and 1’s a computer uses to do all its magic… hex is a step above bits for people readability sake… because a human trying to read 0’s and 1’s = :evilgrin:

UINT32 refers to an unsigned 32-bit number. 32-bits = 4-bytes.

How the left most bit in that number is interpreted: 10000000000000000000000000000000 by the computer determines if that number is negative or positive. An unsigned number (UINT) tells the computer to not treat that last bit as a negative/positive flag, but rather use that bit to expand the possible range of values available.

e.g. 8-bit number where the last bit is used to expand the range of values UINT8 = 2^8 = 256 if you choose to use that left most flag to identify if the number is negative or positive you are reduced to 7-bits for representing the number INT8 = 2^7=128 with the eight bit signifying if its negative or positive.

Summary, different types of numbers use their bits and bytes differently for different purposes/advantage. So, if I think I’m looking for REAL32 but I come across a UINT32…I’m going to get bad results?

Yep (i’ll visually show this below) :cool:

UINT8[80] – Header
UINT32 – Number of triangles
REAL32[3] – Normal vector (12 bytes)
REAL32[3] – Vertex 1 (12 bytes)
REAL32[3] – Vertex 2 (12 bytes)
REAL32[3] – Vertex 3 (12 bytes)
UINT16 – Attribute byte count (2 bytes…S#!7!)

Exactly. Setting the last variable to UINT8[2] would work as well. As for the rest of your question …the hex editor you are using displays bytes in pairs… but in reality they are totally separate. Take a look at a screenshot of the following hex editor…http://www.mediafire.com/i/?p3pmbu9ava9rf03

It displays its file contents one byte a time as opposed to two… some might choose pairs of four for readability…

Anywho tips!

  1. In my last image ( http://www.mediafire.com/i/?p3pmbu9ava9rf03 ) you will notice that hex editor automatically calculates the different possible values at any given byte location saving manual labor of doing it yourself. In that image I am pointing to one of the mesh’s vertices (more specifically the y-component of the first vertex).

On the left next to float it shows 0.99999 (the actual value in blender). You can also see if it tries to decode it into something its not you get junk. Under 32-bit integer it results in 1065353207 (totally bogus junk). Hopefully, that answers your question about what if I’m looking for a REAL32 but its really a INT32.

  1. Some hex editors have fancy smancy features. In your hex editor it tries to decode the file contents one byte at a time (good for finding strings). But, as you discovered what you are looking for now is 4-byte REALs because you want to find mesh data. Take the following snapshot of Visual Studio 2010 Express:http://www.mediafire.com/i/?e73vbd5bjxgb16x

The top half shows what you have seen thus far. The contents of a file decoded one byte at a time. But, look at the popup menu. It allows for you to instantly decode the data differently. In this case I clicked 32-bit floating point because I was interested in finding mesh data. In the bottom half of the image is the result. I highlighted the 3 values representing a face’s normal followed by the 9 values representing a face’s vertices. As you can see it joined bytes into pairs of 4 then instructed the computer to decode them as floating point numbers. Kinda neat.

Scroll up and down the file in this mode and you could potentially quickly spot large chunks of mesh data in their true form.:eyebrowlift:

Ok, this is epic, I may not sleep tonight. How do I get to that pop up? I have MSVC 2008 (for building blender duh…) and I have opened my test file with the binary reader but I can’t seem to find the options for the decoding.

Seems there is a plugin for Rhino that reads the cdt dental files. You can get demos of both. The Rhino demo lets you save 35 times. From there you can export it to some format that blender can read. That will make it a hell of a lot easier to decode the file.

http://www.rhino3d.com/download.htm


http://www.cimsystem.com/trials.php?d=./areafile/installatori/Dental-Solutions/Dental-Shaper

This discussion brings me back to when I was reversing the wings3d file format (about 10 years ago). It looks like you guys are on the right track.

What helped me the most was looking at files with known data. For example, if you had a cdt file of a unit sized cube, you could then look for the 8 vertices, usually followed by 6 faces (or 12 triangles). The first triangle would usually be 3 integers – something like 0000 0001 0002. And that is assuming they used an indexed format without compression. If you don’t have a cube file and you know how many triangles are in each object, you could look for those integers in the first field.

@batFinger and scorpius

I had the Rhino/Dental Shaper add on trials a month or so ago but the Dental Shaper trial is time limited where as the Rhino trial is #saves limited as you stated. So, I do have some files which are correlated, but they are rather large with several meshes. The example I’m working on is a single mesh file, but i don’t have the corresponding stl…paradox

I won’t have access to a scanner again for a few months, but I may have some friends who can hook me up with the simplest scan possible and correlated 3d mesh file (stl, ply or whatever). The journey continues…

@notmybuddy

I have used struct.unpack (by hand in the blnder console) with some different settings to decode 4byte blocks around the region where I think the mesh data should be. So far, I don’t see anything plausible yet. I got a lot really really large and really really small numbers and not many in the range i was looking for. But, at lease I learned how to use the struct.upack with different options (little endian, native, integer, float etc etc). I still can’t figure out how you got MSVC to display the file like that. It looks like you have a memory window open?

Indeed I do :wink:

I don’t appear to have MSVC’s binary reader on my computer (perhaps the Express edition doesn’t ship with it? I dunno). I was lazy and gave up searching after a few mins. :evilgrin:

Instead I decided to just write a very simple c++ program to read the file into memory and look at it there (hence the memory window).

The program simply opens a file, determines its size, and reads it into memory. See screenshot: http://www.mediafire.com/i/?joe7g8ta5aafq08

Memory is quite huge as you know (prob a few gig on your computer) so you will need to know where in memory the file was loaded. The program has two variables for this, start and end. Type either of these into the address field of a memory window and poof it’ll take you to the beginning or end of your file. (see previous screenshot)

Your file contents will only be between these two addresses start and end. Scroll before the start are after the end and you are no longer looking at file contents, but rather just surfing through your RAM… nothing interesting.

Quickly moving to a section in your file:

While MSVC’s memory window has that fancy decoder it seems to lack a search/find feature (or I didn’t see one). So in order to move quickly to a section of your file, for example, field0_1… just open up a hex editor with search functionality and jot down the location offset into the file. See screenshot (I highlighted it in red): http://www.mediafire.com/i/?4rlc38wfnmcz53b

Add this offset to start in MSVC’s address field e.g. start+14260 and you’ll be there. See bottom half of screenshot: http://www.mediafire.com/i/?joe7g8ta5aafq08

scorpius’ advice

I don’t know if you are still able to run the Dental/Rhino thing that bat found… but if so scorpius’ suggestion of creating a simple cube, writing down info about it on a piece of paper such as # of faces, vertices locations etc, then if you export it using the .cdt exporter (assuming it can do that) … you could use this much smaller/simpler file to learn how things are stored (since you know exactly what values to look for) instead of having to surf through the larger 5mb ones that contain I’m guessing a very complex mesh and possibly a lot of other info.

Although, it sounds like you have an idea of what values you are searching for already so maybe you’re good to go …:RocknRoll:

@ notyourbuddy

You will get a kick out of this.

  1. I duplicated your program…monkey see, monkey do (my first C++ program…screw you ‘hello world’)
  2. I goggled how to open a memory window…I need to be in debug mode
  3. I click debug, it shows me my typos, tells me i need to include some other things. No problem. Logically, I fix my stuff and then debug again and it runs through and spits me back out where I started. Leaving me no opportunity to go up to the debug menu to get to the memory window. audible sigh

3.5 No Joke, I click debug and try to open the drop-down to access the memory window before it finishes debugging. Yeah, I tried to race my computer through a 24 line program…twice…dumb and embarrassing.

  1. I examine your image again, see some golden circle next to line 20. I finagle around with some menus and somehow get the idea to “add a breakpoint.” If ever there were something called providence.

  2. I can now read my files as your screen shots have demonstrated!


RE: Correlated STL to data file.

I wish that I could make a “cube” in the file, but because the data in the file is generated from an optical structured light scanner, there is not a way that I can insert artificial data into the file.

Here is how this scanner works. It has a 640 x 480 sensor with a structured light projector. You image/scan a small area at a time. You re-position the scanner and overlap the scans and it registers them together as you go. When you are done, this gets meshed to form a model. So, the simplest file I can think of will have a model derived from one single scan of something more or less planar. That should give us a bunch of points with roughly the same z coordinate (or x or y depending on convention). That should give us something to look for. My colleague is going to send me a scan of something of this nature and the corresponding .stl file (his lab has a commercial software which can extract mesh data from this file).

Shall we recess for 24-48 hours to await new evidence?

Niiiice :o

Here is how this scanner works. It has a 640 x 480 sensor with a structured light projector. You image/scan a small area at a time. You re-position the scanner and overlap the scans and it registers them together as you go.

Ah indeed. Point clouds. I see what you are up to now. :cool:

Shall we recess for 24-48 hours to await new evidence?

Sounds like a plan :RocknRoll: