This is an important subject that is long overdue to be discussed in detail. There are several aspects of numeric precision that are relevant to EVS and our users, but two in particular warrant a detailed description.
- Precision of calculations and deliverable results
- Precision of graphical displays
Both topics above are driven by the way that computers represent numbers, therefore we will address first how computers represent Float and Double numbers. For those of you not familiar with the terms “Float” and “Double” as used in computer jargon, computer programming languages represent numbers in several different ways (in order of increasing precision) including “Short”, “Integer”, “Half Float”, “Float” and “Double”.
In EVS, Float and Double are most used, and the limitations of Float are the most important to understand. The table below lists the Float and Double Precision for Ranges of numbers from 1.0 to 1 trillion.
Let me explain what this table is telling us. For Float Precision, it tells us:
- For numbers in the range of 1 to 2, we can expect precision of 1 part in ~10 million.
- For numbers between 4 to 8 million, our precision goes down to 1 part in 0.5.
- For numbers between 1 to 2 billion, our precision is only 1 part in 128!
What this shows us is that regardless of the magnitude of the numbers, Floats have only 6-7 digits of precision and Doubles have 15-16 digits of precision.
When I look at this table from a non-programmer’s perspective, the obvious question is “why don’t you just do everything in Double Precision?”. Our answer is that we do everything we can in Double Precision, but all computer graphic cards render their displays with Floats.
We do use Double Precision for all internal computations including all of our estimation methods (e.g. kriging), and volumetric computations. However, when it is time to send this “double precise” model to the viewer, we’re subject to the limitations of graphics cards which do everything in Float Precision.
The consequences of this are:
- You don’t need to worry about the precision of analytical results in EVS.
- Volumes and masses in volumetrics will be accurate.
- Sites with large X and/or Y coordinates and small X-Y extents can have some problems in the viewer.
What do we mean by large X-Y coordinate and small extents? With the table above, you can see if your site might have issues. Here is how:
Here’s an example.
- Take the larger of your Maximum X or Y coordinate and find the “Float Precision” in the table above.
- If your Y coordinates are bigger than X coordinates and Ymax = 10,315,442, then the Float Precision is 1.0
- Use file_statistics to get the X-Y extents. Let’s assume Xext = 65.2 and Yext = 62.8. Take the lesser extent = 62.8
- Compute the Precision Ratio by taking the lesser extent divided by the Float Precision
- If this is less than ~400, you’ll likely see some viewer issues. In the above case it would be < 63, so the problems would be very obvious.
- If you zoom into a small region of your site, a Precision Ratio of 400 may not be high enough.
What do we mean by “problems in the viewer”?
- The good news is that your model will look exactly the same (in EVS’s viewer) when your Precision Ratio is 63 or 1,000.
- Problems are only obvious when you try rotating or zooming on the model
- Rotation problems are the most obvious. The model will be jerky when rotating.
- Zooming is only obvious if you use Shift-Middle-button with your mouse. This (normally) gives smooth zooming and it too will be jerky.
- If you tend to use the Az-El controls, you might never see the above issues.
- Making a bitmap animation with rotations and or zooming will likely be a big problem.
- Your bitmap animation (e.g. AVI file) will clearly exhibit these problems to the point that it will likely be unacceptable.
What other problems might you encounter?
- Exporting models to other software (e.g. CAD) may be problematic.
- Tests in Autodesk’s AutoCAD 2020 showed no problems importing DWG files created in EVS with low Precision Ratios. (however, it does not have an option to perform smooth rotations)
- Tests in ZWCAD 2020 Pro showed some problems importing DWG files created in EVS with low Precision Ratios.
- When displaying the model in Wireframe model it showed no issues
- When rendering as surfaces (Gouraud or Flat), there were severe rendering problems.
- Tests in ESRI’s ArcScene 10.7.1 had no major problems (but rotation is slow for modest sized models).
How can you fix this issue for your project?
- If you discover that this is happening to you, there is a relatively straightforward solution:
- Remember, this only happens when you have big coordinates and small coordinate extents!
- Translate the coordinates of your data (all of it) to move the range of your coordinates
- For example, if your:
- X coordinates range from 211,400 to 211,470 (extent 70), and
- Y coordinates range from 6,133,200 to 6,133,260 (extent 60), then
- Your precision is 0.5 with a Precision ratio of 120 (60 / 0.5)
- If you translate the Y coordinates by -6,000,000, then
- Your precision drops to 0.015625
- Your Precision Ratio increases to 3,840
- What are the consequences and limitations in EVS?
- Your model’s coordinates displayed in the viewer will be offset by 6 million.
- Probing will report Y coordinates that are off by 6 million.
- Normally the axes module would label the translated Y coordinates (offset by 6 million)
- However, the axes module has a “Set Axes Origin” toggle. Turn it on and set Y to be -6,000,000.
- This will correct the coordinates displayed by axes.
- For non-ASCII data, use this approach:
- For CAD and Shapefiles, use transform_field to move the coordinates
- For georeferenced images (for use in overlay_aerial), save/convert your image to a format with a world file (e.g. PNG with .PGW or .JPG with .JGW).
- Translate the appropriate coordinate(s) in the world file.
- For example, if your:
C Tech’s Long-Term Plans:
We know how to solve this problem completely, but it will take a major restructuring of EVS and changing many of the 3rd party libraries upon which our software is built. We are moving in this direction.