This document describes HydrOffice QC Tools 4.9.9.
2.3.3.2. How Does It Work?¶
2.3.3.2.1. Flier Height¶
The flier height is used in:
2.3.3.2.1.1. Automatic Estimated Flier Height¶
The automatic determination of the flier height is performed per tile, and is based on the median depth (characteristic depth), the MAD (variability in range), and the standard deviation of the gaussian curvature (roughness).
First, a base height from the median depth of the grid is assigned:
Depth Interval |
Base height |
|---|---|
if < 20 |
1.0 |
if < 40 |
2.0 |
if < 80 |
4.0 |
if < 160 |
6.4 |
if < 320 |
9.6 |
if < 640 |
16.0 |
if < 1280 |
25.6 |
if < 2560 |
38.4 |
if >= 2560 |
51.2 |
Then, the base height is incrementally increased by the level of depth variability and roughness of the grid:
Depth variability is estimated by proxy using the Normalized Median of Absolute Deviation (NMAD) of the grid, which is derived by dividing the absolute difference of depth mean and depth median by depth standard deviation.
The lower the NMAD, the more depth variability we estimate.
An increase in the Base height of the flier search height estimation is warranted if NMAD is: * Less than 0.20 (1-meter increase) * Less than 0.10 (2-meter increases).
Roughness is estimated by the standard deviation of the Gaussian curvature (STD_CURV).
The Gaussian curvature is a measure of concavity at each node, whether concave up (shoal) or concave down (deep).
The higher the STD_CURV, the rougher the surface.
An increase in the Base height of the flier search height estimation is warranted if STD_CURV is greater than: * 0.01 (1-meter or 2-meter increase based on the initial estimate) * 0.1 (2-meter increase) * 3.0 (4-meter increase) * 30.0 (8-meter increase) * 100.0 (16-meter increase)
For example:
If a surface has depth median = 12 m, NMAD = 0.15, and STD_CURV = 0.005, then the Estimated height = 2.0 m.
If a surface has depth median = 75 m, NMAD = 0.04, and STD_CURV = 0.08, then the Estimated height = 10.0 m.
If a surface has depth median = 5337 m, NMAD = 0.41, and STD_CURV = 294.20, then the Estimated height = 83.2 m.
2.3.3.2.2. Checks¶
2.3.3.2.2.1. Laplacian Operator [#1]¶
The Laplacian Operator is a measure of curvature at each node. It is equivalent to summing the depth gradients of the four nodes adjacent (north, south, east, and west) to each node. If the absolute value of the Laplacian Operator is greater than four times the flier search height, the node will be flagged.
In the example below, a 3 m flier search height would register 1 flag, while a 2 m flier search height would register 4 flags, and a 1m search height would register 7 flags.
Depth Layer |
Laplacian Operator |
|||||||
|---|---|---|---|---|---|---|---|---|
9 |
9 |
9 |
9 |
0 |
1 |
0 |
3 |
|
9 |
8 |
9 |
6 |
1 |
4 |
10 |
9 |
|
9 |
9 |
3 |
9 |
0 |
7 |
24 |
9 |
|
9 |
9 |
9 |
9 |
0 |
0 |
6 |
0 |
|
The algorithm is effective, but may be prone to excessive flags, as demonstrated in the above example. Testing showed that it generally did not reveal fliers not already revealed by the other algorithms. For these reasons, it is disabled by default, but is recommended as an additional check in those situations when the other algorithms return very few or no flags.
The example in Fig. 2.8 shows grid nodes (depths in meters) recommended for further examination by the Laplacian Operator (indicated by 1s) and a 6m estimated search height.
Fig. 2.8 Laplacian Operator.¶
2.3.3.2.2.2. Gaussian Curvature [#2]¶
The Gaussian Curvature is a measure of concavity at each node. The gradients are taken in the x and y directions to establish gx and gy, and repeated on each gradient again in the x and y direction to establish gxx, gxy, gyx, and gyy (note that gxy = gyx). The gaussian curvature at each node is then determined by:
(gxx * gyy - (gxy * gyx)) / (1 + (gx^2) + (gy^2))^2
In the example below, a single flier is found (regardless of Flier Height).
Depth Layer |
Gaussian Curvature |
|||||||
|---|---|---|---|---|---|---|---|---|
9 |
9 |
9 |
9 |
-1 |
0 |
-1 |
-.09 |
|
9 |
8 |
9 |
6 |
0 |
-2.3 |
0 |
-.14 |
|
9 |
9 |
3 |
9 |
-2.5 |
0 |
20 |
0 |
|
9 |
9 |
9 |
9 |
0 |
-9 |
0 |
-36 |
|
Testing showed that the algorithm on occasion offered unique value by flagging a flier not captured by other algorithms, while also it is not prone to excessive flags. For these reasons this algorithm is enabled by default.
The example in Fig. 2.9 shows grid nodes (depths in meters) and a deep flier found by the Gaussian Curvature (indicated by the red 2).
Fig. 2.9 Gaussian Curvature.¶
Note
This algorithm uses the standard deviation of the tile’s Gaussian curvature, not the Flier Height.
2.3.3.2.2.3. Adjacent Cells [#3]¶
This algorithm examines the nodes that are adjacent to a single node. There are a maximum of 8 adjacent nodes (N,NW,W,SW,S,SE,E,NE), but there could be less than 8 if the node resides on a grid edge.
The algorithm crawls across empty cells (2 nodes diagonally, and 3 nodes in the cardinal directions) in order to establish neighbors. For example, the image below shows that 6 neighbors were found for the flagged node; previous versions of Flier Finder would only have identified 4 (Fig. 2.10).
Fig. 2.10 Crawling example.¶
The depth is differenced with each adjacent cell identified, and the number of times the difference is greater in magnitude than the Flier Height is tallied. If the ratio of this tally to the number of adjacent cells available is 0.8 or greater, then the node is flagged. [1]
In the example below, a 3 m flier search height would register 2 flags, while a 2m search height would also register 2 flags, and a 1m search height would register 3 flags.
Depth Layer |
Adjacent Cells(3m) |
|||||||
|---|---|---|---|---|---|---|---|---|
9 |
9 |
9 |
9 |
0 |
0 |
.2 |
.33 |
|
9 |
8 |
9 |
6 |
0 |
.13 |
.25 |
1 |
|
9 |
9 |
3 |
9 |
0 |
0 |
1 |
.4 |
|
9 |
9 |
9 |
9 |
0 |
.2 |
.2 |
.33 |
|
Testing showed that the Adjacent Cells algorithm offers unique value by flagging fliers not captured by the other algorithms (especially those residing on grid edges), and is not as prone to excessive flagging as the Laplacian Operator. For these reasons, it is enabled by default.
The example in Fig. 2.11 shows grid nodes (depths in meters) and the type of flier that Adjacent Cells (indicated by the red 3) identifies with particular effectiveness, in this case with a 4m search height.
Fig. 2.11 Adjacent Cells.¶
Footnotes
2.3.3.2.2.4. Edge Slivers [#4]¶
The Edge Slivers algorithm identifies small groups of connected nodes (3 nodes or less) that are detached (but within 5 nodes) from the grid. If the depth difference between the nearest detached node and the valid connection to the grid is greater than half the Flier Height, a flag is registered.
Testing showed that the algorithm offers unique value by identifying the quite common fliers that result in areas of sparse data density. For this reason, it is enabled by default.
The example in Fig. 2.12 shows grid nodes (depths in meters) and the type of detached nodes that Edge Slivers flags, in this case with a 4m search height.
Fig. 2.12 Edge Slivers.¶
2.3.3.2.2.5. Isolated Nodes [#5]¶
The Isolated Nodes algorithm identifies small groups of connected nodes (3 nodes or less) that are detached (but outside of 5 nodes) of the grid. Effectively it is identifying the remaining isolated nodes not caught by Edge Slivers, however, it is independent of Flier Height, meaning that all small groups of isolated nodes will be flagged.
Testing shows that the algorithm offers unique value by identifying nodes far detached from the grid that the reviewer may wish to exclude. Because it is identifying any detached nodes and not considering their associated depth, it is largely considered a separate tool to be used on an “as-needed” basis. Therefore, it is not enabled by default.
The example in Fig. 2.13 shows a grid node far detached from the main grid, found by Isolated Nodes (indicated by a red 5).
Fig. 2.13 Isolated Nodes.¶
Note
This algorithm does not use the Flier Height.
2.3.3.2.2.6. Noisy Margins [#7]¶
Note
Flier check #6 (Noisy Edges) was deprecated in favor of flier check #7 (Noisy Margins) which is an improved version.
The Noisy Margins is tailored to identify fliers along noisy swath edges.
The algorithm crawls across empty cells (2 nodes diagonally, and 3 nodes in the cardinal directions) in order to establish a margin. A margin is identified when a node is missing two neighbors in the surrounding 8 directions (N, NW, W, SW, S, SE, E, and NE).
Once that a margin node is identified, the least depth and the maximum difference with its neighbors are calculated.
The least depth is used to calculate to local Total Vertical Uncertainty (TVU), which is used for the flagging threshold. The TVU is calculated per NOAA HSSD:
where 
for Exceptional, 
for Critical, 
for General 1, 
for General 2 and General 3, and 
for General 4.
The TVU is selected in the Input tab in the Feature Detection drop down.
A noisy margin is flagged when the maximum depth difference with its neighbors is greater than the flagging threshold. To prevent too many flags, the algorithm searches the nearest three nodes and if any of those nodes contain a flag, it will not be flagged. If a flag is not present in a three node area, the flier is flagged.
Fig. 2.14 shows an example of a flagged 4.5m edge node. Since the shallowest node in the neighborhood is 3.7m, the flagging threshold developed from TVU was 0.504m for General 1. The maximum difference between the node and its neighbors is 0.8m, therefore the margin node was flagged.
Fig. 2.14 Noisy margin.¶
Note
This algorithm uses the TVU calculated per NOAA HSSD, not the Flier Height.
2.3.3.2.3. Which Algorithms Should I Use?¶
To assist with this question, we have gathered insights from power users and created a separate, easy-reference document Common Flier Finder Techniques that covers the following topics:
2.3.3.2.4. Masking and Filters¶
2.3.3.2.4.1. Masking¶
Flier Finder does not produce duplicate flags when multiple enabled algorithms flag the same node during processing. If an earlier algorithm in the processing queue has already flagged a node, that existing flag masks any subsequent one that would otherwise be added by a later algorithm for the same node. The result means that a processing queue utilizing, for example, algorithms #2 and #3 together might produce less total flags than running them in separate runs.
When activated, the order of execution of the algorithms is:
This sequence is intentionally arranged from the algorithms that tend to produce the fewest flags (Gaussian Curvature [#2]) to the most (Noisy Margins [#7]).
2.3.3.2.4.2. Filters¶
After the algorithms have completed identifying fliers, an optional final step compares those fliers against point features with a depth (when Use Features from S57 File is checked).
By default, the filters remove a flier candidate when the distance of a point feature is:
Horizontally, less than (or equal to) 1.0 times the resolution (e.g., 1m for 1-m grid).
Vertically, less than (or equal to) 0.01 meters.
It is also possible to modify the above filtering criteria. And, if the user does not want to apply the filters, they may be turned off.
