Dear Bill,

You got me thinking with your questions about the precision of celestial navigation. I have gotten some helpful insights from Frank Reed here and here.

Celestial navigation traditionally (and now I am beginning to understand why) has used the construct of the "celestial sphere": a perfect sphere that surrounds the earth at an infinite distance across which celestial objects move, some (e.g. stars) very slowly, some (e.g. moon, planets, sun) much more quickly. And the entire sphere rotates around the earth once a day.

On this sphere there are hour angle lines (which map to lines of longitude on earth) and declination lines (which map to lines of latitude on earth). There is also a celestial equator, which is precisely above the earth's equator. A celestial object appears on the sphere at some hour angle and degrees of declination, and precisely below that point, at an equivalent longitude and latitude, is the object's geographical position (GP).

Bowditch persists in trying to draw, on a flat page, the celestial sphere, inside which is drawn the earth, with points and lines of all sorts drawn on both. I have found that hopelessly confusing...and have found that you can largely do without the construct of the celestial sphere completely. Just think in terms of the GP rushing across the face of the earth at 15° per hour as one point of a spherical triangle with the north pole and your AP as the other points.

The celestial sphere is perfectly spherical, and spherical trigonometry is completely accurate to however many decimal places you care to calculate it.

The earth is NOT a perfect sphere...but our mapmakers have set up our lines of latitude such that they are mapped precisely from the celestial sphere's degrees of declination. So if you project down those declination lines from the celestial sphere down onto the non-spherical earth, the lines are not precisely the same distance apart when they touch the face of the earth.

We have decided that 1 nm = 1,852 m...but that is at a mid-latitude. A minute of latitude is longer than that at the equator, and shorter than that at the pole. I am coming to think that nailing down the length of the nautical mile to some number of meters was an exercise of second rate minds who couldn't handle the ambiguity of an apparently dimensionless measure of distance.

There is an analogy here to the use of dimensionless Mach numbers to measure aircraft speed...where the actual speed of "Mach 1" in km/hour varies depending on a plane's altitude and the temperature of the air. But I digress.

Celestial navigation deals in degrees and minutes...and where it thinks of nautical miles (regardless of what some land bound scientists have defined) it doesn't THINK in terms of meters. It thinks in terms of...well, those degrees and minutes. Thinking that way lets you work up your fix and position yourself on a chart precisely.

So celestial navigation, built on spherical trigonometry, "works" on an oblate spheroid because the work of resolving the non-spherical shape of the earth has been done at the chart level...by the mapmakers...by varying the width of degrees of latitude as they get closer to the poles. So as long as you can free your mind of the 1,852 strait jacket, celestial navigation works very well indeed, to a high degree of accuracy.

This grounding in the celestial sphere is why we can talk about a spherical triangle, rather than being forced to deal with some hypothetical "spherioidal triangle".

Frank Reed said, "In summary, we don't have to worry about the shift in latitude in celestial navigation due to the spheroidal shape of the Earth because it's already built into the definition of latitude that we use."

Now, if you are calculating the distance from Vancouver to Tokyo by great circle route, you would like your results to be in a kilometers or miles rather than angular degrees and minutes, and you don't want a constantly varying nautical mile. If you want an answer you can get with a slide rule or hand calculator, you use the spherical earth model...and accept several dozen nautical miles worth or error. If you have serious computing power available, you can use a WGS84 model of the earth, and get a better answer.

The biggest remaining inaccuracy of celestial navigation has to do with the fact that gravity varies in its direction from place to place. A big rock has its own gravity and can pull a plumb bob out of vertical, so it is not pointing at the center of the earth. So then, if you are sailing near a big volcano (e.g. Bermuda, any number of Caribbean islands) your view of the horizon is NOT perpendicular to a line leading to the center of the earth. The pull of the mass of the volcano is tipping the ocean's surface so it is "not level". (Seems reminiscent of C.S. Lewis' Ransom meeting Perelandra while on Venus.) Given that it is this ocean surface that you are using as your horizon to measure the height of the sun, you will inevitably have errors of position either near a volcano, or over a big gap like the Marianas Trench.

This is also why if you go to Greenwich Observatory, your handheld GPS does not read 0° longitude when you are standing on the line in the floor there. It is 100 m off to the east. English astronomers spent enormous energy defining 0° longitude there, using a tray of liquid mercury to define "horizontal" and then extending a line perpendicular to to that, going to the very center of the earth...except that because of a local gravitational variation, that line pointing "straight down" actually misses the center of the earth by some 15 cm or so. Run a line "straight down" from the GPS 0° point, and it DOES hit the center of the earth. This assumes the WGS84-defined average shape of the earth, of course.