Calculating the Weather

Weather forecasting is in its third phase.  For thousands of years single observer forecasting was the only possible tool.  John Ruskin in 1839 explained why that approach did not work.

Then from the late 1850s, the synoptic approach was the main tool by which forecasters could "look" at the weather as plotted and analysed on charts, day-in, day-out. This had some, partial success because the

meteorologists could understand what was happening but still could not quantify their understanding. 

Numerical weather prediction has been gaining strength ever since WW2.  It is not a statistical or pattern matching approach but one that seeks to model the atmosphere.  This page tries to make to complexities understandable. To see if succeeds - now, read on...

Introduction

Most sailors will have realized that Numerical Weather Prediction has produced the greatest advances in weather prediction ever. Since my time as a Senior Forecaster ending in the late 70s, the improvements are quite astounding. However, forecasts are still far from perfect. Small scale detail, such as we sailors would like to know about, can still be lacking. The greatest improvements have taken place in predictions over periods up to a week. These are very useful for planning purposes and help greatly in decision making.

But, what is NWP?   First and foremost, it is not a statistical process, nor is it pattern matching. Those have been tried and, simply, do not work.

It all began during WW1 when a British mathematician, Lewis Fry Richardson, first proposed a scheme for calculating the weather. Richardson reasoned that the atmosphere is a fluid subject to the laws of physics. He said that these laws could be expressed in mathematical terms. But what are these processes?

How the Atmosphere Works

In broad and very general terms, a simplified description of how the weather machine works is as follows:

• Heating and cooling of the earth varies with time of day, time of year, latitude and nature of the surface. Oceans will heat up and cool down very slowly. Deserts and other dry areas will heat up and cool down quickly. Conurbations, prairies forests and so on will all behave in different ways. Slopes facing the sun will heat up quicker than those facing away from the sun.

  

• The resulting temperature differences create different air temperatures leading  to pressure differences. Pressure differences lead to air movement - wind.

  

• Winds - movement of air - cause a redistribution of pressure leading to changes in the winds.

• Heating of the oceans, seas, rivers, lakes and vegetation results in the air containing water vapour. Cooling of the air occurs when air is forced to rise by flowing over hills, by convection or by air streams converging. Cooling also can occur when the air is in contact with cold ground.

• As the air cools it is able to hold less water vapour until, eventually the water vapour starts to condense into droplets. This releases latent heat. If the air continues to cool then the drops may freeze to become ice or snow, again releasing latent heat. These effects warm the air. If the

ice melts then  heat is needed. This comes from the air so  cooling takes place. Again. if the water drops evaporate, then latent heat is required from the air and, again, cooling tales place.

• These latent heat effects change the air temperatures and therefore the pressure patterns and, hence, the winds.

 

• The formation of drops of water in the air cause cloud to form. Cloud cuts off the radiation from the sun and reflects it back out to space - that is why cloud appears so bright when you fly over it. But, cloud also absorbs radiation from the ground and re-radiates it back to earth.

• Topography affects weather on all scales. We experience sea breezes and the effects of headlands, cliffs, valleys and hills on a local scale. On the larger scale, mountain ranges, such as the Rockies and the Urals have correspondingly large scale effects. On an even larger scale, continents are important - eg the Roaring Forties of the Southern Oceans result from the relatively small amount of land compared to the Northern Hemisphere.

The atmospheric system, thus, has multiple feedbacks, both positive and negative.  All these physical processes can be expressed mathematically and that concept is the basis of modern forecasting. Richardson was the first - or, certainly, one of the first to realise this and try to develop the ideas scientifically.

Calculating It

Richardson, a Quaker pacifist, was a stretcher bearer in France during WW1. He spent his spare time working on his ideas of numerical weather prediction and tried to calculate the pressure change for 6 hours for one place in Central Europe. He got the answer wrong by a factor of 100 but he still published a book (in 1922) saying what he had done and why.

In this book, he envisaged the forecast room of the future as a spherical building so as to simulate the earth, rather  like the Albert Hall, full of people with hand calculators and slide rules all doing sums at the same time. Conceptually, this is the super-computer of today. Parallel processing is not a new idea.

After WW2, when electronic computing became a reality, Richardson's ideas were dusted off. Ever since then, meteorologists have been keeping ahead  of the computer designers by specifying needs for greater and greater

computing power and speed. The latest, massively parallel processors used by weather services are fulfilling Richardson's dream.

The real problem lies in putting numbers to the many equations. This is, in fact, impossible to do precisely and many approximations and estimates  have to be made. There are also some very clever mathematical computing techniques used. Richardson failed in his very brave attempt on both counts.

In plain language all the physical processes mentioned above have to be estimated and many of the advances in forecasting over the past 30 or more years have been possible because of improvements in these "guesstimates".

That is, at least, in principle. The practice is not as easy as those words might imply.

The Data

Their are several different kinds of data used in NWP. First there are observations made at the surface of the earth  from fixed locations on land. These observations may be entirely made by a human being, totally automatic or a combination.  These can be provided by professional observers, by air traffic controllers, coast guards or, sometimes, willing volunteers. Observations are almost always made at the Main Synoptic Hours of 00, 06, 12 and 18 hours UTC. Observations are often made hourly, particularly from automatic stations on land and at sea. 

There are observations from ships on passage. These are usually made by the officer of the watch. Usually these will be made at one or more of the four main synoptic hours.

At a selection of land stations and from a few ships there are measurements from free flying balloons, known as radio sondes. These are normally made at 00 and  12 UTC for temperatures, hunidities and winds. At 06 and 18 hours UTC a few stations may make full soundings but many more will simply measure wind alone. At an increasing number of locations there are

measurements from a  kind of radar that can detect movement of clear air and so measure winds throughout the atmosphere above the location of the equipment.

All these data are gathered at the Main Synoptic hours. In addition there are other types of observation that can be available at any time of day or night. There are measurements of wind and temperature from aircraft on routine commercial flights. There are data from drifting buoys. There are satellite orbiting on low level polar orbit or in geostationary orbit.

Satellite data are often very difficult to use because satellites are not making in situ observations but measuring the effects of the atmosphere on the radiation from the earth and the effects of wind on the sea.

All weather data are exchanged throughout the world in very quick time. The amount of data received at any major weather centre depends upon the nature of the data. In the UK, for example, data are received hourly from the near continent, three hourly from parts of N America and further parts of the continent and 6 hourly at the main synoptic hours from the rest of the world.

Starting off the NWP

At the major NWP centres, eg the national weather services of the UK, US, France, Germany, Japan, Australia and, the ECMWF etc, the data have, first to be checked for errors. These can occur due to human error, malfunctioning of equipment or in communications. Sometimes this process is entirely automatic and sometimes with human input. Where possible, data are corrected and only rejected if necessary.

Data are assimilated into the NWP process in several ways by the various centres. One method is the use of a 4 dimensional method. In this, starting from 00 UTC, say, all new data are combined with the forecast at a time appropriate to the data, say 01 UTC. The forecast up to 06 UTC is then re-computed. Later, say at 02 UTC the same process is repeated and so on.

At the main synoptic hour of 06 UTC, there are, of course, many more data to  be assimilated. In this way, all the data for the previous 6 hours will have been used to give the analysis for 06 UTC the best possible start.

Data from the various differing sources are of varying quality and this is taken into account by giving variable weight to the way in which they are used. Similarly, when combing the predicted values with new data, the prediction is given a lower weight than new data.

The whole process is, as will be gathered, immensely complex and it is fair to say that as much effort goes into assessing and analysing the basic data as into the prediction process itself. Perhaps, even more, since the old adage of "garbage in garbage out" is very true.

Grid length

One of the most important approximations insofar as we sailors are concerned is the Grid Length. NWP cannot calculate the atmosphere as the continuum which it is. Models have to represent the atmosphere at a  3 - dimensional grid of points. The nearer the points are together, the better will the model be able to represent the atmosphere.

There are two limitations to the grid length spacing. First is the computing power available. Secondly, there is the amount of meteorological and topographic information.  For day to day forecasting with outlooks for the next few days a global model is needed. At the time of writing these have a grid length, typically, about  20 - 30 NM.

For short period, very local forecasting then much smaller grid lengths can be used over small areas. The UK Met Office currently runs a meso-scale, limited area  model for much of the North Atlantic and western Europe.  This has a grid length of about 6 NM.  For more detailed very short period forecasts of about an hour or so ahead then models can be run with shorter grid lengths. The UK Met Office is trialling a model with a 2 NM grid length model just for the area of the UK.

The US Navy runs an operational Coupled Ocean/Atmosphere Meso-scale Prediction System (COAMPS)  model with a 6 NM grid length. Meso-scale models are run by a number of research institutions and by Theyr.net in Iceland and by a group in North America on behalf of ProGRIB.

There are two limitations on the value of such limited or local area models. First, of course, is the computing power that limits the area covered. Computing power requirements increase by a factor of about 16 every time that the grid

length is halved.  Secondly, there is the uncertainty of what is moving into an area and the effects of large scale weather features which are unlikely, in themselves, to be forecast precisely . The model may be very good at dealing with  the local topographic effects on the sea breeze, say, but the results will be invalidated if the larger scale model that is bringing cloud across gets the amount of cloud wrong!

An important consequence of the grid length is the size of weather and topographical features that can be described and modelled. As a rough guide, this is between four and five times the grid length. In other words, 20  NM grid for global models cam only cope with weather and topography some
100 NM size. Such models do not even know that the Isle of Man exists!

The UK and several other Meso-scale models with 6 NM grid lengths can deal with features down to about 25 - 30 NM. They cannot deal with many of our favourite sailing waters such as the Solent or the Clyde. To model the Tor Bay sea breeze would need a grid length of about 0.5 NM or less, The Solent would need a model with grid lengths of around 300 metres or less. To model Start Point would need a grid length measures in 10s of  metres, perhaps 100 metres at most.

Meso-scale Model output

Output from meso-scale models can be obtained for several days ahead from ProGRIB/ Theyr.net, SailDocs (COAMPS) and Météo France. However, I suggest that Meso-scale forecasts should only be used up to around 48 hours ahead for a 6 NM model and for a few hours from models running with smaller grid lengths. But, users should experiment and see how useful they find the data. Accuracy and usefulness are not necessarily synonymous.

When National Met Services run meso-scale models they have a vast amount of raw data - conventional in situ observations, data buoys aircraft in flight data, satellite imagery, radar imagery. They can, also, use human judgement to ensure that the model is using all the data. This enables them to start with analyses that are on a scale comparable with the resolution of the models.  Even satellite and radar techniques can only observe weather on a scale of around 2-3 NM. We sailors know only too well how much variation there can be in wind over such short distances.

When a commercial company or university department is running such meso-scale models, they usually will have to rely upon a starting point provided by the National Met Services. For most, this will be the US GFS model which provides data at 1/2 degree intervals. These data have to be interpolated down to model grid lengths. Such organisations, therefore, start without all the detail that exists in weather patterns. This will compromise attempts at detailed prediction.

What this means, in practice, is that improvements in meso-scale modelling by private services swill not be as great as with the national services. Near land, it will be a few hours into the prediction before the effects of topography really take effect. They will not handle existing small weather features, ie less than around 100 NM size, at all.

There will be some benefits from such meso-scale models because they do model atmospheric physical processes better than global models. However, the limited areas for which these models can be run limits the time ahead for which that data are likely to be useful.

The time ahead at which they will be useful will depend upon the type of weather. In a very mobile weather situation, a meso=scale model with a 6 NM grid length may only give useful additional information at 24 to 36 hours ahead. A model with a finer grid length may only be useful up to 24 hours ahead. NOTE - these are illustrative figures and not at all precise.

However, he proof of any pudding is in the eating. Users with enquiring minds might like to try using meso-scale model forecast and see for themselves how good they are. I would expect that the Météo France meso-scale model, with its better starting point will do better than the commercial services. Were the UK Met Office to release data from its experimental model running at 2  NM grid length, then it, also, would perform better.

Limitations on what can be and cannot be forecast

Reading the previous section, I hope that the reasons for what we all observe is becoming clearer.  That is that forecasts of general weather type for the next few days - perhaps up to seven days - are remarkably good. To my mind, as a user pf forecasts this has been the major and most obvious  improvement since 1977 when I last worked as a Met Office Senior Forecaster at the then Central Forecast Office at Bracknell (now the National Meteorological Centre, Exeter).

It also follows from the above section that very local, detailed forecasting can only be possible for very short periods ahead. How far ahead and what level of detail local forecasting will be possible is, as yet, uncertain. Watch this space!

For the racing yachtsman, whether in dinghies or round-the-cans yachts it will be necessary for years to come to use careful observation, coupled with observation and experience. Anyone who has heard my old friend and

colleague David Houghton will be aware of what can be done along those lines.

For the coastal cruising yachtsman, the general forecasts give excellent advice for the next few days. Used properly, sailors should be able to avoid getting caught out in really bad weather. They should also be able to be in port or good sheltered anchorages when bad weather comes. Importantly, they should be able to avoid being in a position when they are in places that they do not want to be in, in weather that they do not want to go out in.

The blue water sailor making passages lasting weeks can only plan using climatology. His ability to avoid bad weather once out there is strictly limited. His only real tactic is to be aware of warnings and prepare for bad weather before it comes. He has to be fully equipped to do so.

Another limitation

The amount of information that it is and will be possible for forecasters to provide is enormous. Serious thought will have to be given to how the user will access and use  it. Onshore is one matter with broadband communications. Afloat is another. How many sailors will wish to be stuck down below,  nose, to a laptop computer? When will high speed communications become  

affordable for access over radio systems?

One thing is certain and that is that the written or spoken word will and cannot cope with all the variations in the weather that exist, whether  they are well forecast or not.

More

To read more about the operational aspects of NWP see the page on Weather Forecasts for the Sailor.  Also, see  Why do we pay for forecasts? and  my pages on Using Forecasts for cruising decisions. The Met Office has a very good, basic Education page and a Weather Topic page. The Internet has a wealth of pages of varying degrees of complexity., but try first. putting "numerical models" into the Met Office page search engine. Then do the same with Google.

Appendix.  Grid lengths - yet more. Ice pack on head time!

Introduction

So, why are short grid lengths important?

The same arguments apply to the observed data. Where the features of interest are largely forced by topography or synoptic scale structures, the most important observations are those that define the synoptic scale. Indeed, the most important aspect of observations  may one of avoiding distortion of the finer scale structures generated by the model.

For locally generated features, we have to rely on those sources of observations that are truly on the very small scale - eg satellite and radar imagery. These both

give an effective observation spacing of around 5 km.  Over the sea, of course radar data are only available to a fairly short distance from the coast.

In practice, there are, quite simply,  not enough data  that to define the very small weather detail that sailors observe over the sea.  Satellite (QuikScat) winds are not available often enough and have problems near coasts and in cloudy areas anyway. Over land, data from radar can be used to help predict small scale and short lived detail.

There is also the pragmatic observation that reducing the grid length has enabled improved performance in predicting the larger scale evolution. It is for this reason that global forecasting centres  continue to move to finer grid lengths in global models. For example, if  a hurricane can be predicted more accurately by resolving its circulation better, then a correct prediction becomes more likely of whether recurvature will occur, where it will engage in the mid-latitude flow, and hence the synoptic evolution over the UK at 4-5 days ahead.

Therefore, the future is likely to be one of finer resolution global models but with not a great improvement on short term, small detail over the sea.

What does it mean for the user?

1. More realistic representation of the impact of topography on coastal winds and weather on such scales as the strong winds through the Dover Strait.

2. More reliable forecasts of small scale thundery troughs and the like..

3. More reliable forecasts of the development of organised convection systems, with associated severe weather.