Re: case-based reasoning, fuzzy logic & weather prediction

fuzzy_logician@my-deja.com
Fri, 15 Oct 1999 03:00:23 +0200 (MET DST)

.. http://chebucto.ns.ca/~bjarne/cbr

> Wasn't material on this project available online (.PS or .PDF) at some
> point? I recall reading a paper on this and found it interesting.

The online paper is a new how-to paper about how to combine case-based
reasoning (CBR) and fuzzy logic to predict weather. Though, I suppose
the technique could be used for prediction of any sort of natural time
series where perceptiveness, fuzziness, and large databases of past
cases are involved.

> Did you have a more specific discussion in mind?

Three points in my thesis, topics for discussion, are:

Fuzzy k-nn is not fuzzy clustering
... it avoids the cluster validity problem
Fuzzy k-nn is not a fuzzy expert system
... it avoids the rule explosion problem
Fuzzy k-nn is intelligently guided data mining
... it imparts to CBR the discrimation ability of an expert

I'd appreciate any pointers about fuzzy k-nn prediction techniques.

Expert systems and CBR are complementary approaches to AI. Models and
cases both represent knowledge. Fuzzy expert systems have been common
since the early 1980’s. So, I suppose combining fuzzy logic and CBR
opens new opportunities.

In weather prediction, atmospheric models and analog forecasting are two
complementary methods for prediction. Someone on the CBR mailing list
asked

> how does case-based weather forecasting compare
> with the usual number crunching methods?

Two sections in my thesis address this. I've inserted the beginnings of
those sections below. The gist of it is that analog forecasting is
complementary to numerical weather prediction (NWP) and can thereby
improve NWP-based forecasts by postprocessing NWP output.

> Do you find that you can forecast as accurately -
> or as far ahead - for example?

Analog forecasting shows potential to improve accuracy and increase time
range of airport weather forecasts. But ultimately, sensitive
dependence on initial conditions (i.e., chaos) limits the time scale of
both analog forecasting and atmospheric models. The effect is
illustrated at http://chebucto.ns.ca/~bjarne/chaotic

----------------------------
Analog forecasting technique

Weather patterns repeat themselves—this is the basic idea behind the
weather prediction technique called analog forecasting. Analog
forecasting is a meteorological retrieval based form of CBR. Analog
forecasting is simple in theory: make a prediction for the current
situation based on the outcome of similar past situations. However,
development of analog forecasting systems is challenging in practice.

Analog forecasting is by far the oldest weather prediction technique.
Useful weather sayings are based on recurring patterns of weather, and
using recurring patterns of weather is essentially analog forecasting,
thus useful weather sayings are a form of analog forecasting. For
example, the following familiar saying is at least 2000 years old.

"Red sky in the morning, sailors take warning.
Red sky at night, sailors delight." (Anonymous)

The Online Guide to Weather Forecasting describes analog forecasting as
follows.

"It involves examining today’s forecast scenario and remembering a day
in the past when the weather scenario looked very similar (an analog).
The forecaster would predict that the weather in this forecast will
behave the same as it did in the past. ... The analog method is
difficult to use because it is virtually impossible to find a perfect
analog. Various weather features rarely align themselves in the same
locations they were in the previous time. Even small differences
between the current time and the analog can lead to very different
results. However, as time passes and more weather data is archived, the
chances of finding a ‘good match’ analog for the current weather
situation should improve, and so should analog forecasts."
..

Numerical Weather Prediction

Numerical weather prediction (NWP) is a top-down, model-based approach,
whereas analog forecasting is a bottom-up, model-free approach. NWP is
numerical, computer modelling of the weather based on known physical
equations of the atmosphere. NWP is the antithesis of analog
forecasting and, thus, the two techniques are complementary. NWP is a
huge, very active field in meteorological research, so because of
limited space, we can only give a few relevant details here.

Lorenz (1993) describes NWP as a set of simultaneous equations with 5
million variables. Horizontally, the NWP model has a coarse grid of
about 20 kilometers spacing. Initialization points are 100’s of
kilometers apart over land and 1000’s of kilometers apart over the
ocean. The model is initialized with rawinsonde measurements (i.e.,
weather balloon soundings) every twelve hours. Local weather, such as
that at specific airports, operates across far finer space and time
scales. Battan (1984) explains how NWP forecasts only large-scale
weather features.

"The present-day numerical models do a satisfactory job in predicting
the patterns of pressure and wind velocity, especially in the middle
layers of the atmosphere, such as at the 500-mb level [approximately
18,000 feet]. The models still do not adequately predict surface
temperatures, winds, and precipitation. These quantities are
significantly influenced by local geographic features as well as the
patterns of atmospheric pressures and winds. Large bodies of water,
hills, and mountains affect local weather in ways discussed in (Battan
1984). In actual practice, the output of numerical calculations
represent the first step in weather forecasting. Meteorologists
familiar with local influences use the maps produced by numerical models
as guides for producing specific predictions of temperature,
precipitation, and wind."

Weather predictions of large-scale weather systems in the
12-hour-to-5-day timeframe are largely derived from NWP. NWP directly
models synoptic scale (i.e., large-scale) continuous parameters, such as
atmospheric pressure and wind fields. NWP cannot directly model local
scale weather, such as stratus and fog at an airport, as Battan (1984)
explains.

"To an increasing extent, the information on prognostic charts is
employed as input data for statistical methods in which the weather at a
specific locality is related to the values of pressure, temperature, and
humidity at one or more places. Such statistical techniques lend
themselves to prediction of the probable occurrence of certain weather
events."
..

Bjarne

Date: Thu, 14 Oct 1999 14:39:33 GMT
Message-ID: <7u4pv3$m5v$1@nnrp1.deja.com>
References: <7u216r$leu$1@nnrp1.deja.com> <38052C2D.A81@compuserve.com>
NNTP-Posting-Host: 192.75.95.23
X-Article-Creation-Date: Thu Oct 14 14:39:33 1999 GMT
X-Http-User-Agent: Mozilla/4.04 [en] (Win95; I)
X-Http-Proxy: 1.0 x36.deja.com:80 (Squid/1.1.22) for client 192.75.95.23
X-MyDeja-Info: XMYDJUIDfuzzy_logician
Status: R

.. http://chebucto.ns.ca/~bjarne/cbr

> Wasn't material on this project available online (.PS or .PDF) at some
> point? I recall reading a paper on this and found it interesting.

The online paper is a new how-to paper about how to combine case-based
reasoning (CBR) and fuzzy logic to predict weather. Though, I suppose
the technique could be used for prediction of any sort of natural time
series where perceptiveness, fuzziness, and large databases of past
cases are involved.

> Did you have a more specific discussion in mind?

Three points in my thesis, topics for discussion, are:

Fuzzy k-nn is not fuzzy clustering
... it avoids the cluster validity problem
Fuzzy k-nn is not a fuzzy expert system
... it avoids the rule explosion problem
Fuzzy k-nn is intelligently guided data mining
... it imparts to CBR the discrimation ability of an expert

I'd appreciate any pointers about fuzzy k-nn prediction techniques.

Expert systems and CBR are complementary approaches to AI. Models and
cases both represent knowledge. Fuzzy expert systems have been common
since the early 1980’s. So, I suppose combining fuzzy logic and CBR
opens new opportunities.

In weather prediction, atmospheric models and analog forecasting are two
complementary methods for prediction. Someone on the CBR mailing list
asked

> how does case-based weather forecasting compare
> with the usual number crunching methods?

Two sections in my thesis address this. I've inserted the beginnings of
those sections below. The gist of it is that analog forecasting is
complementary to numerical weather prediction (NWP) and can thereby
improve NWP-based forecasts by postprocessing NWP output.

> Do you find that you can forecast as accurately -
> or as far ahead - for example?

Analog forecasting shows potential to improve accuracy and increase time
range of airport weather forecasts. But ultimately, sensitive
dependence on initial conditions (i.e., chaos) limits the time scale of
both analog forecasting and atmospheric models. The effect is
illustrated at http://chebucto.ns.ca/~bjarne/chaotic

----------------------------
Analog forecasting technique

Weather patterns repeat themselves—this is the basic idea behind the
weather prediction technique called analog forecasting. Analog
forecasting is a meteorological retrieval based form of CBR. Analog
forecasting is simple in theory: make a prediction for the current
situation based on the outcome of similar past situations. However,
development of analog forecasting systems is challenging in practice.

Analog forecasting is by far the oldest weather prediction technique.
Useful weather sayings are based on recurring patterns of weather, and
using recurring patterns of weather is essentially analog forecasting,
thus useful weather sayings are a form of analog forecasting. For
example, the following familiar saying is at least 2000 years old.

"Red sky in the morning, sailors take warning.
Red sky at night, sailors delight." (Anonymous)

The Online Guide to Weather Forecasting describes analog forecasting as
follows.

"It involves examining today’s forecast scenario and remembering a day
in the past when the weather scenario looked very similar (an analog).
The forecaster would predict that the weather in this forecast will
behave the same as it did in the past. ... The analog method is
difficult to use because it is virtually impossible to find a perfect
analog. Various weather features rarely align themselves in the same
locations they were in the previous time. Even small differences
between the current time and the analog can lead to very different
results. However, as time passes and more weather data is archived, the
chances of finding a ‘good match’ analog for the current weather
situation should improve, and so should analog forecasts."
..

Numerical Weather Prediction

Numerical weather prediction (NWP) is a top-down, model-based approach,
whereas analog forecasting is a bottom-up, model-free approach. NWP is
numerical, computer modelling of the weather based on known physical
equations of the atmosphere. NWP is the antithesis of analog
forecasting and, thus, the two techniques are complementary. NWP is a
huge, very active field in meteorological research, so because of
limited space, we can only give a few relevant details here.

Lorenz (1993) describes NWP as a set of simultaneous equations with 5
million variables. Horizontally, the NWP model has a coarse grid of
about 20 kilometers spacing. Initialization points are 100’s of
kilometers apart over land and 1000’s of kilometers apart over the
ocean. The model is initialized with rawinsonde measurements (i.e.,
weather balloon soundings) every twelve hours. Local weather, such as
that at specific airports, operates across far finer space and time
scales. Battan (1984) explains how NWP forecasts only large-scale
weather features.

"The present-day numerical models do a satisfactory job in predicting
the patterns of pressure and wind velocity, especially in the middle
layers of the atmosphere, such as at the 500-mb level [approximately
18,000 feet]. The models still do not adequately predict surface
temperatures, winds, and precipitation. These quantities are
significantly influenced by local geographic features as well as the
patterns of atmospheric pressures and winds. Large bodies of water,
hills, and mountains affect local weather in ways discussed in (Battan
1984). In actual practice, the output of numerical calculations
represent the first step in weather forecasting. Meteorologists
familiar with local influences use the maps produced by numerical models
as guides for producing specific predictions of temperature,
precipitation, and wind."

Weather predictions of large-scale weather systems in the
12-hour-to-5-day timeframe are largely derived from NWP. NWP directly
models synoptic scale (i.e., large-scale) continuous parameters, such as
atmospheric pressure and wind fields. NWP cannot directly model local
scale weather, such as stratus and fog at an airport, as Battan (1984)
explains.

"To an increasing extent, the information on prognostic charts is
employed as input data for statistical methods in which the weather at a
specific locality is related to the values of pressure, temperature, and
humidity at one or more places. Such statistical techniques lend
themselves to prediction of the probable occurrence of certain weather
events."
..

Bjarne

############################################################################
This message was posted through the fuzzy mailing list.
(1) To subscribe to this mailing list, send a message body of
"SUB FUZZY-MAIL myFirstName mySurname" to listproc@dbai.tuwien.ac.at
(2) To unsubscribe from this mailing list, send a message body of
"UNSUB FUZZY-MAIL" or "UNSUB FUZZY-MAIL yoursubscription@email.address.com"
to listproc@dbai.tuwien.ac.at
(3) To reach the human who maintains the list, send mail to
fuzzy-owner@dbai.tuwien.ac.at
(4) WWW access and other information on Fuzzy Sets and Logic see
http://www.dbai.tuwien.ac.at/ftp/mlowner/fuzzy-mail.info
(5) WWW archive: http://www.dbai.tuwien.ac.at/marchives/fuzzy-mail/index.html