RE: How to quantify the quality of fuzzy expert system?

From: Ulrich Bodenhofer (ulrich.bodenhofer@scch.at)
Date: Wed Aug 22 2001 - 07:19:45 MET DST

  • Next message: Robert Dodier: "Re: Thomas' Fuzziness and Probability"

    Dear Phayung Meesad,

    indeed all the quality criteria you mentioned are of vital importance
    in fuzzy rule-based modeling. Some of them, however, are IMHO rather
    vague and subjective.

    Among those you refer to, accuracy is the one that appears most often
    (as objective function in methods for constructing fuzzy systems from
    example data; this function most often measures the degree of matching
    between desired and actually obtained output). I cannot refer to
    specific publications, since accuracy (with varying definitions) appears
    is almost every paper dealing with constructing/optimizing fuzzy systems
    according to example data.

    I have the opinion (as I suspect, like you) that accuracy alone is not
    the only criterion that matters. If one only concentrates on accuracy
    without taking the other criteria into account, one ends up with
    functions which are fuzzy systems formally, but do not allow any insights
    into the relationships the system is supposed to model (black-box behavior).

    The quality criterion that fortunately increases in importance is
    interpretability. Although being a subjective and vague matter, too,
    several researchers have encountered that it is a crucial property.
    I do not know all of the people working in this area, but I can give a
    few pointers:

    http://decsai.ugr.es/indexenglish.html
    http://lcewww.et.tudelft.nl/

    Last but not least, our group:

    http://www.scch.at/research/projects/AdvLearn/home/

    You may also check ResearchIndex and search for the criteria
    you would like to know more about:

    http://www.researchindex.com/

    By the way, some group has concrete plans for an edited volume about
    interpretatility and its interplay with accuracy (too early to
    say more about it).

    I hope this information is helpful for you.

    Best regards,
    Ulrich Bodenhofer

    > -----Original Message-----
    > From: fuzzy-mail@dbai.tuwien.ac.at
    > [mailto:fuzzy-mail@dbai.tuwien.ac.at]On Behalf Of Phayung Meesad
    > Sent: Tuesday, August 21, 2001 11:42 AM
    > To: Multiple recipients of list
    > Subject: How to quantify the quality of fuzzy expert system?
    >
    >
    > Dear Experts,
    >
    > I am wondering if anyone has experience on how to evaluate the goodness of
    > fuzzy rules. How to quantify the following qualitative measures of fuzzy
    > rule based systems?
    > Accuracy
    > Completeness
    > Comprehensibility
    > Consistency
    > Interpretability
    > Soundness
    > (Or anything else that can be used to measure the goodness of fuzzy expert
    > systems.)
    >
    > It would be very appreciated if anyone can direct me to papers, textbooks,
    > or websites related to these issues.
    >
    > Thank you in advance.
    >
    > Sincerely,
    > Phayung Meesad
    >
    >
    >
    >
    > ##################################################################
    > ##########
    > This message was posted through the fuzzy mailing list.
    > (1) To subscribe to this mailing list, send a message body of
    > "SUB FUZZY-MAIL myFirstName mySurname" to listproc@dbai.tuwien.ac.at
    > (2) To unsubscribe from this mailing list, send a message body of
    > "UNSUB FUZZY-MAIL" or "UNSUB FUZZY-MAIL
    > yoursubscription@email.address.com"
    > to listproc@dbai.tuwien.ac.at
    > (3) To reach the human who maintains the list, send mail to
    > fuzzy-owner@dbai.tuwien.ac.at
    > (4) WWW access and other information on Fuzzy Sets and Logic see
    > http://www.dbai.tuwien.ac.at/ftp/mlowner/fuzzy-mail.info
    > (5) WWW archive:
    http://www.dbai.tuwien.ac.at/marchives/fuzzy-mail/index.html

    ############################################################################
    This message was posted through the fuzzy mailing list.
    (1) To subscribe to this mailing list, send a message body of
    "SUB FUZZY-MAIL myFirstName mySurname" to listproc@dbai.tuwien.ac.at
    (2) To unsubscribe from this mailing list, send a message body of
    "UNSUB FUZZY-MAIL" or "UNSUB FUZZY-MAIL yoursubscription@email.address.com"
    to listproc@dbai.tuwien.ac.at
    (3) To reach the human who maintains the list, send mail to
    fuzzy-owner@dbai.tuwien.ac.at
    (4) WWW access and other information on Fuzzy Sets and Logic see
    http://www.dbai.tuwien.ac.at/ftp/mlowner/fuzzy-mail.info
    (5) WWW archive: http://www.dbai.tuwien.ac.at/marchives/fuzzy-mail/index.html



    This archive was generated by hypermail 2b30 : Wed Aug 22 2001 - 07:23:32 MET DST