Towards an Assessment of an AI System's Validity by a Turing Test

Authors:

• Rainer Knauf, Faculty of Computer Science and Automation, Dept. of Artificial Intelligence, Technical University of Ilmenau, Germany, Rainer.Knauf@TheoInf.TU-Ilmenau.de
• Avelino J. Gonzalez, College of Engineering, Dept. of Electrical and Computer Engineering, University of Central Florida, Orlando, FL, ajg@ece.engr.ucf.edu
• Ilka Philippow, Faculty of Computer Science and Automation, Dept. of Process Data Processing, Technical University of Ilmenau, Germany, Ilka.Philippow@TheoInf.TU-Ilmenau.de

Abstract:

The authors present some ideas on developing a validity statement based on a Turing-test methodology with a set of "good" test cases. The objective of this is, of course, to make the result of the validation process (the validity statement) more objective.

Furthermore, in an effort to maximize objectivity, the approach described here includes a competence scale for each validator. This is done for each test case separately due to the fact that competence is a property of an expert, which normally isn't distributed homogeneously within the "input space" of the AI system.

Key Words:

Expert System Validation, Competence Assessment of AI Systems, Turing Test