The Questioning Turing Test
- PDF / 1,744,767 Bytes
- 25 Pages / 439.37 x 666.142 pts Page_size
- 1 Downloads / 224 Views
The Questioning Turing Test Nicola Damassino1 Received: 24 April 2020 / Accepted: 3 November 2020 © The Author(s) 2020
Abstract The Turing Test (TT) is best regarded as a model to test for intelligence, where an entity’s intelligence is inferred from its ability to be attributed with ‘human-likeness’ during a text-based conversation. The problem with this model, however, is that it does not care if or how well an entity produces a meaningful conversation, as long as its interactions are humanlike enough. As a consequence, the TT attracts projects that concentrate on how best to fool the judges. In light of this, I propose a new version of the TT: the Questioning Turing Test (QTT). Here, the entity has to produce an enquiry rather than a conversation; and it is parametrised along two further dimensions in addition to ‘human-likeness’: ‘correctness’, evaluating if the entity accomplishes the enquiry; and ‘strategicness’, evaluating how well the entity accomplishes the enquiry, in terms of the number of questions asked (the fewer, the better). Keywords Turing · Turing test · Blockhead · Human-likeness
1 Introduction The soundness of the TT as a test for intelligence has been constantly debated. Hernández-Orallo (2017), among others, argues that: The standard Turing test is not a valid and reliable test for HLMI [Human Level Machine Intelligence].[…] the Turing test aims at a quality and not a quantity. Even if judges can give scores, in the end any score of humanness is meaningless. (129)
* Nicola Damassino [email protected] 1
University of Edinburgh, Edinburgh, UK
13
Vol.:(0123456789)
N. Damassino
My view is that the fault of the TT is one of interpretation and experimental design rather than experimental concept. To show this, I propose a new version of the TT, called QTT. In the QTT, the entity1 must accomplish a yes/no enquiry in a humanlike and strategic way, where ‘strategic’ means with as few questions as possible.2 My claim is that the QTT (i) improves the experimental design of the TT, by minimising both the Eliza Effect3 and the Confederate Effect4; and (ii) prevents both Artificial Stupidity5 and Blockhead6 from passing. The rest of the paper is structured as follows. In the next section, I review two interpretations of the TT: the Original Imitation Game (OIG), advocated by Sterrett (2000); and the Standard Turing Test (STT), advocated by Moor (2001). In Sect. 3, I discuss two problems with the TT: (i) Artificial Stupidity and (ii) Blockhead. In Sect. 4, I introduce the QTT, describe my study, and show the results gained. Finally, in Sect. 5, I consider four possible objections to the QTT.
2 Interpretations of the Turing Test In this section, I review two different interpretations of the TT: (i) the Literal Interpretation, endorsed by the Original Imitation Game (Sterrett 2000); and (ii) the Standard Interpretation, endorsed by the Standard Turing Test (Moor 2001). The former holds that the results of the TT are given by the comparison between the human’s performance and the machine’s p
Data Loading...
