Human misidentification in Turing tests
Full text not archived in this repository.
To link to this item DOI: 10.1080/0952813X.2014.921734
This paper presents some important issues on misidentification of human interlocutors in text-based communication during practical Turing tests. The study here presents transcripts in which human judges succumbed to theconfederate effect, misidentifying hidden human foils for machines. An attempt is made to assess the reasons for this. The practical Turing tests in question were held on 23 June 2012 at Bletchley Park, England. A selection of actual full transcripts from the tests is shown and an analysis is given in each case. As a result of these tests, conclusions are drawn with regard to the sort of strategies which can perhaps lead to erroneous conclusions when one is involved as an interrogator. Such results also serve to indicate conversational directions to avoid for those machine designers who wish to create a conversational entity that performs well on the Turing test.
1. Block, N. (1981). Psychologism and behaviorism. In S.Shieber (Ed.), The Turing test: Verbal behavior as the hallmark of intelligence (pp. 229–266). Cambridge, MA: MIT Press. 2. Chomsky, N. (2008). Turing on the imitation game. In R.Epstein, G.Roberts, & G.Beber (Eds.), Parsing the Turing test: Philosophocial and methodological issues in the quest for the thinking computer (pp. 103–106). Dordrecht: Springer Science+Business Media B.V. OpenURL University of Reading 3. Copeland, B. J. (2004). The essential Turing – The ideas that gave birth to the computer age. Oxford: Clarendon Press. 4. Demchenko, E., & Veselov, V. (2008). Who fools whom? The great mystification, or methodological issues on making fools of human beings. In R.Epstein, G.Roberts, & G.Beber (Eds.), Parsing the Turing test: Philosophical and methodological issues in the quest for the thinking computer. Dordrecht: Springer Science+Business Media B.V. 5. Fowler, H., & Fowler, F. (Eds.). (1995). The concise Oxford dictionary of current English (9th ed., p. 486). Oxford: Clarendon Press. 6. French, R. M. (2000). The Turing test: The first 50 years. Trends in Cognitive Sciences, 4, 115–122. [CrossRef], [PubMed], [Web of Science ®] 7. Harnad, S. (1992). The Turing test is not a trick: Turing indistinguishability is a scientific criterion. ACM SIGART Bulletin, 3, 9–10. [CrossRef] 8. Hayes, P., & Ford, K. (1995). Turing test considered harmful. In Proceedings of the international joint conference on artificial intelligence. Montreal (Vol. 1, pp. 972–977). San Francisco, CA: Morgan Kaufmann. 9. Hodges, A. (1992). Alan Turing: The enigma. New York, NY: Vintage Press. 10. Levesque, H. J. (2009, July 11–17). Is it enough to get the behavior right?In Proceedings of the twenty-first international joint conference on artificial intelligence. Pasadena, USA (pp. 1439–1444). San Francisco, CA: Morgan Kaufmann. 11. Moor, J. H. (1976). An analysis of the Turing test. Philosophical Studies, 30, 249–257. [CrossRef], [Web of Science ®], [CSA] 12. Moor, J. H. (2001). The status and future of the Turing test. In J. H.Moor (Ed.), The Turing test – The Elusive standard of artificial intelligence (2003) (pp. 197–214). Dordrecht: Kluwer. 13. Preston, J., & Bishop, J. M. (Eds.). (2002). Views into the Chinese room (428 pp.). Oxford: Clarendon Press. 14. Purtill, R. L. (1971). Beating the imitation game. Mind, 80, 290–294. [CrossRef], [Web of Science ®] 15. Rickman, P. (1999). The philosopher as joker. Philosophy Now, 25, 10–11. 16. Searle, J. (1997). The mystery of consciousness. New York: New York Review of Books. 17. Shah, H. (2011). Turing's misunderstood imitation game and IBM Watson's success. In Proceedings of the 2nd towards a comprehensive intelligence test (TCIT) – Reconsidering the Turing test for the 21st century, symposium in AISB 2011 convention. York University,UK (pp. 1–5). London: AISB. 18. Shah, H., & Henry, O. (2005, May 11–14). The confederate effect in human–machine interaction. In Proceedings of 5th WSEAS international conference on Information Science, Communications and Applications (WSEAS ISCA). Cancun, Mexico (pp. 109–114). Athens, Greece: WSEAS. ISBN: 960-8457-22-X. 19. Shah, H., & Warwick, K. (2010a, March 29–April 1). From the buzzing in Turing's head to machine intelligence contests. In Proceedings of symposium for 1st towards a comprehensive intelligence test. AISB Convention, De Montfort, UK. London: AISB. 20. Shah, H., & Warwick, K. (2010b). Hidden interlocutor misidentification in practical Turing tests. Minds and Machines, 20, 441–454. [CrossRef], [Web of Science ®] 21. Turing, A. M. (1950). Computing, machinery and intelligence. Mind, LIX, 433–460. [CrossRef] 22. Warwick, K. (2011). Artificial intelligence: The basics. Abingdon/New York: Routledge. 23. Warwick, K. (2012). Not another look at the Turing test? In M.Bielikova, G.Friedrich, G.Gottlob, S.Katzenbeisser, & G.Turan (Eds.), In Proceedings of the SOFSEM 2012. Vol. 7147: Lecture notes in computer science (pp. 130–140). Berlin, Heidelberg: Springer. [CrossRef] 24. Warwick, K., & Shah, H. (2013). Good machine performance in Turing's imitation game. IEEE Transactions on Computational Intelligence and AI in Games, 10.1109/TCIAIG.2283538. [CrossRef] 25. Warwick, K., Shah, H., & Moor, J. H. (2013). Some implications of a sample of practical Turing tests. Minds and Machines, 23, 163–177. [CrossRef], [Web of Science ®] 26. Wheeler, M. (2010). Plastic machines: Behavioural diversity and the Turing test. Kybernetes, 39, 466–480. [CrossRef], [Web of Science ®] 27. Whitby, B. (1996). The Turing test: AI's biggest blind Alley? In P. J. R.Millican & A.Clark (Eds.), Machine and thought: The legacy of Alan Turing (Vol. 1, pp. 53–62). Oxford: Clarendon Press.