Accessibility navigation


A technique for evaluation of interactive evolutionary systems

Shackelford, M. and Corne, D. W. (2004) A technique for evaluation of interactive evolutionary systems. In: Parmee, I. C. (ed.) Adaptive Computing in Design and Manufacture Vi. Springer-Verlag London Ltd, Godalming, pp. 197-208. ISBN 1852338296

Full text not archived in this repository.

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

Abstract/Summary

Very large scale scheduling and planning tasks cannot be effectively addressed by fully automated schedule optimisation systems, since many key factors which govern 'fitness' in such cases are unformalisable. This raises the question of an interactive (or collaborative) approach, where fitness is assigned by the expert user. Though well-researched in the domains of interactively evolved art and music, this method is as yet rarely used in logistics. This paper concerns a difficulty shared by all interactive evolutionary systems (IESs), but especially those used for logistics or design problems. The difficulty is that objective evaluation of IESs is severely hampered by the need for expert humans in the loop. This makes it effectively impossible to, for example, determine with statistical confidence any ranking among a decent number of configurations for the parameters and strategy choices. We make headway into this difficulty with an Automated Tester (AT) for such systems. The AT replaces the human in experiments, and has parameters controlling its decision-making accuracy (modelling human error) and a built-in notion of a target solution which may typically be at odds with the solution which is optimal in terms of formalisable fitness. Using the AT, plausible evaluations of alternative designs for the IES can be done, allowing for (and examining the effects of) different levels of user error. We describe such an AT for evaluating an IES for very large scale planning.

Item Type:Book or Report Section
Divisions:Faculty of Science
ID Code:14449
Publisher:Springer-Verlag London Ltd

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation