THE MEASURE OF ALL MINDS
Evaluating Natural and Artificial Intelligence

José Hernández-Orallo
Cambridge University Press
2017

Beyond human intelligence there is a growing universe of minds, including a variety of machines, enhanced humans, hybrids and collectives. Can we measure all of them? Human psychometrics, comparative psychology, animal cognition and the evaluation tools in artificial intelligence are insufficient to cope with this challenge —if not simply inconsistent when put together. This is calling for a new universal psychometrics.

This monograph focuses on primary research, by formulating new scientific questions and hypotheses, and introducing a whole range of definitions and results. It integrates the fundamentals and most significant research developments in the past two decades, with the aim of providing a unified view of the universal evaluation of cognitive abilities and personality traits. The intended audience includes researchers and academics in cognitive science, artificial intelligence, machine learning, psychometrics, comparative cognition, intelligence research, information theory, psychology and animal behaviour. Given this diversity of fields, a great effort has been made to render the terminology and mathematical content accessible to a wider readership.

Major questions addressed in the book:

1. How can behavioural features be measured in the machine kingdom?

2. How universal and adaptive can a behavioural test be?

3. Are IQ tests valid for any machine and what do they measure?

4. How can abilities be identified, through task breadth or similarity?

5. Is intelligence one or many, objective or subjective?

6. Can we formalise task difficulty without particular populations?

7. Is there a common component for all abilities, a universal g factor?

8. Can general intelligence be independent of the task distribution?

9. Will scales fully range from minimal cognition to supercapacities?

10. Can intelligence be defined solely with computational principles?

11. Should collectives and hybrids be evaluated unlike their individuals?

12. Is intelligence useful for adaptive testing and self-assessment?

13. Can social abilities be measured through situated tests?

14. What kind of ability can capture communication and language skills?

15. How can potential abilities be measured and how do they develop?

16. Can any universal machine and human become arbitrarily intelligent?

17. How can cognitive abilities and personality traits develop in general?

18. Are behavioural features sufficient to characterise personhood?

19. What are the limits and consequences of supercapacities?

20. Can human, animal and AI evaluation benefit from an integration?

.

TABLE OF CONTENTS:

PART I A long-pondered outfit

1 Extended nature
2 Mind the step: scala universalis


PART II The evaluation discordance

3 The evaluation of human behaviour
4 The evaluation of non-human natural behaviour
5 The evaluation of artificial intelligence
6 The boundaries against a unified evaluation


PART III The algorithmic confluence

7 Intelligence and algorithmic information theory
8 Cognitive tasks and difficulty
9 From tasks to tests
10 The arrangement of abilities
11 General intelligence


PART IV The society of minds

12 Cognitive development and potential
13 Identifying social skills
14 Communication abilities
15 Evaluating collective and hybrid systems


PART V The kingdom of ends

16 Universal tests
17 Rooting for ratiocentrism
18 Exploitation and exploration

EXCERPT AND AVAILABILITY

Click here for the Frontmatter, including a detailed Table of Contents, the Preface, parts of chapter 1 and the index.
Go here (and click on Look Inside) for other parts, including the references and the colour figures. The reference list can be found here.

You can order the book on Cambridge University Press or Amazon.

REVIEWS, INTERVIEWS AND BLOG ENTRIES

• PsycCRITIQUES: A Comparative Psychology for the Machine Kingdom. "This is a highly original and important contribution, one that offers a new perspective on the comparison process in psychology and suggests that ambition and audacity can still be found in the study of the mind."
• INVERSE: Give a Robot Mouse an IQ test.
• Financial Times: What the octopus tells us about human intelligence (jointly with Godfrey-Smith's "Other Minds" and Charles Dennett's "From Bacteria to Bach and Back"). "an impressive and important step towards a less human-centred understanding of intelligence"
• Cambridge Blog: Warning! Intelligence Not Included.

A list of errata and other comments will be kept here: here.

(c) José Hernández Orallo and Cambridge University Press, 2017.