Computational Measurement of Intelligence
This corresponds to a project spanning the period 1996-2000. The last update of this page was made in 2000!
This page contains much secondary material and digressions, so if you want to start reading something more concise and round,
please try first the most representative publication of this period: "Beyond the Turing Test", J. of Logic, Language and Information, 2000.
For a more recent project and publications, go to the
anYnt project
.
Encara
us dic que port una Art general
que novament és
dada per do espiritual,
per qui hom pot saber
tota re natural,
segons que enteniment
ateny lo sensual.
A Dret e a Medicina e
a tot saber val
e a Teologia, la qual
m'és mais coral,
e a soure qüestions
nul·la art tant no val,
e a destruir errors per
raó natural;
tenc-la per perduda car
quaix a hom no cal.
Ramon Llull,
Lo desconhort, VIII, XIIIth century.
Presentation:
This page describes the theory and results of the measurement of cognitive
abilities by the use of tests generated from computational constructs derived
from descriptional complexity (more commonly known as Kolmogorov complexity).
Although there are still many problems to solve, its current state
of the art (see e.g. Hernández-Orallo 2000d)
allows the generation of grounded tests of an important factor of intelligence,
the ability to comprehend, which correlates with the g-factor.
The implications of a successful computational measurement of intelligence
are many-fold:
-
its non-anthropomorofic character would allow to measure in a more equal
way human and non-human machines,
-
AI could evaluate its progress objectively in many different areas (according
to different factorial tests) and, finally,
-
many fascinating questions about the nature and limits of intelligence
are posed ahead to be solved.
Current and Past Tests:
The following tests were generated according to the theory developed from
some publications (see below). For its complete understanding, please read
the corresponding paper first.
Some of them are compared with the European Intelligence Test (EIQtest,
see e.g. European IQtest)
This test measures the comprehension ability, and
it is measured in two slightly different forms: predictive and abductive
(forward and backwards respectively). These closely related factors are
measured by two separate tests: the IT-test and the AIT-test.
-
Methodology. How the Test was Generated.
-
Subjects and Administration
-
Summarised Results
We evaluated the test without penalising the
errors, i.e., the function hit evaluated the same for blanks than
for mistakes. We gave all the questions the same value, independently of
its difficulty (e=0). IQ-correlations are illustrated in Table 1.
| |
Prediction
|
Abduction
|
Both (Induction)
|
| High-School |
0.31
|
0.38
|
0.42
|
| University |
0.51
|
0.42
|
0.56
|
| Both Groups |
0.73
|
0.68
|
0.77
|
Table 1. Correlations with EIQ test
The correlation for induction (prediction + abduction)
is of the same order as the usual correlation for induction tests made
by psychologists. The correlation between the abduction and prediction
tests was 0.61, less than expected, which suggests that even problems constructed
by the same generator can be more or less difficult depending on its presentation
(abductive or predictive). The correlation between induction and similarity
was 0.51, which supports the thesis that "the ability of compression" is
different from "the ability of comprehension". Finally, we think that an
analogy test based on our theory would surely round off the study.
With these data and our amateur methods we are
not in conditions to assert more things about the relation between C-tests
and IQ-tests. There is only a thing that has no discussion in the light
of the results, the k-hardness matches fairly well with the difficulty
people found on them, as it is seen in Figure 1:
Figure 1. Hit Rate per Difficulty
-
Extended Results
Would you like to pass the IT-test? Click
Here for an English version of the IT-test.
Quieres pasar el IT-test? Pulsa aquí
para una versión en castellano del IT-test.
Click Here for the solutions of the IT-test.
Would you like to pass the AIT-test? Click Here
for an English version of the AIT-test.
Quieres pasar el AIT-test? Pulsa aquí
para una versión en castellano del AIT-test.
Click Here for the solutions of the AIT-test.
Would you like to collaborate with your or more results? Please send
me an e-mail.
This test attempts to measure the analogical ability.
It is called AT-test. However, the problems are quite simple, and the problems
are contaminated by both inductive and deductive factors, thus it is not
much meaningful.
Would you like to pass the AT-test? Click Here for
an English version of the AT-test.
Quieres pasar el AT-test? Pulsa aquí
para una versión en castellano del AT-test.
Click Here for the solutions of the AT-test.
Would you like to collaborate with your or more results? Please send
me an e-mail.
S-test: Similarity / Compression Test:
This test measures the recognition ability.
Methodology. How the Test was Generated.
Subjects and Administration
Summarised Results
Preliminary results
| |
Simil.
|
| High-School |
0.39
|
| University |
0.35
|
| Both Groups |
0.50
|
Table 2. Correlations of S-test with EIQ test
Would you like to pass the S-test? Click Here for
an English version of the S-test.
Quieres pasar el S-test? Pulsa aquí para
una versión en castellano del S-test.
Click Here for the solutions of the S-test.
Would you like to collaborate with your or more results? Please send
me an e-mail.
On-going Tests:
Methodology. How the Test was Generated.
Subjects and Administration
Would you like to pass the D-test? Click Here for an
English version of the D-test.
Quieres pasar el D-test? Pulsa aquí para
una versión en castellano del D-test.
Click Here for the solutions of the D-test.
Would you like to collaborate with your or more results? Please send
me an e-mail.
Future Work:
Related Publications:
(Hernández-Orallo2000a)
-
Hernández-Orallo, José Constructive Reinforcement
Learning (English, 24 pp.), International
Journal of Intelligent Systems, Volume 15, Issue 3, 2000. Pages: 241-264,
Wiley,
ISSN: 0884-8173
(Hernández-Orallo2000b)
-
Hernández-Orallo, José Computational Information
Gain and Inference, Collegium
Logicum - Annals of the Kurt-Gödel
Society, V.4, Springer-Verlag Wien
New York, to appear 2000, ISSN: 0947-577X.
(Hernández-Orallo and Minaya-Collado 1998)
-
Hernández-Orallo, José; Minaya-Collado, N.: A Formal
Definition of Intelligence Based on an Intensional Variant of Kolmogorov
Complexity (English, 18 pp.), Proceedings of the International Symposium
of Engineering of Intelligent Systems (EIS'98)
ICSC Press 1998, pp. 146-163.
See the electronic versions (ps) of these and other papers here.
On-going Theoretical Work:
Hernández-Orallo, José On the Hardness of an ILP
problem instance.
Hernández-Orallo, José Constructive Reinforcement
Induction vs. Complexity-Based Induction.
Links:
© 1999-2000 José
Hernández Orallo.
A follower of this project can be found here..