Histopathology image classification: Highlighting the gap between manual analysis and AI automation

Charles Edward Spearman, FRS^[1][3] (10 September 1863 – 17 September 1945) was an English psychologist known for work in statistics, as a pioneer of factor analysis, and for Spearman's rank correlation coefficient. He also did seminal work on models for human intelligence, including his theory that disparate cognitive test scores reflect a single general intelligence factor^[4] and coining the term g factor.^[5]

Spearman had an unusual background for a psychologist. In his childhood he was ambitious to follow an academic career. But first he joined the army as a regular officer of engineers in August 1883,^[1] and was promoted to captain on 8 July 1893, serving in the Munster Fusiliers.^[6] After 15 years he resigned in 1897 to study for a PhD in experimental psychology. In Britain, psychology was generally seen as a branch of philosophy and Spearman chose to study in Leipzig under Wilhelm Wundt, because it was a centre of the "new psychology"—one that used the scientific method instead of metaphysical speculation. As Wundt was often absent due to his multiple duties and popularity, Spearman largely worked with Felix Krueger and Wilhelm Wirth, both of whom he admired.^[1] He started in 1897, and after some interruption (he was recalled to the army during the Second Boer War, and served as a Deputy Assistant Adjutant General from February 1900^[7]) he obtained his degree in 1906. He had already published his seminal paper on the factor analysis of intelligence (1904). Spearman met and impressed the psychologist William McDougall who arranged for Spearman to replace him when he left his position at University College London. Spearman stayed at University College until he retired in 1931. Initially he was Reader and head of the small psychological laboratory. In 1911 he was promoted to the Grote professorship of the Philosophy of Mind and Logic. His title changed to Professor of Psychology in 1928 when a separate Department of Psychology was created.

When Spearman was elected to the Royal Society in 1924^[1] the citation read:

Dr. Spearman has made many researches in experimental psychology. His many published papers cover a wide field, but he is especially distinguished by his pioneer work in the application of mathematical methods to the analysis of the human mind, and his original studies of correlation in this sphere. He has inspired and directed research work by many pupils.

Chief amongst these achievements was the discovery of the general factor in human intelligence,^[4] and his subsequent development of a theory of "g"^[8] and synthesis of empirical work on ability.^[9]

Spearman was strongly influenced by the work of Francis Galton. Galton did pioneering work in psychology and developed correlation, the main statistical tool used by Spearman.

In statistics, Spearman developed rank correlation (1904),^[10] a non-parametric version of the conventional Pearson correlation, as well as both the widely used correction for attenuation (1907), and the earliest version of a 'factor analysis' (Lovie & Lovie, 1996, p. 81).^[3] His statistical work was not appreciated by his University College colleague Karl Pearson and there was a long feud between them.

Although Spearman achieved most recognition in his day for his statistical work, he regarded this work as subordinate to his quest for the fundamental laws of psychology, and he is now similarly renowned for both.

Charles Spearman always insisted that his work be applied in psychiatry and urged so in his Maudsley lecture to the Royal Medico-Psychological Association. While some work has been made on these lines by pupils and associates of his, the development of factor analysis as a tool of psychiatry followed a different path than he had intended. Regardless, his indirect contributions towards psychiatry were considerable.^[11]

Spearman's life both began and ended in the city of London. He had three daughters, and a son who died in 1941 in Crete.^[1]

A record of Spearman's views on g (and also those of Godfrey Thomson and Edward Thorndike) was made in the course of the Carnegie-sponsored International Examinations Inquiry Meetings.^[12]

Here, Spearman gives a compact summary of his findings and theory of g:

When asked what g is, one has to distinguish between the meanings of terms and the facts about things. g means a particular quantity derived from statistical operations. Under certain conditions the score of a person at a mental test can be divided into two factors, one of which is always the same in all tests, whereas the other varies from one test to another; the former is called the general factor or g, while the other is called the specific factor. This then is what the g term means, a score-factor and nothing more. But this meaning is sufficient to render the term well defined so that the underlying thing is susceptible to scientific investigation; we can proceed to find out facts about this score-factor, or g factor. We can ascertain the kind of mental operations in which it plays a dominant part as compared with the other or specific factor. And so the discovery has been made that g is dominant in such operations as reasoning, or learning Latin; whereas it plays a very small part indeed in such operation [sic] as distinguishing one tone from another. . . g tends to dominate according as the performance involves the perceiving of relations, or as it requires that relations seen in one situation should be transferred to another. . . . On weighing the evidence, many of us used to say that this g appears to measure some form of mental energy. But in the first place, such a suggestion is apt to invite needless controversy. This can be avoided by saying more cautiously that g behaves as if it measured an energy. In the second place, however, there seems to be good reason for changing the concept of energy to that of "power" (which, of course, is energy or work divided by time). In this way, one can talk about mind power in much the same manner as about horse power. . . . . . .g is in the normal course of events determined innately; a person can no more be trained to have it in higher degree than he can be trained to be taller. (pp. 156 –157).

There was also another co-factor as proposed by Spearman that was special intelligence. The special intelligence was for individuals who accomplished high success results in the same tests. However, later Spearman introduced group factor that was particular to those correlations that were not a result of factor g or s. His ideas were in 1938 criticized on paper by psychologist Louis L. Thurstone who argued his own experiments showed that intelligence formed seven primary categories: numerical, reasoning, spatial, perceptual, memory, verbal fluency and verbal comprehension. Thurstone ultimately agreed with Spearman that there was a general factor among ability measures. Subsequently, Raymond Cattell supported a version of the general ability concept theorized by Spearman but highlighted two forms of ability, distinguished by their noegenetic properties: fluid and crystallized intelligence.^[13]

As time progressed, Spearman increasingly argued that g was not, from a psychological point of view, a single ability but composed of two very different abilities which normally worked closely together. These he called "eductive" ability and "reproductive" ability. The former term comes from the Latin root "educere" – which means to "draw out" and thus refers to the ability to make meaning out of confusion. He claimed that to understand these different abilities "in their trenchant contrast, their ubiquitous cooperation, and their genetic interlinkage" would, for the study of "individual differences – and even cognition itself" – be "the very beginning of wisdom."^[9]

Despite Spearman arguing that g was what emerged from a large battery of tests, i.e., that it was not measured perfectly by any single test, the fact that g-theory suggested that much of ability could be captured in a single factor, and his suggestion that "the eduction of relations and correlates" underlay this general factor led to the quest for tests of this general ability. Raven's Progressive Matrices might be regarded as one of these although Raven himself clearly stated that his tests should not be regarded as "intelligence" tests.

While arguing consistently that g accounted for much of individual differences in "ability" (as measured by tests which had "no place in schools"), Spearman also acknowledged that "Every normal man, woman, and child is … a genius at something … It remains to discover at what …" He thought that detecting these areas of genius required procedures very different from "any of the testing procedures at present in current usage", though he felt these to be capable of "vast improvement".^[14]

Spearman felt that though g could be detected in any sufficiently-broad set of cognitive measures, he felt that the tests from which his g had emerged "had no place in schools" because they "deflected" teachers', pupils', parents' and politicians' attention from the business of education which, as the Latin root of the word implies, should be concerned with "drawing out" whatever talents a student may have.^{[citation needed]}

He presented a digest of his views in the entry "Abilities, general and special" in the 14th edition of the Encyclopaedia Britannica.^[15]

Spearman's model was influential, but was also critiqued by others, such as Godfrey Thomson.^[16] In particular the move from a psychological g to a biological g – that is a unitary biological mechanism or mechanisms has remained a matter of active research. Nonetheless, Thomson's disagreements with Spearman had more to do with methodology and epistemology than data or interpretations thereof. It was Thomson who authored Spearman's biographical memoir^[1] for the Royal Society.

Factor analysis is a statistical test that is used to find relationships between multiple correlated measures and Spearman played a clear part in its development. Spearman coined the term factor analysis and used it extensively in analyzing multiple measures of cognitive performance. It was factor analytic data which led Spearman to postulate his original general and specific factor models of ability.^[17] Spearman applied mathematical procedures to psychological phenomena and moulded the outcome of his analysis into a theory – which has greatly influenced modern psychology.^[11] Factor analysis and its modern relations confirmatory factor analysis and structural equation modelling underlie much of modern behaviour research.

• Spearman, C. (1987). "The proof and measurement of association between two things. By C. Spearman, 1904". The American Journal of Psychology. 100 (3–4): 441–471. doi:10.2307/1422689. JSTOR 1422689. PMID 3322052.
• Spearman, C. (2010). "The proof and measurement of association between two things". International Journal of Epidemiology. 39 (5): 1137–1150. doi:10.1093/ije/dyq191. PMID 21051364.
• Spearman, C. (1907). "Demonstration of Formulæ for True Measurement of Correlation". The American Journal of Psychology. 18 (2): 161–169. doi:10.2307/1412408. JSTOR 1412408.
• Gregory, Robert J. (2011). Psychological Testing: History, Principles, and Applications (Sixth ed.). Boston: Allyn & Bacon. ISBN 978-0-205-78214-7.
• Mackintosh, N. J. (1998). IQ and Human Intelligence. Oxford: Oxford University Press. ISBN 978-0-19-852367-3.
• Charles Spearman at Human Intelligence

• Charles Spearman on the Portraits of Statisticians page.
• John Raven, Spearman on Intelligence, 14 May 2011
• Charles Spearman, 1927 The abilities of man : their nature and measurement