Mental Images

You may also download a MS-Word 97 version of this file.

Introduction 
Consider this, if you were asked how many ten öre coins are there in a one-krona coin? You would probably answer, without too much conscious thought or visual imagery, that there are ten ten öre coins in one krona. But what if you are then asked to descri be exactly what the two sides of the one krona look like, without looking at a physical coin. You would probably find yourself visualising the coin in your mind. This is what we would call seeing things in the "minds eye". In other words you would probabl y use your conscious thought to visual a one-krona coin. You might even rotate the coin to visualise both sides of the coin, even thought the coin is not present in a physical form.

The purpose of this paper is to give the reader an account of some current theories of Mental Images. I shall account for some of the theoretical and experimental references to mental imagery. In my account of mental images I have started with an account of the theory of mental images followed by an account of some of the experiments into mental imagery. I have rounded off my paper with a discussion and an outline of an illustration of mental imagery. Throughout out the paper I have referred to the name v isual images instead of mental images because they are both related. The word visual image is related to the physical world rather than the more abstract word of mental images, which are images that are perceived in an individual person's mind.

Theory of Visual Images 

Cognitive penetrability criterion. 
The claim that visual imagery is a distinct c omponent of the cognitive architecture, requires that it must be shown that the visual representations a person experience as images have to bind characteristics that are not easily alterable as a function of b eliefs. In his proposed test, Pylyshyn's ² cognitive penetrability criterion, which is a test for when a representation or process is biologically built into the cognitive architecture. What this criterion means is that if imagery is part of the architect ure then it should always operate in the same way and therefor not be changed by one's beliefs or knowledge of the real world.

For example:
There has been experiments carried out that have produced evidence that mental images can be scanned in away similar to the way an individual actually scans a visual image. But the scanning operation might be cognitively penetrable. It may be that in expe riments, subjects scan their mental image just because they know that they would scan a real visual image. But under different circumstances it might be that they can shift their attention between or answer questions about two locations within a image wit hout scanning. In that case it can be argued that image scanning is neither built into the architecture or to bind component process in performing certain tasks. Instead, it might be that subjects can choose whether or not to employ cognitively penetratio n in order to mimic their perceptual experiences, but discard it if they think that perceptual mimicry is not called for.

Intuition 
Intuition is not a strong source of hypotheses for constructing a theory of imagery. If someone gave you an abstract verbal description of an object that you can easily encode it propositionaly and remember it. However, when that object is imagined, it se ems that a number of properties must be added to the description of the object. Yet these added properties fall short of all those properties that would be present in an actual visual scene.

For example ³, if someone tells you to search in a box of toy blocks for a three-inch cube that has two adjacent blue sides, you would have no problem remembering the description. Now if you were to imagine a cube in your minds eye, there would be certain ly some additional information added to the description of the cube in your image. In other words the cube will have a particular position relative to your point of view. The top, front and right side of the cube maybe visible. Two particular sides will be blue: perhaps the top and right sides. None of this information was included in the original description and none of it was required to form and remember a representation of the description. But the information is required to form a mental image of the cube. But the actual scene of the cube would contain much more information, i.e. ⁴ the surface of the cube would have texture, the cube would have some form of support and not forgetting how the cube is illuminated, resulting in casting a shadow of the cube, and so on.

So the above suggests (but does not determine) that imagery makes use of representations and processes that are specific to the visual system. The assumption that sensory systems feed into central systems but that the central systems do not feed back must be altered (See the diagram 2.1).

fig 2.1

If the hypotheses are to be correct then we should have the following figure.
fig 2.1

Stillings et al report explained then by Kosslyn ⁵ developed a theory that visual imagery involves the visual system. Their view is that visual imagery involves unique representational formats and dedicated process, which operate during visual perception. They also proposed that visual imagery is the result of an active representation in a short-term visual buffer, which is generated from the long-term memory and that this short-term visual buffer fades rapidly with time unless constantly refreshed. So what is the visual buffer? Stillings ^6a describes the visual buffer by comparing a bit-map memory of a computer to the visual buffer. He describes it in the following way.

Think of the visual buffer as something similar to a block of memory in a computer that is organised to be read out directly onto the computer screens (a bit-mapped memory). Each cell in the memory represents a point (or pixel for picture element) in the two dimensional picture display on the monitor. Therefore the cell has two-dimensional geometric properties: a position (x, y co-ordinates), adjacent neighbour cells, and so on. Geometric, or graphics, procedures can be programmed to operate on the cells, i.e. to draw a line between two points, etc. A spatially organised area of memory plus a set of graphic routines constitute a specialised subsystem in the computers with graphic facilities.

The human visual buffer is viewed to be a similar specialised subsystem. The buffer is a short-term memory structure with the two dimensional spatial properties. But the human visual buffer has a number of characteristics that the computer version does not possess. For example, in vision and imagery, the centre of visual buffer has the highest resolution, and there is a special focus of attention that can be moved around within the buffer. This results in that we can direct our processing on a particular location.

The Function of Visual Imagery 
Stillings ^6b states that the clear reason for the existence of visual imagery is to make the computational resources of the visual system available for reasoning about the shapes and the spatial arrangement of the objects in the absence of visual input (page 46).

What this means is that while we are problem solving with goal orientated thought we draw on our long-term memory. Our long-term memory contains information about objects, but this information does not include explicit spatial detail that is contained in an actual image. The built in central processes (fig 2.1) are not designed to processes this detailed spatial information. The result is that when spatial reasoning is required, we call on our long-term memory to construct an image. Our long-term memory o f the image contains the spatial information needed by the specialised processes available in the visual system.

Stillings then continuous by saying, that the theory is that process of visual perception and attention are used on the image of an object just as they would be in looking at an actual object. With out external input to the visual buffer image perception is made difficult. The image of an object must be continually refreshed, and details of a particular object must be generated as we shift our attention on the object. This is quite taxing and can lead to subtle interplay between imagery and reasoning.

For example: One might solve a problem by scanning a image from our long term memory and then use our built in central processes to facilitate reasoning about the image. The limited capacity of visual imagination seems to be a joint product of the limited ca pacities of the central processing and of visual attention.

Experiments of Visual Imagery 

This means that when our attention is focused on one region of an image and then is shifted to focus on another region, the shift involves a scan across the space represented in the image. Now if scanning takes place at a constant rate, the shifts between regions that are further apart should take longer.

Kosslyn, Ball, and Reiser ⁹ tested the scanning assumption experimentally. But Pylyshyn (1984) and Finke (1985) criticise the results of the experiment carried out by Kosslyn. The results that Kosslyn found could reflect the subjects tactical knowledge of visual scanning rather than the operation of a scanning function that is a part of the cognitive architecture.

One approach to strengthen the evidence of built-in scanning operation is to try to set up a situation in which subjects have to answer some questions about two spatial locations but are not told to or encouraged to mimic visual scanning. If scanning is a primitive operation then response time should still be affected by distance. Finke and Pinker (1992) conducted such an experiment.

Here is what they had to do. On each trial a pattern of four dots appeared on a screen, and the test subject formed an image of the dots. The pattern was turned off and after a two-second delay an arrow appeared on the screen in an unexpected position and orientation. The test subject's task was to decide as quickly as possible whether the arrow was pointing at one of the previously shown dots.

The result of the experiment should that the subjects used a scanning operation, even though they were instructed not to scan and the demand of the task was to answer a question as fast as possible.

Mental rotation 
The term refers to the ability to imagine objects rotating in space. This provides a second example of the use of response times to study the properties of mental imagery. Take a look at the fig 2.4 below.

The question is which of these fives are backward? In order to answer this question you would probably mentally rotate, in turn, each of the fives to there correct position. Thus, the fives that involve turning them backwards to be able to read them corre ctly, are them-selves backwards, and as a result the answer to the question. It would be quite difficult to answer this question with out doing mental rotation. Thus mental rotation is hypothesised to one of the built-in operations that can be applied to visual images.

Stillings ¹⁰ says, the experience of mental rotation suggests the hypotheses that mental rotation is similar to the physical rotation in the real world (page 49). This means that during mental rotation the object represented must pass through different st ates that correspond to the actual path of the physical rotation of the subject. Stillings continuous by writing, that an another hypothesis, that is similar to the scanning case, is that mental rotation occurs at a constant rate.

Example:
Cooper ¹¹ studied the mental rotation with the illustrated figure below.
fig 2.5

Try to decide yourself whether each of the test stimulus is standard of reflected? As you make your decision you will probably experience mental rotation. "Coopers´s resulting data from the experiment confirmed the hypotheses that subjects make their judg ements based on mental rotation. Mental rotation does occur at a constant rate". See fig 2.6 to view the results.
fig 2.6

"Cooper´s experiments provided evidence that mental rotation is a precise analogue of physical rotation". The data from Cooper´s experiments agrees "with the hypotheses that the rotation operation is part of the cognitive architecture and that it operates on a visual buffer". But we cannot ignore the fact that tactical knowledge may influence the mental rotation experiments.

Conclusion with experiment 
All the literature that I have read so far has reflected the difficulty of determining if a particular process is biologically built into the cognitive architecture. This is made clearer by thinking about the following. Imagine if you met an old friend that has changed their appearance? Your long-term memory of them will be out of date so you might not recognise them straight away. You might find yourself scanning the old image of them focusing on some detail that is common to the actual visual image and the long term memory image.

Experiment
Test:
Draw your self-portrait (or any motive that suites your choice) blindfolded.

Goal:
The goal of this test is to illustrate how one would use and experience mental images. I am not trying to prove any theories with this experiment, I am just giving an illustration of how a subject might experience mental / visual images. This would be an ideal introduction to "mental/Visual images" before any theory is presented to the test subjects.

Equipment:
All test subject(s) should posses the following pieces of equipment. A pen or pencil to write with, a piece of paper in which to write on and a blindfold.

Instructions:
The instructor is to instruct the test subject(s) to draw their self-portrait. But the test subject(s) must be blindfolded while they are drawing their portrait. The only time the test subject(s) is allowed to remove their blindfold is when the test subject(s) deems that he/she has completed their task. Upon completion of the task the test subject(s) may remove their blindfold and view their results. The instructor may then ask the following questions or other questions that may be reverent to the subject of mental images.

Question:
How did you (i.e. the test subject(s)) solve the problem?
Perhaps the instructor could stir the test subjects response by using their prior knowledge of mental and visual imagery. But the basic idea is to illustrate that perhaps the test subject(s) may have used their cognitive architecture and intuition while drawing their self-portrait. The test subjects may also have experienced what is called the "minds eye". In other words the test subject(s) may have visualised their physical actions of drawing their self-portrait in their minds eye. The results of the subjects drawings should be of poor quality. Especially if we keep in mind just how difficult it is to recall all the details that are included in a image and just how difficult the task is for the cognitive architecture. The difficulty factor for the cognitive architecture is also increased if the test subject(s) lift their pen from the paper. This would result with that the test subject imagines that they are replacing their pen at the correct position on the paper for to continue they're drawing. The underlining intention of this test is to act as an introduction into mental imagery and the theory that surrounds this subject.

Literature 
Cooper, L. A. (1975). Mental rotation of random two-dimensional shapes. Cognitive Pyschology 7, 20-43.
Finke, R. A. (1985) Theories relating mental imagery to perception. Psychological Bulletin 98, 236-259.
Finke, R. A, and Pinker, S. (1992). Spontaneous imagery scanning in mental extrapolation. Journal of Experimental Psychology: Learning Memory and Cognition 8, 14$
Kosslyn, S. (1980) Imagery and mind. Cambridge, Mass.: Harvard University Press.
Kosslyn, S & Koenig, O, 1992 Wet mind: The new cognitive neuroscience.
Kosslyn, S, Ball, T. M, and Reiser, B. J. (1978) Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance 4, 47-60.
Pylyshyn, Z. (1984) Computation and cognition: Toward a foundation for cognitive science. Cambridge, Mass. : MIT Press.
Simon, H. (1972) What is visual imagery. An information processing interperation. In L. W. Gregg, ed., Cognitive learning and memory. New York: Wiley.
Stillings, Neil A. (editor) (1995) Cognitive Science an introduction (Second edition) Massachusetts Institute of Technology.

1 See 2.4 Propositional Representation, Cognitive Science, Neil A. Stillings (editor) (1995)
2 See Pylyshyn, Z. W. Computational and cognition: Toward a foundation for cognitive science (1994)
3 Consider the example adopted from Simon, H. What is visual imagery? (1972)
4 I.e. = id est.; in other words.
5 See Kosslyn, S. Image and mind (1980); Kosslyn, S & Koenig, O. Wet mind (1992)
6a / 6b see 2.7 Theory of Visual Images, Cognitive Science, Neil A. Stillings (editor) 1995
7 see 2.7 Theory of Visual Images, Cognitive Science, Neil A. Stillings (editor) (1995)
8 outlined in 2.7 Theory of Visual Images, Cognitive Science, Neil A. Stillings (editor) (1995)
9(Kosslyn, S. M., T. M. Ball, and B. J., Reiser 1978). Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance 4, 47-60
10 (see 2.7 Experiments of Visual Imagery, Cognitive Science, Neil A. Stillings (editor) 1995)
11 In Cooper's (1975) experiment subjects first learne the standard and reflected version of eight forms at fixed orientation, as in the examples above. Test stimuli appeared either at the trained orientation of at one of five angular departures from the training orientation. Examples of rotated test stimuli are given in the third row of the figure. In each case the test stimuli is a rotation of either standard or the reflected form that appears above it.

Umeå Universitet	Kognitionsvetenskap
Datavetenskap	97-12-10
Clive Leddy (dvacly@cs.umu.se)	Contents.