Hierarchical Complexity Scoring System (HCSS)
How to Score Anything1

Michael Lamport Commons, Harvard Medical School; Patrice Marie Miller, Harvard Medical School; Eric Andrew Goodheart, Harvard University; Dorothy Danaher-Gilpin

Commons, M. L., Miller, P. M., Goodheart, E. A., & Danaher-Gilpin, D. (2005).  Hierarchical Complexity Scoring System (HCSS):  How to Score Anything.  Unpublished Scoring Manual Available from Dare Institute, 

© 1991-2005 Dare Association, Inc.  Cambridge, MA 02138

Abstract

The Model of Hierarchical Complexity presents a framework for scoring reasoning stages in any domain as well as in any cross cultural setting.  The scoring is based not upon the content or the participant material, but instead on the mathematical complexity of hierarchical organization of information. The participant’s performance on a task of a given complexity represents the stage of developmental complexity. This paper presents an elaboration of the concepts underlying the Model of Hierarchical Complexity (MHC), the description of the stages, steps involved in universal stage transition, as well as examples of several scoring samples using the MHC as a scoring aid. 

Introduction

The Basis of Scoring Performance and Constructing Tasks:  The Issues

	The assessment of stage of development would seem like a straight forward task.  One might look at the resposes to questions and place them into categories.  Likewise one might construct questions to obtain responses that succeed or fail in addressing that item.  But the issue is not so simple.  These previous ways have led to great difficulties and endless controversies.

	There are three prerequisites.  First, one needs to understand the difference among experience, appearance, and reality.  Second, one needs to understand the difference among Analysis, Phenomenology and Empiricism.  And third and last is to understand the difference between independent and dependent variables as set forth by Aristotle and modified by Descartes into stimulus and response.

History

	The following is adapted from   Edger Brown (2004).  It is important that any "stage" theory and the accompanying scoring  scheme have a mathematically and logically developed basis.  The Greek philosopher and scientist,  Thales (640 - 546) of Miletus, who had knowledge of Egyptian geometry and Babylonian astronomy, is credited with founding mathematics as a deductive science, that is, organizing mathematics around demonstrating by logical arguments the correctness of one’s assertions and calculations.

	But if one does not understand the difference between the ideal and the real one can get into trouble.  The failure of the Pythagorean school rested with its need to make its assertions absolute.  How could one conduct science or have knowledge in general without the possibility that this knowledge corresponds with reality?  Plato handled this problem by rejecting the correspondence account of truth.  We cannot ever know the truth in its complete and pure form.  Anything we can say about reality is only a likely story of the ideal truth.  Here the ideal truth is the mathematical forms.

	We know that an essential element of science is direct observation  and interaction with the world.  But, Plato set forth a very different doctrine, to the effect that knowledge cannot be derived from the senses; real knowledge only 
has to do with concepts. The senses only deceive us; hence we should, in acquiring knowledge, ignore sense impressions and develop reason.
 	Aristotle (384-322), in codifying logical reasoning, set down rules of inference and recognized the importance of axioms for logic, postulates for the subject at hand, definitions of terms and the importance of giving logical arguments starting with the postulates.  The model of hierarchical complexity follows in that tradition. Combining of Aristotle's precise formulation of logic with Thales' method, the main elements of modern science were then in place

	The Model of Hierarchical Complexity on which scoring and problem construction is based, is a mathematical theory of the ideal.  It is a perfect form as Plato would have described.  It is like a circle.  Once one draws it, it is no longer perfect.  The lines have width, it is not perfect.  If can be be perfectly round.

Events
	
	Scientific accounts of behavior are built out of both analytical and empirical accounts of events.  One problem that continually arises is what perturbations to consider as existing, or in other words, what constitutes an event.  There only seems to be one necessary restriction on saying that something exists.  The restriction is rather weak compared to those required by operationalism but strong with respect to intuitionism and phenomenonology.  With the quantitative behavioral developmental theory that follows, we have to consider events as the basis.  This notion is less restrictive than behaviorists' notions of stimuli and responses and so allows the theory to consider events that may not be clearly stimuli or responses.  On the other hand, we do not want to make the mistake of Piagetians that  thoughts, "schema," and verbalizations that belong to mental structures are the only causes of actions.

	How do we know that something is an event?  Events are potentially detectable perturbations.  Perturbations are classed as events when they achieve some potential to be observed, witnessed, and in some way distinguished from the remaining noise by two independent paths of detection.  The term event is used here to include all such perturbations, both public and private.  The notion of paths of detection is not deniable or reducible lest we get into an infinite regress.  These paths do not require direct observation.  Note also that more experiencers or more experiences do not count as more independent paths.  
	
	Potential events may be inferred as long as there are two distinct paths leading to that inference, such as the case with electrons. Electrons may be detected through a multitude of paths by which inferences as to the existence of an "electron event" can be made.  One can measure the magnetic moment of a single electron moving along a path in a magnetic field, the electric charge in an electric field, or the ionizing potential in a liquid hydrogen bubble chamber.  There are numerous other ways of detecting the electron.
 
	The reason two paths are required for events is because one path alone could mean that the perturbation could serve as its own causal explanation of itself.  Some perturbations are deemed as having the status of being only singly detectable by one path.  For example, if someone reports that the president is talking to them, there is one path, their report.  They do not have a radio, telephone or any other such device and the president is nowhere close by.  One other path is necessary to confirm that the president is actually talking to them and they are not reporting a hallucination.  Behaviors and causes detected from a personal experience alone have this character.  Robert Stickgold (personal communication, 1999) has shown that people think that of what they think, see, and dream as "real" while thinking, seeing and dreaming.  The status of events and perturbations is even more complex when activity is not potentially observable, as is with gyrations of the soul or will.  These perturbations may be studied in theological and theosophical terms ( Lowenthal, 1989).  The best we can do within science is to discuss the report of these perturbations as data to be explained or refer to these perturbations in metaphorical terms.
	
	Behavioral constructs (such as stimuli, behaviors, or consequences) are events.  In the case of a verbal report, an observer may hear it.  A microphone and meter will show it. There is a difference between the appearance of a perceived event and the actual event.  Perceptual activity can transform events.  Illusions refer to those instances where people report the appearance of stimuli in ways that distort the physical properties of the objects or events.  Let us say one was looking at a color patch and the person said, "I see the color brown." But the color brown has no unique
Three Ways of Knowing about Development 

	With the definitions of perturbations and events, it is possible to show what are the minimum conditions necessary for having a quantitative behavioral developmental theory.  One needs to recognize the different ways in which we might know and understand development.  The argument is very simple. There are three ways of knowing:
Three Ways of Knowing about Development

	With the definitions of perturbations and events, it is possible to show what are the minimum conditions necessary for having a quantitative behavioral developmental theory.  One needs to recognize the different ways in which we might know and understand development.  The argument is very simple. There are three ways of knowing as shown in Table 1.  Knowledge is treated in a much more complex manner in philosophy.  Here, the number of paths needed for detecting a perturbation is associated with the field and methodology that claims knowledge. 

Table 1
Ways of Knowing

Ways of Knowing	Example of Fields Utilizing These Ways of Knowing	Number of Paths of Detections of Perturbations	
1. Analytic: Proved material always true no matter what "data" or "experience" shows	Mathematics, Logic, Parts of Philosophy	No paths of detections of perturbations	
2.  Phenomenological: Experienced material a property of organisms and sometimes organisms interacting with environments.	Religion, Law, Art, Literature, Dance and Music	One independent path of detection. This means that if one observes an action and hypothesizes a cause, such as free will, then the putative cause may represent one path of detection.  Detecting the behavior, however, does not prove that the hypothetical "causal" event is an actual event.  If only one path is available, that is, if only one effect can be detected–that is the experience (and its report), there is no way to determine the cause of that experience.  The experience is sometimes erroneously said to "cause itself."  	
3.  Empirical: Resultant material from investigations moves scientific towards the truth.	Science, History	Two independent paths.  An event can be said to be real in a scientific sense if and only if it is detectable by two independent paths.  An independent second path for detecting the hypothesized causal event must be found.	

	There can be combinations such as 1 and 3, which define most of science.  Problems arise with combinations of 2 with 1 (Folk Psychology of Aristotle), 2 with 3 (current mixes of experimental and phenomenonological accounts of free will such as   Libet's, 1985).  These can lead to various dangerous policies and practices.  That does not mean that 2 is not prized for itself.  It is.

There were three further developments necessary

	The first, we are familiar with.   Copernicus (1530) showed that the sun is the center of the solar system.  He used mathematics to represent the orbits of the planets.  In some sense, this was the first mathematical model.  Second, modern thinking about the brain and behavior began with the French philosopher René Descartes (1596-1650).  According to  Descartes (1637), all action is a response to an event.  He thereby introduced the notion of the stimulus and the response.  Descartes suggested that 'animal spirits' flowing through the nerves of animals or humans served a similar function in automatic behavioral responses in man and animals or reflexes.  The term 'reflex' is derived from the notion that the flow of animal spirits produced by a stimulus is somehow reflected by the brain into an outgoing flow which eventually produces some behavior.    G. T.   Fechner (1860) laid the basis for the application of the experimental method to psychology.  His establishment of psychophysics through his publication of Elements of Psychophysics in 1860.  He showed introduced the psychophysical scale and showed how to relate psychological variable to stimulus ones.  This is exactly what the model of Hierarchical Complexity does.  It relates stage of performance to the order of hierarchical complexity of tasks.  Lastly, in the early 1960, Krantz, Luce, Suppes, and Tversky (  Krantz, Luce, Suppes, and Tversky, 1971);   Suppes, Krantz, Luce & Tversky, 1989);   Luce, Krantz, Suppes, & Tversky, 1990) introduced the representational theory of measurement.  It is the basis for the model of hierarchical complexity.  

Tasks

	One major basis for this developmental theory is task analysis.  The study of ideal tasks, including their instantiation in the real world, has been the basis of the branch of stimulus control called Psychophysics.  Tasks are defined as sequences of contingencies, each presenting stimuli and requiring a behavior or a sequence of behaviors that must occur in some non-arbitrary fashion.  Properties of tasks (usually the stimuli) are varied and responses to them measured and analyzed.  In the present use of task analysis, the complexity of behaviors necessary to complete a task can be specified using the complexity definitions described next.  One examines behavior with respect to the analytically known complexity of the task.

History of Stage Scoring

	Ever since the introduction of the idea that development proceeds in discrete stages, scientists have argued over the framework for defining and analyzing such stages.  Many models were presented to conceptualize development, including the mentalistic theory of Jean   Piaget (1954), a pioneer in the field of developmental psychology.  Though Piaget’s theory did not define all stages precisely, it clearly established that there is one invariant pathway along which stage development proceeds irrespective of content or culture (  Piaget, 1976).  Other developmental models followed Piaget’s, and each usually focused on development within a particular domain of information. As more content-oriented models were introduced, the “theme of uniqueness [of each model] was increasingly dropping out” (  Kohlberg, 1990).  Because the varying informational frameworks of different domains have often concealed the common underlying process of stage development, standardization of research methods has been difficult to achieve.  Nevertheless, researchers soon recognized the need for a broadly applicable model of developmental assessment that is necessary in order not only to better conceptualize the patterns and themes of development, but also to conduct comparable cross-cultural studies.

Model of Hierarchical Complexity

	The Model of Hierarchical Complexity (MHC) developed by Commons (  Commons, Trudeau, Stein, Richards, & Krause, 1998) offers a standard method of examining the universal pattern of development.  This model is based on a theory of General Stage development (  Commons & Richards, 1984a, 1984b).  The MHC states that all stages are hard distinct stages varying only in the degrees of hierarchical complexity. To counter the possible objection of arbitrariness in such an inclusive and uniform definition of stages, the MHC stage orders are grounded in the hierarchical complexity criteria of mathematical models (  Coombs, Dawes, & Tversky, 1970), and information science (  Commons & Richards, 1984a, 1984b;   Lindsay & Norman, 1977;   Commons & Rodriguez, 1990, 1993).  The Model of Hierarchical Complexity is not based on the assessment of domain specific information, but instead on the analysis of the complexity of the participant’s attempted solution to a task of a specific complexity.  That is, the participant’s successful performance on a task of a given order of complexity represents the stage of development achieved by that participant. The stage score is derived from the evaluation of performances on tasks.

	The MHC has a broad range of applicability. The mathematical foundation of the model makes it an excellent research tool to be used by anyone examining performance that is organized into stages.  It is designed simply to assess development based on the level of complexity which the individual utilizes to organize information. The MHC offers a singular mathematical method of measuring stages in any domain because the tasks presented can contain any kind of information. The model thus allows for a standard quantitative analysis of developmental complexity in any cultural setting. Other advantages of this model include its avoidance of mentalistic or contextual explanations, as well as its use of purely quantitative principles which are universally applicable in any context. Cross-cultural developmentalists and animal developmentalists; evolutionary psychologists, organizational psychologists, and developmental political psychologists; learning theorists, perception researchers, and history of science historians; as well as educators, therapists, and anthropologists can use the MHC to quantitatively assess developmental stages.

The MHC and Skill Theory have ordered problem-solving tasks of various kinds, including:

Social perspective-taking (  Commons & Rodriguez, 1990; 1993)
Informed consent (    Commons & Rodriguez, 1990, 1993).
Attachment and Loss (  Commons, 1991;   Miller & Lee, 1998)
Workplace organization (  Bowman, 1996a; 1996b)
Workplace culture (  Commons, Krause, Fayer, & Meaney, 1993)
Political development (  Sonnert & Commons, 1994)
Leadership before and after crises (  Oliver, 2004)
Honesty and Kindness (  Lamborn, Fischer & Pipp, 1994) 
Relationships (  Cheryl Armon, 1984a, 1984b)
Good Work (  Cheryl Armon, 1993)
Good Education (  Dawson, 1998)
Good interpersonal (  Armon, 1990)
Views of the “good life” (Armon, 1984c; Danaher, 1993; Dawson, 2000;     Lam, 1994)
Evaluative reasoning (  Dawson, 1998)
Epistemology (  Kitchener & King, 1990;   Kitchener & Fischer, 1990)
Moral Judgment (      Armon & Dawson, 1997;   Dawson, 2000)	Language stages (  Commons, et. al., 2004)
Writing (  DeVos & Commons, unpublished manuscript)
Algebra (Commons, in preparation)
Music (Beethoven) (  Funk, 1990)
Physics tasks (  Inhelder & Piaget, 1958)
Four Story problem (    Commons, Richards & Kuhn, 1982;   Kallio & Helkama, 1991)
Balance beam and pendulum (  Commons, Goodheart, & Bresette, 1995)
Spirituality (  Miller & Cook-Greuter, 2000)
Atheism (Nicholas Commons-Miller, in preparation)
Animal stages (  Commons and Miller, in press)
Contingencies of reinforcement (Commons, in preparation)
Hominid Empathy (  Commons & Wolfsont, 2002)
Hominid Tools Making (  Commons & Miller 2004)
Counselor stages (  Lovell, 2004)
Loevinger’s Sentence Completion task (  Cook-Greuter, 1990)
Informed consent (  Commons, Rodriguez, Cyr, Gutheil et. al., in preparation)
Report patient’s prior crimes (  Commons, Lee, Gutheil, et. al., 1995)
Orienteering (  Commons, in preparation)	

	Most of the earlier scoring schemes have not presented ways of assessing to what extent the quality of a participant’s performance on a task should influence the stage score independent of the content of the participant’s discussion. To remedy this problem, the MHC presents a framework that quantifies the order of hierarchical complexity of a task based on mathematical principles of how the information is organized, not what information is presented. The hierarchical complexity of a task to be solved is determined by the mathematical analysis of task demands.  The order of performance on the task, or the stage, is also derived by analyzing the mathematical complexity of successful performance, not merely by observing what the participant does or says. The scores of the MHC indicate the stage achieved by the participant as indicated by his ability to successfully meet task demands of varying degrees of complexity. Results are not subjectively weighted based on the considerations of culture or the environment.

	Hierarchical complexity refers to the mathematical complexity of the task presented to the participant, but not directly to the complexity of the participant’s performance that will successfully complete the given task.  Every task contains a multitude of subtasks (  Overton, 1990).  When the subtasks are carried out by the participant in a required order, the task in question is successfully completed. Therefore, the model asserts that all tasks fit in some sequence of tasks, making it possible to precisely determine the hierarchical order of task complexity.  Tasks vary in complexity in two ways: either as horizontal (involving classical information); or as vertical (involving hierarchical information).

	Horizontal (Classical Information) Complexity

	Classical information describes the number of “yes-no” questions it takes to do a task.  For example, if one asked a person across the room whether a penny came up heads when they flipped it, their saying “heads” would transmit 1 bit of “horizontal” information.  If there were 2 pennies, one would have to ask at least two questions, one about each penny.  Hence, each additional 1-bit question would add another bit.  Let us say they had a four-faced top with the faces numbered 1, 2, 3, or 4.  Instead of spinning it, they tossed it against a backboard as one does with dice in a game.  Again, there would be 2 bits.  One could ask them whether the face had an even number.  If it did, one would then ask if it were a 2.  Horizontal complexity, then, is the sum of bits required by just such tasks as this.               

	Vertical (Hierarchical) Complexity 

	Specifically, hierarchical complexity refers to the number of recursions that the co-ordinating actions must perform on a set of primary elements.  Actions at a higher order of hierarchical complexity: a) are defined in terms of actions at the next lower order of hierarchical complexity; b) organize and transform the lower-order actions; c) produce organizations of lower-order actions that are new and not arbitrary,  and cannot be accomplished by those lower-order actions alone.  Once these conditions have been met, we say the higher-order action co-ordinates the actions of the next lower order.  Stage of performance is defined as the highest-order hierarchical complexity of the task solved.  Using   Rasch (1980) analysis,   Commons, Goodheart, and Dawson (1995; 1997) found that hierarchical complexity of a given task predicts stage of a performance, the correlation being r = .92 (hierarchical complexity of the task that is completed).

	The nonarbitrary organization of several lower order actions constitutes one action of a higher order of complexity.  For example, completing the entire operation 3 x (4 + 1) constitutes a task requiring the distributive act.  That act non-arbitrarily orders adding and multiplying to coordinate them.  The distributive act is therefore one order more hierarchically complex than the acts of adding and multiplying alone and it indicates the singular proper sequence of the simpler actions.  Although someone who simply adds can arrive at the same answer, people who can do both display a greater freedom of mental functioning.  Therefore, the order of complexity of the task is determined through analyzing the demands of each task by breaking it down into its constituent parts. the hierarchical complexity of any complex task is thus mathematically determined The participant is scored at the stage this complexity when he successfully completes the task using the integrated approach of coordinated combination of lower order actions.

	The hierarchical complexity of a task refers to the number of concatenation operations it contains. An order-three task has three concatenations operations. A task of order three operates on a task of order two and a task of order two operates on a task of order one (a simple task).

	Tasks are also quantal in nature.  They are either completed correctly or not completed at all.  There is no intermediate state. For this reason, the General Stage Model characterizes all stages as hard and distinct.  The orders of hierarchical complexity are stepped like the rings around the nucleus.  Each task difficulty has an order of hierarchical complexity required to complete it correctly.  Once again, since tasks of a given order of hierarchical complexity require actions of a given order of hierarchical complexity, the stage of the participant’s performance is equivalent to the order of complexity of the successfully completed task.  The quantal feature of tasks is thus particularly instrumental in stage assessment because the scores obtained for stages are likewise discrete.

	Hierarchical complexity of actions refers to the number of recursions that the coordinating actions must perform on a set of primary elements. Like tasks, actions at a higher order of hierarchical complexity:

	1)	are defined in terms of the actions at the next lower order of hierarchical complexity;
	2)	organize and transform the lower order actions;
	3)	produce organizations of lower order actions that are new and not arbitrary, and cannot be accomplished by those lower order actions alone.

	The hierarchical complexity of tasks and the actions they require to be successfully completed provide the mathematical foundation for deriving scores for stages of reasoning.  The MHC, however, does not dismiss the influences of the environment on one’s reasoning stage development, it simply does not quantify contextual variables during the scoring process as do other scoring manuals which are designed to measure stages in a particular domain of information and may give more weight to the overall score if particular issues are addressed by participants, regardless of the manner in which the references are made.

Stages

    The notion of stages is fundamental in the description of human, organismic, and machine evolution.  Previously it has been defined in some ad hoc ways; here we describe it formally in terms of the model of hierarchical complexity. Given a collection of actions A and a participant S performing A, the stage of performance of S on A is the highest order of the actions in A completed successfully, i.e., it is

	stage(S, A) = max {h(A) | A ∈ A and A completed successfully by S}.

Thus, the notion of stage is discontinuous, having the same gaps as the orders of hierarchical complexity.  This is in agreement with previous definitions (Commons et al, 1998; Commons & Miller, 2001, Commons & Pekker, submitted).
	
Relationship between Piaget and Commons notions

	There are some common elements between Piaget and Commons notions of stage and many more that are different.  In both one finds: 

	1.  Higher order actions defined in terms of lower order actions
		This forces the hierarchical nature of the relations and makes the higher order tasks include the lower ones
	2.  Higher order of complexity actions organize those lower order actions
		This makes them more powerful		

What Commons et al have added includes:

3. 	Higher order of complexity actions organize those lower order actions in an non-arbitrary way
		This makes it possible for the organization to meet real world requirements, including the empirical and analytic
	4.  Task and performance are separated	
	5.  All tasks have an order of hierarchical complexity
	6.  There is only one sequence of orders of hierarchical complexity.
		Hence, there is structure of the whole for ideal task actions
	7.  All orders of hierarchical complexity are equally spaced
	8.  There are gaps between the orders of hierarchical complexity
	9.  Stage is most hierarchically complex task solved. 
	10. There are gaps in Rasch Scaled Stage of Performance
	11. Rasch Scaled Stage of Performance are also equally spaced 
	12. Performance stage is different task area to task area
		There is no structure of the whole – horizontal decaláge for performance.
		It is not inconsistency in thinking within a developmental stage.
		Decaláge is the normal modal state of affairs

Stages of Development

	The MHC specifies 14 order of hierarchical complexity and their corresponding stages, showing that each of Piaget’s substages, in fact, are hard stages.  Commons also adds three postformal stages. The sequence is as follows: (0) computory, (1) sensory & motor, (2) circular sensory-motor, (3) sensory-motor, (4) nominal, (5) sentential, (6) preoperational, (7) primary, (8) concrete, (9) abstract, (10) formal, (11) systematic, (12) metasystematic, (13) paradigmatic, and (14) cross-paradigmatic.  The first four stages (0-3) correspond to Piaget’s sensorimotor stage at which infants and very young children perform.  The sentential stage was added at Fischer’s suggestion.  Adolescents and adults can perform at any of the subsequent stages.  MHC stages 4-6 correspond to Piaget’s  pre-operational stage; 6-8 correspond to his concrete operational stage; and 9-11 correspond to his formal operational stage.  The three highest stages in the MHC are not represented in Piaget’s model.  Because MHC stages are conceptualized in terms of the hierarchical complexity of tasks rather than in terms of mental representations (as are Piaget’s stages), the highest stage represents successful performances on the most hierarchically complex tasks rather than intellectual maturity.
	Stages 0-5 normally develop during infancy and early childhood in people.

	At the calculatory stage (0), machines can do simple arithmetic on 0s and 1s.

	At the sensory and motor stage (1), infants may see or touch shapes, make generalized discriminations, as well as babbling vocalizations.

	At the circular sensory and motor stage (2), reaching and grasping actions occurs.  These actions generate simple gestures. 

	At the sensory-motor stage (3), the actions become associated with vocalizations.  For instance, an infant may hold up an object and make sounds while doing so.  

	At the nominal stage (4), first single words are formed.  These words such as “cup” or “water” relate concepts to others.

	At the sentential stage (5), toddlers form short sentences and phrases.  The use pronouns, and say numbers and letters in order as well.  Sentences might be “want water,” “cup of water,” etc.

	At the preoperational stage (6), these sentences are organized into paragraph long utterances.  

	At the primary stage (7), these paragraph long utterances are organized into stories which may be matched to reality. 

	At the concrete stage (8), two primary stage operations may be co-ordinated.  For example, children think that a deal is fair after looking at from the perspective of simple outcomes for each person who is entering the deal.  Negotiations make sense but there are not social norms for setting prices or values.  

	At the abstract stage (9), variables, stereotypes, personalities, traits, etc are introduced.  Quantification words like “everyone in my group,” What would other’s think?” appear.  The dimensionalized qualities may be used to express preferences.

	At the formal operational stage (10), discussions are logical and empirical support is logical brought.  Words like “if ...then,” “in every case, it turned out the same,” “the reasons were” occur. 

	Few individuals perform at stages above formal operations. More complex behaviors characterize multiple system models (  Kallio, 1995;   Kallio & Helkama, 1991).  Some adults are said to develop alternative to, and perspectives on, formal operations. They use formal operations within a “higher” system of operations and transcend the limitations of formal operations. In any case, these are all ways in which these theories argue and present converging evidence that adults are using forms of reasoning that are move complex than formal operations.

	At the systematic stage (11), the new concepts are referred to as 3rd order abstractions. These coordinate elements of abstract systems. Words like bureaucratic, capitalist, functional, and structural  are common. The systematic stage concept, structure, for example, can be employed to ask whether the structure of camp helps instill the qualities we want in future citizens. The logical structure of this stage coordinates multiple aspects of two or more abstractions, as in: “relationships are built on trust and though we can’t always keep them, making promises is one way we build trust, so it’s generally better to make promises than not to make them.”  Here, the importance of trust to relationships, building trust, and the possibility that promises can be broken,  are all taken into account while formulating the conclusion that promises are desirable.

	At the metasystematic stage (12), the new concepts are referred to as 1st order principles. These coordinate formal systems. Words like autonomy, parallelism, heteronomy, and proportionality  are common. The metasystematic stage concept of parallelism, for example, can be employed to compare the structures of the military and of camp as institutions. The logical structure of this stage identifies one aspect of a principle or an axiom that coordinates several systems, as in: “contracts and promises are articulations of a unique human quality, mutual trust, which coordinates human relations.” Here, contracts and promises are seen as the instantiation of a broader principle coordinating human interactions.

	At the paradigmatic stage (13), people create new fields out of multiple metasystems.  The objects of paradigmatic acts are metasystems.  When there are metasystems that are incomplete and adding to them would create inconsistences, quite often a new paradigm is developed.  Usually, the paradigm develops out of a recognition of a poorly understood phenomenon.  The actions in paradigmatic thought form new paradigms from supersystems (metasystems).     

	Paradigmatic actions often affect fields of knowledge that appear unrelated to the original field of the thinkers. Individuals reasoning at the paradigmatic order have to see the relationship between very large and often disparate bodies of knowledge, and co-ordinate the metasystematic supersystems.  Paradigmatic action requires a tremendous degree of decentration.  One has to transcend tradition and recognize one's actions as distinct and possible troubling to those in one's environment.  But at the same time one has to understand that the laws of nature operate both on oneself and one’s environment—a unity.  This suggests that learning in one realm can be generalized to others.
			
	At the cross-paradigmatic, paradigms and coordinated.  This is the fourth postformal stage.  Cross-paradigmatic actions integrate paradigms into a new field or profoundly transform an old one.  A field contains more than one paradigm and cannot be reduced to a single paradigm.  One might ask whether all interdisciplinary studies are therefore cross-paradigmatic?  Is psycho biology cross-paradigmatic?  The answer to both questions is ‘no’.  Such interdisciplinary studies might create new paradigms, such as psychophysics, but not new fields.

	This order has not been examined in much detail because there are very few people who can solve tasks of this complexity.  It may also take a certain amount of time and perspective to realize that behavior or findings were cross-paradigmatic.  All that can be done at this time is to identify and analyze historical examples. 
	
	Several tables are provided to help the reader better understand the concepts of stages as defined by the MHC. Tables 1, 4a, and 4b present the definitions of stages with examples of task demands of respective complexities. Table 1 particularly explains how behavior may form classes and how stimuli may be place into classes both functionally and analytically.   This table gives examples of behaviors as they may be observed, and each stage behaviors is also broken down into substeps, showing the organizing functions of varying complexities.  Table 1 is especially useful for scoring behaviors up to stage 12. 

	Table 2 provides examples of the kinds of vocal remarks made by various family members performing at specified stages. This table is a particularly useful tool for evaluating stages among related individuals which may be helpful in studies examining development and controlling for hereditary factors. In addition, in many developing nations families tend to play particularly important roles in lives of individuals, and increases in complexity might be particularly evident if the content of the assessment deals with topic of family.  Family affairs constitute a practical source of universally relevant content that could be used to evaluate stages in any cross-cultural setting. Familiarity with table 2, therefore, is particularly useful for researchers.

	Table 3 exemplifies various verbal relationships formed by vocalizations characterizing different stages.  Table 3 is especially helpful since it shows some key grammatical structures associated with each stage, as well as the key phrases used by people performing at given stages.  Though an in-depth mathematical analysis is necessary for an accurate assessment of stage, familiarity with Table 3 will assist the scorer in initial classification of behavior.

	Tables 4a and 4b describe the orders of hierarchy and sequence of stages, respectively. That is, the table elaborates the nonarbitrary coordination process underlying MHC stages. This table explains how concepts are constructed and vocalized at each stage with increasing complexity. These table also clearly show how each subsequent discriminations is vocalized at the subsequent verbal relationship is formed. Understanding the examples provided in Tables 4a and 4b enables the researcher to classify stage based on complexity of vocalizations and the verbal relationships they form. 

	The new concepts formed at each stage can be viewed as “summaries” of the constructions of previous stages. Although the MHC proposes no mental model to explain this process (  Halford, 1999), suggests that this summarizing or “chunking” makes advanced forms of thought possible by reducing the number of elements that must be simultaneously coordinated to produce an argument or a conceptualization at a higher order of hierarchical complexity.  Interestingly, at the preoperational, abstract, and metasystematic stages of the MHC  the new concepts not only coordinate or modify constructions from the previous stage, but they are also  qualitatively distinct conceptual forms: representations, abstractions, and principles, respectively. The appearance of each of these conceptual forms ushers in three repeating logical forms: definitional, linear, and multi variate. Other researchers have confirmed these distinct conceptual forms and repeating logical structures (  Fischer, 1980;   Fischer & Kenny 1986;   Fischer & Lazerson, 1984).

External Influences

	Psychological, sociological, and anthropological data address why the participant’s performance develops in a given manner.  However, why development takes place is linked to how participants can demonstrate the stage of development. The successful completion of a task requires an ideal action of a given order of hierarchical complexity which had developed as a result of influences by psychological and sociological variables.  For example, Table 5 shows how stimulus control with or without support can change the relative difficulty of a task. The level of support during task completion, therefore, changes the scored order of performance. Other models have often used the participant’s reference to an informational set as an index of stage development without considering such variable as the level of support. We believe that this approach is oversimplified. Accurate, consistent results could only be obtained when the system of evaluation is based on a universally applicable groundwork, such as the mathematical foundation of the MHC.  According to the MHC, the participant’s approach to a given task is  quantified to produce a score for the stage of reasoning in any domain.  Inferences regarding the factors influencing the performance can be made independent of obtaining  the stage scores.

	The Model of Hierarchical Complexity posits that individual’s perceptions of the world (and the stimuli in it) are influenced by frameworks. These frameworks embody the individual’s conditioning history, including cultural, educational, religious, political, and social backgrounds, among other factors.  These combined frameworks are referred to as one’s perspective.  Perspectives differ in terms of hierarchical complexity. As the hierarchical complexity of an individuals’s response to task demands increases (i.e., as complexity of performance goes up), the individual is increasingly likely to have taken many such perspectives into account (  Commons & Rodriguez, 1990). 

	There are task demands that certain professions require of individuals.  Although the job demands of a secretary may not exceed formal stage of complexity, those of managers or judges often require development beyond the systematic stage. Tables 9 and 10 provide examples of types of social organizations and professional settings which require development to various stages of complexity.

Conceptualizing Stages

	Each of Piaget’s stages is defined by a set of formal properties that constitute a structure d’ensemble, or a structure of the whole. This has sometimes been taken to mean that the entire knowledge system forms a single unified global structure (  Fischer, 1980). In some interpretations of stage transitions based on the notion of structure d’ensemble, development is characterized by abrupt global reorganizations of the knowledge system which is modeled as a single staircase.  However, studies of performances on various tasks do not provide  evidence for this type of a global structuring of knowledge.  Instead, assessment models such as the MHC posit that several analogous structures of knowledge exist, however, they do not appear to develop in parallel. This is especially true of analogous structures in different knowledge domains (  Fischer, 1980;   Fischer & Kenny 1986;   Fischer & Lazerson,  1984). In fact, there is no evidence whatsoever supporting a single, global, stepwise pattern of development. Instead, it has been argued that the cognitive system can best be conceptualized as a set of interrelated dynamic knowledge systems (  Fischer, 1980; Fischer & Kenny, 1986; Fischer & Lazerson,1984;    van Geert, 1991), each developing in a hierarchical manner. As discussed earlier, the MHC does not propose a direct link between mental processes and performance. However, inferences can be made about mental processes on the basis of patterns of performance, and these inferences can inform research into mental functioning when cultural and other factors influencing performance are evaluated along with the actual actions.-
	
Task Theory

	To further elucidate the concepts involved in quantifying task demands as the basis for generating stage scores we present an overview of how tasks are constructed. Task demands increase along a continuum of complexity: from preoperational, concrete, and abstract, to formal, systematic, and metasystematic.

	Series of tasks in different domains

	Each task can only be correctly addressed at a given point in development. If the successful completion of the task requires a higher stage then one at which the person is performing, the scored stage will be lower than if the task demands actions at the reasoning stage the participant has already achieved.  Fischer has shown that presenting a task that is above the participant’s stage of performance depressed the performance index below the actual stage for reasons related to emotional development (  Fischer, 1980;   Fischer & Kenny, 1986;   Fischer & Lazerson, 1984), to be additionally discussed in section called Stage Transition.  Therefore, using only a stage task that’s too demanding may result in underscoring performance.  Presenting a task demanding the response that the participant  can actually display  is a more accurate method of assessment. At the outset of the study, this stage is hard to predict. The most efficient way to assess stage, therefore, is to administer several tasks of varying complexity for the participant to attempt, including tasks of low orders of complexity. The completed task of the highest order of hierarchical complexity of all the tasks presented would most accurately represent the actual stage of the participants’ reasoning. In other words, the Model of Hierarchical Complexity not only does not focus on any particular domain of knowledge for reasoning stage assessment, but it also recommends that several tasks from multiple domains are presented in order to obtain the most accurate results. The stage scores may differ in each domain depending upon the mathematical complexity of performance.

	Dimensions of tasks

	 Tasks are comprised of three basic dimensions: action, description or reflection upon that action (  King & et al., 1989;   Tappan, 1990), and the number of element that a person can work with at a given time which are required to perform that action and to report on it.  The theory underlying the development of tasks is that different tasks require different levels or values of each of the three dimensions. The values of each dimension are important in assessing the stage at which a person is able to successfully execute a task.  Often, these three dimensions are ignored and only one measurement, stage of action, is specified. This oversimplified process does not yield comparable measures of stage across tasks because the scoring is based on different values in one or more of the other three dimensions.  In other words, the action demands of executing a certain task in one domain may differ from the action demands of executing a task in another domain.  The same would be true for the demands of reflection required  in performing a task  in one domain versus another; and for the amount of memory required to execute a task in one domain versus another. The MHC is primarily concerned with the first dimension of task, the action dimension, because it interprets the stage of reasoning to correspond other stage of performance.  However, the stages may differ in different domains because task demands also differ.

	In making comparable stage interpretations across tasks, each of the values in the dimensions of action, reflection and memory should be specified.  In other words, when discussing stage one must be specific about the reference to the dimensions of action, reflection, and memory. 

		I. Dimension 1: Action
	The dimension of action consists of a number of requirements for a series of concatenated actions to form a stage hierarchy of actions.  The chain of steps may not be rearranged.  If doing the action was at the sensory motor stage1, reporting on the action would be at the nominal stage, reporting on why one chooses that particular action would be at preoperational stage, and justifying those justifications would be at the primary stage.  That is, more complex tasks and actions coordinate lower order tasks and actions in a nonarbitrary fashion, yielding the process to quantitative analysis.

	For example, children might be told to put their toys into the toy box.  Putting toys into the toy box is an action that a sensory motor child might perform.  At the nominal stage they might say "Put toys," or "Put toys away."  Preoperational children might say, "We are putting the toys away so we can get some cookies."  Primary operational children might justify putting the toys away by saying, "We must put the toys away now before we do the next thing because that is the rules."

	The order of hierarchical complexity of tasks composed of subtasks is easily determined.  When the tasks are from the same domain, if one task operates on the other, the order of complexity increases.  The same is true across domains.  When tasks from different domains are added to one another to form a new task, the number of required concatenations of actions also add.  This assumes that stage requirements form an interval scale.  The order of hierarchical complexity  required by a task is written as o.   Hence, for stage:

o = the order n requirements - order n - 1 requirements where the order n requirements is the order of hierarchal complexity required by the task.
  Hence for stage:
	stage n requirements - stage n - 1 requirements =
	stage n - k - stage n - k - 1
The predicted stage required by a task is written as o.
	o  = order n - k - order n - k - 1

This assumption also holds for describing action and reflecting upon the description.

		Dimension 2: Reflection

	The dimension of reflection on action consists of the following steps:
 	1. Doing the action
	2. Reporting on doing the action (shadowing)
	3. Reporting on why one chooses that particular action
	4. Reporting on why that justification is good
	5. Reporting on why that system of justifications is good
Each step requires the previous step.  The question is whether the fact that each step requires a previous step represents a change of stage.

		Dimension 3: Memory

	Remembering an action in order to reflect upon it requires non-structural actions that increase the task difficulty.  Little children can describe what they are doing before they can describe what they have done earlier (    Piaget, 1976;     Karmiloff-Smith, 1986) although their exact report of what they have done may differ from what they actually did.  Karmiloff-Smith clearly explains that there are mechanisms of thought in operation before the child comes to be able to report on those actions.  Recalling previous actions may or may not require an extra stage depending on how the recall is triggered.  For example, if the recall is in the sensory-motor stage as is the remembering of a comfortable sleeping position in order to attain that level of comfort again, one stage is not required to solve the problem, one simply moves around until that position is again attained.  Yet, the explanation of what that position is requires additional stages.  If the recall depends upon having a sense of time (i.e., recalling something ordered by time) it might require the attainment of at least one stage of development.

	A given developmental stage represents a measure of successful performance on tasks of the same order of hierarchical complexity.  The General Stage Model (  Commons & Richards, 1984a, 1984b;   Commons & Rodriguez, 1990;   Commons & Rodriguez, 1993) defines the stage in terms of task performance.  When people successfully perform tasks of a given order of hierarchical complexity, they are performing at the stage of the equivalent order.  However, the dimensions of reflection and memory also influence the performance or action and are shaped by the developmental environment of the individual.  The MHC incorporates ideas about how task performance develops and how transition progresses from one stage to the next.

Dimension 4  Familiarity

Task can vary in their important both between different cultures and within cultures.  Individuals may have more interest or training in certain tasks.  Familiarly affects the difficulty of tasks.  With practice, support and reinforcement, the effects of Familiarly maybe wiped out.

Dimension 5	Placement of Key Information within tasks

Information place at the beginning or end of tasks are more easily remember and sometimes seen.

Dimension 6	Degree of symbolization provided

Surprisingly, mathematical problems are the easiest in educated populations because they come in a compact symbolized form.  That form requires a minimum of coding by the participants

Dimension 7 Level of Support

Stage Transition

	Measuring transition is extremely important.  Many interventions do not produce a change of a complete stage.  Some population only vary between transition-to-the-next stage and the next stage such as professors at research universities who study ethics.         

	There are two forms of stage transition.  One is transition steps.  These steps represent how two lower stage behaviors alternate increasingly rapidly.  At step -1 or 4, the same stage behavior there is no alternation.  The rate of alternation is 0.  Then at step 0, which is getting off the dime when the present stage is seen to fail, to using an alternative same stage action, to alternating such an action with the previously more used action to finally smashing elements of both previous stage actions together – an infinitely fast alternation rate.

 	The second form of transition if the proportion of current and next stage action as Theo Dawson shows using Rasch analysis.  If one looks at where the person scores fall, they fall between the stage marked by item those two stages or scores for those two stages.

	In order to understand how the dimension of performance increases in hierarchical complexity we must examine the factors implicated in driving stage transition.  That is, we must examine the various contingencies that promote the development of performance at higher reasoning stages which is only possible when the dimensions of reflection and memory coordinately increase in complexity along with the dimension of action.  There are a large number of such contingencies.  They include but are not limited to providing reinforcement or support for next stage behavior, showing contradiction for present stage behavior, exposing people to models of next stage behavior and the reinforcement that such behaviors attain.  Here we also consider emotional and various environmental factors that shape the individual’s transition from one stage to the next.

	Every participant’s behavior could be categorized to a transition step between stages.  Varying factors such as the impact of emotions, personality, and environment, etc influence how long someone may stay at each step-. Most people only traverse up to 12 stages by the age of 24.  Evidence shows people may transition every two years at most, sometimes even less.  The only time when fast transitions occur is perhaps during infancy.  Again, the participant’s performance on a task can only be scored at a given stage of complexity when the task of a corresponding order of complexity is successfully completed.  Table 7, for example, focuses on four types of personalities associated with various transition substeps.  Adults are simply not meant to “get stuck” at these substeps, and the examples provided are often associated with psychological or personality disorders.  This knowledge of transitional mechanisms underlying development is a great asset for therapists and psychiatrists to possess because it could be so useful in diagnosing patients.  Because reinforcement moves people along the substeps toward the successful achievement of stage, using various modifiers of reinforcement would help clinicians treat patients.  The crucial insights of the MHC, therefore, are clearly applicable not only in research, but in medical practice as well. 

	When one successfully completes a task of a given order of hierarchical complexity, one is performing at that stage of complexity.  Therefore, static coping is what occurs when one is not required to perform above one’s characteristic stage of performance.  Often one must meet or solve other problems successfully, or assume additional perspectives and skills in order to change stages.  In those cases, dynamic coping occurs during stage transition and it involves several steps.  During steps 0-2, deconstruction of previous stage behavior occurs (e.g.   Swan & Benack, 2002), during steps 3-4, new stage behavior is constructed.

	At the beginning of each transition the perceived rate of reinforcement drops.  The more one confronts failure, the more one might expect avoidance.  In fact,   Commons, Grotzer, and Davidson (in preparation) found that feedback alone in higher stage tasks led to a decrease in stage of performance, rather than an increase. Perhaps its defensive behavior, with is fear accompanying transition through the steps, decreases the stage of performance. Another explanation could be that one does not see a stage of performance higher than one’s own in others and this impedes learning through support.  Please refer to Table 5 which elaborates the role of support in stage change. In any case, it is important to note that emotions are usually associated with transition of stages.

Transition steps

	At step 0, the demands for performance beyond the final step of the last stage are perceived.  Without changing performance from step 4 of the previous stage, there is a perceived reduction of reinforcement for task performance. This characterizes step 0.  A person feels stupid and upset, sometimes even angry, while failing to fulfil a task. One may also feel elated about task mastery of the previous stages tasks.

	At step 1, the person feels dejection in addition to the previous feeling of sadness (or anger). In both of these first transitional steps, one may want to “give it all up” and forget about it all. These are defense mechanists, ways of switching the point and rejecting frustration.

	At step 2, relativism becomes the key concept.  One sees the possibility of solving a problem but does not necessarily know the right means of doing it.  Someone can be seen as competent for a special task, but not to any task. Relativism has to do with contexts, and because contextualization is a sort of concretizing, it is an attempt to cope with each better way.  But concretizing is not the same as coordinating. One just knows there is a way of comparing situations and means, but not how to do so.  Keep in mind that actions of the full higher order of hierarchal complexity not only put together actions of the lower order, but organize them in a non-arbitrary fashion. Random contextualization, therefore, is characteristic of a transitional step from one stage of performance to another.

	Table 6a explains the steps involved in stage deconstruction, also providing the logical scheme underlying this process.  The substeps organizing each deconstruction step are provided in dialectical form, even though the organization is based on mathematical laws.  The construction of each step out of substeps is written out to facilitate the readers’s understanding of the very mathematical notions involved in organizing complexity. 

	At this point during transition, between the deconstruction and construction steps, one may feel conflicted, anxious and not sure of anything, because the individual does not perceive any control over the situation.  People may ask themselves whether they are independent or dependent, but they most likely can not find an answer.  Who is the one that really holds the reins?  One might enjoy the excitement of the uncertainty, such as s tourist feels upon visiting a strange land and experiencing other cultures for the first time.  One might defend the relativism as a necessary reality and feel that it justifies one’s behavior.

	At step 3, the first step in constructing new stage behavior,  people begin to show more creativity in handling problems.  There are several conditioning substeps comprising this step:
a) The first substep is described as "getting chaotic".  One simply tries anything to get going.  What is often done is just smashing (or lumping) of all the existing systems of acting together without any formal integration.  Smashing has an aggressive and desperate tone characterizing attempts to "survive"– i.e.  building a life raft out of anything.  On the first substep, people feel somewhat manic as part of a normal process.

b) The second substep is the "learning what to do” substep.  Templates are formed that are inclusive.  The instance of the relationship at the target stage will be detected and used.  This second substep brings with it a beginning in producing correct results.  One is not able to eliminate those acts that do not bring good solutions, but the right direction is at hand.  The most common feelings experienced at this point are excitement and a sense of frustration because of making errors.

c) The third conditioning substep is "learning when and where to do" each subset of action.  People know what to do but not when to do it.  They may feel uncomfortable and confused, but not helpless.  One knows what to do, but not when.  On the other hand, people who do not know what to do, may have a feeling of deep incompetence and helplessness.  When people feel both confused and helpless, they have no sense of power nor the ability to act progressively.  One learns to eliminate over generalization errors.  Everything has to be compulsively cleaned up.  One may be obsessive, fussy, and "sticking."  Templates constructed here exclude rather than include.  There is reconstruction.  One is just not meant to get stuck here.

	During the final step 4 which completes the construction of new stage behavior, inclusion and exclusion templates are finally coordinated.  One feels glorious for combining right elements successfully.  A post-reinforcement pause may follow.  At this step the closure makes one feel personally satisfied.  As   Rosenberg  points out, how this momentary stability is perceived will effect how one feels socially (  Rosenberg, 1979).  Quite often the demands for further development occur.  This affects how long such positive feeling persist.

	When entering a score into an analysis, we use the following:

Stage	Stage #	
Concrete	8	
Abstract	9	
Formal	10	
Systematic	11	
Metasystematic	12	

To the previous stage, we add the following for the transition steps.

Step	Points	
0	.2	
1	.4	
2	.6	
3	.8	
4	1.0	

For example, a performance transitional to metasystematic at step 2 would be 11- 2 or 11.6 points.  Performing fully at the Metasystematic stage is therefore 11-4 or 12= 11 + 1 points 

	Table 6b explains the deconstruction steps, also elaborating the logical scheme involved.  Like in Table 6a, the substeps are clearly written out to clarify the mathematical principles of organization. 

	Reinforcement moves everyone along the substeps according the melioration law (  Herrnstein, 1982,   Herrnstein & Vaughan, 1980), which dictates that behavior progresses at a rate proportional to an increase in reinforcement.  Increases in hits increase the likelihood of making hits, which reinforces the generalization.  Decreases in over generalization also increase reinforcement. This law also explains how and why the lack of satisfaction reinforcement that occurs when tasks are completed which are below the actual stage of the individual-- underestimate the stage since the individual may actually perform at a lower stage due to the lack of such reinforcement. Indeed, as explained in Table 7, emotional states and personality types affect stage transition, and this factor must be considered when formulating a complete conception of the processes involved in stage development.

	Knowing  how stage transition occurs is important in the proper application of the quantitative methods of the MHC.  Since stage is assessed from performance, the best performance must be elicited properly.  The failure of the researchers to administer the tasks so as to provide an adequate environment for the expression of ability may result in underscoring stage.  Therefore, researchers must understand the psychological and sociological variables not only of how performance on tasks develops, but also how it can be demonstrated during assessment procedures.	

How to Measure Transition

	Transition can be measured using four different methods:

1.  Scoring interviews directly for statements that reflect transition
2.  Finding the rate and acceleration of alternations of old stage and newer stage actions.
3.  Finding the proportion of new stage versus old stage behavior.
4.  Determining the hierarchical complexity of stimulus items (or tasks) and using a Rasch analysis to show that they form a continuous scale.  The Rasch analysis scales performance and items on the same log linear line.  Transitional performance is shown by the mixtures of performances at different stages.  The mixtures range from 0% at the higher stage to 100%.  We call 95% at a stage consolidated performance and 0% up to 95% transitional.  The advantages of the Rasch analysis are that: (a) it reduces measurement variance to a minimum;  (b) This yields direct comparability which is useful in assessing both the possible natural number and nature of the items and the corresponding performances (  Mislevy & Wilson, 1996;   Wilson, 1989).

Acquisition of New Stage Behavior

	To overcome the huge gap between the lower stage behavior and the higher-stage behavior, Piaget suggested two processes: assimilation of new behaviors and performances to the present stage; accommodation to the higher stage performance.  In both cases, we argue that the laws of learning apply.  Different forms of instruction produce both assimilation and accommodation.  The general finding is the more solid the performance at the lower stage behaviors, the more easily the new stage behavior may be acquired.

	We describe five ways of advancing stage change as discussed with respect to adult development.  First is the didactic method of teaching about higher stage behaviors.   Second is the Piagetian notion of immersion and the use of contradictions.  There are a wide range of programs and variation on this theme (See   Adey, in preparation;   Brendel, Kolbert, & Foster, in press;   Lovell, 1999;   McAuliffe, in press).  Third is the use of reinforcement for correct answers and outcomes.  Fourth is the use of support.  Last is the use of direct instruction and charted performance as feedback.

	Didactic teaching has many variants.  The most common is show and tell.  At the high school level and above, this is referred to as lectures.  Lecturers seem to have been derived in form from sermons.  The information is imparted by speaking to the multitude.  Seeing films or videos, DVD’s or other electronic form of one-way media including seeing films, or listening to tapes, are all variants.  Sometimes there is a lecture followed by a discussion section, which may include more detailed lectures with some possibility for questions.

	A second and related form to lectures is reading material.  Not surprisingly, it is more effective.  It allows for self-pacing, reviewing, and highlighting.  Reading is a much more active process.  The order from least active to most active is as follows: show and tell, listening, and reading.

	Whereas the Piagetian notion of immersion works well for children and adults who care about contradictions in academic settings, less motivated children do not change stage very readily under these conditions (  Commons & Miller, 1998).  In one experiment, performing the correct task lets the children earn points.  The children’s points are then pooled for different teams and then the teams are put into competition between each other.  These competitions for points led 75% of fifth and six grade students to acquire formal operations on a number of Piagetian tasks.

   	Fischer (personal communication) reports that various forms of support–providing examples or prompts for what is the correct response--leads to the acceleration of the acquisition of new stage behavior.  This is probably due to the fact that such types of support reduce the required task demands by one order of hierarchical complexity (see Table 5).  This makes it possible to perform the higher stage task.  Repeated performances at the higher stage are reinforced and therefore acquired.

	Finally, fields such as Precision Teaching offer actual training of new actions.  Two basic notions in Precision Teaching are elements (components) and compounds (combinations) of those element behaviors.  Precision Teachers first train individuals on the elements or components, and only later on combining them.  In Precision Teaching one makes decisions about the effectiveness of current instructional interactions based on charted performances.  The chart shows the rate of completing tasks and compares the rates to how experts would perform.  Fluency training on the element behaviors consists of getting those behaviors to occur at an extremely rapid rate.  When the rate of behavior reaches a maximum, that is, it most closely matches the rate of an expert –the behavior is considered fluent.  If it is learned to the extent that very little effort or special attention is required, that is,  the performance is automatic.  Fluency training on the elements seems to increase the speed at which compounds are acquired from elements.  The implications of this work are that Precision Teaching in behavior analysis provides an empirical account of development.

The Upper Limits of Stage Transition

	The discussion of stage transition may give the impression that under ideal conditions no factors in the stage transition theory necessitate an upper limit on stage.  The current formulation of stages includes 14 orders of complexity, suggesting that the number of times a series of elements can be turned into a higher order combination is 14.  This may, in fact, be the upper limit, at least for human beings.  There have been an increasing number of empirical reports claiming that  a limit exists on the number of times a series of elements can be turned into a combination.  These reports can be found in training studies, which show that at a given age, there are limits to how much training is effective in bringing about change.  We also know from training graduate students that no matter how much training one provides for this group, some students will never move beyond the systematic stage in their problem solving because of getting stuck somewhere in the transition, not because of an inadequate testing environment.

	It is also suggested that whatever the upper limit may be for a particular individual, that ceiling is almost totally heritable.  For example, there does not seem to be any variation among identical twins who have been provided with similar training.  Providing additional training to both twins merely causes acceleration of transition  in the slower twin, but only up to the limit achieved by the other twin, not beyond.

	This theory of stage transition makes six predictions regarding the stages, all of which (  Dawson, Commons, & Wilson, in review;   Dawson et al., 1999, June) have confirmed:

1.	 	There is perfect sequentiality.
2.	 	There is absolutely no mixing of stage items.
3.	 	There are gaps in difficulty between stages.  A Saltus model (  Wilson, 1989) shows that there is no continuity between the stage items. 
4.	 	These gaps are pretty equal, showing that the task demands of transition from one stage to another are similar regardless of the particular transition.  These gaps have been shown using a Rasch analysis with a Saltus model (Dawson & Commons, in preparation).   This result is consistent with our argument here about the quantal nature of transition..   
5.	 	People generally perform in a uniform manner regardless of the domain.  Most performances are predominantly at a single stage. 
6.	 	The distribution of person’s abilities within each transition is strongly skewed toward the higher stage. Comparatively few people exhibit only a little reasoning at their highest stage.  For example (  Dawson & Commons, in preparation), there are fewer participants performing in transition on Kohlberg’s Heinz and Joe dilemmas and more who perform at lower consolidated stages.  Whether a participant’s performance was in transition was measured psychometrically by the proportion of new stage versus old stage behavior.

	The Model of Hierarchical Complexity allows an explanation for performance and behavior that may apparently be “stuck” between stages. Stages of performance are, indeed, hard, and distinct, as the General Stage Model dictates; however, understanding the steps involved in transition between stages, and the influences of emotional states on the participant’s actions as well as on other dimensions of performance on tasks allows for a more complete understanding of stages. The higher the stage, the more complex the behavior it requires, therefore, at higher stages, transition behavior is more likely to be observed than at lower stages which require simpler behavioral patterns.

	The mathematical foundation of the MHC also presents a more concrete framework for assessing development that can be used to make cross-cultural studies and comparisons in order to elaborate the factors involved in human development. However, it is also necessary to thoughtfully construct the format of presenting the tasks to the participants, because the format may have additional effects on performance.

Steps to scoring interviews using the Model of Hierarchical Complexity

The Hierarchical Complexity Scoring System (HCSS) entails several steps for assessing performance on a task:
1.	Transcribe the interview or the material and put it onto a disk in a file.  The file should be continuous.  There should be a participant number, age and sex.  The interview should be recorded verbatim.  The analysis follows the interview.  It is done systematically.
2.	Divide the interview into individual statements about an issue in a domain.  In an interview, each statement is numbered.  These are usually propositions.  Number the propositions a1,  a2,  a3, ...  ak etc.	
3.	The individual statements are coordinated to build bigger statements.  For example, two abstract stage statements, a1, a2, may be coordinated to form one formal operational statement, f1.  Two or more formal operational statements, f1, f2.  may be coordinated into a system, s1.  Two or more systems, s1, s2, may be coordinated into a metasystem, m1.  Hence, the beginning statements may appear low in stage.  The overall statement that is being scored usually ends with the highest stage coordination.  The overall concluding statement is used to determine overall complexity for a number of reasons:
a.		The mixture of lower order items distort the score of a statement or action.
b.		All higher order statements require such lower order substatement.
c.		Hence, scores the most integrative statement or action because it is the only consistent way to score
Sometimes the last coordination comes when the participant is asked why something is not true, caring, fair, beautiful, important, etc.  The overall statement is the series of sub-statements the ends with the highest stage coordination.

3.	Statements are classified as scorable or unscorable.  A scorable statement consisted of the assertion of a solution to what the participant perceived as a problem (often an interview question) and the justification for that assertion.  Statements are considered unscorable if no justifications were given regardless of whether solutions were asserted.

4.	Scorable statements contain either positive or negative assertions.  An assertion was a positive if it affirmed some position or relationship.  A assertion was a negative if it denied or rejected a position or relationship.

5.	To determine the category in choice theory (hit, miss, correct rejection, or false alarm) into which a answer falls the correct and incorrect information is combined with the positive and negative assertions.

6.	To determine whether the statement's conclusion is correct or incorrect for the stage of reasoning the participant used to make the conclusion,

a.	the hierarchical complexity of the implicit task a participant is trying to perform was systematically abstracted;
b.	the most complex task attempted in a statement was identified;
c.	the participant's argumentation was compared with the criteria for the stage.

7.	Whether an assertion resulted from successful or unsuccessful reasoning was distinguished in two ways, depending on whether the assertion was positive or negative.  A positive assertion which affirmed a conclusion that is correct for the stage of reasoning it used was called a hit.  If the conclusion was incorrect, by the criteria of the stage, the positive assertion was labeled a false alarm.  A negative assertion which correctly rejected a conclusion that is false for the stage of reasoning used was a correct rejection.  And if the conclusion was incorrectly rejected, by the criteria of the stage, it was called a miss.

8.	The last step is to calculate an index of sensitivity and a Rasch estimate of both item (score given to a statement) difficulty and participant proclivity.  These are related to the given order of hierarchical complexity of the item.  The nominal traditional categories of domain, issue, and norm are also recorded.
													
Problems with other forms of scoring

I.	Scoring manuals have been domain specific for the most part.    Colby and Kohlberg's (1987a, 1987b) scoring system is reliable only for those dilemmas contained in the scoring manual.  The Washington Sentence Completion Test (WSCT) scoring system is reliable only for those sentence stems contained in the scoring manual.  This restriction follows from the fact that the logic of interstage relationships has not been made explicit.

a.	bootstrapping method of developing the scoring system
b.	manuals generated by applying a particular stage theory to pilot data
c.	manual consists of standard dilemmas with representative answers for each stage
d.	criterion judgments for each issue in the argument are reviewed, and matched to participants' responses
e.	the matching process requires scorer to be familiar with the manual and to make fine discriminations between arguments
f.	matches are ultimately based on the particular conceptual content of elements employed by the participant, rather than upon the relations among these elements.  Standard Issue Scoring is thus limited by being content bound.									

Definition of Common Terms across Stage Models

Concatenation:  In a concatenation, a coordinating action is performed on two or more elements.  The products of this action then become the elements and the action is performed again on the new elements.  The products of the second performance of the action are taken as elements and the action is performed again.  In theory, such a concatenation may be extended indefinitely.

Domain:  Domain describes a set of tasks that share certain qualities in common.  Such tasks are similar in both their actions and the objects acted upon (content).

Downward assimilation:  Participants sometimes give answers or solutions that are derived from a higher orders of reasoning than the one the participant uses to justify the answer.  This suggests that people may be attracted to the arguments of a given stage even if they themselves are unable to generate them.  The adoption of such arguments is downward assimilation.  For example, there is a social domain

Interview instrument:  An interview instrument is a construction such as the Heinz Dilemma.  It presents a framework around which the interview can be directed.

Order of Hierarchical Complexity of Performance:  A Participant's stage of performance is the order of hierarchical complexity of a successfully performed task.

Variable:  A variable is defined as an element with more than one possible solution (value).  Variables can be continuous (like size) or discrete (like age in years).  Discrete variables can be binary (dichotomous), like clean/dirty or new/old, or they can be multivalent, like number of dependents.

Instruments of assessment

	In assessing development there is always the question as to whether the tasks should be presented to participant as a series of problems or in an interview format.  In either case, the task may be used to examine various issues, such as moral reasoning, social-perspective taking, attachment, causality detection, etc. The participant may deal with each issue at a different stage, depending on the order of the performance on the task connected to each issue.  When the three task dimensions earlier described  are uniformly taken into account, both types of the assessment instruments, the interview and problem set,  yield equivalent stage results because stage is a single measure of the hierarchical complexity of the task that the participant is solving. As long as the task demands presented in an interview or in a problem set are the same, performance stage should not vary.  Though the format of the task can possibly add demands not related to stage of performance, the stage scores remain unchanged because the task related to stage is what must be correctly completed. Like all answers during an interview solve some implied task, all solutions to a problem series solve an issue at a stage of a particular hierarchical complexity (  Commons, Kantrowitz, et al., 1984).2

	Once a task has been constructed and administered, the scoring scheme specifies the relevant data that the researcher evaluates to produce a stage score.  Since most scoring schemes use standard assessment tools, the  implied tasks that the participant carries out at a given stage are clear to the researcher.  Once the implied tasks for each stage are uncovered from analyzing participants’ responses, a direct problem with more categorical answers is constructed.

	Interview answers are relatively less constrained than stimulus driven problems.  Problems and dilemmas always have specific contexts and ways of assessing performances.  Because interview answers are much less constrained, one would expect that the plausible responses are essentially infinite.  The validly of the answers may be great because the probability of selecting a non-self-representative response is virtually nil. On the other hand, dilemma presentation risks missing the responses that a participant would choose if less constrained by the instrument of assessment.  Hence, we suggest that both dilemma and response sets are used as measurement tools during the evaluations.  In some cultures, on the spot social discussions may also prove useful and necessary.  Keep in mind, that presenting a variety of tasks of varying orders of complexity comprise the most efficient method of accurately assessing stage.

	The two forms of measurement are mirrors of one another.  With fixed problems, one can study the processes of transition and acquisition of new stage behaviors, as well as the specific ways in which problems are solved.  Small variations may be introduced into the fixed problems administered to the participants, by varying single aspects of interest each time an assessment tool is used.

	Turning interview questions into problems

	In addition, though this involves a considerable effort,  researchers can turn the interview responses into problems.  The main difficulty lies in delineating the implied tasks and in showing how the actions that are needed to carry out a desired sequence of tasks order the more complex stage sequence.

	P

	There is a number of reasons for choosing whether or not to use interviews. Open ended interviews, for instance, create variability, delaying the operationalization of the variables that the researcher thinks are important. During such interviews the participant may or may not choose to discuss a particular topic which may be important for raising the stage score according the system based on informational references. While solving specific problems, on the other hand, the participant is more likely to obtain clearer instructions from the researcher regarding which particular topics it’s important and relevant to address. The MHC does not make this contextual distinction and any combination of open-ended questions and more specific problems may comprise an acceptable format of testing stage.

	Problems commit the researcher to an operationalization of the issues. They often do not measure the actual proficiency of the participant, because they contain demands that are not central to the concerns of the researcher and therefore may also underestimate the stage score.  As many testing manuals point out (  Anastasi, 1982), more than one form of the problem has to be generated in order to help the researcher make appropriate evaluations. So-called warm-up effects reflect the transfer of competence from one domain to the one being examined. Unless the transfer process is also examined, initial results of single items can be very misleading (  Commons & Ducheny 1979).

Examples of scoring

Attachment A.   Male, Age 8, Stage 7 Primary-Step 3 Smash, Substep 2: Over generalization: Transition to Concrete: 

	Interviewer: What happened to the toy that your cousin lost?
Child: Yeah.  He threw it up someplace.  It must've landed in a gutter or in the streets.

	Analyses:  He talked about his own point of view in an earlier response.  Now he has reversed and is talking about his cousin (who threw his toy “some place”).  He is at least considering what his cousin did and how that affected not being able to find the toy.  However there is no specific co-ordination between what the cousin did and the fact that the toy cannot be found.  The substep of transition that he is showing is over generalization.  He would blame his cousin for anything, so he does not have “correct rejection” strategies - just a large number of hits.

	Attachment N.  Male, Age 9, Stage 7 Primary-Step 3 Smash: Transitional to Concrete

	Interviewer: Why weren’t you very mad when your friend moved?  
Child: Because I did have a say in it, sort of.  I asked them to stay, but he said Oh we’ve been planning to move for about a year.

	Analyses: This child spoke about these experiences almost entirely in primary order terms.  But he made three statements approaching the concrete order (of which this is one).  This statement involves more than just himself.  He recognizes that others have points of view, but he does not really refer to their point of view.  As a result this was coded as being at the transition substep smash.  

	Attachment K. Female, Age 8, Stage 8 Concrete-Step 2 Relativism:	Transition to abstract

[When asked whether she was afraid due to the loss of her hat:]
Child:  I just wasn't afraid.  Because, I don't get afraid when I lose something.  But if it's something very, very special to me, really, really important and I always loved it, then I would be a little more scared and worried that I lost it.  I'll never see it again.

	Analysis: She seems to have two ideas: some things that are not very important do not make her get very afraid; other things that are very, very important would make her get more afraid.  She seems to be beginning to deal with different values of “importance” and of “fear” and relating them to each other, but she is not doing the relating explicitly.  She also seems to be thinking hypothetically; she does not have a specific thing in mind but says “If it’s something…” As far as what step of transition she is showing here, it is relativism: she has both points of view, but does not co-ordinate them, instead she alternates between them.

	Attachment C.  Female, Age 9, Stage 8 Concrete-Step 3 Smash: Transition to abstract

[When her cat died:]
Child:  It made me feel like I had to do something because I wasn't taking it that hard and like, the other two were.  
Analysis:  This is a story, with specified roles: self and other family members.  She is stating what sounds like a social norm, but it is not a general social norm.  It is specific: because these two people are upset, she should be upset.  This is transitional to abstract, because generalized social norms are abstract.  The transition substep is smash because the social norm is not free, it is stuck on these two people and this situation.

	Therapy Stage 10 Formal, Step 4 (0):

	Participant:  I play slowly enough to anticipate each upcoming section of the music.

	Analysis: Formal, stage 10.  An implied “if…  then” relationship logically connects two abstract variables.  The first variable is the speed of playing, and the second is the anticipation of upcoming sections of music.
	Therapy Stage 10 Formal-Step 1 Negation: Transition to Systematic

	Participant:  “He sees intimacy in a different way than me.”

	Analysis: This is a functional relation: “If he sees intimacy as ‘x’ then I see it as ‘y’ and vice-versa.”  This is a comparison between two abstract propositions.

	Therapy Stage 10 Formal-Step 3 Smash: Transition to Systematic 

	Participant: “Need to explore and respect each other’s wants and desires and function as a team [to build intimacy].”

	Analysis: The adult has a “needs to do” list of the conditions required for building intimacy.  This is multiple causation; the conditions are combined in an additive fashion at the formal stage.  The social relationship, as a system that builds intimacy, is not explained as a co-ordinated system of viewpoints that balances individual with common needs or desires.  “Explore and respect each other’s wants and desires” indicates a notion of maintaining independence and “function as a team” dependence, but the adult doesn’t account for how to co-ordinate them.  

	Attachment M.  Male, Age 41, Stage 11 Systematic-Step 0 Failure: Transition to Metasystematic

      Participant:  I lost my car, my marriage, my job, my health and a whole lot of other things at that same period of time so I can’t say, you know, it was point 0.  0 centimeters of sadness associated with losing my motorcycle.

	Analysis: Systematic because there was this whole system of losses impacting on him that he cannot point to one event or one variable as the cause of his sadness.  It is seen as transitional step 0 because it is just loss with nothing else.

	Therapy Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic

	Participant: I need to understand that John is a man of few words when it comes to love.  
Analysis: The adult is negating blame.  She is taking responsibility for constructing her view of John as an element of her overall understanding of building intimacy instead of blaming him (entirely) for blocking the process.  (Negating blame is a rejection of a formal, linear view of causality.  ) The systematic level of complexity as explained here involves a context (i.e., “when it comes to love” is a distinct context) in which the self (i.e., “I need to understand that …”) takes a view of the other’s view of love (i.e., “John is a man of few words when it comes to love”).

	Attachment D. Female, Age 41 Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic

	Interviewer: How did you come to change your mind [about your whole way of looking at life, as a result of living through the war in your country]?  Was it just the fear of death?   …  
	Participant: Well, it was the fear of death.  [and somewhat further down in the same statement:]
Things like this, you can’t have pink ideals when the situation around here is like that.  And you have to live day by day.  You just cannot plan anything not even for a week.  Lack of water, lack of sometimes bread…  

	Analyses:  The reason that she changed her mind is partly because of the fear of death.  Her whole way of looking at life changed as a result of death becoming so immediate.  This was coded as being at the Systematic stage.  She was talking about having one view of life (her first system) before the war, and having a second and totally different view of life (her second system) after the war.  Further down, she is saying you can’t choose a system, you can’t have ideals - and the war is what made her that way - she couldn’t choose the path - the war made her see things on a day-to-day basis.  So, this is a comparative statement about two systems: the way things would have been, and the way they ended up, but there is no explicit comparing.  She articulates each one, but alternates between describing one or the other; or rather she mainly describes the new system and leaves the interviewer to understand that the old system had none of this.  This was scored as relativism.  

	Attachment  E. Male, Age 23, Stage 11 Systematic-Step 2, Relativism: Transition to Metasystematic

[When asked to decide what was his greatest loss:]
	Participant: ... but that hasn’t [happened] to me yet, though, ummm, and I’m not sure I feel comfortable saying that the biggest loss I’ve ever had in my entire life, I think for some it might be easy to do.  They could tell right off, but I really have different experiences, so I could say that something was a big loss, but I’m not sure it was the most…so is that going to be…?

	Analyses:  He talks about one system: his set of experiences that have given him a particular set of losses.  He also refers to potential other systems containing the experiences of others that may produce other losses or more losses.  Because he does not co-ordinate these two together, but talks on the one hand about his experiences, and on the other hand about the experiences of others, he is at the relativism substep.  

	Therapy  Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic

	Participant:  I understand that it may not be possible to be both at the same time [to be a friend and pastor to an individual], and that what I am looking for from an individual at each particular time will be different as I am friend and pastor.

	Analysis: The adult succeeds in bringing together the two roles of friend and pastor within the same individual.   He alternates them in a systematic fashion so they do not conflict with one another.  The adult does this by placing himself with the parishioner into two different temporal contexts.  Here, there are two perspectives (from the same person) but they are not fully integrated.  The transition process is not yet complete for this stage.  

	Attachment  J.  Female, Age 25, Stage 11 Systematic-Step 3	Smash: Transition to Metasystematic

[When asked to describe her emotions after breaking up with her boyfriend:]
	Participant:  And yeah, I was angry too.  I was angry at him because…  because I knew there were some things about him that were wrong, and created these adverse reactions in me, and I didn’t really know what they were, but I was really mad at him for just being himself.

	Analyses:  She was angry for at least two reasons: he did things or had characteristics that were wrong, but there was almost something about her that had adverse reactions to the things he did.  So she is describing a kind of multi-variable system that determines her emotion, in this case, anger.  Also, just the phrase “being himself” is a systematic notion; it consists of multiple behaviors occurring at multiple times and occasions.  But this is not fully metasystematic because she does not know what is driving her nuts.  She does not fully specify either her self system enough, or the “other” system enough to have a clear sense of what is wrong.  She is at step 3, smash, in the transition to metasystematic, and most likely at substep 1.  

	Therapy Stage 11 Systematic-Step 3 Relativism:  Transition to Metasystematic

	Participant: I relax.  I breathe.  I visualize the pages ahead and the intent of playing and the feeling for the music to be expressed, all from a calm place.  I keep practicing all the little snags to smooth out.  I get plenty of rest, do Brain Gym and chi activities.  I visualize success and calmness.  I pay attention to the music and not the worry.  I breathe some more, and repeat.  I remember this and create it from out of my larger goals and purposes.
Analysis: The adult explains that he integrates success and calmness through visualizing both together.  Also, coming from a “calm place” promotes visualization, and paying attention to the music and not the worry.  Success and calmness are two “systems” that he is co-ordinating.  At this point, he appears to be overgeneralizing how the two are combined.  As systems, he explains how success depends on calmness, but not clearly how calmness depends on success.  

	Therapy Stage 12 Metasystematic-Step 4 (0)

	Participant: “To find things easily is to pursue a thought until it intersects perceptions or images of the misplaced item, to sense its presence without clearly identifying it and then it appears as if to fulfill some sense of it being there already.”

	Analysis: The adult integrates “locating” and “identifying” into a subjective, intuitive system of thinking about an object.  This is a strategy of “scanning and zeroing in” on the misplaced item.  In this process, the adult first thinks about an item, which elicits perceptions or images of it.  Then, the adult implies that he looks around for the item.  The scanning is done by first using the “ground” to sense an object intuitively, and then by disembodying the object more fully so it becomes the “figure.  “ At this point, the appearance of the object is gradually matched against the images of the thought about the object until the client becomes fully conscious of the object’s presence.  At the end of his explanation, the client is just starting to construct the related system of objective appearances and locations.  The variables of thinking about, locating, and identifying a misplaced item are fully co-ordinated into a strategy for finding misplaced items.  When looking about, he first senses the item's presence without fully identifying it.  At the end of the process, the item is fully located and identified, confirming the adult's “sense” of the item's location.

	Good Education Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic Stage

	Participant: Teacher says look, we're going to tell you things that you can write down.  And if you forget you can look them up in the textbook.  I promise I won't tell you anything that's not in a book you can look up.  And you write them down and memorize them.  And then we're going to have an exam.  And you tell us back and we'll check off whether you told us right and whether you told us everything.  And if you did, then we'll give you an A.  

	Now this is a very safe process for all concerned.  There's very little risk for the faculty.  I mean anybody can give an adequate lecture of that type.  So it won't reveal you as a bad teacher unless you just don't prepare.  It's also very safe for the student, right?  Because if you do your homework, if you don’t screw around and play tennis and waste your time, you can pass almost any course that is taught that way.  And the conspiracy is that neither party, neither the professor nor the student do anything to reveal that not much learning has gone on.  What do I mean by not much learning has gone on?  What I mean is, when you confront people with problems for which the knowledge you have transmitted is supposed to be useful later on, they can't solve them.

	Analysis: Participant includes components from an educational system based on predictable outcomes and rote memorization, as well as components from a system with risk.  Participant does not co-ordinate these components and gives as examples subsets of different ways of education and assessing students at the Kennedy School of Government.

	Good Education Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic Stage

	Participant: “I want to say one more thing about this.  One of the ways, that it is very difficult to evaluate a process like this, is to ask the victims or participants of it at the time.  It's characteristic of every experiment that we have made that…(in my view)…in every experiment that we have made that really involves learning, that the students and this includes mid-career adult students, hate it.  Or say they hate it.  They say, oh, don't do that.  That's a terrible idea.  They plead with us to teach statistics by the lecture method.  They…we asked them for example to grade each other in class performance.  We don’t do it any more.  They grade each other and that's half their course grade.  So they are responsible for each other and responsible to the classroom and we are no longer the policeman of classroom behavior.  And they have six dozen different elaborately reasoned explanations of why that's inappropriate and unethical and why we shouldn't do that.  And it's our job to grade them and so on….

	Analyses:  Participant includes components from an educational system based on traditional lecture-type teaching and teachers doing all the evaluation, as well as components from a system based on the case method and students’ evaluation of one another.  Participant produces hits at stage 12 by describing components of an ideal system of student empowerment, but overgeneralizes by relying too much on examples and not co-ordinating components into a cohesive system –
	You know I think the best example is of Ulysses tying himself to the mast.  You know, where he sails past the sirens?  Do you know the story?  He sails past the sirens and he knows that they're going to sing and lure the ship to its death so he says, “OK, sailors, stuff your ears with wax so everybody can't hear anything.  OK now tie me to the mast and unstuff my ears.  So they sail past the sirens and he hears the sirens but he can't do anything about it and they don't hear it so they keep rowing.  SO he's the only man in the world who's heard them and survived.  And he knew that when he heard them that if he didn't tie himself to the mast, right, if he didn't restrict his behavior.  Somehow we have to trust the students that have made an agreement like this.  At the same time they can not want to do this week's homework and hope that we will not listen to them.  At that moment.  Tricky problem.”
	
Moral Reasoning Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic

	Participant: Alright, a business is trying to provide some product or some service to the society.  Trying to make a profit.  The university is trying to educate people.  So obviously, the activity is going to be different…  I suppose you could say that there's a certain kind of interaction that's crucial for a university's place students and faculty which doesn't have a comparable place in other kinds of institutions. 

	Analysis:  Participant does not fully describe or co-ordinate the systems of business and education

	Good Education Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic Stage

      Participant:  “Well, there are some people who like to think of everything as a business.  It's a metaphor people use.  And maybe there's some point to that, but it also can be misleading.  There's a point to it in the sense that the university should worry about using its resources efficiently and should make sure that it's accomplishing its mission.  But it's misleading in the sense that you can't simply take the standards that apply in a business enterprise and transfer them wholesale to a university, judge it by the same standards.  You've got to realize that there's a different kind of enterprise going on.”

	Analysis:  Participant does not fully describe or co-ordinate the systems of business and education.

	Good Education Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic Stage

	Participant: Well, because, I guess these goals are sort of Aristotelian [in] that the truth is always somewhere in between.  And if we used all the resources of higher education merely to prepare for a career for example then the career wouldn't be worth having because life wouldn't be worth living.  I mean if what you learned was how to do your job, then there wouldn't be much point in doing the job because you couldn't enjoy anything else.  Conversely, if all you did was get very good at reading books and consuming experience, you wouldn't be any good to anybody else so why we should we care that you are having a good time?

	Analysis: Participant alternates between the “preparing for a career” and “getting very good at reading books and consuming experience,” without co-ordinating them.

	Good Education Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic Stage

	Participant: So if higher education is in the service of these three objectives, and if turns out to only be good for one of them, then, I'd say it won't even advance that one.  That these aren't additive.  You can't say, well, this is the Harvard School of Getting an MBA and Making a Million Dollars.  The accusation that's leveled at the Business school is that all it does is teaches people to make a lot of money and not to be good people in some other sense.  And then it turns out to be not worth it that they are making a lot of money.  Because they are deficient in these other dimensions.  So that I can't separate the quality of interdependence and talk about any one of these by themselves in isolation.

	Analysis:  Participant alternates between students to make money and becoming a good person without co-ordinating them.
	Good Education Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic Stage

	Participant: .., any large institution needs governance, which is to say that some people have to be in a position to make decisions.  So the university's no different from other institutions in that respect.  What may be characteristic about universities is that much of the authority is decentralized, much so than any other social institutions.  And that's because it's very important to allow faculty members to have a good deal of autonomy.

	Analysis:  Participant does not co-ordinate the two systematic goals of governance and autonomy  

	Good Education Stage 11 Systematic-Step 2: Transition to Metasystematic Stage

	Participant:  Whereas, if you want to call what the university does is producing a product, the product is the very interaction between students and faculty.  I: Ok.  What should that interaction be and why?  I: What quality should it have?  There are some very general features one might say as openness, willingness to explore new ideas, a willingness to question, to debate, to provide support for claims that are made...no one has monopoly or a lockhold on the truth.  And if for every individual to a considerable extent has to make a judgement for him herself as to what's true.  What makes sense.  What connects to what. 

	Analysis:  Participant produces a hit at stage 12, but basis for exclusion what is or isn’t an element in the ideal university system is not sharp			

	Good Education Stage 11 Systematic, Step 2: Transition to Metasystematic Stage

	Participant:  I think that everyone has an inclination, maybe even a desire to be certain about how the world is.  Among other inclinations.  And what you want to do is lead a person to perceive that they are adults, so you explore the beliefs that people have and you show where there are questions and why one has to keep a certain openness in regard to those things because they are questions.

	Analysis: Participant produces a hit at stage 12, but basis for exclusion what is or isn’t an element in the ideal system of training is not sharp.	

	Good Education Stage 11 Systematic, Step 3 Smash: Transition to Metasystematic Stage

	Participant:  I don't mean by that there should be some separate course in a professional school because that seems to me a way of simply keeping ethos concerns marginal …  in law there is a guiding ideal that the course system should be doing justice…  when a lawyer helps two parties write a contract, the lawyer should have some understanding of what a fair contract is and what equal bargaining is.  I: Why?  W: Those are moral notions.  Because this is a way of treating people with respect.”

	Analysis:  Participant correctly rejects reducing moral concerns to a marginal level, but under generalizes the need for respect to one professional area, the law.

	Good Education Stage 12 Metasystematic, Step 0 (4) 

	Participant:  It's important because in almost all aspects of the society where people are involved day by day, the professional business bureaucratic demands upon people--let us say for the moment, relatively educated people in the entirely conventional sense of having gotten a certain number of degrees or something--are such that they neither make nor are given the time and I think soon lose the energy to find the time to think critically of what they are doing and learn a variety of ways to absorb the mainstream lessons of their society or to work within the alternatives that the society considers respectable and suitable alternatives, as whatever, as the Democrats and the Republicans or between this college or that, or this or that job, whatever it might be.  So that there are few institutions which have the luxury as it were, or the capacity to get people to think outside the context of any one pursuit and to think about themselves and their society better with that extent distanced.

	Analysis:  Pass at Metasystematic Stage 12.  The participant clearly defines her/his ideal system of a moral education with a definitive goal of integrating the self-system with society, although this goal should be accomplished along with the student receiving a practical skill.  The participant passes at Stage 12 because they are able to describe a system which has a clearly defined method integrating these two goals– “ ideally the institutions should make one competent to work within the profession and it's too obvious to mention when you get to areas like surgery or building bridges, but the notion of competence for the law is far more complex, but there is clearly a sense of an ability to deal with the materials to grasp and to understand their various possibilities and so on.  On the other hand, simply to train people to that competence which would enable them to play the professional role and serve any set of interests, which in my mind would be an inadequate way of thinking of the role of the school as such, that is, it should have the role of making the student not only aware of the professional tradition, but critical of the professional tradition.  

	Good Education Stage 12 Metasystematic, Step 0 (4)  

	Participant:  Whereas if you stay within a narrow conception of technique and professionalism---which inevitably, you do, in any professional school.  If you're studying the bone structure of the body, you don't want to develop the forms of democratic government necessarily in the same course.  But a school that offered no opportunity to see what it was doing relationally, too, in the sense of what life the individual ought to lead or the understanding of the character of your own society.  What I think failed by making the professional think that professionalism and work consisted of the boundaries where the boundaries or the boundaries of technique taught in the school.  There were no larger issues present.

	Analysis:  Pass at Metasystematic Stage 12.  Participant successfully describes a cohesive system with sharply defined criteria for what determines a good or bad education.  Participant successfully integrates the goal of professional competence with the goal of broadening the professional student’s understanding of his/her role in society.

	Good Education Stage 12 Metasystematic, Step 0 (4) 

	Participant:  All are theories that rest upon different ethical assumptions of what society is and what the role of law and society is and what either has achieved which often blends into what people think it ought to achieve and people create their histories which are open to many interpretations which often, you know, consist of the way they think of what the society ought to be.  So it's just become a prevalent way of thinking about law.  To understand why the rule says one thing rather than another or the standard or the principle or why the Constitution or whatever interprets it to mean this rather than can't be understood with any strict system that extrudes history, morals, political theory, even psychology, sociology, a whole variety of perceptions that inform the lawmaker, whether you're talking with a judge or a legislator. 

	Analysis:  Pass at Metasystematic Stage 12.  Participant successfully describes a cohesive system with sharply defined criteria for what determines a good or bad legal education.  Participant successfully integrates the goal of legal competence with the goal of broadening the law student’s understanding of his/her role in society.

	Good Government Stage 12 Metasystematic, Step 0 (4)  

	Participant:  I mean there's a sense in which we respect ourselves more and respect each other to the extent we are self-governing rather than taking orders.  Um.  There is a sense a purely efficiency sense in which you tend to keep the most corrupt and most venal and most crooked out of office---at least you can bounce them from office if there's some conception of electoral or democratic control as opposed to having no control over the guys.  And there's a conception of self-development through, you know, not thinking what your life can be entirely your business and private and other people are going to run the joint so as you start to participate and you start to think more of the polity and we start to think more the society you're living and ideally, we take that as part of your responsibility to think about others who think about the society and not simply say, my domain is entirely my life.
Analysis:  Pass at Metasystematic Stage 12.  Participant successfully describes a cohesive governmental system with sharply defined criteria for what determines a good government.  Participant successfully integrates the goal of legal competence with the goal of self-development and the citizen’s responsibility to society.

	Good Government Stage 12 Metasystematic, Step 0 (4) 

	Participant:  That denied people important possibilities of participation that dictated rather than saw itself in some fashion of the community of the people themselves that did evil things, that denied respect, that humiliated, that discriminated…  
Interviewer: Why would that be evil?
	Participant: Well, you're back to fundamental, I mean, formal premises intuitively because it would create systemic official differences, discriminations among people in a way that denied to them, denied some basic notion of equal worth…  you ask why say that, why not some more than others and in many respects, some are better than others.  Morally, or in talents or in one thing or another.  But in modern terms of respect as human beings which they inherit so that would deny that, would I think true to some, hierarchically, traumatically whatever superior to others.  And deny participation to all or some and to deny respect by crushing all of those possibilities of human expression or discovery or gaining more freedom that we talked about earlier.

	Analysis:  Pass at Metasystematic Stage 12.  Participant describes a system which has sharply defines a bad versus and good government.  Participant successfully integrates the two themes of superiority in some individuals with the theme of equal respect .

	Good Education Stage 12 Metasystematic, Step 0 (4)
 
	Participant:  I'm not certain that I would want some veto power but it would probably be on the grounds similar to Constitutional grounds vetoing what Congress does.  If it violates fundamental principles.  So I'm unclear what the relationship between the president and the faculty should be, but it would move more heavily towards faculty participant to very few restraints and faculty participation going toward a great voice whether or not it would fully [allow] election of the dean and faculty committees and things of that sort.  But I think the president is functioning.  I think the faculty could get all locked up.  They could all become one thing and refuse to hire anyone outside who's not part of that model.  Or they could develop vendettas and become very destructive and drive people out on personal or ideological grounds…  It would become a closed institution rather than an open institution.   And I think part of the role of the president might be to assure that a university remains, not equally open to everything, but remains an institution in which ideas have a chance to develop and there's no formal closure to any of the competing the set of ways of thinking about a participant.

	Analysis:  Pass at Metasystematic Stage 12.  Participant describes a system which has sharply defines a bad versus and good university government.  Participant successfully integrates the two themes of openness with the theme of structure.

	Good Education Stage 12 Metasystematic, Step 0 (4) 

	Participant:  If you believe in what an institution should be, then you should act for it.  Because there's a problem when you're hierarchically in the administration under someone's dean for a lot of reasons I don't someone at MIT who's going to be publicly criticizing me.  Fax friend Peter.  I want someone's who's supporting me.  That's my team.  I gotta get things done.  Gotta count on you.  But a faculty member's not anyone's team.  A faculty member is an independent functioning human being.  It has a responsibility, I think, certainly the right, whether or not if they have the responsibility, to act in that way…  If I were vice-president or something of the sort.  In an important way.  I wouldn't stop criticizing, but I'd do it internally more than externally.

	Analysis:  Pass at Metasystematic Stage 12.  Participant successfully describes an ideal system of university government which integrates the goals of supporting the institution and being a responsible critic of same.

	Good Education Stage 12 Metasystematic, Step 0 (4) 

	Participant:  The reasons for it may lie in the subconscious, but at least we all know, it's---you know you're lying.  I think that you deny your respect for me.  You know, that really bothers me.   I: Why?   S: You're manipulating me.  You're not treating me as someone real.   And you may be doing it— if you've got a point to 
fight out on the university paper, I think that you learn early in life that you do it directly.  You don't get around it by lying.  You say, Mr.  Administrator, thanks, but no thanks or yes, I will bow to pressure if you don't want it published.  Rest assured, I'm a goody, I'm not.  If your superiors lie to you.  That's why the whole idea of censoring the thing is bad.  You learn that when you're in authority, you're going to crush someone who threatens to reveal something that's embarrassing to you.  Going to publish.  Or you say, keep your hands off our papers, if you try to censor this at all, we're going to be responsible, or we're going to try to be or whatever you're going to try to say.

	Analysis:  Pass at Metasystematic Stage 12.  Participant correctly rejects lying, particularly by those in authority, as inconsistent with an ideal university system based on respect and responsibility.

	Good Education Stage 12 Metasystematic, Step 0 (4) 

	Participant:  Although the university, I think did seem to be acting abusively-the way you put it---around the facts that I've imagined.  If it were anything more than inquiry and urging you to take great care with this story and assure that it's correct if there going to publish it and to think hard about it.  Stonewall?   I think it's a bad way to get out things.  It’s a hard way to get out of things.  You…  it’s a very…  and you're teaching a lesson by doing that.”

	Analysis:  Pass at Metasystematic Stage 12.  Participant correctly rejects lying, particularly by 
those in authority, as inconsistent with an ideal system of morality based on truth – “

	Good Education Stage 12 Metasystematic, Step 0 (4)
 
	Participant:  if it passes for acceptable for personal authority is able then, to denigrate particular constituents.  If the uneducated, the poor, the minorities, women, whatever he wants to do.  So that if it passes in the classroom, it's, uh, you know.  It's like, um, whatever, um,…  a president of this country denigrating a particular other people's for this or that or denigrating all Communists or whatever.  I: Ok S: Whatever it might be.  Or a Palestinian denigrating all Jews or an Israeli leader denigrating all Palestinians.  It legitimates that fine dilemma.  I:
Interviewer:  Why?
Participant: Because it's from a person in a position of authority.  
Interviewer: Why is that important---that he's in authority.
Participant: The legitimation because when people with formal authority speak others listen more than they do to someone they can dismiss down the street and that person gains an audience and the press has a lot more power to disseminate and you may have the power, even more significant, to write your views into some form of official conduct by your policies in the classroom.

	Analysis:  Pass at Metasystematic Stage 12.  Participant correctly rejects denigration and lack of respect, particularly by those in authority, as inconsistent with promoting and supporting an ideal system of morality based on truth. 

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	Participant:  Like you could maybe go to jail for stealing, you know.  That's a little bit like civil disobedience and Thoreau and so on, not quite the same thing, but you're standing up for the principle which human life is more important on this particular equation more than property and are willing to take the consequences even if it means that you're going to jail…  And I think that this notion of say being willing to take the consequences, if there are, such there are social consequences for, and the willingness to make the reparation to the druggist.  I mean, there's the problem of an unjust price, in this situation, there are so many complex moral issues—There are many ways in 
which you could express your willingness to comport with a social system and still not outrage you in various ways.

	Analysis:  Pass at metasystematic Stage 12.  Participant successfully describes an ideal system of morality with preservation of life over property in a hierarchal value system, but integrates this theme with the theme of preserving social order.

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	Unless you say that this is an idiosyncratic occasion.  I mean, it's never going to happen again and you've got the whole human tragedy there and it's not going to hurt you to give and I suppose that I would feel in that circumstance, I'd consume all of that, yes.  I should give.  I can't bear the responsibility for not doing it.  But I'd also feel a great rage.  Why am I doing it in my society?  Why am I called upon?  Why shouldn't the society be doing it in some structured way?  Through tax systems, welfare systems, whatever it might be.  And I find it hard to believe that this would be that isolated case, you see.

	Analysis:  Pass at Metasystematic Stage 12 – Participant describes an ideal system which would meet peoples’ needs in emergencies rather than relying on the unpredictable and haphazard intervention of individuals, contrasting this ideal system with one that places an unreasonable and sometimes impossible burden on any particular individual who may be confronted with such situations.

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	Participant:  The market is not functioning by definition when you're price-gouging and since most justification for prices rests on some assumed structure of competitiveness applying demand and if you don't like my drug prices, go to someone else, if you don't like my towing service in the road, go to someone else.  All of that collapses when you're broken down on the road and I'm the only towing company that's willing to come out and get you for a thousand bucks and you're the only person with this rating on them--that's all.  We're talking about what an economist would call a monopolist situation and monopoly pricing raises serious issues with respect to the public good.  Why give one person all that power to profit self at the expense of much public suffering in this situation elsewhere?

	Analysis:  Pass at Metasystematic Stage 12 – Participant describes an ideal economic system devoted to the public good, and correctly rejects any monopolistic system as being inconsistent with such a system.

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	Participant:  Entrepreneurial skill, running out of the game, taking a risk, all of those seem, simply on a utilitarian basis, which I would come back to the diminution of welfare to the druggist seems to be a relatively trivial matter if you were still allowing some normal business and profit compared with the enormous harm you're creating for others.  If people try to justify this out of ignorance, you have whole elaborate attempts at social or political theory that been to justify one or another set of arrangements.  I think where you're dealing with, particularly in matters like health which hurt you very strongly, life and death, it seems to go back to some fundamental lack of respect or any perception of equal humanity to allow one person to appropriate so much that could be helping so many others in a vital way

	Analysis:  Pass at Metasystematic Stage 12.  The participant describes an ideal economic system that places life and health over promoting entrepreneurial skill and risk-taking.

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	Participant: He should repent and embrace him like a brother…well, he should certainly try to understand why Heinz is doing this.  I suppose that would be…or Heinz should try to get him to understand why he is doing 
this…there's always the possibility that he will understand Heinz's.  .  the death of Heinz's motivations and see why Heinz used this as a moral act and perhaps that point of view may be persuasive to him.  He can empathize with Heinz to some degree and see it as he would if it were his wife and if he didn't have the money then he may rethink the morality of his own conduct and wonder if he should not act differently in the circumstances…  Ideally, I mean,…what I'm stressing is that it would be best if we had a character that would empathetically identify with and see the dilemma and think it through by feeling and understanding the thing rather than someone who saw it clearly in terms of property rights and property right protection.
	Analysis:  Pass at Metasystematic Stage 12 – The participant describes an ideal personal moral system for the druggist based on empathy and placing life over property.

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	What right has the stranger to say Heinz, you lower your welfare by participanting yourself to a prison sentence by stealing this drug to me.  I think relationships here are vitally important the way we understand our moral responsibilities.  We have to---either, I think they are personal and individual through love or affect or one or another close, or they are systemic and social.  Handling the social problem as a whole.  I find it very hard to work out the moral in between.  Between the, you know, two, four, eight, twenty, whatever they are, people who would be within our intimate circle and the millions and millions and millions who might be making claims on us.
Analysis:  Pass at Metasystem Stage 12.  Participant describes an ideal system which would integrate the two themes of personal and social responsibility.

	Moral Reasoning Stage 12 Metasystematic, Step 0 (4) 

	Participant: The doctor inevitably gives part of his welfare to patients maybe by subjecting self to disease by working with a communicable disease pairing.  That's part of the effort of helping.  Suppose the doctor simply asks for the four thousand dollars to pay for the drug.  It's hard for me again to see why individually, this doctor is under this responsibility with this person and why not the same responsibility to every other poor patient who comes in.  I think the doctor is under an obligation, I would say, to work toward some system which will make it possible for the people he's seeing to have a possibility of getting these jobs or getting help.  And again, I don't think that he has to expose his whole welfare and bankbook and family or whatever, to the demands of individual patients.  And I wouldn't know again where it stopped.  to I think that there are limits to what his sense of responsibility is.

	Analysis:  Pass at Metasystematic Stage 12 – The participant describes an ideal system integrating the doctors’ personal moral responsibility for providing some of her/his patients with free services with the general responsibility of society for providing medical treatment for all.

Table 1
Order of Hierarchical Complexity	Name	Example	
0	Calculatory	Simple Machine Arithmetic on 0's and 1's	
1	Sensory & Motor	Either seeing circles, squares, etc. or instead, touching them.  O   #	
2	Circular Sensory- Motor	Reaching and grasping a circle or square.  O   #	
3	Sensory-Motor	A class of filled in squares may be formed  # # # # #	
4	Nominal 	That class may be named, “Squares”	
5	Sentential	The numbers, 1, 2, 3, 4, 5 may be said in order	
6	Preoperational	The objects in row 5 may be counted.  The last count called 5, five, cinco, etc
	* * * * *				# # # # #
	O O O O O			# / "} Q	
7	Primary	There are behaviors that act on such classes that we call simple arithmetic operations
	1 + 3	=  4			
	5 + 15	= 20
	5(4) 	= 20		
	5(3)	= 15
	5(1)	=   5	
8	Concrete	There are behaviors that order the simple arithmetic behaviors when multiplying a sum by a number.  Such distributive behaviors require the simple arithmetic behavior as a prerequisite, not just a precursor
5(1 + 3) = 5(1) + 5(3)  =  5 + 15 = 20	
9	Abstract	All the forms of five in the five rows in the example are equivalent in value, x = 5.  Forming class based on abstract feature	
10	Formal	The general left hand distributive relation is
x * (y + z) = (x * y) + (x * z)	
11	Systematic	The right hand distribution law is not true for numbers but is true for proportions and sets.
x + (y * z) = (x * y)  + (x * z)	
x   (y   z) = (x   y)   (x   z)		
12	Metasystematic	The system of propositional logic and elementary set theory are isomorphic
       x &  (y or z) = (x & y) or (x & z) Logic
     x     (y     z) = (x    y)    (x    z) Sets
      T(False)   φ    Empty set
      T(True)    Ω    Universal set	

Table 2 Stages described in the Model of Hierarchical Complexity

Order or Stage	What they do	How they do it	End result		
0	calculatory	Exact–no generalization 	Human made program manipulate 0, 1 	None	
1	sensory & motor	Discriminate in a rote fashion, stimuli generalization, move   	Move limbs, lips, eyes, head  View objects and movement	Discriminative and conditioned stimuli	
2	circular sensory-motor	Form open-ended classes   	Reach, touch, grab, shake objects, babble	Open ended classes, phonemes	
3	sensory-motor	Form concepts	Respond to stimuli in a class successfully	Morphemes, concepts	
4	nominal	Find relations among concepts  Use names	Use names and other words as successful commands	Single words: ejaculatives & exclamations, verbs, nouns, number names, letter names	
5	sentential	Imitate and acquire sequences  Follows short sequential acts	Generalize match-dependent task actions.  Chain words	Pronouns: my, mine, I; yours, you; we, ours; they, them	
6	preoperational	Make simple deductions Follows lists of sequential acts  Tell stories	Count random events and objects  Combine numbers and simple propositions	Connectives: as, when, then, why, before; products of simple operations	
7	primary	Simple logical deduction and empirical rules involving time sequence  Simple arithmetic	Adds, subtracts, multiplies, divides, counts, proves, does series of tasks on own	Times, places, counts acts, actors, arithmetic outcome from calculation 	
8	concrete	Carry out full arithmetic, form cliques, plan deals	Does long division, follows complex social rules, takes and coordinates perspective of other and self	Interrelations, social events, what happened among others, reasonable deals, 	
9	abstract	Discriminate variables such as Stereotypes; logical quantification; (none, some, all)	Form variables out of finite classes  Make and quantify propositions	Variable time, place, act, actor, state, type; quantifiers  (all, none, some); categorical assertions (e.g. “We all die ")	
10	formal	Argue using empirical or logical evidence  Logic is linear, 1 dimensional 	Solve problems with one unknown using algebra, logic and empiricism	Relationships are formed out of variables; words: linear, logical, one dimensional, if then, thus, therefore, because; correct scientific solutions	
11	systematic	Construct multivariate systems and matrices  	Coordinates more than one variable as input  Consider relationships in contexts   	Events and concepts situated in a multivariate context;  systems are formed out of relations; systems: legal, societal, corporate, economic, national	
12	metasystematic	Construct multi-systems and metasystems out of disparate systems	Create supersystems out of systems Compare systems and perspectives Name properties of systems: e.g. homomorphic, isomorphic, complete, consistent, commensurable	Supersystems and metasystems are formed out of systems of relationships	
13	paradigmatic	Fit metasystems together to form new paradigms	Synthesize metasystems of 	Paradigms are formed out of multiple metasystems 	

14	cross-paradigmatic	Fit paradigms together to form new fields	Form new fields by crossing paradigms	New fields are formed out of multiple paradigms	

Table 3  The Model of Hierarchical Complexity and Skill Theory (Fischer, 1980) have ordered problem-solving tasks of various kinds, including:

Social perspective-taking Commons & Rodriguez, 1990; 1993)
Informed consent Commons & Rodriguez, 1990, 1993).
Attachment and Loss Commons, 1991; Miller & Lee, 1998)
Workplace organization (Bowman, 1996a; 1996b)
Workplace culture (Commons, Krause, Fayer, & Meaney, 1993)
Political development ( Sonnert & Commons, 1994)
Leadership before and after crises ( Oliver, 2004)
Honesty and Kindness ( Lamborn, Fischer & Pipp, 1994) 
Relationships ( Cheryl Armon, 1984a, 1984b)
Good Work ( Cheryl Armon, 1993)
Good Education ( Dawson, 1998)
Good interpersonal (Armon, 1990)
Views of the “good life” (Armon, 1984c; Danaher, 1993; Dawson, 2000; Lam, 1994)
Evaluative reasoning (Dawson, 1998)
Epistemology (Kitchener & King, 1990; Kitchener & Fischer, 1990)
Moral Judgment,.Armon & Dawson, 1997; Dawson, 2000)	Language stages (Commons, et. al., 2004)
Writing (DeVos & Commons, unpublished manuscript)
Algebra (Commons, in preparation)
Music (Beethoven) (Funk, 1990)
Physics tasks (Inhelder & Piaget, 1958)
Four Story problem (Commons, Richards & Kuhn, 1982; Kallio & Helkama, 1991)
Balance beam and pendulum ( Commons, Goodheart, & Bresette, 1995)
Spirituality ( Miller & Cook-Greuter, 2000)
Atheism ( Nicholas Commons-Miller, in preparation)
Animal stages ( Commons and Miller, in press)
Contingencies of reinforcement ( Commons, in preparation)
Hominid Empathy ( Commons & Wolfsont, 2002)
Hominid Tools Making ( Commons & Miller 2004)
Counselor stages ( Lovell, 2004)
Loevinger’s Sentence Completion task ( Cook-Greuter, 1990)
Informed consent ( Commons, Rodriguez, Cyr, Gutheil et. al., in preparation)
Report patient’s prior crimes ( Commons, Lee, Gutheil, et. al., 1995)
Orienteering ( Commons, in preparation)	

Table 3b (LoCicero, Personal Communication, 2005)

Model of Hierarchical Complexity	Kegan Stages	Kegan Descriptions	WWK Descriptions	Womens’ Ways of Knowing Stages	
Stage 9
Abstract	Stage 3: Interpersonal	Able to coordinate own needs and interests with others; unable to prioritize. 	Distrusts conventional authorities; trusts experience, intuition	Position 3: Subjective	
Stage 10 Formal	3-4 Transition	Shaky sense of self-as-authority, self-sufficiency.	 Beginning to appreciate objectivity; procedures for sharing and evaluating knowledge. 	3-4 Transition	
Stage 11 Systematic	Stage 4: Institutional	Self-sufficiency, authority. Identification secure. Knows where one stands.	Concerned with methods and procedures for obtaining and sharing knowledge. Interested in objectivity. Learns by doubting game (separate) or believing game (connected).		Position 4: Procedural 
(Separate and Connected)	
	4-5 Transition

	Sees limits of self-sufficiency. Somewhat aware of isolation. Moving towards intimacy and directness.	Beginning to see self and others as creators of knowledge. 
	4-5 Transition	
Stage 12 Meta-systematic	Stage 5: Interindivid-ual	Relates to others directly. Not role-bound. Intimacy. Recognizes reality as co-constructed and self as continuously created by multiple relationships.	Knowledge as situated. Integration of subjective and procedural.  Sees self and others as creating and co-creating knowledge.	Stage 5: Constructed	

Table 3c 

Table 11.2
Comparative Table of Concorded Theories of Early to Postformal Stages

Researchers	Abstract	Formal	Systematic	Meta-systematic	Paradigmatic	Cross-Paradigmatic	Transcen-dental	
 Commons & Richards (1984)	9 (= 4a) 	10 ( = 4b)	11 ( = 5a)	  12 ( = 5b)	 13 ( = 6a)	  14 ( = 6b)		
 Sonnert & Commons (1994)	group	bureaucratic	institutional	universal	dialogical			
 Inhelder & Piaget (1958)	formal III-A	formal III-B	postformal	polyvalent logic; systems of systems				
  Fischer, Hand, & Russell (1984)	7	8	9	10				
 Sternberg  (1984)		first-order relational reasoning		second-order relational reasoning				
 Kohlberg (1981)	3 mutuality	3/4	4 social system	5 prior rights/ social contract
6 universal ethical principles				
 Benack (1994)	4	5	6	7				
 Pascual-Leone (1983)	late concrete	formal and late concrete	pre-dialectical	dialectical			transcendental	
 Armon (1984)	3 affective mutuality	3/4	4 individuality	5 autonomy		6 universal categories		
 Powell (1984)	early formal	formal	stage 4a/ interactive empathy	category operations [?]				
 Labouvie-Vief (1984) 		intra-systematic	inter-systematic	autonomous				
 Arlin (1984)	3a low formal (problem- solving)	3b high formal	4a postformal
(problem-finding)	4b relativism of thought
4c over-generalization, 4d displace-ment of concepts	4e late postformal (dialectical)			
 Sinnott (1984)		formal	relativistic/ relativize systems, metalevel rules	unified theory: interpretation of contradict-tory levels				
 Basseches (1984)	phase 1b: formal early foundations	phase 2 intermediate  dialectical schemes	phase 3: 2 out of 3 clusters of advanced dialectical schemes	4. advanced dialectical thinking				
 Koplowitz (1984)		formal	systems	general systems		unitary concepts		
 King & Kitchener (2002)	4	5	6	7	       	         		
 Torbert (1994)	diplomat	technician	achiever	ironist				
 Kegan (1994)	3:interpersonal	3/4 	4: institutional	5 interindividual				
 Loevinger (1998)	conformist-conscientious	conscientious	individualistic	autonomous integrated 1				
 Cook-Greuter (1990)	3/4	4	4/5	5	5/6		6	
 Gray (1999, June, personal comm.)	early formal	formal	systematic	metasystematic				
 Bond (1999, June, personal comm.)	early formal	formal	systematic	metasystematic				
 Dawson (1998)	9	10	11	12	13	14		
 Kallio (1991, 1995)	formal 1	formal 2	formal 3 generalized formal	postformal				
  Demetriou (1990; 1995)								
 Broughton (1977; 1984)	3: person vs. inner self	4. dualist or positivist; cynical, mechanistic	5. inner observer differentiated from ego	6. mind & body experiences of an integrated self				
Belenky, Clinchy, Goldberger,  Tarule women’s ways of knowing (1997)		position 3 subjective	transition	position 4 procedural separate and connected)	position 5 constructed				

Part II

Designing Task Sequences

Michael Lamport Commons

Part II consists of two major sections.  The first section covers the analyses and construction of tasks sequences for people who can talk or read..  The second section covers the same for other animals.

10/01/2005 12:46:37
53:19:12;06-02-05-20 (Sunday, February 6, 2005; 12:19:53 pm)
55:14:15;16-02-05-20 (Wednesday, February 16, 2005; 3:14:55 pm)

Draft

Section A: Analysing Tasks and Constructing Tasks Sequences Requiring Talking Or Reading 

	Constructing tasks is probably the activity that defines most of the important creative aspects of a research project.  It is one of the most likely places for creativity to enter.  Most people use other people’s measures and tasks.  This is useful if there is a really adequate tool out there and that it matches the present purposes of the research.  But tasks sequences have much more face validity and allow for a psychophysical use of Rasch Analysis to check the adequacy of the task sequence and items therein using the model of hierarchical complexity   The form of the model of hierarchical complexity used here is from Commons and Pekker, (submitted;   Commons, Pekker, & Adams, in press).

	The idea for using task sequences was suggested by Kurt W. Fischer (Fischer & Bidell, 1998; Fischer, Bullock, 1981; Fischer, Knight & Van Parys, 1993).  Fischer pointed out that it would be efficient to have a sequence of tasks, each requiring a given stage.  The tasks would then be presented from easy to hard.  One could then determine the stage of performance with the content in the task by just seeing what was the highest stage task completed.  Two early sets of task series was created by Commons and Goodheart in the early 1990s.  They consisted of the Balance Beam and the Laundry series of problems.  They were administered by Commons.  Portions of a set of resulting papers were presented by Goodheart, Dawson, Draney & Commons (1997); Commons, Richards, Trudeau, Dawson, and Goodheart (1997) and the same authors at The Association for Behavior Analysis in May 1994 in Atlanta.  

	The method of constructing tasks has historically been called bootstrapping.  One goes back and forth from what one knows about the theoretical basis of a task and task sequence and what one knows about actual performance of an organism.  I just think things through and ask myself what I want to know.  In scoring verbal performances, one has to infer the task that the person is attempting to solve.  With task sequences, one just has to know whether the task was successfully accomplished.  One also should check to make sure the task one order of complexity down was successfully solved and the task above was not.  A Rasch analysis will show if one has made an error in constructing the items because the rasch scored will not be in the order predicted by the hierarchical complexity of the tasks.

	With people, I then make an interview that asks directly about the issues I want to know.  I interview just a very few people trying out various questions to make sure they get at what I am interested in. one starts with interviews to find out how they answer general questions or one reads interviews on knowledge collect by others.  With animals, observe, watch videos or film or read accounts.  Then think of tasks I might designs.

	For people, one then looks over the trial questions and the participants answers.  One then looks for the most hierarchically complex ones.  If one has a metasystematic order task then, one reconstructs it into the systems that are coordinated to form the metasystems, and the relations that form the systems, and the variables that form the relationships among the variables.  One roughly constructs stories with just the concrete items that are generalized into the variables, and then just containing variables without any logic or empiricism.  One then tries to build a story that has just one system and maybe one; their relationship but is missing the second full system.  Finally, One builds back up to a story that has two systems  and are coordinated.  Then one goes back to the same participants and give it to them.  Ask them to write down the reasons for their answers.  Revise to make things clearer.

	Make up three sets of items, usually 1 item per stage for each set.  Use 6 alternative.  Then give to about 30 participants and do a Rasch Analysis.  Try to figure out what goes wrong with the items that are out of sequence or have large infit errors.  Sometimes the items at the ends of the scale have large infit errors.  This is probably OK.

	My background in task design grows out of three traditions, experimental psychology, especially psychophysics, behaviour analysis, and psychometrics, in particular   Rasch (1980) analysis.  There are a few rules.  Identify the domain, subdomain and general task.  For example, in the multi-story ordering problem, the first task designed was the four story problem (  Commons, Richards & Kuhn, 1982).  Think through how you want the problems answered.

1.	 	There must be the same number of answers for all the different order of hierarchical complexity problems.
2.	 	Are you going to have exact answers (multiple choice and ratings) or open ended?
3.	 	Do you want each exact answer to represent a solution that works at some given stage?
4.	 	Can you get the number of stages in the sequence down to 6, because this makes it easier to do a Rasch analysis? 
5.	 	 Do you want to use a rating scale?  Can you use a scale from 1 to 6?  A Rasch analysis will work on rating scales because participants randomly answer above the stage of their understanding.  They do not show preferences and do not differentiate among the answers above their stage.
6.	 	Do not use a ranking scale, however.
7.	 	Make sure the questions are as 
a.	 
	simple
b.	 
	familiar
c.	 
	easy as possible for that order of complexity.
8.	 	If one would like to increase variability, do so systematically.
9.	 	Number the increases in horizontal complexity so that variable can be kept track of.
a.	 
	First, give the problem a unique number.
b.	 
	Then, after a period, dash or colon, put the hierarchical complexity number.
c.	 
	Then, put the horizontal complexity number.  This is for the key, to keep track of what is happening with the different problems.  Carefully try to score the order of hierarchical complexity of the task.  Then extend down and up in order of hierarchical complexity.  Score each task carefully as one creates them.  Is the higher order task a)  Defined in terms of the lower order task actions; b) Does it organize those task actions; Is that organization non-arbitrary?  Extend this test this up and down two orders.  Try them out on a few participants.  How do they do?  Are there any anomalies?  After correcting the problems, give it to at least 50 people.  Then do a Rasch Analysis

The structure of adult developmental stage problems

31:28:08;13-01-05-20 (Thursday, January 13, 2005; 8:28:31 am)

	The history of these problem sequences starts with the four story of Commons and Richards (  Commons, Richards & Kuhn, 1982); and Livia problems of   Kuhn, Pennington, and Leadbeater (1983).  Rodriguez and I had been working on Selman and Social-Perspective Taking for his Qualifying paper (  Rodriguez, 1989).  The four story problem was at the metasystematic order.  We had produced a sequence of little stories at each stage after Fischer’s suggestion to have one task per stage.  We then partially illustrated the stages of Social Perspective taking in High School.  This was published as a response to Moshman in Developmental Review.  For his dissertation, Rodriguez (1993) thought Doctor Patient sequence with increasing in informed consent would better fit the Program in Psychiatry and the Law.  The sequence had the following structure which Commons generalized to the laundry and balance beam problems. 

Abstract Order

Start with variable v1, v2, v3, v4, v5, v6,   These are used at the abstract stage usages 

Concrete Order

One may specify the variable producing concrete instances, events, places, actors, etc

c1, c2, c3, c4, c5, c6,   These being actual specified events

So the stories or vignette have to have at least 4 of these elements plus and lead in and outcome.

At the abstract order, these will all be variables and the values of the variable will make it clear they come from variables

Formal

Now at this order, just one relationship, vnRn + 1, is operative although other non-related variables may be present

v1 Rv2, v3, v4, v5, v6, 

Systematic

2 or more relationships between variables which form a system

v1R1v2, v3R2v4, v5, v6,

Metasystematic

(v1R1v2) R3(v3R2v4), v5, v6,

Paradigmatic

Ask them to combine two metasystems.  Attempts to combine do not work
The problems do not have a consistent or complete representation of the Metasystematic

Sections B:  Using Hierarchical Complexity to Create and Analyze Tasks to Compare Animal Cognition 

A scoring manual for scoring task hierarchical complexity

	There is a great deal of work on how to score the stage of responses to interview on all manner of things.  The Model of Hierarchical Complexity  (  Commons & Pekker, submitted;   Commons, Trudeau et al 1998) allows for interspecies comparisons.  The Model of Hierarchical Complexity of tasks allows for the scoring of the tasks themselves rather than the performances on those tasks.  Scoring a task means to find out its order of hierarchical complexity.  It does so by taking the actions and tasks that machines, animals and humans engage in, and putting them into an order based upon how hierarchically complex they are.  

	Using the axioms (Commons & Pekker, submitted), the order of hierarchical complexity is obtained by counting the number of coordinations that the action must perform on each lower order action 1) that it defined in terms of and 2) That it organizes until one reaches a set of elementary order actions.  3) The organizing of the lower order actions has to be shown to be non-arbitrary.  This later condition keeps the organizing of action to form just a chain of behavioral tasks.  Also, the non-arbitrary condition means that the order of lower order actions may match the requirements of the real world in that manner the higher order task action will actually accomplish something.  It also suggests that we look into real world actions and what they accomplish to find to what sequence of actions that a  particular task action might belong (in response to a question of Meredith Root-Bernstein, Personal communication, January 18, 2005).  Order of hierarchical complexity is a equally spaced ordinal scale.  Order of hierarchical complexity is absolute, and independent of species, particular task, context and content.  Stage of performance has the same name and number as the corresponding order of hierarchical complexity of the task it correctly completes.
 
	The reason one might like to score for their order of hierarchical complexity these tasks is simple.  If the hierarchical complexity of tasks is known, then becomes possible to compare different animals and species as to their stage of performance.  The process is as follows
  
1.	Find the highest order of hierarchical complexity of a task a given species of animals can do.  This should be a task that most favors the animal.  Favoring means training the animal, using a niche, sensory and response modality that favors the animal.

2.	One can then get not only a profile of that animals strengths but one can see how inter-correlated those strengths are.

3.	One can compare different animals best performances on the task with the highest order of hierarchical complexity they can do.

This will yield an interesting metric -- one that order species by the order of hierarchical of complexity of tasks they successfully address.

Examples of Finding the Order of Hierarchical Complexity

	A task supplied by Tony Dickenson (personal communication, March 28, 2004) is scored next.  Dickenson has been conducting studies involving human (adult and infant), Cebus monkeys and pigeons.    Dickenson and McGonigle (1996) have shown the monkeys successfully exhaustively search  hierarchically organized arrays of up to and including 12 items, simultaneously displayed on an interactive touch screen.  One starts with the most hierarchically complex task and then breaks that overall task in to subtasks, then breaks the subtasks into subsubtasks, etc.  Later one examines the task one order above.  The task that roughly matches the way the monkeys do it requires Preoperational order 6 action.  Counting objects in an array coordinates counting with pointing making it preoperational.  Counting organizes Sentential order 5 representations by stringing together those representations in a fixed manner.  The representations of the objects are of the Nominal order 4.  For example, given a sequence of knocks or beeps or flashes of light, people accurately and quickly tell how many there are, up to five or six (  Davis & Perusse, 1988).  If the items were randomly organized, the counting task would be a primary order 7 task.  But it is preoperational (order 6, preoperational) because of the hierarchically organized arrays.   In humans, placing objects in an order (transitive) would also be at the sentential order 5.  The organization goes like this.   Placing objects in order 5.  Matching the counts to sequenced objects would be order 6.  Doing the same and keeping track of what has been counted is 7.  Naming the random array that has been counted forms a class that is names, order 8.

	For pigeons, they are not counting in the sense that was just described.  For them, the tasks is subitizing.   Subitization is the process of directly perceiving the number of a collection of items, as distinct from counting.  Humans may subsidize up to around seven objects.  For example, if one makes the objects vary greatly in one or more of the following, size, position, type of object and density, one can ruin their performance.  They are discriminating one or more various dimensions of the array but not counting.  They may have the concept of amount (not number), which would be sensory-motor order 3 task.

	That can be either a order 2 (circular sensory motor) task if they are using area or one order higher (order 3 sensory motor).  In humans, subitizing is usually an order 3 task because small numbers of objects are directly appreciated as a concept.

	At three or four days, a neonate discriminates between collections of two and three items (  Antell & Keating, 1983).  There are certain conditions under which they distinguish three items from four (  Strauss & Curtis, 1981;   van Loosbroek & Smitsman, 1990).  This is a sensory or motor order 1 task.	

Horizontal Complexity

	The horizontal complexity varies in counting tasks as well in other tasks.  Think about how many bits the numbers represent.  Counting to 2 is one bit, 4 is 2 bits, 8 is three bits, 12 is 3.6 bits, 16 is 4 bits, 32 is 5 bits. Dickenson (personal communication, 2004) says that pigeons typically achieved with success at 4-5 items, with 2-3 categorical exemplars.  Human adults topped out at 25-items, with young children coping with between 4-9 depending upon their age. 

Other Things Affect Task Difficulty

	In comparative animal cognition, one might like to extend the use of hierarchical complexity and the stage to a variety of tasks.   It makes it possible to get a pretty good profile on how well animals do in various tasks and various niches with various modalities, sensory and motor.  There are other things that need to be measured as well.  These might include the following: 

1)	Other forms of complexity, mainly horizontal complexity
2) 	How many pretraining steps, training steps and testing steps are there 
3) 	What kind of representations are available to the organism.  Do the organisms have to learn the representations or do they already know them?  So familiarity with the materials has shown to be an extremely important non-hierarchical complexity predictor of difficulty. 
4)	Other things that should be measured have to do with how complex the visual display is.  The spacial frequency of the display might be a variable.  Pattern discrimination of the same order of hierarchical complexity are influence by the spacial frequence in the patterns to be discriminated. 

	For instance, if one gives a problem, like learning how to measure the hierarchical complexity of tasks, people fail everything but the concrete order tasks.  People need actual examples and actual instances at first.  They do not need actual materials.  They can use, but either do concrete operational participants do not need actual materials, they need pictures of materials and by the abstract stage they can use descriptions of materials perfectly well.  We have done a study which we have not analyzed comparing pictures and words in formal operational tasks and we will know the answer to what the cost is for various ages because they are all the same stage problem.  

Measuring other Forms of Difficultly Using Order of Hierarchical Complexity as a Benchmark

	Lots of the things studied in animal behavior are same order of hierarchical complexity.  Because there are so many other ways of making things more difficult, the order of hierarchical complexity  is a very good source for measuring all these other dimensions of how to make things difficult.  One can use order as a metric against which to measure the other ways.  One varies the amount of the other difficulty producing variable, until it produces a decrement in performance equivalent to lower the order of complexity by one.  Then one knows that that amount is the equivalent of one order of hierarchical complexity for that organism in that situation.

	Let us say one gets, what is true about hierarchical complexity is that the sequence, the numbers and the properties of the orders of complexity are the same irrespective of task, content, method, modality, organism or anything.  They have nothing to do with the empirical world.  They are analytic truths, but that does not mean the performances are all identical at all.  In fact, the performances should be highly different in each animal depending on each of the characteristics of the tasks that do not vary in hierarchical complexity, but hierarchical complexity is so powerful that when one look across organisms and tasks, it should be, the prediction is that it should explain the most variance.  Let us say one did not know what the task was and one gave it to an organism, one did not know anything about the organism or the task or the niche or anything, one randomly selected them from a common set of tasks, the best predictor would be hierarchical complexity that they can generally do.  

The stage of performances and G

	G, The General Intelligence Factor, is found from applying factor analysis to a set of performances.  Non-primate animals most often respond successfully to lower orders of hierarchical complexity tasks.  They are clearly are more specialized at solving niche problems than organism that solve higher order tasks.  The higher the stage of performance of an organism, the more likely it is to be able to solve a variety of tasks and one thing that we find is that social animals generally do better even on arithmetic tasks than nonsocial animals.  So there is a general G and if I had to guess I would say it has to do with social communication in a flexible manner.  That the more complex the social communication that that would be the best measure of G and attempts to look at G in animals in visual spatial tasks will have a similar history as it does in humans.  It does not do very well and obviously a few things such as assortativeness in insects so that they only mate with their own kind and so forth would be highly specialized, but most organisms have to solve that problem of who to mate with and there quite often is in pretty low level organisms there is competition for mates.  A lot of it is ritualized, but we would get a much better understanding of what organisms are about if we had these profiles and we could see what kind of skills hang out with what kind of skills and which ones do not.  And we could test these hypotheses, but to me that is what comparative is and there is at the moment, other than measuring sensory sensitivities and some very basic kind of sensory and perceptual processes.  That is not really about cognition, that is about sensory and perception and this is really the only measure that allows us to compare cognition. 

Tasks, Domains and Subdomains

	This is about the pattern and profile of multiple tasks with possible different  hierarchical complexity.  Now,  in a homo-centered way, the tasks have already been classified as a visual task or as a spatial task.  It would be good to further break down the domains into subdomains.  Empirically, what is needed is test for any transfer across the tasks.  If one gets better at one task, then is there an improved performance or improved acquisition on another task.  If there is, then the two tasks are related and they would be functionally in the same subdomains.  What the domains and subdomains are can vary in different organisms, different niches, different modalities.  Let me give some extreme examples.  Humans do not get to order one, the discrimination and habituation of ultra violet light or inferred light.  When it comes to many smells, humans do not get to order one.  When humans have no sensitivity, we do not get even to order one.  That part of the profile in a human would show absolute idiocy.  

	We are not ranking organisms per se for each task.  But when we want to compare organisms, one thing that we do is say we use what is called the maximum rule. **[needs to be edited from here to the next bracket]  We take the maximum complexity of any task that this animal can do in any domain with any modality.  The organism is characterized by the max of this.  That is fair to the organism because they are not compared in different domains that they do not function well in.  For example, using mice with something that involves smell and probably spatial would probably a niche that they normally function very well and flexibly.  
	One should characterize a number of things about the tasks.  One, one has to choose the simplest version of the task at a given order of complexity.  For example, if the issue is whether pigeons recognize that pictures represent objects, one have to make it very simple.  One way to make it idiotically simple, is to put the same frame around the object and the picture so if one show it to a human being they have a hard time telling them apart.  One is sure to get it correctly.  But that is a degenerate situation.  What one is really asking is how different can the representation or the picture be from the object ?   It is a much more complicated issue.  One would not know that unless one had some sort of model of how one test these things and this is such a model.
	
Level of Support and the Hierarchical Complexity of a Task That Can Be Addressed 

	Seven levels of support or lack of support that have been identified (  Commons & Miller, 1998).  

Level 1 of support is imitation, this is widely used by Fischer (  Fischer & Lazerson, 1984).  This consists of showing the animal how to do the task and then after showing them and they do it, with reinforcement.  

  	Commons, Miller and Kuhn (1982) presented such material over and over.  There was an improvement in performance.    Inhelder and Piaget (1958) were probably wrong not to present the problems over and over for at least a few trials with reinforcement for correct solutions.  There is a difference in doing it that way.  People work harder, they learn to what to attend.    Commons, Grotzer and Davidson (in preparation) found that about 75% of fifth and sixth graders performed at formal stage after three or four trials, whereas for Inhelder and Piaget, only 50% of tenth graders did so.  It is interesting even just adding reinforcement and practice, but it has to be reinforcement. It is not enough just to use feedback because we found that was a disaster with human beings. One should follow good animal behavior rules that the animals have to be reinforced appropriately and the situations have to be designed appropriately.  One can make tasks hard in a lot of stupid ways

Level 0 of support is illustrated with formal stage behavior by   Inhelder and Piaget (1958), who just presented problems and did not use any reinforcement nor with any repetition.

Level -1 is problem finding.  So finding out what one needs to do is really even harder than formal operations. 

Starting a Fire Example and Levels of Support

Let us take the task of starting a fire, which hominids did. 

Level 1: Imitating somebody starting a fire.  If somebody says I want one to build a fire and someone else says this is how I start a fire, it would be one level of support. 

Level 0:  If one wants someone to discover how to start a fire, but tells them it is possible even though that person never saw it done, that person could fool around for a long time.  But it would not be the simplest instance of a problem because there is a number of steps and a number of tools that have to be constructed.  For instance, one has to collect tinder if one are using the striking stone method.  So one would first have to collect tinder and then one would have to have the right stones and then one would have to hit them together in exactly in the right way to produce a spark and then one would have to blow on the tinder, have to be able to organize the fire, the tinder on the bottom, the small sticks next and then the bigger sticks and one would have to learn.

	A simple formal operational level 0 task would be that if one found fire one could feed it with wood.  This involves trying out various materials.  One would find out that there are some things burn and others do not.  Burning versus not burning  is one variable, plant material versus non-plant material is another.   that stones do not work, sand does not work, wet word barely works, dry wood works well, small pieces of wood burn easily, big pieces last long, etc.  Making a fire is much harder than maintaining the fire and it is very interesting that humans have evolved to love, to watch fire.  Kids love to build fires and the fact that it is reinforcing is interesting in and of itself.  

	We can use nonverbal sequences that really are very complex, especially things like starting and maintaining a fire, but by the time fire evolved there already was some language so we do not know that was the way humans did it, but one could imagine some other animal doing it, but it already started at a stage higher than a stage the chimps get to.  We could show that.

	The usual range of support varies from no support, minus one -- having them discover what the problem

	The use and maintenance of fire can be a nonverbal activity.  It could possibly be accomplished by chimpanzees but  it is not.  But  making fire is formal order task finding it the first time is a much more difficult task.

	First, no one understands that fires cook meat and tubers.  Finding the problem is -1 level of support.  The task of finding that fire cooked meat is a lot easier to eat than raw meat is formal order.  It requires finding that meat in the fire is more tender and meat out of the fire is not.  Finding meat that is tender is reinforcing.  More of the meat in a carcass may be eaten.  It is easier to get off of the animal.  The first hominid would have to discover the utility of fire, keeping themselves warm, cooking meat.  Then one might figure out that cooking certain tubers makes them edible whereas before they are not.  That way hominids with fire have a much wider supply of food.  It also means one they do not need such a large stomach because cooked tubers and meat do not require as huge a stomach as uncooked tubers and meat do to digest.  The higher the caloric value of the food, it can support a bigger brain, it can require less effort and social animals tend to have higher values of caloric food than non-social animals.  But there are solitary animals like bears that fish and eat incredibly high caloric food.  They actually skin and only eat the skin of the fish, not even the meat when they are very plentiful.  

	Developing an explanation of why cooking food entailed even less support.

For Every Task, There Are a Number of Sequences. 

  	Meredith Root-Bernstein (Personal communication, January 18, 2005) in Kacelnik laboratory presents an interesting example.  Consider the end task of making a spider web.  The important lesson here is that one must construct the task sequence, not just the end task.  For a human and for a spider, the end looks somewhat similar.  But the task belongs to two entirely different sequences.  The human probably cannot make a spider web with a tool dispensing the web material until they perform at the concrete stage.  For the human, it is a complex task.  They have to understand the structure of the web – that it has axle arms connected by straight connectors between the arms.  They have to start the tool at the arm and stop it at the next.  This is somewhat akin to model car and airplane making.  But for the spider, this is just a circular sensory motor order 2 task.  All they have to do is squirt the suff out of the correct spinaret at the back (an sensory or motor order 1 task) and coordinate it with making sure the tension is constant, using their legs to gauge that tension (a second order 1 sensory or motor task).  The web silk is made inside the spider and comes out of a spinaret at the back of the spider. This is an organ that makes the protein come out in thin fibers.  It just has to coordinate the squirting of the sild with the tension the gauge between their legs.  They start their web by sending out a thread that floats in the air until it sticks to a branch or leaf. Then more thread that sticks to other branches and also goes from the center to the sides. They keep spinning around and around in circles. Soon after, The flat round web is finished.

Preliminary analysis and scoring the crow example

Weir, Chappell & Kacelnik (2002) observed New Caledonian crows performing what will be shown to be a Preoperational order 6 action.  The way sequences are built, is from the top down.  So the preoperational actions has to organize two sequences.  Each of the sequences has to organized in a different manner the same or different elements from the nominal.

At the Preoperational order, the crows have to coordinate two sentential order 5 sequences of action that finally result in the crow using a bent wire to obtain food from a crooked clear plastic tube.

The first Sentential order 5 straight-wire tool-use sequence consists of action steps:  1, 2, 3, 4,  N
The second Sentential order 5 bent-wire tool-making sequence consists of action steps:  1, 2, 3, 4, K 

Many birds pick up twigs.  They also weave their nests with twigs.  These require circular sensory-motor actions if they do not need a concept of a nest to do it.  Instead of a concept of a nest, there is something that guides the process.

Pre actions – Where does this fit in?  What sequence does this belong to?  Is it necessary or just sufficient?

I.  The first Sentential order tool use sequence consists of two Nominal order 4 representations.   The “template” or representations are the nominal order  names. 

A.  The first Nominal action is that Crows have to represent that sticking wire in the tube to get the food.  

1.  The first concept is that stinking stick like things into cavities to get at things.   [Where two instances of concept are there here? ] 

Normally pecking at food obtains it (Circular Sensory Motor).  Pecking at the food in the tube does not obtain the food (Circular Sensory Motor).  Sticking their beak down the tube fails also  (Circular Sensory Motor).  Failing at obtaining the food in a tube, leads to seeing that an extension of their own beak is needed (Nominal).  They might have to visualize the wire as an extension of their beak. (Nominal order 4).  This picturing of an extension is a representation at the  nominal stage.

B.  The second Nominal action is that the crow must see that the wire might poke the food out of the tube.  That is, the crow needs to form a representation of the wire as a pusher of food out of the tube. (Nominal)   They do not have practice doing that. 

But seeing the food in the tube would not lead them poke with out without a representation at the nominal order 4 of some sort.

Distinguishing putting a stick into a tube to push out food from other tool use actions (putting a different stick into a hole to push out something from there) is the same as naming the two different tool use actions.]

Crows stick sticks and other things into a tube (circular sensory motor, order 2)
Crows pick up straight wires or sticks (circular sensory motor, order 2)

Crows see food in a clear plastic tube. (Sensory or motor)
Crows peck at objects with their beaks (Sensory or motor)

II.  The second tool making sentential order sequence also consists of two nominal order 4 representations.  
A.  The first nominal order actions is seeing that using a straight wire failed because of the bend in the tube  – straight wires do not go around bends.  They form a picture of the bend in the tube and a picture of the straight wire. 

B.  The second nominal order actions is that the crow then forms a representation or template for the bend in the tube.  From the representation of the bend in the tube, the crow then bends a wire.
Sentential order 5 They are
substep 3,  Sticking it into the tube, 
substep 4.  Pushing wire around a curve, and 
substep 5.  Reaching the food and getting it

3.  Sticking it into the tube;
4.  Pushing wire around a curve; 
5.  Reaching the food and getting it.  

So it is not a concept that has been formed by trial and error.  (Sensory Motor) 

The first sentential order 5 tool use sequence of actions then consists of substeps:  1, 2, 3, 4.  
Maybe trying to get the food with the beak (subsubstep 1) 
Failing leads to, seeing that an extension is needed (nominal).  
Picking up a straight wire or stick (circular sensory motor, order 2)
Sticking it into a tube (circular sensory motor, order 2)
Push the food out of the tube (circular sensory motor, order 2) producing a concept at sensory motor order 3. 

How to Construct a Task at the Next order of Complexity.

	Let us say one have a ordered complexity task and I have advised people tests, where the testing one above and they are not pushing the organism, if the organism can do one above, they have to do two above until the organism fails to do the task. To make sure that one is in the right sequence and things are not weird, one do one below.  So one do one above and one below.  And they better fail the top one.  

	One can really write up a profile and of course, much of this work has already been done and what we need to do is to mine it.  One way one make tasks more complex, is one ask organisms to reflect on the task.  Let us say there subitizing how many pellets they get.  One can ask them to tell 5 from 6 and that has been done, one can go 7 from 8 and one can try different spacing and one can have a whole thing.  That would be a first ordered test. Are these tests more about whether things, what rule they are using, Brannon (  Brannon & Terrace, 1998) suggests the following.  Let us say one are doing arithmetic and there are two rules either multiply or add and if one wanted to do a very simple concrete task one would ask them, one would ask them very simply a left press would mean I am doing addition, a right press would mean I am doing multiplication.  And since they are reflecting on the operation that is a concrete task.

Variability of stage of performance and stage

	One would expect that the variability in organisms that do not get to very high levels of order of complexity is very low.  The variability should be across tasks of different sorts and of tasks with different modalities and different sensory systems

Today is 3-27-2004.  Driving from Melbourne Florida to Orlando Airport.

Multisystems Tasks
Michael Lamport Commons and Francis Asbury Richards

For Educational Purposes

Consent Form

I, the undersigned, understand that the basic intent of these tasks is to understand reasoning and how it develops.  This is entirely voluntary and I may stop at any time. I understand that the information I have given regarding my personal history, as well as the responses I give to the tasks, will be kept strictly confidential. I understand that my identity and personal history will not be divulged in any discussion, student report, lecture, address, or publication derived from this project.

                                               	             |             |            
Signature                                      	 Month      Day      Year

Instructions

Please answer all parts of this questionnaire.  Read each section and answer the questions in the order given.  Do not go to the following section before you have finished the previous section.  Remember this is not a test of your ability as an individual.  Rather, we wish to know how adults, in general, reason about the issues presented here.  The order of answering is essential to this study about adult reasoning. 

What is most important is not only what you consider to be good, but the reasons for your choices.  Therefore, it is essential that you state your reasons why you make the choices that you make. 

Participant Number: _______
Date of Birth: month _______ day _____

Here are four stories.  After you read them, you will answer questions on which are most similar and which most different.  Use the "greater than"  symbol ">" to indicate the order of things.  For example, indicate "Brown prefers Oregon over Texas," by O > T.  Only attend to order.

1. On counter earth, Richard Reagan has been elected President of the United States.  As a gesture of gratitude to the people of his home state, California, Reagan has convinced them to leave California and either wander around with no state, or take a combination of one, two, or three of the following states: Oregon, Washington, and Indiana.  Reagan thinks that the economic value of Oregon and Washington are equal, and that the value of either is less than that of Indiana.  The boundary and passport bill that he submitted to Congress, requiring the barbed wiring and mining of the boundaries at considerable expense, did not pass.  

Instead, Congress specified that the funds asked for in that bill were to go to the states selected to be used as the states saw fit.  Only Washington and Oregon have common boundaries and, therefore, a union of these two states would receive a smaller amount of the fortification funds.  Therefore, even with the value of their combined economies, Reagan thinks that the pair Oregon and Washington is less valuable than the pairs Oregon and Indiana or Washington and Indiana.  Because Indiana makes boats that can be used on the Columbia River, Reagan thinks that the economic benefits of pairing Washington and Indian are slightly greater than the benefits of pairing Oregon and Indiana.  Reagan thinks that three states are worth more than any combination of 0, 1, or 2 states, and any 2 states are worth more than 0 or 1 states, except he cannot decide which is worth more, Indiana or the pair Oregon and Washington.  Reagan also thinks that any state is worth more than no state at all.

2. Bad Bart ambles into the local casino and converts his gold watch into chips of the following colors: silver, bronze, and gold.  Bart likes to play the one-chip candy machines.  He likes the chips in the following order:  gold better than silver, silver better than bronze, and gold better than bronze, and any chip over none.  Bart also likes to play the one-armed bandit machines which use combinations of two chips.  He likes the two-chip combinations in the following order: golds and silvers first, golds and bronzes second, and silvers and bronzes third.  With one exception, Bart knows he likes to play with any two chips over one or none; he is not sure about a gold versus a silver and a bronze.  Because Bad Bart likes to play with three chips best, there is one machine that he likes better than any other: the washing machine, and it takes a silver, a bronze, and a gold chip.
                                                                                                                                                                                                                                                                                                                                                                                     
3. In Madras, India, V. P. Vanktesh, a man of habit and variable income, has a favorite restaurant.  Although his tastes never vary, the food he can afford does.  Of the three foods the restaurant serves, he prefers curry, birani, and alu paratha, in that order.  Also, he likes curry better than alu paratha, and anything better than nothing.  When V. P. has more money he buys two dishes, except it is not known whether he would choose the curry over both birani and alu paratha.  He likes the combination of curry and birani better than curry and alu paratha, and he also likes curry and alu paratha better than birani and alu paratha, and birani and alu paratha better than curry and birani.  Although a temperate man, at festivals, given the means, he has all three dishes instead of any single dish or pair.

4. A jeweler has three boxes, the first containing different kinds of broken 18-carat gold necklaces, the second various scratched earrings, and the third different kinds of 18-carat gold pins that are broken.  He keeps the old jewelry because he occasionally uses the gold.  To get the approximate amount of gold he needs he weighs and then melts down a combination of objects.  To do the weighing he uses a simple, somewhat variable balance-beam scale.  It consists of a beam that pivots in the middle and two pans hanging from each end of the beam, equidistant from the pivot.  The beam is almost level when the pans are empty.

The jeweler can place combinations of from 0 to 3 object types into one pan, but never more than one of each object type in a pan.  Beginning with empty pans, he notices that whenever he puts any combinations containing at least one object into a pan, that pan sinks down, indicating that it is heavier than the empty pan.  Using this same method, he finds that any pin is heavier than any earring.  Necklaces are always heavier than pins.  He has discovered two rules that reduce how many combinations he needs to try to find out how the weights of the combinations are ordered.  First, he notices that if the combination in the right pan is heavier than the combination in the left pan, and a single object type not already in either of the pans is added to both pans, the right pan remains heavier than the left.  Second, if he weighs three combinations of objects, he finds that the following is always true: if the first combination is heavier than the second, and the second heavier than the third, then the first is heavier than the third.
                                                                                                                                                                                                                                                                                                                                                                                     
5.	Brown tries to help a competent patient to choose a treatment for some serious problems.  Brown offers the patient a series of treatment options.  Brown thinks that the treatments have the following order of effectiveness: surgery better than chemotherapy, chemotherapy better than pills.  No treatment is the worst.  But quite often the problems respond better to two or more treatments.  Brown thinks that combined treatments are effective in the following order: surgery and chemotherapy first, surgery and pills second, and chemotherapy and pills third.  In all cases but one, Brown thinks that combined treatments are more effective than single treatments.  Brown does not know if surgery is superior to a combination of chemotherapy and pills.  But Brown is pretty aggressive and thinks that combining all the treatments is most powerful and the most pays Brown the most.  Brown's patient thinks seriously about what Brown has said.  Feeling that Brown knows best, Brown's patient prepares to undergo the treatment.  Of course, Brown's patient gets much worse.

Now that you have read the stories and are familiar with them, answer the questions below.  Make your comparisons on the basis of properties of the orderings found in each story, in the best and most complete way that you can.

Write out all the comparisons that you can in a systematic way.  Use symbols to represent the order of things, and explain what the symbols stand for.  Also include an explanation in English.  You may want to use a chain; for example, Oregon → Texas, in addition to an ordering, Oregon > Texas.  Then explain what are the most important similarities and differences in the stories, and explain how you arrived at deciding the relative importance of these similarities and differences.  You may refer to more than one framework.

Make sure to give the strongest, most thorough, broad, inclusive, and complete explanations possible for the similarities and differences.  It is necessary that you show all of your work in forming the orders and making the comparisons, as well as your commentary in English.

How similar are the following pairs of stories?

	Totally				Totally
	Dissimilar			Similar

1.	1 & 2		1 2 3 4 5 6
2.	1 & 3		1 2 3 4 5 6
3.	1 & 4		1 2 3 4 5 6
4.	1 & 5		1 2 3 4 5 6
5.	2 & 3		1 2 3 4 5 6
6.	2 & 4		1 2 3 4 5 6
7.	2 & 5		1 2 3 4 5 6
8.	3 & 4		1 2 3 4 5 6

9.	3 & 5		1 2 3 4 5 6
10. 4 & 5		1 2 3 4 5 6

6.	Casey has such a busy practice that Casey has developed judgements as t to what part of the informed-consent procedure is important is planning treatment and avoiding getting sued.  Casey decides that getting sued is least likely if the patient has been told of the treatment options, discusses them, and is given the opportunity to consent to them.  Yet there is not always time.  Doing nothing is most dangerous.  Getting the patient to decide on the treatment seems most important, followed by discussing treatments, and last is telling the patient about the treatments.  When there is a little more time, Casey thinks that discussing the treatments and getting consent is more important than telling about the treatments and getting consent.  Also, telling about the treatments and discussing them is as important as getting consent.  Yet, even with all these strategies, Casey got sued.

7.	During the revolution, Politician Drake is up for reelection decides the only possible successful strategy is to offer the voters a series of bills.  Drake thinks that the voters support the bills to different degrees as described as follows.  There is most support for allowing the death penalty for suicide attempters because they tried murder.  There is some support for reducing the sentence of prisoners that kill death-row inmates in fair fights.  Drake thinks that least popular is to have the death sentence for letting people die who could possibly have been saved.  Yet Drake thinks that various crime packages contain these bill also have differing support.  The package of all three bills are strongest, followed by any two.  Single bills are least supported.  After offering the bills which all pass, Drake gets very depressed that there are no other things to do in life and attempts suicide.

8.	Evans is a very slopping eater and yet very compulsive about doing the laundry but extremely poor.  When a piece of clothes is stained Evans finds that using all three ingredients: bleach, booster, brightener gets the stain out most often.  Alone, bleach works better than booster, and booster better than brightener.  In pairs, bleach and booster works better than bleach and brightener.  Using none of the ingredients leaves the stains in most often.  Because bleach alone is so powerful, Evans uses it often and has dissolved a great number of pieces of clothing and now dresses in tatters.

9.	Planner Greys has to attract industry to the community.  Yet all the industries are polluting.  The nuclear test area seems most devastating yet most profitable.  Next is the smelting plant followed by a waste treatment plant.  The community is desperate but how much pollution can it take?  All three industries together produce the most pollution.  But if the waste treatment plant could process some of the smelting plant effluvia (waste), that would be less polluting than the nuclear alone and that in turn less than a combination of the nuclear and waste treatment industries.  Which is worse, the nuclear and smelting industries or the nuclear and waste treatment industries is anyone's guess.  Greys is in a pickle.  When the city council finds that the Chamber of Commerce members own stock in these industries and the land on which they are to built, Greys is fired.

10.	Under managed care, Leader Heath runs a large number of group sessions a day and is worn out.  Yet Heath knows the Groups' Followers respond increasingly more positively to Heath doing: nothing, listening, recognizing follower's concerns, empathically caring about their dilemmas.  Combinations even do better.  Listening and recognizing do worse than listening and caring, which in turn do worse than doing all three.  Heath tries to do all three but often does less.  Heath's followers complain about the groups and Heath is removed from the preferred providers list.

Now that you have read the stories and are familiar with them, answer the questions below.  Make your comparisons on the basis of properties of the orderings found in each story, in the best and most complete way that you can.

Write out all the comparisons that you can in a systematic way.  Use symbols to represent the order of things, and explain what the symbols stand for.  Also include an explanation in English.  You may want to use a chain; for example, Oregon → Texas, in addition to an ordering, Oregon > Texas.  Then explain what are the most important similarities and differences in the stories, and explain how you arrived at deciding the relative importance of these similarities and differences.  You may refer to more than one framework.

Make sure to give the strongest, most thorough, broad, inclusive, and complete explanations possible for the similarities and differences.  It is necessary that you show all of your work in forming the orders and making the comparisons, as well as your commentary in English.

11.	6 & 7		1 2 3 4 5 6
12.	6 & 8		1 2 3 4 5 6
13.	6 & 9		1 2 3 4 5 6
14.	6 & 10		1 2 3 4 5 6
15.	7 & 8		1 2 3 4 5 6
16.	7 & 9		1 2 3 4 5 6
17.	7 & 10		1 2 3 4 5 6
18.	8 & 9		1 2 3 4 5 6
19.	8 & 10		1 2 3 4 5 6
20.	9 & 10		1 2 3 4 5 6

© 1982, 1987, 1994, 2005 Michael L. Commons, Francis A. Richards, Dare Association, Inc. 234 Huron Avenue, Cambridge,

21.	How do you know that the order you have indicated is correct (Fischer)?

22.	What did you think of what Richard Reagan proposed to do with the people in the various states in story one?  What is important in that first story to you?

23.	Why is it important?

24.	What do you think of the social, and political consequences of Richard Reagan's proposal?

25.	How fair what Richard Reagan wanted to do with the states?

	Not at all 1 2 3 4 5 6 Totally

26.	Why?

27.	Why not?

28.	How might you deal with such a proposal and why would you deal in that manner (Gutheil, Appelbaum, Pauline, & Commons, in preparation)?  Sorry 

29.	How much did you care about what happened to the people in these states and especially the ones you know (Gilligan, 1982)?

	Not at all 1 2 3 4 5 6 Totally

30.	Why?

31.	Why not?

32.	To what extant do you consider this an issue (Gilligan; Kegan)?

	Not at all 1 2 3 4 5 6 Totally

33.	"Should Vanktesh have been able to choose between x and y?" ___No ___Yes

34.	Why should Vanktesh been able to choose or why not?

35.	In the fourth story about weighing jewelry, could it ever be that even with a good scale one might see (get) similar results (King & Kitchener)?  ___No ___Yes

36.	Why or Why not?

37.	What is it about weighing, taking states, chips and food preference that are all similar or different?

38.	What is their core meaning?

39.	How do you make sense of it (Kegan)?

40.	If things are not decidable is it because they are relative (Sinnott, 1984; Kramer, 1990). 

© 1982, 1987, 1994, 2005 Michael L. Commons, Francis A. Richards, Dare Association, Inc. 234 Huron Avenue, Cambridge, MA 02138-1328.

Table 3 General Description of Sequence 

	Vocalizations and words	Verbal relationships	Examples		
0	calculatory	None	None	None	
1	sensory & motor actions	Babbling (Universal)	None	None	
2	circular sensory-motor actions	Phonemes		Gestures, “aa”	
3	sensory-motor	Morphemes	Phoneme action relationships	Holding cup out and saying “wa”	
4	nominal	Single words:  ejaculatives & exclamations, verbs, nouns, number names, letter names	Relating concepts to others through a word	Says "water"	
5	sentential	Pronouns	Incomplete sentences; then complete sentences, crude dichotomies,  ordered numbers & letters	States a rule "Don't touch"	
6	preoperational		Multiple sentence stories with logically order sequenced events.  Counting materials laid out in a line	Says "The water is filling the bathtub"	
7	primary		Stories with logically order sequenced events connected solidly to reality with some specified times, places, acts, actors.  Either perspective of self or other. 	The water is filling the   bathtub so I can take a bath	
8	concrete	times, places, acts, actors	Stories about things, incidents, events, actors, actions, places in the context of the interaction between self and other. 	If I turn on the faucet, the water will run out and fill the bathtub.   My parents like it when I fill the tub myself.  They are more willing to buy me clothes when I am always clean.	
9	abstract	Variable time, place, act, actor, state, type; Quantifies (all, none, some) play an important role in the preadolescent's formulation of categorical assertions (e.g., "All teachers do that!").  	Variable time, places, acts, actors, state, type
Variables such as Stereotypes, 
Logical Quantification Words: all, none, some, never, sometimes, always, no one, somebody, everyone  

Propositions with variables	The water can run out quickly or slowly and the temperature may get colder after awhile.
All teachers do that!  
	
10	formal	Words: linear, logical, one dimensional 
	Variables connected by "if...  then" and other relations.
Empirical, or logical evidence and argument.  Relations among Variables

Single variables on input side
Empirical, or logical evidence and argument.	Turning of a faucet changes the size of the opening of the passage way and thereby controlling the amount of flow. 	
11	systematic	Systems words appear in adult's utterances: legal system, society, our company, the economy, the country.
	Systems of relations.  Relations among relationship among variables	For slow flows, amount of flow is controlled by 1) Water pressure, 2) size of faucet opening 3) smallest of pipe size to source.	
12	meta-systematic	Metalogical, meta-analytic words appear.  Properties of systems can be considered: homomorphic, isomorphic, incomplete, inconsistent system, consistent system, incomplete system, complete system, incommensurable system, orders of complexity of the system, commensurable systems, etc. 			
	Supersystems of systems	The systems that describe non-turbulent water flow and electric flow (amperes) are isomorphic.  Flow equals pressure (volts) x resistence (ohms)	
13	paradig-matic	Newly created paradigmatic words are used whether metasystems may be integrated.		There are many metasystems that belong to non-relativistic and non statistical mechanics	
14	cross-paradig-matic	Newly created word about integration of paradigms.		By crossing mathematics and physical experimentation, classical mechanics arises.	
Table 4a General Description of Hierarchy
Order or Stage	Discriminations	Vocalizations		
0	calculatory	Exact–no generalization 	none	
1	sensory & motor actions	Rote, Generalized      	Babbling (Universal)	
2	circular sensory-motor actions	Open-Ended Classes      	Phonemes	
3	sensory-motor	Concepts	Morphemes	
4	nominal	Relations among concepts	Single words:  ejaculatives & exclamations, verbs, nouns, number names, letter names	
5	sentential	Imitates and acquires sequences.  Follows short sequential acts	Pronouns: my, mine, I; yours, you; we, ours; they, them	
6	pre-operational	Simple deduction but contradiction is not excluded. Follows lists of sequential acts	Connectives: as, when, then, why, before	
7	primary	Simple logical deduction and empirical rules involving time sequence.  Simple arithmetic	Times, places, acts, actors	
8	concrete	Full arithmetic	Interactions, social events, what happened among others	
9	abstract	Discriminates variables such as Stereotypes; Logical Quantification; (all, none, some)
	Variable time, place, act, actor, state, type; Quantifies (all, none, some); Categorical assertions (e.g., "All teachers do that!").  	
10	formal	Argue using empirical or logical evidence.  Logic is linear, 1 dimensional. 	Words: linear, logical, one dimensional, if... then, thus, therefore, because.	
11	systematic	Constructs multi variate systems and matrices, coordinating more than one variable.  Events and ideas situated in a larger context. 	Systems: legal system, society, our company, the economy, the country.
	
12	metasystematic	Integrate systems to construct multisystems.  Compare systems and perspectives in a systematic way (across multiple domains).  Reflects on systems.
	Metalogical, meta-analytic.  Properties of systems named: homomorphic, isomorphic, incomplete, inconsistent or consistent, incomplete or  complete,  commensurable, incommensurable	
13	paradigmatic	Discriminate how to fit metasystems together to form new paradigms		
14	crossparadigmatic	Discriminate how to form new fields by crossing paradigms.		

Table 4b General Description of Sequence 

Kohlberg		Discriminations	Vocalizations and words	Verbal relationships	Examples	Receptive speech examples		
0	-1	calculatory	Exact–no generalization 	none	none	none	none	
1	0	sensory & motor actions	Rote, Generalized      	Babbling (Universal)	none	none	none	
2	0	circular sensory-motor actions	Open-Ended Classes      	Phonemes		gestures, “aa”	Follows one act directions 	
3	0/1	sensory-motor	Concepts	Morphemes	phoneme action relationships	Holding cup out and saying “wa”	Follows simple commands	
4	1	nominal	Relations among concepts	Single words:  ejaculatives & exclamations, verbs, nouns, number names, letter names		Says "water"	Follows simple chain of  commands such as:	
5		sentential	Imitates and acquires sequences.  Follows short sequential acts.	Pronouns	incomplete sentences; then complete sentences, crude dichotomies,  ordered numbers & letters	States a rule "Don't touch"	Follows instructions: "Turn off VCR when tape is over"	
6	½	preoperational			Multiple sentence stories with logically order sequenced events.  Counting materials laid out in a line	Says "The water is filling the bathtub"		
7	2	primary	Simple logical deduction and empirical rules involving time sequence.  Simple arithmetic	times, places, acts, actors	Stories with logically order sequenced events connected solidly to reality.	The water is filling the   bathtub so I can take a bath	Can follow directions without being supervised for an extended period of time.	
8	2/3	concrete	Full arithmetic	interactions, social events, what happened with among others.	Stories with some specified times, places, acts, actors, and the relation among the above.  Things, Incidents, Events, Actors, Actions, Places	If I turn on the faucet, the water will run out and fill the bathtub.
		
9	3	abstract	Discriminates variables such as Stereotypes; Logical Quantification; (all, none, some)
	Variable time, place, act, actor, state, type; Quantifies (all, none, some) play an important role in the preadolescent's formulation of categorical assertions (e.g., "All teachers do that!").  	Variable time, place, act, actor, state, type
Variables such as Stereotypes, 
Logical Quantification Words: all, none, some, 

Propositions with variables	The water can run out quickly or slowly
and the temperature may get colder after awhile.
All teachers do that!  
	All teachers do that!	
10	3/4	formal	Argumentation contains empirical or logical evidence.
Logical reasoning is linear, logical, one dimensional. 
	Words: linear, logical, one dimensional 
	Variables connected by "if...  then" and other relations.
Empirical, or logical evidence and argument.  Relations among Variables

Single variables on input side
Empirical, or logical evidence and argument.			
11	4	systematic	The adult constructs multi variate systems and matrices, coordinating more than one variable as input.  Events and ideas can be situated in a larger context. 	Systems words appear in adult's utterances: legal system, society, our company, the economy, the country.
	Systems of relations.  Relations among relationship among variables			
12	5	meta-systematic	Some adults can integrate systems to construct multisystems.  Comparison of systems and perspectives occurs in a systematic way (i.e., across multiple domains).
The metasystematic adult reflects on systems.
	Metalogical, meta-analytic words appear.  Properties of systems can be considered: homomorphic, isomorphic, incomplete, inconsistent system, consistent system, incomplete system, complete system, incommensurable system, orders of complexity of the system, commensurable systems, etc. 			
	Supersystems of systems			
13	6	paradig-matic	There are a few adults who can discriminate how to fit metasystems together to form new paradigms					
14	7	cross-paradig-matic	Historically there a few people who have discriminated how to form new fields by crossing paradigms.					

Levels of Support

Expanding upon   Vygotsky's (1966a, 1966b) notion of scaffolding, we introduce seven values of stimulus control (Dimension 3).  Think of these as  levels of independence of control by stimuli  autonomy in responding to stimuli.   Each level changes the relative difficulty of a task.  Level of support or demand given during problem solving derived from     Arlin (1975, 1984),   Fischer et al. (1984),   Gewirtz (1969), and   Vygotsky (1962; 1966).  The name of support is stated and how it changes the measured order of performance relative to unaided problem solving.  Then the action with respect to the participant is stated.  The explanations follow.

Table 5 Level of support
Support  Name	Change in measured complexity 	Form of support	Action	Description	
.
Manipulation	-3	Being moved though each step.	Literally being moved through each step of how to solve a problem.	Part of the stimulus is the push that guides the movement.	
Transfer of stimulus control	-2	Being told each step (direct instruction).	Do a task based on a set of verbal instructions or other direct stimuli telling one what to do.	Train a discrimination with one set of stimuli on one task.  Use the same set of stimuli to control performance in another task.  Slowly remove first set of stimuli. This is like an errorless learning procedure (Moore & Goldiamond, 1964; Terrace, 1963).	
Pervasive imitation	-1	Being shown.	Includes delayed imitation or observational learning (  Gewirtz, 1969). The imitated action may be written, depicted or otherwise reproduced.  	Fischer and Lazerson (1984) call this form of control the optimal level. 	
Direct	0	No help or support is given.  	Problem-solving or hacking (without support).	  Fischer and Lazerson (1984) call this the functional level.  Most of Piaget’s work was at this level.  	
Problem finding	1	In addition, to not getting help, one must discover a task to answer a known question.	Persons are given an issue and they are asked to give a example of a problem that reflects that issue.  	      Arlin (1975, 1977, 1984) introduced postformal complexity (systematic order) by requiring the construction of a formal-operational problem without aid or definition.	
Question finding	2	In addition, to not getting help and having to discover, one must
discover the question 	With a known phenomenon, people  find a problem and an instance in which to solve that problem.  	One has to discriminate  the phenomenon clearly enough to create and solve a problem based on that discrimination.	
Phenomenon finding	3	No direct stimulus control is possible without a description of  phenomenon. 	Discovering a new phenomenon.	No reinforcement history with phenomenon.	

Table 6a Deconstruction in the Transition Steps
Step	Sub-step	Relation	Name	Dialectical Form	
0 (4)		a = a' with b'	Temporary equilibrium point (thesis)	Previous stage synthesis does not solve all tasks.  (Deconstruction Begins) Extinction Process	
1		b					Negation or complementation (antithesis)	Negation or complementation, Inversion, or alternate thesis.  Participant forms a second synthesis of previous stage actions). (antithesis)	
2		a or b	Relativism (alternation of thesis and antithesis)	Relativism.  Alternates among thesis and antithesis.  The schemes coexist, but there no coordination of them). (alternation of thesis and antithesis)	

Table 6b Construction in the Transition Steps
3		a and b	Smash (attempts at synthesis)	The following substeps transitions in synthesis.	
	1		Hits and Excess False Alarms and Misses 	Elements from a and b are included in a non-systematic, non-coordinated manner.  Incorporates various subsets of all the possible elements.  	
	2		Hit and Excess False Alarms.	Incorporates subsets producing hits at stage n.  Basis for exclusion not sharp. Over generalization	
	3		Correct Rejections and Excess Misses	Incorporates subsets that produce correct rejections at stage n.  Produces misses.  Basis for inclusion not sharp. Under generalization	
4(0)	4	a with b	
	New temporary equilibrium (synthesis and new thesis)	New temporary equilibrium (synthesis and new thesis)

Personality and Transition in Performance Table 7

In personality, steps in adults tend to be stable but substeps are not.

Step 0	Fault finders	At low orders of performance in the social domain, this may result in antisocial behavior.  These people perceive tremendous unfairness.  People get stuck on this here because much of their current order behavior is maintained by “punishment” that reinforces the failure behavior.  They have experienced a huge drop in perceived reinforcement rate because they see the failures of the behavior of the present order to obtain what others do.  They quite often have unshakable negative depressive scripts.  	
Step 1	Nay sayers	May consist of persons who enter therapy, rebels, radicals, discontents.  They have given up their old ways.  If A is wrong, then the opposite of A is right.  There is the second largest drop in reinforcement rate because they drop their previous successful behavior from A and substitute behavior from B for it.  	
Step 2	Relativists	Clinically at its worst, they may be Narcissists (Situationalists) or Borderlines (That nothing works proves no one loves them).  In this culture, it is quite often the largest group.  They fill academia.  They stop progress by insisting that there is more than one way to look at things but cannot decide.  There is some gain in reinforcement but the conflict between whether to chose A or B produces anxiety, angst, mood swings and uncertainty about roles, values etc.	
Step 3	Movers	They are moving from smash to consolidate.  They create great trouble for themselves and others by throwing ideas and actions together in a creative haphazard way, taking a great deal of risk.  	
Step 4	Unshakables	Everything is OK if it is not OK.  They avoid conundrums, apparent contradiction, comparisons to people they look up to.  Everything is good enough.	

Layers of Contingency Setting
Michael Lamport Commons

Institutional atmosphere and Cultural atmosphere refer to the dynamic relationship between an institution and those individuals who comprise it.  Atmosphere includes the contingencies that affect individual behavior within an organization and the methods by which contingencies are set.  We define a contingency as a relationship between events (i.e., behaviors or responses) and outcomes.  Consequences that increase the likelihood of the event that they follow are termed reinforcers.  Consequences that a participant will avoid what have followed behavior are termed punishers and negative reinforcers.  What the environment contributes to behavior, we suggest, can be described in terms of contingent relations among events.  We also maintain that the reasoning of individual members within any institutional or family setting has significant bearing upon organizational atmosphere.  As reasoning develops in complexity, individuals are increasingly discriminating the perspectives of others, and of evaluating and integrating competing perspectives.  These skills are integrated into the formal and informal policymaking and policy enforcement structures of the organization.  The explicit statements of perceived organizational contingencies are referred to as verbalized causal rules or contingencies (  Commons, Kantrowitz, Buhlman, Ellis, & Grotzer, 1984).  The implicit perceptions of causality are the perceived causal rules or contingencies.

	In order to characterize atmosphere at the level of organizational macro structure, we believe that it is necessary to examine the individual contingencies embodied in atmosphere, which constitute the organizational micro structure (  Goffman, 1963).  Micro structure and macro structure are inseparable.  Atmosphere is, therefore, a characterization of the sum of individual contingencies operative within an organization and as   Kohlberg emphasized, their justifications (  Kohlberg, 1984).  Atmosphere, therefore, refers to the manner in which the institution and individuals either constrain or motivate the development of individuals and the development of the organization.

Table 8 Talk Radio Example 
Michael L. Commons, Ph.D., Patricia Roy

Whose Contingencies	Stated reasons for behavior and for conditioning	Stimulus	Behavior	Outcome	
Listener	Received some punishment	Listens to other saying they feel alienated	Calls in	Gets listened to	
Talk-Show Host	Money and Axe to Grind	Gets call	Engages Audience, Listen, Comments, Soliloquizes	Audience comment, Money	
Producer	Money and Power	Auditions, previous experience, recommendations	Set contingencies for Talk Show Host.  Informs Host of Ratings, Problems, Feedback, Studies’ Results	Talk show host behaves better, Station renews show, Producers increased price of package O.K.’D, % to producer	
Radio Station
Manager,
CEO, Board of Directors
Lumped here but could be separated	Money, Name recognition, spill over to other shows	Proposal and data from producer	Gives Contract to Producer and Host, Pays producer Advertises Show, Sells time to Advertisers	Ups ratings, Sells more advertising to adjacent shows, gets more money.	
Advertising Agency 	Contract for Advertising product.  More business.  More power, money by selling product.	Demographics of Audience, Previous experience with outlet.	Buys Air time, Put on advertisements	Contract renewed if product sales and/or name recognition improves	
Company	Increase Sales and Name recognition	Proposal, previous performance, reputation of Agency	Contract with Advertising Agency	Increase in sales, market share, profitability, name recognition	
Public	Prefers Good Products and services and lower risk for trying such, prefers to be entertained	Hears advertisement while listening to show	Buys product or Services	“Enjoys” the effects of products or services	

Table 9
Classification of organizations, and positons as minimally requiring the following stages.

Stage	Type of Organization
Activity	Positions
Person	Social
Armon
Kegan
Kohlberg
Selman	Social Structure	
Primary 7	Individual who works for others and is closely supervised	Cleaning person, Errand runner	2	Individual	
Concrete 8	Individual vendors	Individual physical laborer	2/3	Clique	
Abstract 9	Family or small number of individual's
company	Filing Clerks, Typists, Elementary School Teachers	3	Small Group	
Formal 10	Single niche company	Secretaries, Technicians, High School Teachers, Whistle Blowers	3/4	Bureaucracy	
Systematic 11	Regular small to large companies and organizations		Managers, Professionals, College Professors		4	Institution	
Metasystematic 12	Learning and developing companies and organizations (See Linda Morris) 	Innovators & Leaders, Research Scientists and Professors, Major Artists, Appellate Court Judges	5	Universal	
Paradigmatic	Organizations that reinvent themselves by adopting new paradigms		6		

Table 10: Levels of Reflection of Actions Governed by Implicit and Explicit Rules

Basis for Reasoning	Implicit
Rules
	Explicit
Rules	Tappan's Corresponding analysis as Modified by Commons:	
Action & Decision
Including Use of
Implicit Rules   			Use of Someone Else's Rules 	
Present
(Expressed Decision)
(Melioration Theory)
(No memory)			Action Controlled by someone else past rules & Action is Shadowed
(Reflective)           						
Past Action
(Expressed Decisions)
(Melioration Theory) 
Memory			Past Action
Description of Actions
Guides Action
(Reflective)
	
Prospective Action
(Expressed Decision)
(Melioration Theory)
(Memory applied)					Prospective Action
Pragmatic Description of
Action Guides Action
(Reflective)	
Prospective Action
With Explanation
(Would)			Prospective Action 
Pragmatic Explanation of
With Explanation of Value & Structure in Cognitions
(Meta-reflective)	
Prescriptive Action
(Ought or Should)			Normative: Combination
of Value & Structure in Cognitions 
(3d Order Reflective)	
Reflective Actions (How to Prescribe)
			Justification of Normative system
(4th Order Reflective)
Epistemic Cognitions

	

My current idea is that there is a simpler table.  But what kind of reflection and metacognition interacts with stage.
So the problem of refection just generating all the higher stages is maybe not a problem.  In some sense they do but once they are automatized they do not require such computational space.

References

Binder, C.  (May, 2000).  Component/composite analysis and programming from the “real world.”  Presented at the Association for Behavior Analysis, Washington D.C.
Bowman, A. K.  (1966a).  The relationship between organizational work practices and employee performance:  Through the lens of adult development.  Unpublished doctoral dissertation.  The Fielding Institute, Santa Barbara, CA.
Brogden, H. E. (1977). The Rasch model, the law of comparative judgement and additive conjoint measurement. Psychometrika, 42, 631-634. 
Campbell, R. L. (1991).  Does class inclusion have mathematical prerequisites?  Cognitive Development, 6(2), 169-194.
Campbell, R. L., & Bickhard, M. H.  (1986).  Knowing levels and developmental stages.  Human Development, 16 Entire Issue.

     Commons, M. L., Goodheart, E. A., & Bresette, L. M. with Bauer, N. F., Farrell, E. W., McCarthy, K. G., Danaher, D. L., Richards, F. A., Ellis, J. B., O'Brien, A. M., Rodriguez, J. A., and Schraeder, D.  (1995).  Formal, systematic, and metasystematic operations with a balance-beam task series: A reply to Kallio’s claim of no distinct systematic stage.  Adult Development, 2 (3), 193-199.
Commons, M. L., Krause, S. R., Fayer, G. A., & Meaney, M. (1993). Atmosphere and stage development in the workplace. In J. Demick & P. M. Miller (Eds.). Development in the workplace (pp. 199-220). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Commons, M. L., Lee, P., Gutheil, T. G., Goldman, M., Rubin, E. & Appelbaum, P. S. (1995).  Moral stage of reasoning and the misperceived "duty" to report past crimes (misprision).  International Journal of Law and Psychiatry, 18(4), 415-424.
Commons, M. L., & Miller, P. M.  (1998).  A quantitative behavior-analytic theory of development.  Mexican Journal of Experimental Analysis of Behavior. 24(2) 153-180. 
Commons, M. L., & Miller, P. A.  (2001).  A  quantitative behavioral model of developmental stage based upon hierarchical complexity theory.  Behavior Analyst Today, 2(3), 222-240.
Commons, M. L., & Pekker, A. (submitted). Hierarchical Complexity:  A Formal Theory.  Journal of Mathematical Psychology.
Commons, M. L., & Richards, F. A. (1984). A general model of stage theory. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 120-140). New York: Praeger.
Commons, M. L., & Richards, F. A.  (2002).  Organizing components into combinations: How stage transition works.  Journal of Adult Development, 9 (2): 159-177.
Commons, M. L., Richards, F. A., Trudeau, E. Dawson , T. L., & Goodheart, E. A.  (1997, March).  The Psychophysics of Stage: The General Stage Model analytically measures hierarchical task complexity and the Rasch Model statistically models corresponding stage of performance.  Presented at the International Objective Measurement Workshop, Judd Hall, University of Chicago.
Commons, M. L., Rodriguez, J. A. (1990). “Equal access" without "establishing" religion: The necessity for assessing social perspective-taking skills and institutional atmosphere. Developmental Review, 10, 323-340.
Commons, M. L., Rodriguez, J. A. (1993). The development of hierarchically complex equivalence classes. 	Psychological Record, 43, 667-697.
Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A., & Krause, S. R. (1998).  The existence of developmental stages as shown by the hierarchical complexity of tasks.  Developmental Review, 8(3), 237-278.
Commons, M. L., & White, M. S.  (2003).  A complete theory of tests for a theory of mind must consider hierarchical complexity and stage: A commentary on Anderson and Lebiere target article, The Newell Test for a theory of mind.  Behavioral and Brain Sciences, 26, 587–64, 20-21.
Cook-Greuter, S. R. (1990). Maps for living: Ego-development theory from symbiosis to conscious universal embeddedness. In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.). Adult Development: Vol. 2, Comparisons and applications of adolescent and adult developmental models (pp. 79-104). New York: Praeger.
Danaher, D.  (1994)  Sex role differences in ego and moral development: Mitigation with maturity.  Unpublished doctoral dissertation, Harvard University, Cambridge, MA.
Dawson, T. L. (2000). Moral reasoning and evaluative reasoning about the good life. Journal of Applied Measurement, 1, 372-397.
Dawson, T. L. (2002).  A comparison of three developmental stage scoring systems.  Journal of Applied Measurement. 3(2) 2002, 146-189.
Dawson, T. L., Goodheart, E. A., Draney, K., Wilson, M., Commons, M. L. (1997, March). Concrete, abstract, formal, and systematic operations as observed in a “Piagetian” balance-beam task Series.  Paper presented at the annual meeting of the AERA. Chicago.
Dawson, T. L., Commons, M., & Wilson, M. (in review). The shape of development.
Draney, K. L. (1996).  The polytomous Saltus model: A mixture model approach to the diagnosis of developmental differences. Unpublished doctoral dissertation, University of California at Berkeley, Berkeley, CA.
Fischer, G. (1968). Psychologische Test Theorie. Bern, Switzerland: Huber. 
Green, D. M. & Swets, J. A.  (1966).  Signal detection theory and psychophysics.  Huntington, NY: Krieger.
Goel. A., & Chandrasekaran, B.  (1992).  Case-Based Design: A Task Analysis In Artificial Intelligence Approaches to Engineering Design, in Tong and D. Sriram (editors). Volume II. Innovative Design, pp. 165-184.  San Diego, CA: Academic Press..
Goodheart, E. A., Dawson, T. L., Draney, K., & Commons, M. L.  (1997, March).  A Saltus analysis of developmental data from the laundry problem task series.  Presented at the International Objective Measurement Workshop, Judd Hall, University of Chicago.
Guttman, L. (1950) The basis for Scalogram analysis. In Stouffer et al. Measurement & Prediction, The American Soldier, Vol IV. New York: Wiley.
Holland, J.G. & Skinner, B.F. (1961). The analysis of behavior. New York: McGraw-Hill.
Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. New York: Basic Books.
Kauffman S. A. (1993). The origins of order: self-organization and selection in evolution.  New York: Oxford University Press. 
Keats, J. A. (1967). Test theory. Annual Review of Psychology, 18, 217-238.
Keats, J. A. (1971).  An Introduction to Quantitative Psychology. Sydney, Australia: John Wiley.
Kubina, R. M., & Morrison, R. (2000). Fluency in education. Behavior and Social Issues, 10, 83-99.
Linacre, J. M.  (2004).  Winsteps: Multiple-Choice, Rating Scale and Partial Credit Rasch Analysis. Chicago, IL: MESA  Press.
Mislevy, R. J., & Wilson, M. (1996).  Marginal maximum likelihood estimation for a psychometric model of discontinuous development. Psychometrica, 61, 41-47.
Pelton, T. W., & Bunderson, C. V. (2003). The Recovery of the Density Scale using a Stochastic Quasi-realization of Additive Conjoint Measurement. Journal of Applied Measurement. 4 (3) 269-281.
Perline, R., Wright, B.D. & Wainer, H. (1979). The Rasch model as additive conjoint measurement. Applied  Psychological Measurement, 3, 237-256.
Rasch, G. (1960). Probabilistic model for some intelligence and attainment tests.  Copenhagen, Denmark: Danish   Institute for Educational Research. Expanded edition, 1980. Chicago: University of Chicago Press.
Rasch, G. (1966). An item analysis which takes individual differences into account. British Journal of Mathematical  and Statistical Psychology, 19, 49-57
Rost, J. (2001).  The Growing Family of Rasch Models, In A. Boomsma, M. A. J. van Duijn, & T. A. B. Snijders  (Eds.).  Essays on item response theory. (Pp. 25-42).  New York: Springer-Verlag. 
Siegler, R. S. (1986). Children's thinking. Englewood-Cliffs, NJ: Prentice-Hall. 
Tomasello, M., & Farrar, M. J. (1986). Joint attention and early language. Child Development, 57, 1454-1463.
Wilson, M.  (1984).  A psychometric model of hierarchical development.  Unpublished doctoral dissertation,  University of Chicago.
Wilson, M. (1989).  Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105, 276-289.
Wolfram, S.  (2002). A new kind of science: Notes for Chapter 10: Processes of perception and analysis:  Section:    defining complexity.  (Pp. 1067- 1069).  Champaign, IL: Wolfram Media.
Wright, B. D., & Stone, M. H. (1979).  Best test design. Chicago, IL: MESA Press.

Adey, P.  (In preparation).  Promoting the development of formal operations.  Journal of Adult Development
Alexander, C. N., Druker, S. M., & Langer, E. J.  (1990).  Major issues in the exploration of adult growth.  In C. N. Alexander & E. J.  Langer (Eds.), Higher stages of human development : perspectives on adult growth  (pp. 3-34).  New York: Oxford University Press.
Anastasi, A. (1982). Psychological testing. 5th Edition.  New York, NY: Macmillan Publishers.
Andrich, D. (1988).  Rasch models for measurement. Thousand Oaks, CA: Sage Publications, Inc. 
Antell, S., & Keating, D. (1983). Perception of numerical invariance in neonates. Child Development, 54, 695-701.
Arlin, P. K. (1975).  Cognitive development in adulthood: A fifth stage?  Developmental Psychology, 11, 602-606.
Arlin, P. K.  (1977).  Piagetian operations in problem finding.  Developmental Psychology, 13, 247-298. 
Arlin, P. K. (1984).  Adolescent and adult thought:  A structural interpretation. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development. New York: Praeger.
*Armon, C. (1984). Ideals of the good life and moral judgment:  Ethical reasoning across the life span. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development (pp. 357-380). New York: Praeger.
*Armon, C. & Dawson, T. L. (1997).  Developmental trajectories in moral reasoning across the life-span. Journal of Moral Education, 26, 433-453.
*Baer, D. M. & Rosales, J. (1994, May).  Development cusps: A relevant concept for behavior analysis. Paper presented at the Association for Behavior Analysis, Atlanta, GA.
*Baldwin, J. M.  (1906).  Thoughts and things or genetic logic (3 Volumes). New  York: Macmillan.
*Baldwin, J. M.  (1906).  Social and ethical interpretations in mental development.  New York:  Macmillan.
*Baltes, P. B.  (1987).  Theoretical propositions of life-span developmental psychology: On the  dynamics between growth and decline.  Developmental Psychology, 23, 611-626.
*Basseches, M. A. (1984). Dialectical thinking as a metasystematic form of cognitive organization.  In M. L. Commons, F. A. Richards, and C. Armon (Eds.), Beyond formal operations: Vol. 1.Late adolescent and adult cognitive development. (pp. 216-238). New York: Praeger.
*Benack, S., & Basseches, M. A.  (1989).  Dialectical thinking and relativistic epistemology: their relation in adult development.  In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 1, Comparisons and applications of adolescent and adult developmental models [(pp.   ).  New York: Praeger.
*Bidell, T. R., & Fischer, K. W. (1992).  Beyond the stage debate: Action, structure, and variability in Piagetian theory and research. In, Berg, C. A. & Sternberg, R. J. (Eds.),  Intellectual development. New York:  Cambridge University Press.
*Binder, C. (2000).  Component/composite analysis and programming from the “real world..”  Presented at the association for Behavior Analysis, Washington D.C.
*Binet, A., & Simon, T.  (1905/1916).  The development of intelligence in children.  Baltimore: Williams & Wilkins.
Bond, T. G, & Fox, C. M. (2001).  Applying the Rasch model: Fundamental measurement in the human sciences.  Mahwah, NJ:  Lawrence Erlbaum Associates, Inc. 
Bowman, A. K. (1966b). Examples of task and relationship 4b, 5a, 5b statements for task performance, atmosphere, and preferred atmosphere. In M. L. Commons, E. A. Goodheart, T. L. Dawson, P. M. Miller, & D. L. Danaher, (Eds.) The general stage scoring system (GSSS). Presented at the Society for Research in Adult Development, Amherst, MA. 
*Brainerd, C. D.  (1978). Learning research and Piagetian theory. In L. S. Siegel & C. J. Brainerd (Eds.), Alternatives to Piaget, critical essays the theory (pp.   ). New York: Academic Press.
Brannon, E. M., & Terrace, H. S. (1998). Ordering of the numerosities 1 to 9 by monkeys. Science, 282, 746-749.
*Brendel, J. M., Kolbert, J. B., & Foster, V. A.  (In press).  Promoting student cognitive development.  Journal of Adult Development
*Broughton, J. M. (1984).  Not beyond formal operations but beyond Piaget. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations:  Vol. 1. Late adolescent and adult cognitive development (pp. 395-411).  New York: Praeger.
Brown, E. H.  (2004).   The "Two Cultures", Mathematics and Postmodernism.   September 10, 2004  http://people.brandeis.edu/~buchsbau/FACSEMPAPERS/2Cult.pdf.
*Campbell, R. L., & Bickhard, M. H.  (1986).  Knowing levels and developmental stages.  Contribution to Human Development, 16.
*Campbell, R. L., & Bickhard, M. H. (1987).  A deconstruction of Fodor's anticonstructivism.  Human Development, 30(1), 4, 8-59.
*Campbell, R. L., & Richie, D. M.  (1983). Problems in the theory of developmental sequences: Prerequisites and precursors.  Human Development, 26, 156-172.
*Case, R.  (1974).  Structures and strictures: Some functional limitations in the course of cognitive growth.  Cognitive Psychology, 6, 544-573.
*Case, R.  (1978).  Intellectual development from birth to adulthood: A Neo-Piagetian interpretation.  In R. Siegler (Ed.), Children's thinking: What develops?  Proceedings of the Thirteenth Annual Carnegie Symposium on Cognition.
*Case, R.  (1982).  The search for horizontal structures in children's development.  The Genetic Epistemologist, 11(3), 1-12.
*Case, R.  (1985).  Intellectual development:  Birth to adulthood.  Orlando Florida: Academic Press.
Colby, A., & Kohlberg, L. (1987a).  The measurement of moral judgement: Vol. 1.  Theoretical foundations and research validation.  New York: Cambridge.
Colby, A., & Kohlberg, L. (Eds.) (1987b).  The measurement of moral judgment:  Vol. 2.  Standard form scoring manuals. New York:  Cambridge University Press.
*Commons, M. L., & Calnek, A. D. (1984). On the empirical undecidability between the hypotheses that stage change is or is not a discrete, discontinuous, stepwise process. The Genetic Epistemologist, 14(2), 11-16.
*Commons, M.. L., Danaher, D., & Meaney, M. (June, 2000). Transition and stage in performance of Harvard University faculty and staff.  Data presented at Society for Research in Adult Development Symposium, Pace University, New York City, New York.
Commons, M. L., & Ducheny, J. R.  (1979). The relationship between perceived  density of reinforcement in a schedule sample and its reinforcement value. Paper presented at the Second Harvard Symposium on Quantitative Analyses of Behavior: Matching and Maximizing Accounts, Harvard University, Cambridge.
Commons M. L. Goodheart E. A. Danaher D. L. 1995. Scoring individual and organizational material for moral  stage using the  General Stage Scoring System (GSCSSS) and Rasch analysis. How we score almost any text or discourse in any domain for stage in ½ stages with 95% reliability and transition step. AME, New York. 
*Commons, M.L., Goodheart E. A., Danaher, D. L., Miller, P.M., Broderick, M. A., Rodriguez, J.A., Weinstein, G. (1994). Scoring text and discourse and choice for stage and transition using The General Stage Scoring System.
Commons, M. L., Goodheart, E. A., & Dawson T. L. 1997. The general hierarchical complexity scoring system (GHCSS): How to score anything. Scoring individual and organization material for hierarchical complexity of performance using the General Hierarchical Complexity Scoring System and Rasch Analysis: How to score almost any text or discourse in any domain for hierarchical complexity of performance (stage) with 95% reliability and transition steps.
*Commons, M. L., & Grotzer, T. A.  (1990).  The relationship between Piagetian and  Kohlbergian stage:  An examination of the "necessary but not sufficient relationship."  In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 205-231).  New York: Praeger.
Commons, M. L., Grotzer, T. A., & Davidson, M. N. (In preparation).  The necessity of reinforcing problem solutions for transition to formal operations: An examination of Piaget's equilibration theory of stage change.
*Commons, M. L., & Hallinan, P. W., with Fong, W., & McCarthy, K. (1990). Intelligent pattern recognition:  Hierarchical organization of concepts.  In M. L. Commons, R. J. Herrnstein, S. M. Kosslyn, & D. B. Mumford (Eds.), Quantitative analyses of behavior: Vol. 9. Computational and clinical approaches to pattern recognition and concept formation (pp. 127-154).  Hillsdale, NJ: Lawrence Erlbaum Associates.
*Commons, M. L., Jenisch, J., & Straughn, J. B.  (submitted).  Operant conditioning and the sense of free will: A response to the libertarian position.
Commons, M. L., Kantrowitz, S., Buhlman, R. A., Ellis, J. B., & Grotzer, T. A.  (1984).  Immediately detecting a causal-relation stimulus embedded in similarly appearing possible causal relations.  In M. L. Commons, M. C. Davison, J. A. Nevin, (Eds), Models of Behavior: Vol. 11, Signal Detection.  Hillsdale, NJ: Lawrence Erlbaum Associates.
*Commons, M. L., Krause, S. R., Fayer, G. A., & Meaney, M.  (1993).  Atmosphere and stage development in the workplace.  In J. Demick & P. M. Miller.  Development in the workplace (pp. 199-220).  Hillsdale, NJ, Lawrence Erlbaum: Associates.
*Commons, M. L., Lee, P., Gutheil, T. G., Goldman, M., Rubin, E. & Appelbaum, P. S. (1995).  Moral stage of reasoning and the misperceived "duty" to report past crimes (misprision).  International Journal of Law and Psychiatry., 18(4), 415-424.
Commons, M. L., & Miller, P. M. (1998).  A quantitative behavior-analytic theory of development.  Mexican Journal of Experimental Analysis of Behavior. 24(2) 153-180.
*Commons, M. L., Miller, P. M., & Kuhn, D. (1982). The relation between formal operational reasoning and academic course selection and performance among college freshmen and sophomores. Journal of Applied Developmental Psychology, 3, 1-10.
*Commons, M. L., & Richards, F. A. (1995).  Behavior analytic approach to dialectics of stage performance and stage change.  Behavioral Development, 5(2), 7-9.
Commons, M. L., & Richards, F. A. (1984a). A general model of stage theory.  In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development (pp. 120-140). New York: Praeger.
Commons, M. L., & Richards, F. A.  (1984b). Applying the general stage model. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development (pp. 141-157). New York: Praeger.
*Commons, M. L., Richards, F. A., & Armon, C. (Eds.). (1984).  Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development.  New York: Praeger. 
*Commons, M. L., Richards, F. A., & Armon, C.  (1986).  More than a "valuable scorecard."   Reply to Keating, D. B.  (1986).  Beyond Piaget: The evolving debate.  Contemporary Psychology, 30(6), 449-450.]  Contemporary Psychology, 31(6), 470-471.
*Commons, M. L., Richards, F. A., & Kuhn, D. (1982). Systematic and metasystematic reasoning: A case for a level of reasoning beyond Piaget's formal operations. Child Development, 53, 1058-1069.
Commons, M. L., & Rodriguez, J. A.  (1990).  "Equal access" without "establishing" religion:  The necessity for assessing social perspective-taking skills and institutional atmosphere.  Developmental Review, 10, 323-340.
Commons, M. L., Rodriguez, J. A. (1993).  The development of hierarchically complex equivalence classes.  Psychological Record, 43, 667-697.
*Commons, M. L., Stein, S. A., & Richards, F. A.  (April, 1987).  A general model of stage theory: Stageof a task.  Paper presented at the Society for Research on Child Development, Baltimore, MD.
*Commons, M. L., Strachan, D. D., Rodriguez, J. A., & Barnett, M. A. (1987, January). The effect of moral judgment stage on the utility function form in young adults: The decision to sacrifice one's own grades to keep tutoring commitments.  Paper presented at the symposium "From moral action to judgement and back: The relationships between action and stage" in Cambridge, MA.
Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A., & Krause, S. R. (1998).  The existence of developmental stages as shown by the hierarchical complexity of tasks.  Developmental Review, 8(3).  237-278.
*Commons, M. L., Woodford, M., & Ducheny, J. R. (1982). The relationship between perceived density of reinforcement in a schedule sample and its reinforcing value. In M. L. Commons, R. J. Herrnstein, & H. Rachlin (Eds.), Quantitative analyses of behavior:  Vol. 2. Matching and maximizing accounts (pp. 25-78). Cambridge, MA: Ballinger.
*Cook-Greuter, S. R.  (1990).  Maps for living: Ego-development theory from symbiosis to conscious universal embeddedness.  In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 2. Comparisons and applications of adolescent and adult developmental models (pp. 79-104).  New York: Praeger.
Coombs, C. H., Dawes, R. M., & Tversky, A.  (1970).  Mathematical psychology:  An elementary introduction. Englewood Cliffs, New Jersey:  Prentice-Hall.
Copernicus, N. (1530/1543/1992),  On the Revolutions,[De revolutionibus orbium coelestium] edited and translated by E. Rosen, MD, Baltimore: The Johns Hopkins University Press,
*Damen, P.  (1977).  Piagetian psychology: Cross-cultural contributions.  New York: Gardner Press.
*Dasen, P. R. (1980) Psychological differentiation and operational  development: a cross-cultural link.  The Quarterly Newsletter of the Laboratory of Comparative Human Cognition, 2, 81-86.
Davis, H., & Perusse, R. (1988). Numerical competence in animals: definitional issues, current evidence, and a new research agenda. Behavioral and Brain Sciences, 11, 561-615.
*Dawson, T. L. (2000). Moral reasoning and evaluative reasoning about the good life. Journal of Applied Measurement, 1 (372-397). 
*Dawson, T. L. (in press). New tools, new insights: Kohlberg  moral reasoning stages revisited. International Journal of Behavioral Development.
Dawson, T. L., Commons, M. L., & Wilson, M. (in review). The shape of development.

Dawson, T. L., Commons, M. L., Wilson, M., & Xie, Y. (1999, June). The General Model of Hierarchical Complexity Scoring System: Refinements and additions. Paper presented at the annual symposium of the Jean Piaget Society, Mexico City, Mexico.
*Demetriou, A.  (1990).  Structural and developmental relations between formal and postformal capacities: Towards a comprehensive theory of adolescent and adult cognitive development.  In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol. 2., Models and methods in the study of adolescent and adult thought (pp. 147-173).  New York: Praeger.
*Demetriou, A., & Efklides, A.  (1985). Structure and sequence of formal and postformal thought: General patterns and individual differences. Child Development, 56, 1062-1091.
*Denton, K. L., & Krebs, D. L.  (1990).  From the scene of the crime:  The effect of alcohol and social context on moral judgement.  Journal of Personality and Social Psychology, 59(2), 242-249.
Descartes, R.  (1637)   Discourse on Method.  Cambridge University  Press, edited Haldane and Ross; first five parts.
*Dromie, E.  (1984).  The one word period as a stage in language development:  Quantitative and qualitative accounts.  In I. Levin (Ed.), Stage and structure (pp.  ).  Norwood, NJ: Ablex.
*Draney, K. L. (1996).  The polytomous Saltus model: A mixture model approach to the diagnosis of developmental differences. Unpublished doctoral dissertation, University of California at Berkeley, Berkeley, CA.
*Ellis, J. B., Commons, M. L., Rodriguez, J. A., Grotzer, T. A., & Stein, S. A. (June, 1987).  Immediately detecting a causal relation stimulus embedded in similarly appearing possible causal relations.  Paper presented at the Tenth Symposium on Quantitative Analyses of Behavior Held at Harvard: Signal Detection, Cambridge, MA.
*Ennis, R. H.  (1978).  Conceptualization of childrens' logical competence:  Piaget's propositional logic and an alternative proposal.  In L. S. Siegel, C. J. Brainard, Alternatives to Piaget:  Critical essays on the theory.  (pp.  ).  New York:  Academic Press.
Fechner, G. T.  (1860/1966).   Elements of Psychophysics, vol. I (D. H. Howes and E. G. Boring, eds.), trans. H. E. Adler (New York: Rinehart and Winston, ).
*Erickson, E. H.  (1959).  Identity and the life cycle.  Psychological Issues Monograph, I.
*Erickson, E. H.  (1978).  Adulthood.  New York: Norton.
*Erickson, E. H.  (1982).  The life cycle completed: A review.  New York: Norton.
*Feldman, D. H.  (1980).  Beyond universals in cognitive development.  Norwood, NJ: Ablex.
Fischer, K. W.  (1980).  A theory of cognitive development: The control and construction of hierarchies of skills.  Psychological Review, 87, 477-531.
Fischer, K. W., & Bidell, T. R. (1998). Dynamic development of psychological structures in action and thought. In R. M. Lerner (Ed.), & W. Damon (Series Ed.), Handbook of child psychology: Vol. 1. Theoretical models of human development (5th ed., pp. 467-561). New York: Wiley.
Fischer, K. W., & Bullock, D. (1981). Patterns of data: Sequence, synchrony, and constraint in cognitive development. In K. W. Fischer (Ed.), New Directions for Child Development: Vol. 12. Cognitive Development (pp. 69-78). San Francisco: Jossey-Bass.
Fischer, K. W., Hand, H. H., & Russell, S.  (1984).  The development of abstractions in adolescence and adulthood. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development (pp. 43-73). New York: Praeger. 
Fischer, K. W., & Kenny, S. L.(1986).  Environmental conditions for discontinuities in the development of abstractions.  In R. A. Mines & K. S. Kitchener (Eds.), Adult cognitive development:  Methods and models.(pp. 57-75). New York:  Praeger.
Fischer, K. W., Knight, C. C., & Van Parys, M. (1993). Analyzing diversity in developmental pathways: Methods and concepts. In W. Edelstein & R. Case (Eds.), Contributions to Human Development: Vol. 23.  Constructivist approaches to development (pp. 33-56). Basel, Switzerland: S. Karger.
Fischer, K. W., & Lazerson, A.  (1984).  Human development:  From conception through adolescence.  New York:  W. H. Freeman.
*Flavell, J. H.  (1981).  Monitoring social-cognitive enterprises: something else that may develop in the area of social cognition.  In J. H. Flavell & L. Ross (Eds.), Social cognitive development: Frontiers and possible futures (pp. 272-287).  New York: Cambridge University Press.
*Flavell, J. H.  (1979).  Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry.  American Psychologist, 34, 906-911.
*Flavell, J. H. (1971).Comments on Beilin's "The development of physical concepts."  In T. Mischel (Ed.).  Cognitive development and epistemology. (pp. 121-128).  New York: Academic Press.
*Flavell, J. H.  (1963).  The developmental psychology of Jean Piaget.  Princeton: Van  Norstrand.
*Gardner, R., Sainato, D. M., Cooper, J. O., Heron, T E., Heward, W. L., Eshleman, J. Grossi, T. A.  (1994).  Behavior Analysis in Education Focus on Measurably Superior Instruction.  California  Pacific Grove.  
*Gesell, A. L., & Amatruda, C. S. (1964).  Developmental diagnosis (2nd ed.). New York:  Hoeber.
Gewirtz, J. L.  (1969).  Mechanisms of social learning: Some roles of stimulation and behavior in early human development.  In D. A. Goslin (Ed.), Handbook of socialization theory and research (pp. 57-212).  Chicago:  Rand McNally. Goodman in preparation.
*Gewirtz, J. L.  (1991).  Identification, attachment, and their developmental sequencing in a conditioning frame.  In J. L. Gewirtz & W. M. Kurtines (Eds.), Intersections with attachment (pp. 247-255).  Hillsdale, New Jersey: Erlbaum.
*Gibbs, J. (1977).  Kohlberg's stages of moral judgment: A constructive critique.  Harvard Educational Review, 47, 43-61.
*Gibbs, J.  (1979).  Kohlberg's moral stage theory: A Piagetian revision.  Human Development, 22, 89-112.
*Gödel, K. (1977).  Some metamathematical results on completeness and consistency; On formal undecidable propositions of Principia Mathematica and related systems I; On completeness and consistency.  In J. Heijehoort (Ed.), From Frege to Gödel: A Source Book in Mathematical Logic 1879-1931 (pp. 592-617).  Cambridge: Harvard University Press, 1977 (Originally published 1930, 1931, 1931, respectively).
Goffman, E. (1963). Behavior in public places; notes on the social organization of gatherings.  New York, NY: Free Press of Glencoe. 
*Grotzer, T. A. McCarthy, K. G., Frome, K. W., & Commons, M. L. (June, 1987).  The process of stage change.  Paper presented at Beyond Formal Operations 3: Positive development during adolescence and adulthood.  The Development of Adolescent and Adult Thought and Perception, Harvard University, Cambridge.
Halford G. (1999). Developmental of mental representation: theories and applications. Sigel I. (Ed.). Mahwah, NJ, US: Lawrence Erlbaum Associates, Inc. Publishers. 
Herrnstein, R. J.  (1982).  Melioration as behavioral dynamism.  In M. L. Commons, R. J. Herrnstein, & H. Rachlin (Eds.), Quantitative analyses of behavior: Vol. 2. Matching and maximizing accounts (pp. 433-458). Cambridge, MA: Ballinger. 
Herrnstein, R. J., & Vaughan, W., Jr.  (1980).  Melioration and behavioral allocation. In J. E. R. Staddon (Ed.), Limits to action: The allocation of individual behavior (pp. 143-176). New York: Academic Press.
*Holland J. G., & Skinner, B. F.(1961).  The analysis of behavior: a program for self-instruction.  New York, McGraw-Hill, 1961.
*Inhelder, B., & Piaget, J.  (1958). The growth of logical thinking from childhood to  adolescence: An essay on the development of formal operational structures. (A. Parsons, & S. Seagrim, Trans.). New York: Basic Books (originally published 1955).
*Jackson, M. A., & Malott, R. W.  (1994).  Helping high-risk Black college students.  In R. Gardner, D. M. Sainato, J. O. Cooper, T. E. Heron, W. L. Heward, J. Eshleman, & T. A. Grossi (Eds),  Behavior Analysis in Education Focus on Measurably Superior Instruction (pp. 349-363).  California Pacific Grove.  
Kallio, E. (1995).  Systematic reasoning: formal or postformal cognition? Journal of Adult Development, 2, 3, 187-192. 
Kallio, E., & Helkaman, K (1991). Formal operations and postformal reasoning: A replication. Scandinavian Journal of Psychology, 32, 1, 18-21. 
*Kantrowitz, S., Buhlman, R. A., & Commons, M. L.  (April, 1985).  Application of  signal-detection theory to the measurement of stage of development found in performance on formal-operational isolation-of-variables problems. Paper presented at the annual meeting of the Eastern Psychological Association, Boston, MA.
Karmiloff-Smith, A.  (1986).  From meta-processes to conscious access:  Evidence from children's metalinguistic and repair data.  Cognition, 23 95-147.
*Keating, D. B.  (1985). Beyond Piaget: The evolving debate.  [Review of Beyond formal operations: Late adolescent and adult cognitive development].  Contemporary Psychology, 30(6), 449-450.
*Kegan, R.  (1982).  The evolving self.  Cambridge: Harvard University Press.
King, P. M., Kitchener, K. S., Wood, P. K., & Davison, M. L. (1989).  Relationships across developmental domains: A longitudinal study of intellectual, moral, and ego development.  Adult Development: Vol. 1. Comparisons and applications of adolescent and adult developmental models (pp. 57-72).  New York: Praeger.
*Kitchener, K. S.  (1983).  Cognition, Metacognition, and epistemic cognition: A three-level model of cognitive processing.  Human Development, 26, 222-232.
*Kitchener, K. S., & King, P. M. (1990).  Reflective judgement: Ten years of research.  In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Beyond formal operations: Vol. 2.  Models and methods in the study of adolescent and adult thought (pp. 63-78).  New York: Praeger.
*Klahr, D., & Wallace J. G.  (1976).  Cognitive development: An information-processing view.  Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
*Kohlberg, L.  (1971).  From is to ought:  How to commit the naturalistic fallacy and get away with it in the study of moral development.  In T. Mischel (Ed.), Cognitive development and epistemology.  New York:  Academic Press.
*Kohlberg, L.  (1981).  The philosophy of moral development: Vol. 1. Moral stages and the idea of justice.  San Francisco: Harper and Row.
Kohlberg, L. (1984). Essays on moral development:  Vol. 2. The psychology of moral development:  Moral stages, their nature and validity. San Francisco: Harper & Row. 
Kohlberg, L.  (1990).  Which postformal levels are stages?  In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott  (Eds.). Adult Development. Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 263-268). New York: Praeger. 
*Kohlberg, L., & Armon, C.  (1984).  Three types of stage models used in the study of adult development.  In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development (pp. 357-381). New York: Praeger.
*Kohlberg, L., & Ryncarz, R. A.  (1990).  Beyond justice reasoning:  Moral development and considerations of a seventh stage.  In C. N. Alexander, & E. J. Langer (Eds.), Higher stages of human development:  Perspectives on adult growth (pp. 191-207).  New York: Oxford University Press.
Krantz, D. H., Luce, R. D., Suppes, P., & Tversky, A. (1971). Foundations of measurement: Vol. 1. Additive and polynomial representations. New York: Academic.
*Krebs, D. L., Denton, K. L., Vermeulen, S. C., Carpendale, J. I. & Bush, A.  (1991).  Structural flexibility of moral judgment.  (Personalty Processes and Individual Differences).  Journal of Personality and Social Psychology, 61(6), 1012-1024.
Kuhn, D., Pennington, N., & Leadbeater, B. (1983).. Adult thinking in developmental perspective: The sample case of juror reasoning.   In P. Baltes & 0. Brim (Eds.), Life-span development and behavior, Vol. 5 (pp.  )  New York: Academic Press.  
*Lahey, L., Souvaine, E., Kegan, R., Goodman, R., & Felix, S. (1982).  A Guide to the Subject-Object interview:  Its administration and interpretation.  Unpublished manuscript available from Robert Kegan, Harvard Graduate School of Education.
Lam, M. S. (1995).  Women and men scientists' notions of the good life:  A developmental approach.  Unpublished doctoral dissertation, University of Massachusetts, Amherst, MA.
*Levinson, D. J.  (1986).  A conception of adult development.  American Psychologist, 41, (pp. 3-13).  Lindsay, P. H., & Norman, D. A. (1977).  Human information processing:  An introduction to psychology, (2nd Edition), New York: Academic Press.
Libet, B. (1985). Unconscious cerebral initiative and the role of conscious will in voluntary action. Behavioral and Brain Sciences, 8:529-566.
Lindsay, P. H., & Norman, D. A. (1977).  Human information processing:  An introduction to psychology, (2nd Edition), New York: Academic Press.
*Loevinger, J.  (1970).  A theory of ego development.  San Francisco:  Jossey-Bass.
*Loevinger, J., & Blasi, A.  (1976). Ego development: Conceptions and theories.  San Francisco: Jossey-Bass.
*Loevinger, J., & Wessler, R.  (1970).  Measuring ego development.  San Francisco: Jossey- Bass. 
Lovell, C. W.  (1999).  Development and disequilibration: Predicting counselor trainee gain and loss scores on the Supervisee Levels Questionnaire.  Journal of Adult Development.
Luce, R. D.  (1959).  On the possible psychological laws. The Psychological Review, 66(2) 81-95.
Luce, R. D. (1963). Detection and recognition.  In R. Luce & R. Bush, (Eds.), Handbook of Mathematical Psychology (pp. 103-190).  New York: Wiley.
Luce, R. D., Krantz, D. H., Suppes, P., & Tversky, A. (1990). Foundations of measurement: Vol. 3. Representation, axiomatization, and invariance. San Diego, CA: Academic.
Luce, R. D. & Tukey, J. W., (1964). Simultaneous conjoint measurement. Journal of Mathematical Psychology, 1, 1-27.
*Luchins, A. S., & Luchins, E. H.  (1965).  Logical foundations of mathematics for behavioral scientists.  New York: Holt, Rinehart and Winston.
McAuliffe, G. J.  (In press).  Student changes, program influences, and adult development in one program of counselor training: An exploratory inductive inquiry.  Journal of Adult Development.
*Miller, P. M., Lee, S. T., & Commons, M. L. (June, 2000).  Scoring for stage and transition of losses in children and adult’s lives.  Data presented at Society for research in Adult Development Symposium, Pace University, New York City, New York.
*Miller, P. M., &. Lee, S. T. (June, 2000).  Stages and transitions in child and adult narratives about losses of attachment objects.  Paper presented at the Jean Piaget Society.  Montreal, Québec, Canada. 
Mislevy, R. J., & Wilson, M. (1996).  Marginal maximum likelihood estimation for a psychometric model of discontinuous development.  Psychometrica,  61, 41-47.
*Moerk, E. L. (1992).  A first language taught and learned.  Baltimore, MD: Paul H. Brookes Publishing 
Moore, R., & Goldiamond, I.  (1964).  Errorless establishment of visual discrimination using fading procedures.  Journal of the Experimental Analysis of Behavior, 7, 269-272.
*Munsinger, H.  (1970).  Light detection and pattern recognition:  Some comments on the growth of visual sensation perception.  In L. R. Goulet & P. B. Baltes (Eds.), Life-span developmental psychology (pp.  228-247).  New York: Academic.
*Newell, A., Shaw, J. C., & Simon, H. A.  (1964).  Elements of a theory of human problem- solving.  In R. J. C. Harper, C. C. Anderson, C. M. Christensen, & S. M. Hunda (Eds.), The cognitive processes:  Readings (pp. 339-356).  Englewood Cliffs, NJ: Prentice Hall.
*Newell A., & Simon, H. A.  (1972).  Human problem solving.  Englewood Cliffs, NJ: Prentice  Hall.
Overton, W. F. (1990).  Reasoning, necessity, and logic:  Developmental perspectives.  Hillsdale, NJ: Lawrence Erlbaum Associates.
*Pascual-Leone, J.  (1970).  A mathematical model for the decision rule in Piaget's developmental stages.  Acta Psychologica, 32, 301-345.
*Pascual-Leone, J.  (1984). Attention, dialectic, and mental effort:  Towards an organismic theory of life stages.  In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1.  Late adolescent and adult cognitive development (pp. 182-215). New York: Praeger.
Pepperberg, I. M. (1987). Evidence for conceptual quantitative abilities in the african grey parrot: labeling of cardinal sets. Ethology, 75, 37-61.
*Perry, W. G.  (1968, 1970).  Forms of intellectual and ethical development in the college years: A scheme. New York: Holt, Rinehart and Winston.
*Perry, W. G.  (1981).  Cognitive and ethical growth: The making of meaning.  In A. Chickering (Ed.), 
The modern American college (pp. 76-116).  San Francisco: Jossey-Bass.
*Piaget, J.  (1953).  Logic and Psychology. (W. Mays and F. Whitehead, Trans.).  New York: Basic Books.
Piaget, J.  (1954).  The construction of reality in the child.  (M. Cook, Trans.).  New York: Ballantine Books.
Piaget, J.  (1976).  The grasp of consciousness: Action and concept in the young child.  Cambridge: Harvard University Press.
*Piaget, J., & Inhelder, B., with Sinclair-de Zwart, H.  (1973).  Memory and intelligence.  A. Pomerans (trans.).  New York:  Basic Books.
*Piaget, J., Inhelder, B. & Szeminska, A.  (1960)  The child's conception of geometry.  New York: Basic Books.
Rasch, G. (1960/1980). Probabilistic model for some intelligence and attainment tests. Copenhagen, Denmark: Danish  Institute for Educational Research. Expanded edition, 1980. Chicago: University of Chicago Press.
*Rest, J. R., Turiel, E., & Kohlberg, L.  (1969).  Level of moral development as a determinant  of preference and comprehension of moral judgement made by others.  Journal of Personality, 37, 225-252.
*Richard, D. C., Unger, C. M., & Commons, M. L. (1988, April 22).  Strategies as knowledge:  Subjects' methods of determining causality during the shift from concrete to formal operations.  Paper presented at the annual meeting of the Eastern Psychological Association at Buffalo.  Friday, April 22, 1988, 8:20 am (Available from the Dare Institute, 234 Huron Avenue, Cambridge, MA 02138).
*Richards, F. A., & Commons, M. L. (1990b).  Applying signal detection theory to measure subject sensitivity to metasystematic, systematic and lower developmental stages.  In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, and J. D. Sinnott (Eds.), Adult Development: Vol 2. Models and methods in the study of adolescent and adult thought (pp. 175-188).  New York: Praeger.
*Riegel, K. F.(1973).Dialectic operations:  The final phase of cognitive development.  Human Development, 16: 346-370.
Rodriguez, J. A.  (1989).  Exploring the notion of higher stages of social perspective taking.  Unpublished Qualifying Paper, Harvard Graduate School of Education.
Rodriguez, J. A.  (1992).  The adult stages of social perspective-taking: assessment with the doctor-patient problem.  Unpublished doctoral dissertation, Harvard Graduate School of Education, Cambridge, MA.
*Rosales, J., & Baer, D. M.  (in press).  A behavior-analytic view of development.  In E. Ribes & S. W. Bijou (Eds.), Recent approaches to behavioral development.  Guadalajara, Jal. México:  Editorial, Universidad de Guadalajara.
*Rosales-Ruiz, J. (May, 2000).  Divide and teach.  Presented at the Association for Behavior Analysis, Washington D.C.
*Rosales-Ruiz, J., & Baer, D. M..(1997).  Behavioral cusps: A developmental and pragmatic concept for behavior analysis.  Journal of Applied Behavior Analysis. 30(3), 533-544. 
Rosenberg, M.  (1979).  Conceiving the self.  New York: Basic Books.
*Scandura, J. M.  (Ed.) (1977).  Problem solving: A structural/process approach with instructional implications.  New York: Academic Press.
*Schrader, D. E.  (1988).  Moral metacognition and its relation to moral judgment:  A description and analysis.  Unpublished doctoral dissertation, Harvard University.
*Schrader, D. E. (1987, June 30-July 4).  Overview of moral development.  Presented at the annual Moral Judgment Interview Scoring Workshop, Harvard graduate School of Education, Cambridge, MA (Available from the author, Cornell University).
*Selman, R. L.  (1980).  The growth of interpersonal understanding.  New York: Academic Press.
*Siegler, R. S.  (1981).  Developmental sequences within and between concepts.  Monographs of the Society for Research in Child Development, 46, (Whole No. 189).
*Siegler, R. S.  (1986).  Children's thinking, What develops?  Englewood Cliffs, NJ: Prentice Hall.
Skinner, B. F. (1931). The concept of the reflex in the description of behavior. Journal of General Psychology, 5, 427-458.
*Skinner, B. F., & Vaughan, M. E. (1983).  Enjoy old age: A program of self-management.  New York: W. W. Norton.
*Spada, H., & McGraw, B. (1985). The assessment of learning effects with linear logistic test models. In S. E. Embretson (Ed.), Test design. Developments in psychology and psychometrics. Orlando, FL Academic Press.
*Sonnert, G., & Commons, M. L.  (1994).  Society and the highest stages of moral development.  Politics and the Individual, 4(1), 31-55.
Sonnert, G., & Commons, M.L. (1994). Society and the highest stages of moral development. The Individual and   Society, 4(1), 31-55.
*Stein, S. A., & Commons, M. L. (May, 1987).  Verbal description and explanation appear lower in stage than nonverbal performance on a formal operational task.  Paper presented at the meeting of the Jean Piaget Society, Philadelphia, PA.
Strauss, M., & Curtis, L. (1984). Development of numerical concepts in infancy. In C. Sophian (Ed.), The origins of cognitive skills. Hillsdale, NJ: Erlbaum.
Suppes, P., Krantz, D. H., Luce, R. D., & Tversky, A. (1989). Foundations of measurement: Vol. 2. Geometrical, threshold, and probabilistic representations. San Diego, CA: Academic
Swan, T. & Benack, S.  (2002).  Narcissism in the Epistemological Pit.  Journal of Adult Development, 9(3). 
*Tappan, M. B.  (1990).  The development of socio-moral cognition during young adulthood: A three-dimensional model.  In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 2. Comparisons and applications of adolescent and adult developmental models. New York: Praeger.
*Terman, L. M., & Merrill, M. A.  (1937).  Measuring intelligence.  Boston: Houghton Mifflin.
*Terrace, H. S.  (1961).  Discrimination learning with and without "errors".  Unpublished doctoral dissertation, Harvard University, Cambridge, MA.
Terrace, H. S. (1963).  Discrimination learning with and without "errors."  Journal of the  Experimental Analysis of Behavior, 6, 1-27.
*Tappan, M. B.  (1990a).  The development of justice reasoning during young adulthood: A three-dimensional model.  In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 235-248).  New York: Praeger.
*Tappan, M. B.  (1990b).  The development of socio-moral cognition during young adulthood: A three-dimensional model.  In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 2. Comparisons and applications of adolescent and adult developmental models. New York: Praeger.
*Terman, L. M., & Merrill, M. A.  (1937).  Measuring intelligence.  Boston: Houghton Mifflin.
*Terrace, H. S.  (1961).  Discrimination learning with and without "errors".  Unpublished doctoral dissertation, Harvard University, Cambridge, MA.
Terrace, H. S. (1963).  Discrimination learning with and without "errors."  Journal of the Experimental Analysis of Behavior, 6, 1-27.
Van Geert, P.  (1991).  Theoretical problems in developmental psychology.  In P. Van Geert, &  L. P. Mos (Eds.), Annals of Theoretical Psychology, 7 (pp. 1-54).  New York: Plenum Press.
Van Loosbroek, E. & Smitsman, A. (1990).  Visual perception of numerosity in infancy. Developmental Psychology, 26, 916-922. 
Vygotsky, L. S. (1962) [or (1986) rpt.]. Thought and language.  Cambridge, MA: MIT Press.
Vygotsky, L. S. (1966). Development of the higher mental function. In Psychological Research in the U.S.S.R. (pp. 44-45). Moscow: Progress Publishers.
*Walker, L. J.  (1986).  Experiential and cognitive sources of moral development in adulthood.  Human Development, 29(2), 113-124.
*Weinreb, N., & Brainerd, C. H.  (1975).  A developmental study of Piaget's Groupment Model of the emergence of speed and time concepts.  Child Development, 46, 176-185.
*Weir, A. A., Chappell, J., Kacelnik, A.  (2002).  Shaping of hooks in New Caledonian crows.  Science. 9;297(5583): 981.
*Wexler, K. A. (1982).  Language acquisition: The state of the art.  Cambridge: At the University  Press.
Wilson, M. (1989).  Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105, 276-289.
*Wolfsont, C.   (June, 2000).  The brain gym and scoring adult developmental transition and stage.  Data presented at Society for research in Adult Development Symposium, Pace University, New York City, New York.
Young, F. W. (1972).  A model for polynomial conjoint analysis algorithms.  In R. B. Shepard, A. K. Romney, & S.  B. Nerlove (Eds.), Multidimensional scaling. Theory and applications in the behavioral sciences.  (69-104).  New York: Seminar Press
Zheng Yan, Z,  Fischer, K. W.  (2002).  Always Under Construction.  Dynamic Variations in Adult Cognitive Microdevelopment, Human Development 45(3):141-160 

Adey, in press 7, 9, 12, 18
Anatasi, 198  22
Antell & Keating, 1983 46
Aristotle (384€322) 2
Arlin (1975, 1977, 1984 68
Arlin (1975, 1977, 1984) 68
Arlin (1975, 1984 68
Arlin (1984) 39
Armon (1984) 39
Armon & Dawson 6
Armon & Dawson, 1997 6
Armon, 1990 6
Basseches (19 39
Benack (1994) 39
Bond (1999, June, personal comm.) 40
Bowman, 1996a; 1996b 6
Brannon & Terrace, 1998 51
Brendel, Kolbert, & Foster, this issue 18
Broughton (1977; 1984) 40
Cheryl Armon, 1984a, 1984b 6, 37
Cheryl Armon, 1993 6, 37
Colby and Kohlberg's (1987) 20
Commons & Ducheny 1979 22
Commons & Miller 2004 6, 37
Commons & Miller, 1998 18, 48
Commons & Pekker, submitted 45
Commons & Richards (1984) 39
Commons & Richards, 1984a, 1984b 5, 14
Commons & Rodriguez, 1990 6, 11
Commons & Rodriguez, 1990, 1993 5, 6
Commons & Rodriguez, 1990;  14
Commons & Rodriguez, 1990; 1993) 6
Commons & Rodriguez, 1993 14
Commons & Wolfsont, 2002 6, 37
Commons and Miller, in press 6, 37
Commons, et. al., 2004 6
Commons, Goodheart, & Bresette, 1995) 6
Commons, Goodheart, & Bresette, 1995)  37
Commons, Goodheart, and Dawson ( 1995; 1997 7
Commons, Grotzer and Davidson (in preparation) 48
Commons, Grotzer, and Davidson (in preparation 15
Commons, in preparation 6, 37
Commons, in preparation)  37
Commons, Kantrowitz and Buhlman, 1984 21
Commons, Kantrowitz, Buhlman, Ellis, Grotzer, in preparation 71
Commons, Krause, Fayer, & Meaney, 1993 6
Commons, Lee, Gutheil, et. al., 1995) 6, 37
Commons, Miller and Kuhn (1982 48
Commons, Pekker, & Adams, in press). 42
Commons, Richards & Kuhn, 1982 6, 42
Commons, Richards & Kuhn, 1982);  44
Commons, Rodriguez, Cyr, Gutheil  37
Commons, Rodriguez, Cyr, Gutheil et. al., in preparation 6
Commons, Trudeau et al 1998 45
Commons, Trudeau, Stein, Richards, & Krause, 1998) 5
Cook-Greuter, 1990 6
Cook€Greuter (1990) 40
Cook€Greuter, 1990)  37
Coombs, Dawes, & Tversky, 1970 5
Copernicus (1530) 4
Davis & Perusse, 1988 45
Dawson (1998) 40
Dawson & Commons, in preparation 19
Dawson, 1998 6, 37
Dawson, 2000 6
Dawson, Commons, & Wilson, 1999, June 19
Dawson, Commons, & Wilson, in review; Dawson, Commons, & Wilso 19
Demetriou (1990; 1995) 40
Descartes (1637 4
DeVos & Commons, unpublished manuscript 6
Dickenson and McGonigle (1996) 45
Edger Brown (2004) 1
Fechner (1860 4
Fischer (1984 68
Fischer (personal communication 18
Fischer & Kenny 1986 11
Fischer & Kenny, 1986 12
Fischer & Lazerson,  1984 11, 12
Fischer & Lazerson, 1984 11, 48
Fischer and Lazerson (1984 68
Fischer, 1980 11, 12
Fischer, 1980; 1986; 1984 12
Fischer, Hand, & Russell (1984) 39
Funk, 1990 6
G. T. Fechner (1860) 4
Gewirtz (1969b 68
Gewirtz, 1969 68
Goffman, 1967 71
Gray (1999, June, personal comm.) 40
Halford, 1999 11
Herrnstein & Vaughan, 1980 17
Herrnstein, 1982 17
Inhelder & Piaget (1958) 39
Inhelder & Piaget, 1958 6
Inhelder and Piaget (1958) 48
Kallio (1991, 1995) 40
Karmiloff-Smith, 1986 13
Kegan (1994) 40
King & Kitchener (2002) 40
King & Kitchener, 1989 12
Kitchener & Fischer, 1990) 6
Kitchener & King, 1990 6
Kohlberg (1981) 39
Kohlberg (1985) 71
Kohlberg, 1984 71
Kohlberg, 1990 5
Koplowitz (1984) 40
Krantz, Luce, Suppes, and Tversky, 1971 4
Kuhn, Pennington, and Leadbeater (1983 44
Labouvie€Vief (1984)  39
Lam, 1994 6
Lamborn, Fischer & Pipp, 1994 37
Lamborn, Fischer & Pipp, 1994) 6
Libet's, 1985 4
Lindsay & Norman, 1977 5
Loevinger (1998) 40
Lovell, 2004 6, 37
Lovell, this issue 18
Lowenthal, 1989 2
Luce, Krantz, Suppes, & Tversky, 1990 4
McAuliffe, this issue 18
Meredith Root-Bernstein (Personal communication, January 18, 2 49
Miller & Cook-Greuter, 2000 6
Miller & Cook€Greuter, 2000 37
Miller & Lee, 1998) 6
Mislevy & Wilson, 1996 18
Nicholas Commons€Miller, in preparation 37
Oliver, 2004 6, 37
Overton, 1990 6
Pascual€Leone (1983) 39
Piaget (1954 5
Piaget, 1973 5
Piaget, 1976 13
Powell (1984) 39
Rasch (1980 42
Rodriguez, 1989 44
Rosenberg (1979 16
Rosenberg, 1979 16
Sinnott (1984) 39
Sonnert & Commons (1994) 39
Sonnert & Commons, 1994 6, 37
Sternberg  (1984) 39
Strauss & Curtis, 1981 46
Suppes, Krantz, Luce & Tversky, 1989 4
Swan & Benack, 2002 15
Tappan, 1990 12
Thales (640 € 546 1
Torbert (1994) 40
van Geert, 1991 12
van Loosbroek & Smitsman, 1990) 46
Vygotsky (1966a; 1966b 68
Vygotsky's (1966a, 1966b 68
Wilson, 1989 18, 19