Reliability & Validity of Psychological Type Assessments

Reliability

What is reliability? Reliability is how consistently a test measures what it attempts to measure. Why is consistency important? Because when you measure something with an instrument two times, you want it to come out with the same answer (or close to it) both times. (This is called test-retest reliability, and it is an important measure of any kind of scientific testing.)

Personality is qualitative and therefore difficult to measure, so psychological instruments cannot have the same consistency you would expect from, say, a ruler. But there are generally accepted standards for psychological instruments. The MBTI® instrument meets and exceeds the standards for psychological instruments in terms of its reliability.

Facts about the MBTI® instrument reliability:

  • Reliability (when scores are treated as continuous scores, as in most other psychological instruments) is as good as or better than other personality instruments.
  • On retest, people come out with three to four type preferences the same 75% to 90% of the time.
  • When a person changes type on retest, it is usually on one of the dichotomous pairs (e.g., E-I or S-N), and in a dichotomy where the preference clarity was low.
  • The reliabilities are quite good across most age and ethnic groups.
  • When the MBTI instrument is used with groups where reported reliabilities are lower or data are lacking, caution should be exercised and the professional should evaluate appropriate use.

        -- myersbriggs.org:

Validity

Validity is the degree to which an instrument measures what it intends to measure, and the degree to which the “thing” that the instrument measures has meaning.

Why is this important? If personality type is real (or rather, if it reflects the real world with accuracy), then we should be able to use MBTI type to understand and predict people's behavior to some degree. Type should help us differentiate the values, attitudes, and behaviors of different people.

Many studies over the years have proven the validity of the MBTI instrument in three categories:
  1. the validity of the four separate preference scales; (2)
  2. the validity of the four preference pairs as dichotomies; and
  3. the validity of whole types or particular combinations of preferences.

Many of these studies are discussed in the MBTI® Manual (published by CPP).

        -- myersbriggs.org:

Value and Usefulness

Let’s make one thing perfectly clear: in our view, asking if a specific personality test is “valid” is a red herring -- this question misleads or distracts from the relevant issue. The fact is that the [system] is not a test with right or wrong answers and thus the statistical concept of “validity” is being misapplied. The [system] is a TOOL for self-identification and self-awareness.

Of course when using a tool, there are many questions one could ask: “Is it safe?”, “Is it practical?”, “Is it the right tool for the job?” and so on... However, we invite you consider one of the most important questions when using a tool: “Is it useful?”

Is a Hammer Valid?

Consider for a moment that you are about to use a hammer and someone stops you and asks, ”Is that hammer valid?” You would likely look at them puzzled and respond, “What do you mean, is it valid?” Your next inquiry might be, “Valid for what?” It seems like a rather strange question in that context, doesn’t it?

However, if someone asked you if a hammer is useful we can all agree that the answer is a resounding YES! The use of hammers dates back to 2,400,000 BCE as a way to deliver impact to an object. Common uses of hammers are for driving nails, fitting parts, and breaking up objects. Hammers are often designed for a specific purpose, and vary widely in their shape and structure. However, a hammer is NOT useful for screwing in a Philips head screw, painting, sawing lumber, tightening a bolt, twisting electrical wires together, etc. So, in the right hands, doing a job it was intended to do, we can all agree that a hammer is a useful tool indeed!

Similarly, the use of personality systems dates back to 460 BC as a way to identify traits associated with different individuals. Common uses of personality systems are self-identification and awareness, improving communication and relationships, process improvement for organizations, and much more. Personality systems are often designed for a specific purpose, and vary widely in their structure and applicability.

        -- Mary Miscisin (PersonalityLingo)

It Is a Test

The question of validity often arises around the mention of MBTI, Myers-Briggs. Yes - it is a fundamentally flawed concept. More people fail it than pass the Myers-Briggs test. By passing it, we mean that they come up with the right answer.

It relies too heavily on self-awareness. How many people do you know who can truly assess themselves objectively? This is the weakness of the MBTI assessment instrument - so it gets easy blame.

If we gave everyone a cello and said play us some music, some could, and some could not. Those who could not would be unlikely to blame the instrument. When it comes to a self-assessment, however, if we fail, we find a warm sense of comfort in blaming the instrument.

... Is the sixteen Personality Types model the best? Perhaps not; it presumes a healthy mind - a balance of introspective and extrospective cognitive functions. It seems a good start; and it is extremely valuable in counseling individuals, and in predicting couple-compatibilities for sustainable bliss, and in predicting most fulfilling career choices.

Because there actually is a right and a wrong result, I finally gave in; so rather than echoing the long-held phrase "it's not a test", I give it back. I may now say: "Yes; it's a test. It's like recognizing yourself in a mirror, but a lot more difficult. Can you pass?"

        -- Gary Howard, (in Facebook; Thursday, July 28, 2016)


References

-- Main.VickiBrown - 18 Aug 2016
Topic revision: r2 - 21 Aug 2016, VickiBrown
Copyright © by the contributing authors. All material on this site is the property of the contributing authors. Please follow referenced links.
Myers-Briggs®, Myers-Briggs Type Indicator® and MBTI® are trademarks or registered trademarks of MBTI® Trust, Inc., in the United States and other countries.
This site is built using Foswiki. -- Send feedback to webmaster.