MAPP™ Construct Validity Study

Following is a letter from Laura Gilbert, Ph.D. to Assessment.com with the results of a validity study between the MAPP career test and the Strong Interest Inventory.  The letter outlines the correlations between the two tests, conducted with a test group of 30 individuals.

September 8, 1997

International Assessment Network

Re: Construct Validity Study Between MAPP™ and Strong

This document sets out the process that was used to conduct the construct validity study which compares the MAPP to the Strong Interest Inventory.  The current analysis yielded correlations which suggest similarity between the underlying constructs of the MAPP and those of the Strong.  Additional studies with larger sample populations are expected to further illuminate this relationship as well as satisfy future reviews by educational professionals.

Process for analysis of construct validity between the MAPP™ and the Strong

In the first step of analysis of construct validity between the MAPP and the Strong, predictions were made as to the occupational matches between MAPP and Strong.  Predictions were made and noted by three senior managers with an average of 17 years experience with job analysis, career development, and performance management.  Each represented a different discipline (human resources, finance, and technology).  Differences were discussed and final agreement was reached on each match.  Although predicted matches for each MAPP occupation were held to a standard of three Strong occupations or less, agreement was reached for a single or dual occupation prediction for the majority of MAPP occupations.  Four exceptions were made to allow for more than three predicted Strong matches in the case of single, broad MAPP occupations such as "Scientific Research" which could equally be predicted to match Strong occupations "Biologist", "Chemist", Physicist", and "Geologist".  (see Appendix A)

Both instruments were then administered to 30 individuals ranging in age from 26-67, with 12 men and 18 women.  Scores were calculated and a full correlation of all MAPP occupations with all Strong occupations was conducted.  (see Appendix B) Correlations for the occupation-match predictions were identified and a single best predicted match was determined.  (see Appendices C and D) Of the 80 Strong occupations with a predicted MAPP match, 5 correlations were shown to be reflective of opposing instrumental themes (e.g. Mental Health v. Social) and had correspondingly low correlations (.14-.41).  The remaining 75 correlations between the predicted occupational matches ranged from .50-.92 with a median correlation of .67.  This is particularly noteworthy given that this average correlation is nearly as high as the average correlation found in an earlier study between the Strong and another assessment tool which was designed to measure the same key dimensions as the Strong (Harmon, 1994, p. 59).

An earlier comparative study between the Strong and yet a different instrument found only 3 correlations (out of 182) to be over .30 (Harman, 1994, p. 59).  Approximately 85% of the totality of correlations in the current study fall above this marker.  The earlier study did find several interesting directional correlations between itself and the Strong (e.g. Strong "Artistic" correlated negatively with 'Order" and positively with 'Change' on the comparing instrument).  Similar and stronger directional correlations were observed between the MAPP and the Strong.  These relationships are found in the form of negative as well as positive correlations. For example, MAPP's "Consulting, Business Services" correlated positively to the Strong's "Realtor" (.73), "Corporate Trainer" (.72) and "Chiropractor" (.69) and negatively to "Farmer" (-.65) and "Plumber" (-.75).  These directional correlations reflect the specific instrument's definitions of these occupations.  A chart including all directional relationships with correlations > + or -.65 within the current study can be seen in Appendix E.  The cutoffs of + or -.65 were chosen to be reflective of the median correlation between predicted occupational match items.

Further analysis of the full correlation matrix between MAPP and Strong occupations revealed additional correlations greater than .65 or less than -.65 beyond those predicted prior to testing.  These correlations were identified for each individual MAPP occupation (see Appendix #) and hypotheses were formulated regarding potential underlying themes using pattern matching among these highly correlating occupations (see Appendix F).  These hypotheses might serve as the basis for further research exploring potential additional or specific cross-utilizations of the MAPP and Strong.  It must be noted, however, that within the limits of the current study these hypotheses are merely exploratory in nature.

Laura Gilbert Ph.D.

Organizational Development and Research Consultant

Lgilbert@gw.hamline.edu