Test Validity Test validation studies or test validity for a pre-employment assessment is only an objective measure that provides evidence that the test or personality assessment actually measures what it purports to measure. Validation is not a stamp of approval by any governmental agency but rather a study undertaken and directed by the test publisher in accordance with certain professional standards. The Achiever employment assessment has been established and validated in accordance with the procedures described in "Standards for Educational and Psychological Testing," which is referred to in paragraph 2 It is therefore not discriminatory and is in compliance with E.
There are five forms of validity: Construct validity refers to the extent in which dimensions with similar names on different tests relate to one another. Two things that correlate highly on a personality test are not necessarily identical, but do provide reassurance that they are related and are a "construct" or part of the makeup like honesty, dependability, sociability, etc.
Concurrent validity is that approach whereby people who are successful within a given job within the same company or industry are evaluated and generally grouped Top Third, Middle Third, and Bottom Third. The assessment scores of the people who fit each of these ranges are then compiled and Job Benchmark Standards of the Top Third are used to hire, train or manage.
Within six months, or any appropriate period of time later, the pre-employment assessment is scored and benchmarks are established of the people who were hired in the new jobs who are still with the employer and whom the employer considers successful.
Job Benchmark Standards are thus established through the Predictive approach. Content validity represents job function testing, i.
Content validity is an appropriate strategy when the job domain is defined through job analysis by identifying the important behaviors, tasks, or knowledge and the assessment or test is a representative sample of behaviors, tasks or knowledge drawn from that domain. The Uniform Guidelines on Employee Selection Procedures state that in order to demonstrate the content validity of a selection procedure, a user should show that the behaviors demonstrated in the selection procedure are a representative sample of the behaviors of the job in question or that the selection procedure provides a representative sample of the work product of the job.
Face validity is the simplest form of validity which basically tells us that the personality test or other assessment instrument appears on the face of it to measure what it is supposed to measure. Simply put, a test that would be composed of accounting problems would have face validity as a measure of the ability to succeed as an accountant. Face validity is not very sophisticated because it is only based on the appearance of the measure. Be careful because the market is flooded with personality testing that has only face validity.
Information should be gathered in each step of the standard hiring process to have specific and measurable data to utilize in making a final hiring decision. The pre-employment assessment used should count for no more than one-third of the hiring decisions. The preliminary interview, job history check, in-depth interview results and evaluation of education, experience and other pertinent factors should be considered as well.
Consequently, there are three optional approaches to using assessments: Establish your own successful employee Job Benchmark Standards by conducting a concurrent validation by job classification. By tying job-related criteria to the aptitudes and personality dimensions of the assessment, the ultimate in validation and job relativity is assured. Also, the Job Benchmark Standards simplify the interpretation and use of the pre-employment assessment in the hiring process, since it establishes a model for hiring, promotion and training purposes.
Establish Job Benchmark Standards by job classification by answering job-related questions on the requirements of the job. This rule was designated by the E. Out of job applicants comprised of 80 white and 40 minority , 48 whites were hired and 12 minorities were hired. Return to Top Do validity studies guarantee accuracy? No, they do not.
Validity and reliability go hand in hand. I have taken a number of assessments with varied results. Many were very far off target but all of them were supposedly validated instruments. Let's take a look at how this frequently happens. Let's say that a company has designed a test that measures communication styles and that the personality assessment is very effective. The validation studies for any assessment instrument are only an objective measure that evidences that the test actually measures what it purports to measure, and in this particular case it is communication styles.
Let's say that this particular personality test is later given certain external modifications so that it can also be sold as a pre-employment assessment. The personality test is still backed by validity studies, but unless new validity studies are done, there are no validity studies to support the use of the assessment for its intended purpose as a pre-employment assessment.
In this example, the intended use is quite clear to measure communication styles. The Uniform Guidelines on Employee Selection Procedures specifically state that the evidence of validity and utility of the selection procedure must support its operational use.
Now let's take another look at where validity studies can be very misleading. All pre-employment assessments that measure behaviors are based on certain theoretical models. Some of those models are very simplistic because they are used more for training purposes than anything else. Common sense tells us that human behavior is actually very complex but for training type applications we need to keep things simple. If we look at the interpretative manual for one of these personality assessments, we will find out a little information about the behavioral model that was used.
Such people are often tenacious, tough-minded types that lack empathy and are often uncooperative. This personality assessment may prove very effective in training situations but its limitations are obvious when applied to a pre-employment assessment context. If people that are high in dominance are very ingenious, then we would also have to assume that submissive individuals would be found to be lacking in mental ability. From a practical standpoint, we know that there are no strong correlations between cognitive ability and dominance.
We also know that there are highly dominant individuals that have low ethical standards and submissive individuals that have high ethical standards. From a practical standpoint we can also say that highly dominant people are not necessarily tough-minded, competitive or planful. I have seen a similar personality test used in a number of pre-employment situations and I can tell you that the results are often very misleading.
In one situation, the personality assessment results were indicating that all of a company's employees were highly dominant. The amusing part was that those employees were all being put through training designed to try to lessen the negative effects of their supposedly high assertiveness. From what I could observe, it would have been more effective to put them through assertiveness training!
I was able to test those same employees a short time later with a validated pre-employment assessment The Scoreboard and the assessment results confirmed my observations. Very few of the employees were high in dominance. They mostly scored low to mid-range.
There are obvious disadvantages to simplistic behavioral models in that a group of separate behaviors are clumped together. In this case the personality test was not measuring true dominance. It relied heavily on ethics and competitiveness to measure dominance. In the preceding situation, it turned out that all of the employees in that particular job had very high character strength or ethics. What was actually being measured was ethics flexibility and not dominance.
Return to Top How valid is the validity study? The fact that a personality test is backed by validity studies means very little in itself. Some of the validation techniques are quite weak. Some personality assessments are very simple in that they will contain a listing of certain descriptors such as friendly, outgoing, agreeable, competitive and will ask the respondent to circle each descriptor that describes themselves.
How is that possible? Actually the whole process can be very simplistic so let's take a look at the total process totally hypothetical, of course. The candidate is given a test and instructed to circle a list of descriptors that he feels accurately describes himself. The testing company takes those descriptors and expands on their definitions and then gives the report back to the candidate. On the last page of the report is a questionnaire sheet that asks the candidate to rate the accuracy of the personality test report and mail it back to the test publisher.
All of the responses that are received are then entered into a database that is used as an ongoing validity study. The main advantage of this hypothetical personality test is that on the surface it is fast and cheap. While I would question the overall effectiveness of such a program, it could offer a few advantages.
It would probably be a little more accurate and objective in most cases than when an interviewer directs the applicant to "tell me a little bit about yourself. What I would have to question in relation to such a test would be whether or not the validity studies would meet the requirements of the "Uniform Guidelines On Employee Selection Procedures" as they pertain to the professional standards for validity studies. Validity studies are not really that comprehensible unless you have a good solid background in statistics.
If you are anything like most people, you are probably suspicious of statistics to begin with. Start with one simple question. Were the procedures used in validation consistent with generally accepted professional standards such as those described in the "Standards for Educational and Psychological Tests? Secondly, you should be aware of one very important fact.
Just because a testing instrument was written by someone with a PhD, does not necessarily mean that the instrument was professionally developed or that it will meet the generally accepted professional standards that have been previously referenced. Be cautious of any personality test that claims to have been written by a professional, and then immediately tries to lead you to the conclusion that it was professionally developed without referencing any validation or reliability studies.
The two concepts do not necessarily go hand in hand. I once visited a web site that used this tactic very effectively. It then followed up with a very long article under the heading of Validity.
After endless scrolling through the long article, it concluded without ever mentioning validity, except in the title. Some people are slicker than cow guts on a door knob!