How is the CLS360 Evidence Based?
When considering whether a leadership diagnostic is evidence based, three criteria must be considered:
1. Theory
2. Statistics
3. Publications
How the CLS360 meets these criteria follows.
Criterion 1. THEORY
Is the diagnostic based on sound and tested theory; that is, has the theory been tested multiple times and found to be convergent with, or at the very minimum not contradictory to, other scientific domains, e.g., evolutionary biology, evolutionary psychology, or physics?
The underlying theory of the CLS360 (the interpersonal circumplex) draws heavily on the most recent scientific developments. From this, researchers discovered leadership could be captured in an interpersonal model: the Leadership Circumplex.
Is the diagnostic based on sound theory that has empirically tested hypotheses through the process of confirmation or falsification/refutation? Since its first publication in 1957, the interpersonal circumplex theory has been researched and confirmed for more than 60 years. It is now considered a universal model to represent human interpersonal behaviour and is in line with findings from other scientific fields such as evolutionary biology: Social (or group living) species are driven by two metamotives—to compete (Agency) and to collaborate (Communion).
The CLS360 was used for repeated hypotheses testing; the underlying circular (circumplex) structure of leadership behaviour was confirmed in six European studies and one Australian study.
The following sections discuss (psychometric) reliability and validity, but both are statistical methods that don’t say anything about the theory itself. The criterion of a good theory is an exclusion criterion.
Criterion 2. STATISTICS
Operationalisation is the process of making the theory measurable. So if you build a new test, does the test construction meet the highest psychometric (psychological measurement) standards?
Two criteria are used to judge the quality of a psychometric test—reliability and validity.
i. Reliability
Psychometric reliability is a measure of a test’s consistency, measurement errors, and its replicability. In other words, does the test measure correctly (only a small error margin). Two common methods are internal reliability and test-retest stability.
a. Internal (consistency) reliability of scales
Do the items or questions that make up the scales of the model show high levels of internal consistency? It is undesirable to have items that are not correlated in the same scale; if this is the case they may not even belong in a particular scale. All of the items should measure the same general construct (e.g., inspirational leadership). An acceptable scientific standard is .70 (Cronbach’s alpha coefficient).
The CLS360 scales exceed this standard as they range from .77 (Directive) to .91 (Coaching), which is considered good to excellent.
b. High test-retest stability
When leaders are asked to repeat the survey, are the survey results from the first and second administration similar? It is undesirable for the results to be significantly different from the first to the second administration of the diagnostic within four to six weeks. Several measurements under the same circumstances should produce only small differences in test results. In other words, it is a stable test. Psychometric test-retest reliability is expressed as a correlation coefficient. An acceptable scientific standard is .70.
The CLS360 scales exceed this standard as they range from .75 (Distrustful) to .87 (Inspirational), indicating high test-retest reliability, which is considered good to excellent.
ii. Validity
Does the test actually measure completely and solely what it claims to measure? Two typical ways to test this are convergent validity and predictive validity.
a. Convergent validity
Do the diagnostic’s scales ‘converge’ or align with other existing evidence-based questionnaires measuring theoretically closely related concepts? It is typically expressed as correlation coefficients.
The CLS360 scales show strong convergence with a range of other scientifically accepted scales and has demonstrated higher reliability.
b. Predictive validity
Does the diagnostic ‘predict’ hypothesised outcomes or criteria?
The CLS360 strongly predicts leadership outcomes such as performance and effort.
Criterion 3. PUBLICATIONS
Has the research on which the diagnostic is based been peer-reviewed and published in a respected scientific journal or high impact A1 journal (i.e., a peer-reviewed international magazine) or at least in an A3 paper (i.e., a peer-reviewed national magazine)?
To be published in A1 journals, the research needs to meet the ‘gold standard,’ including hypothesis testing and replication, and must show a contribution to the advancement of the science. Ideally, the research should be included in broader meta-analyses of similar research over time to confirm its efficacy across a large number of scientific trials.
The CLS360 research has met the ‘gold standard’ by being the outcome of a PhD study (Marleen Redeker, Free Amsterdam University) and more importantly has been published in a peer-reviewed A1 journal, the European Journal of Work and Organizational Psychology (2012). Click here to download: http://dx.doi.org/10.1080/1359432X.2012.738671
Click here to download the CLS360 Evidence-Based Criteria