Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Saville Wave professional styles 2014

1,600 views

Published on

British Psychological Society's assessment of Saville Wave Model of personality psychometric assessment

Published in: Leadership & Management
  • Login to see the comments

  • Be the first to like this

Saville Wave professional styles 2014

  1. 1. Test Review Wave Professional Styles (WAVE) The British Psychological Society © 2014. All rights reserved. No part of this publication may be reproduced or transmitted, in any form or by any means, without permission. This test has been granted registration as a psychological test by the British Psychological Society, Psychological Testing Centre. Permission has been granted to the distributor / publisher named above to distribute copies of this review in paper or PDF file format so long as such copies are not amended or changed in any way from the original version published by the BPS. 1
  2. 2. Wave Professional Styles (WAVE) Reviewers: Barry Cripps & Stephen Fisher Consulting Editor: Paul Willner Senior Editor: Iain Coyne GENERAL INFORMATION AND DESCRIPTION OF THE INSTRUMENT Test Name: Wave Professional Styles Authors of the original test: P. Saville, R. MacIver and R. Kurz License for Distribution Issued To: Saville Consulting UK Ltd Local test distributor / publisher: Saville Consulting UK Ltd Publisher of the original version of the test: Saville Consulting Group Ltd Date of Publication of Current Review/Edition: 2005 Date of Publication of the Original Test: 2005 Date of Current Review: 2014 ISBN: 9781854336910 General Description of Test: The Saville Consulting Wave Styles assessments are part of a multi−dimensional assessment suite for use in a wide variety of applications (selection, individual development, coaching, leadership assessment, team development, organizational talent audit, business growth, self−awareness and career planning). The Styles assessments were developed to separate talents from underlying predispositions or motives, and measure motivation, talent, competency, potential and preferred culture in one questionnaire. The Styles questionnaires were designed specifically for the Internet. They operate as both trait and type instruments and rely on a hierarchical model of work performance that is aligned to the Big Five personality factors and the Great Eight competencies. Wave adopts a hierarchical structure called the Wave Performance Culture Framework, made up of Behaviour (work behavioural styles and competencies), Ability (assessing a person’s abilities to perform certain intellectual or cognitive tasks), and Global (describing broad overall effectiveness characteristics of performance at work such as applying expertise, accomplishing objectives and demonstrating potential). The instrument measures an individual’s ‘Professional Style’ from a total of 216 items representing 108 ‘Facets’, each of which includes 2 items probing motive (will do) and talent (can do) components. The Facets are hierarchically organized into 36 ‘Dimensions’, which are further grouped into 12 ‘Sections’ (Evaluative, Investigative, Imaginative, Sociable, Impactful, Assertive, Resilient, Flexible, Supportive, Conscientious, Structured and Driven), which combine to create 4 ‘Clusters’ (Thought, Influence, Adaptability and Delivery). Four ‘Response Style’ summary scales are also available. A compact Saville Wave Professional Styles © 2014 BPS Page 2 of 28
  3. 3. Consulting Wave Focus Styles questionnaire comprises 72 items measuring 36 behavioural facets that were selected for their strong validity. The Handbook provides very little information about this short form of the instrument. The Handbook provides detailed descriptions of clusters, sections, dimensions and facets. Throughout the Handbook, the four Clusters are colour coded for ease of reference. Wave is computer administered, scored and normed. The Professional Styles questionnaire takes about 40 minutes to complete, while Focus Styles is typically completed in 13 minutes. Completing the Wave requires test−takers to respond to a combination of ‘Likert’ type ratings together with forced−choice items. The scoring integrates rating and ranking responses and results in a combined profile that highlights differences between ipsative and normative scores on the profile. 12 different computer−generated reports are available. These are: − An ‘Expert Report’, intended for trained users, providing a full set of profiles, ‘Competency Potential Profile’ and ‘Predictive Culture/Environment Fit’. − A ‘Personal Report’ providing a summary of results intended for the test−taker. − A ‘Line Manager Report’, intended for trained or untrained stakeholders such as potential managers. Its use is supervised by a trained user. − The ‘Interview Guide’ provides areas of interest based on Saville Consulting’s Wave Professional Styles, Focus Styles and Work Strengths model that might be pursued in interview. − ‘Summary Development’ and ‘Premium Development Reports’ provide suggestions concerning an individual’s development needs. − A ’Types Report’ classifies individuals according to Saville Consulting Wave Typology. − A ‘Team Roles Report’ indicates an individual’s most and least preferred role in a team. − A ‘Leadership Report’ predicts situational leadership effectiveness. − A ‘Reflections Report’ provides a profile designed to give an opportunity for test−takers to reflect on their workplace styles. − A ‘Sales Report’ includes selling styles and sales potential indicator profiles. − An ‘Entrepreneurial Potential Report’ provides a set of profiles based on the ‘Entrecode®’ model. The Styles assessments were developed as work relevant, high validity, international tools suitable for a range of job roles across different industry sectors. They can be used with most adult working populations and are available in 28 languages. Classification Content Domains: Personality − State Personality − Trait Motivation Intended or main area(s) of Use: Work and Occupational Counselling, Advice, Guidance, and Career Choice Intended mode of use (conditions under which the instrument was standardised and validated): Controlled unsupervised administration. Control over conditions (timing etc) and some control of identify of the test taker (e.g. tests administered over the Internet but only to known individuals − password restricted access) Description of the populations for which the test is intended: This questionnaire can be used with most working adult populations. It is most commonly used for the recruitment and development of professionals, managers and graduates, as well as other occupations. The questionnaire is available in 28 languages other than English, with norm data generated for specific countries and occupational groups. Wave Professional Styles © 2014 BPS Page 3 of 28
  4. 4. Number of scales and brief description of the variable or variables measured by the test: The WAVE Wave Professional Styles questionnaire comprises 108 ‘Facets’ grouped hierarchically into 36 ‘Dimensions’ 12 ‘Sections’ and 4 ‘Clusters’. Four ‘Response Style’ summary scales are also included. The 4 Clusters are ‘Thought, ‘Influence’, ‘Adaptability’ and ‘Delivery’, which comprise the following ‘Sections’: − − Thought: Evaluative, Investigative, Imaginative − Influence: Sociable, Impactful, Assertive − Adaptability: Resilient, Flexible, Supportive − Delivery: Conscientious, Structured, Driven At the base of the hierarchy ‘Sections’ are comprised of the following ‘Dimensions’: − Evaluative: Analytical, Factual, Rational − Investigative: Learning Orientated, Practically Minded, Insightful − Imaginative: Inventive, Abstract, Strategic − Sociable: Interactive, Engaging, Self−promoting − Impactful: Convincing, Articulate, Challenging − Assertive: Purposeful, Directing, Empowering − Resilient: Self−assured, Composed, Resolving − Flexible: Positive, Change Oriented, Receptive − Supportive: Attentive, Involving, Accepting − Conscientious: Reliable, Meticulous, Conforming − Structured: Organized, Principled, Activity Oriented − Driven: Dynamic, Enterprising, Striving. Items format: Likert ratings Forced choice, mixed scale alternatives (ipsative) Number of test items: There are a total of 216 items, two for each of the 108 Facets. Administration modes: Computerised Web−based application − unsupervised/self−assessment• Computerised Web−based application − supervised/proctored• Response mode: Computerised Time: Preparation: N/A Administration: 35 mins. Advised time; 40 mins. Modal completion time. Scoring: N/A Analysis: N/A Feedback: 30 minutes − 2 hours Different forms of the test: Focus Styles is a short form with 72 questions covering 36 of the 108 Professional Styles facets, grouping them in the same higher order clusters and sections. For both versions, there are parallel forms for supervised and unsupervised adminstration. Wave Professional Styles © 2014 BPS Page 4 of 28
  5. 5. Measurement and Scoring Scoring procedure for the test: Computer scoring with direct entry of responses by test taker Bureau−service − e.g. scoring by the company selling the test Scores: Respondents are presented with blocks of six items and indicate their responses on a 9−point Likert scale. Item rankings are initially determined by examining the rating value assigned to each of the six items, with ties broken by a further forced−choice question. Each item therefore receives a normative score ranging from 1 to 9 and an ipsative score ranging from 1 to 6. Scale scores are normed separately at each level of the hierarchy. The final overall Wave Styles trait score for any scale at any level in the Wave model is based on both the normative and ipsative scores. Separate measures are given for ipsative and normative responses, motive and talent responses and facet ranges, which give details of the 108 Facets. A Facet scale is composed of two items (ipsative and normative scores combined) and Dimension scores are thus composed of 6 items (3 facets with 2 items each; with ipsative and normative scores combined), while Motive and Talent scores at the dimensional level are composed of 3 items (one for each facet). Ipsative and normative scores at the dimensional level are based on differences from derived ipsative and normative scores. Competency Potential scores and Culture Prediction scores are also calculated, which are complex ratings of different Facet scores based on their criterion−related validity. Scores are also given for Acquiescence, Consistency, and Normative−Ipsative and Motive−Talent Agreements, based on averaging responses across the profile and standardising them. Sten scales are used throughout for reporting of normed scores. Score transformation for standard scores: Normalised − scores obtained by use of normalisation look−up table• Scales Used: Standard scores: Stens• Computer−Generated Reports Are computer generated reports available with the instrument?: Yes• Number of Computer Generated Reports available: 12 Do distributors offer a service to correct and/or develop computer generated reports?: Yes• Report Name: Expert report Wave Professional Styles © 2014 BPS Page 5 of 28
  6. 6. Brief description of Reports.. Media: Integrated text and graphics• Complexity: Complex• Report Structure: Scale based• Sensitivity to context: One version for all contexts• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Directive and guidance• Intended recipients: Qualified test users• Report Name: Personal report Brief description of Reports.. Media: Integrated text and graphics• Complexity: Simple• Report Structure: Scale based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Clear linkage between constructs, scores and text• Style and tone: Descriptive• Intended recipients: Test takers• Report Name: Line Manager Report Wave Professional Styles © 2014 BPS Page 6 of 28
  7. 7. Brief description of Reports.. Media: Integrated text and graphics• Complexity: Medium• Report Structure: Criterion based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Descriptive• Intended recipients: Qualified system users• Third parties• Report Name: Entrepreneurial Potential Report Brief description of Reports.. Media: Integrated text and graphics• Complexity: Medium• Report Structure: Construct based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Descriptive• Intended recipients: Qualified test users• Test takers• Qualified system users• Third parties• Report Name: Types Report Wave Professional Styles © 2014 BPS Page 7 of 28
  8. 8. Brief description of Reports.. Media: Integrated text and graphics• Complexity: Complex• Report Structure: Construct, factor and criterion based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Descriptive• Intended recipients: Qualified test users• Test takers• Qualified system users• Third parties• Report Name: Summary Development Report Brief description of Reports.. Media: Integrated text and graphics• Complexity: Complex• Report Structure: Criterion based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Guidance• Intended recipients: Qualified test users• Test takers• Qualified system users• Report Name: Premium Development Report Wave Professional Styles © 2014 BPS Page 8 of 28
  9. 9. Brief description of Reports.. Media: Integrated text and graphics• Complexity: Complex• Report Structure: Criterion based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Guidance• Intended recipients: Qualified test users• Test takers• Qualified system users• Report Name: Leadership Report Brief description of Reports.. Media: Integrated text and graphics• Complexity: Complex• Report Structure: Construct based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Guidance• Intended recipients: Qualified test users• Report Name: Interview Guide Wave Professional Styles © 2014 BPS Page 9 of 28
  10. 10. Brief description of Reports.. Media: Integrated text and graphics• Complexity: Simple• Report Structure: Scale and criterion based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Clear linkage between constructs, scores and text• Style and tone: Directive• Intended recipients: Qualified test users• Third parties• Report Name: Reflections Report Brief description of Reports.. Media: Integrated text and graphics• Complexity: Medium• Report Structure: Construct based• Sensitivity to context: One version for all contexts• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Guidance• Intended recipients: Qualified test users• Test takers• Report Name: Sales Report Wave Professional Styles © 2014 BPS Page 10 of 28
  11. 11. Brief description of Reports.. Media: Integrated text and graphics• Complexity: Complex• Report Structure: Construct based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Mixture of clear/concealed linkage between constructs, scores and text• Style and tone: Guidance• Intended recipients: Qualified test users• Report Name: Team Roles Report Brief description of Reports.. Media: Integrated text and graphics• Complexity: Simple• Report Structure: Construct based• Sensitivity to context: Pre−defined context−related versions• Clinical−actuarial: Based on empirical/actuarial relationships• Modifiability: Not modifiable• Degree of 'finish': Publication quality• Transparency: Clear linkage between constructs, scores and text• Style and tone: Guidance• Intended recipients: Qualified test users• Test takers• Qualified system users• Third parties• Supply Condition and Costs Documentation provided by the distributor as part of the test package: User manual Supplementary technical information and updates (e.g. local norms, local validation studies etc) Technical (psychometric) manual Books and articles of related interest Wave Professional Styles © 2014 BPS Page 11 of 28
  12. 12. Methods of publication: Paper Live Internet (test runs in a web browser) Internet download Start−up costs: £670 to transfer from another ‘Level B’ instrument. No start−up costs for bureau system. £2240 for own system implementation (Saville Consulting Oasys Platform). Recurrent costs: There are no licence, lease or system maintenance fees. Prices for a report generated by user installed software: Where the Oasys online administration system has been purchased: Expert Report £55.80 Personal Report £25.20 Line Manager Report £33.50 Entrepreneurial Report £44.70 Types Report £44.70 Summary Development Report £55.80 Premium Development Report £111.60 Leadership Report £89.40 Interview Guide £44.70 Reflections Report £44.70 Sales Report £44.70 Team Roles Report £44.70 Prices for a report generated by the Internet services: Expert Report £83.70 Personal Report £37.80 Line Manager Report £50.30 Entrepreneurial Report £67.10 Types Report £67.10 Summary Development Report £83.70 Premium Development Report £167.40 Leadership Report £134.10 Interview Guide £67.10 Reflections Report £67.10 Sales Report £67.10 Team Roles Report £67.10 Test−related qualifications required by the supplier of the test: Accreditation in general personality and assessment: measures of typical behaviour, attitudes and preferences • Test specific accreditation• Wave Professional Styles © 2014 BPS Page 12 of 28
  13. 13. Professional qualifications required for use of the test: Holder of BPS Certificate of Competence in Occupational Testing Level B• Evaluation of Test Materials Key to symbols: Inadequate No longer used Adequate/Reasonable Good Excellent [N.r.i.o.r] (for updates only) Item was not rated in original review Quality of the explanation of the rationale, the presentation and the quality of information provided: Overall rating of the Quality of the explanation of the rationale: i) Theoretical foundations of the constructs: ii) Test development procedure: iii) Thoroughness of the item analyses and item analysis model: iv) Explanation of content validity: v) Summary of relevant research: Adequacy of documentation available to the user (user and technical manuals, norm supplements etc): i) Rationale: ii) Development: iii) Standardisation: iv) Norms: v) Reliability: vi) Validity: Wave Professional Styles © 2014 BPS Page 13 of 28
  14. 14. Quality of the Procedural instructions provided for the user: i) For test administration: ii) For test scoring, norming etc: N/A iii) For interpretation and reporting: iv) For providing feedback and debriefing test takers and others: v) For providing good practice issues on fairness and bias: vi) Restrictions on use: vii) References and supporting materials: Quality of the materials: i) General quality of test materials (test booklets, answer sheets, test objects, software, etc): ii) Test quality of the local adaptation (if the test has been translated and adapted into the local language): N/A iii) Ease with which the test taker can understand the task: iv) Ease with which responses or answers can be made by the test taker: v) Quality of the items: Reviewer's comments on the documentation (comment on rationale, design, test development and acceptability): Rationale The primary assessment goal of Saville Consulting Wave Styles is to accurately forecast people’s performance at work throughout the employee life−cycle. The 744 page Handbook explains in detail all aspects of the instrument. Much of the document is rightly devoted to the nature of the ‘Professional Styles’ model with detailed explanations of the relationships between the scale descriptions and their interpretation. A supplementary manual referring to the ‘Leadership Report is also available. The comprehensive nature of the ‘Handbook’ is both its strength and weakness: without extensive training on the model the less sophisticated test user might find it difficult to digest. There is no discussion of the pros and cons of using the full (Professional) or short (Focus) version of the test. Design The documentation for the Saville Consulting Wave Styles is professional and detailed, and makes good use of colour and diagrams. The instrument is built around an integrated theoretical model and innovative online format integrates rating and ranking responses and results in a profile that highlights differences between ipsative and normative scores. The Handbook clearly explains these features, and includes advice on how to use the instrument for different purposes (e.g. selection or team development) and illustrative case studies. The Handbook also includes explanations of basic statistical and psychometric concepts, which will be useful for the non−expert reader, and an adequate reference list, albeit that referencing throughout the manual is somewhat patchy. Considering its length, the Handbook would benefit greatly from an index. A ‘quick−reference guide’ would also be helpful. Part 1 of the Handbook, ‘Overview’, goes Wave Professional Styles © 2014 BPS Page 14 of 28
  15. 15. some way to fulfill this need but lacks a summary of some of the basic psychometric data that are presented in detail in later sections. Also omitted is information concerning the Oasys online administration system, which is offered as an alternative to the more expensive email/phone and internet bureau service. Test development Development of the Wave Professional Styles was based on both inductive and criterion−related validity at both the predictor and criterion ends of assessment. Neither method focused on empirical validity until late in the development process when a validation−centric approach to item selection identified items or item groupings (facets) that had the best criterion−related validity, rather than selecting items on the basis of how they correlate with other items or how they contribute to a scale’s internal structure. The authors claim that an advantage of this approach is the ability to create short and concise but highly valid scales. Chapter 16 of the Professional Styles Handbook thoroughly outlines the development stages, beginning with preliminary concept trials and running through to supporting materials. However, little information is presented about the item and factor analyses that were conducted during development. Acceptability The questionnaire is clearly presented on screen with easily understandable instructions suitable for those familiar with everyday computer use. Items in the test are relevant to work and likely to be acceptable to those being assessed for employment or career development. Readability statistics are satisfactory and an adequate ‘test−takers’ preparation guide is available to download. In summary the candidates’ experience of taking the questionnaire should be a good one. From a test user viewpoint, the Handbook is impressive in its size and appearance, and appears to contain all of the necessary information, but it is not an easy read. Norms, Validity & Reliability Evaluation of technical information: Norms or reference group information: Overall adequacy: i) Appropriateness for local use, whether local or international norms: ii) Appropriateness for intended applications: iii) Sample sizes: iv) Procedures used in sample selection: Norm tables based on operational use, UK Management, US Management, International Management Mixed Occupational, Graduates, Australian Professionals and Managers v) Quality of information provided about minority/protected group differences, effects of age, gender etc: Reviewers' comments about the norms: The Wave Styles ‘Handbook’ provides details of several norm groups: two sets of data were collected for the standardization group (‘Invited Administration’ and ‘Supervised Administration'), and all others were Wave Professional Styles © 2014 BPS Page 15 of 28
  16. 16. collected under ‘Invited Administration’ conditions. Normative data for different samples were collected between 2005 and 2009. Details of each sample are given in Appendices to the ‘Handbook’. These include a breakdown of the gender, cultural background, education, work function and work experience of each sample, but the tables are not stratified by any of these variables because differences in the mean scores were relatively small. Norms (mean and SD) are provided at the Dimensions level of the Wave Professional Styles hierarchy only, not at the Cluster, Section or Facet levels, and not for Focus Styles. When generating reports the norm group used is determined by the Oasys administrator; there is no facility for the user to nominate a norm group. The reports provided by the Oasys system describe performance in terms of Sten scales. These are more informative for the reader than raw scores, which are not available to the test user, but no information is provided about the mapping of raw scores into Sten scales, so the extensive norm tables in the Handbook are useful only for reference, not for interpreting outputs. Norms generated from the standardization of the instrument are based on 1153 ‘Professionals’. Females comprised 53% of the standardization sample, with 5% of the sample reported as being from Indian, Pakistani, Chinese, Asian Other, Black Caribbean Black African or mixed cultural backgrounds. Highest qualifications ranged from 12% postgraduate to 2% no qualifications. 37% had a degree (12% postgraduate, 25% undergraduate), 15% professional qualifications, 7% HNC/HND or equivalent, 35% school level qualifications and 4% Other qualifications. Over a third of the sample had been employed for more than 20 years with fewer than a third working for less than 10 years. The other norm sets are − UK (total N=33730): Senior Managers and Executives (n=4,276); Professionals & Managers (n=9,884); Mixed Occupational Group (n=10,953); Graduates (n=4,021); Individual Contributors (n=3,190). US (total N=4271): Senior Managers and Executives (n=597); Professionals & Managers (n=1,849); Mixed Occupational Group (n=2,143); Graduates (n=685); Individual Contributors (n=323). International (total N=13,333): Senior Managers and Executives (n=953); Professionals & Managers (n=2,600); Mixed Occupational Group (n=3,095); International Graduates (n=1,423); Individual Contributors (n=2,202). Australia: Professionals & Managers (n=474). The ‘Handbook’ reports that Wave Styles is being produced in over 25 languages and users are invited to contact their local office for details. It is not clear however whether the instrument is simply being translated into other languages or whether norms are also being generated and standardized. Despite reservations concerning the possible need for re−standardizing the test for some groups there are appropriate norms for most applications of the questionnaire. However, it should be noted that the norm samples were heavily biased toward more educated participants. Validity: Overall adequacy: Construct Validity (overall adequacy): i) Designs used: Correlations with other instruments and performance criteria Intra−scale (item−rest correlations) Exploratory Factor Analysis Confirmatory Factor Analysis ii) Sample sizes: Wave Professional Styles © 2014 BPS Page 16 of 28
  17. 17. iii) Procedure of sample selection: Incidental iv) Median and range of the correlations between the test and other similar tests: v) Quality of instruments as criteria or markers: vi) Differential Item Functioning (DIF) analyses: N/A Criterion−related validity: overall adequacy: i) Description of the criteria used and characteristics of the populations: Concurrent Predictive ii) Sample sizes: iii) Procedure of Sample selection: Incidental iv) Median and range of the correlations between the test and criteria: Reviewers' comments about validity: The authors discuss the importance of validity and provide numerous examples of the patterns of correlations of Wave Style measures and other established tests. Quite rightly the emphasis is on is on explaining to the reader what the Wave scales are measuring. Results from ‘Project Epsom’ a study involving 308 participants tested with the Wave Professional Styles questionnaire are quoted as evidence of construct validity. Factor analyses support an alignment of the Wave Dimensions with the Great Eight work performance styles suggested by Kurz and Bartram (2002), with very high item−factor loadings and very little cross−loading. Subsequent analyses show correlations of these performance styles and five other instruments, the Saville PP, OPQ®, NEO−PI−R, Hogan Personality Inventory, 16PF, DISC, DISCUS and MBTI. Although extensive evidence of construct validity is quoted a clear picture suitable for interpretation by an average user does not always emerge. In particular, separate tables for each of the 36 Wave dimensions show that they vary greatly in the extent of correlations with other instruments (from zero to significant correlations with almost all other scales); this is not discussed. There is a similar lack of discussion in relation to the factor analysis aligning the standardization sample to the Great Eight performance styles. Concurrent validities are again derived from the ‘Project Epsom’. Additional US (n=399) and Mexican (n=120) evidence is also presented. In Project Epsom, the Wave Professional and Focus Styles and all of the other instruments used were evaluated against the SHL Great Eight competency framework. Mean correlations of Wave Professional Styles with external ratings of work performance competencies were 0.20 and 0.22 at the Dimension and Section levels, respectively, but these values rose to 0.38 and 0.44 when corrected for criterion unreliability. Correlations with measures of global overall performance, adjusted for criterion unreliability, were .49 for Professional Styles and .38 for Focus Styles. On both forms of the Wave, correlations for measures of Wave−derived Competency Potential scores were marginally higher than Styles scores. Again, the results are presented with minimal discussion. Evidence of predictive validity is less extensive, this being largely restricted to a study of 108 individuals rated externally on Wave Professional Styles and the ‘Wave Performance 360’ questionnaire. The median correlation with external ratings of work performance competencies was 0.17, rising to 0.33 after adjustment for reliability. These data are presented at the Section level only owing to the smaller sample size. In addition to scale validities separate concurrent and construct validity coefficients are quoted for the measures derived for Types, Team Roles, Leadership, Reflections, Sales, and Entrepreneurial Potential. Wave Professional Styles © 2014 BPS Page 17 of 28
  18. 18. Evidence of construct validity is based on correlations with either the Hogan Development Survey (n=306) or with the competency scores derived from the OPQ32 (n=308). Concurrent validity is based on correlations with external ratings of performance. Overall, Wave shows good evidence of both construct and criterion validity. However, whilst adequate evidence of concurrent validity is presented Wave could benefit from more predictive validation studies against a more extensive set of criteria. Reliability: Overall adequacy: i) Data provided about reliability: Reliability coefficients for a number of different groups Standard error of measurement given for a number of different groups Internal consistency: i) Sample size: ii) Median of coefficients: Test retest stability: i) Sample size: ii) Median of coefficients: Equivalence reliability: i) Sample size: ii) Median of coefficients: Reviewers' comments on Reliability (comment on confidence intervals for reliability coefficients and provide Spearman Brown equivalents for a 30−item scale): At the Dimension level, Internal consistency statistics based on Cronbach’s Alpha and using the 1,153 standardization sample are presented for the combined normative−ipsative for both ‘Invited Access’ (range .58 to .86, median .76) and ‘Supervised Access’ administrations (range .65 to .87, median .78). A US sample of 2,102 at the ‘Dimension’ level demonstrates alphas ranging from .52 to .84 with a median of .74. Although not high, these are acceptable levels of consistency. Higher levels of consistency (medians > .8) are reported at the Section level of the hierarchy. Test−retest reliability data for the Wave Professional Styles are quoted for a sample of 100 ‘Invited Access’ participants with an 18 month period between test administrations for both ‘Dimension’ (range .58 to .85, median .74) and ‘Section’ (normative scoring: range .76 to .86, median .79) levels. Six month test−retest reliabilities are also reported for the thirty−six facet scales that are used in the Wave Focus Styles assessment (n=214) with results ranging from .58 to .84 (median .72). Although involving smaller samples than used in the internal consistency studies the test−retest results can be viewed as being good. Alternate form reliability coefficients for normative, ipsative and combined normative−ipsative dimensions are quoted for both ‘Dimension’ and ‘Section’ levels. The alternate forms of the instrument are two independent sets of 216 questions that are used in different ways of administering the questionnaire – ‘Invited Access’ where candidates take the test unsupervised and ‘Supervised Access’. The median ‘alternate form’ of combined normative and ipsative ‘Dimension’ scores Wave Professional Styles © 2014 BPS Page 18 of 28
  19. 19. was .87. ‘Section’ combined scores yielded a median reliability of .92 with a minimum of .86. It is not clear whether the order of administration of the two methods of administration was controlled. A further study was conducted to compare the 36 Wave Professional Styles Dimensions with the 36 Facets of the Wave Focus Styles (n=308). This study reported a median correlation of .71 (range .50 to 84), indicating, perhaps unsurprisingly, that the short form of the Wave (Focus Styles) is less reliable than the long (Professional Styles) version. As with validities in addition to scale reliabilities further figures are quoted for the derived measures, in the respective chapters of the Handbook. − For People Types and Task Types, alternative form reliabilities were .95 and .93 and test−retest reliabilities were .84 and .83. − For eight Team Roles all alternative form reliability coefficients exceeded .91 and Test−Retest reliabilities ranged from .79 to .85 (median .80). − For twenty−four Leadership Style scales alternative form reliabilities ranged from .81 to .95 (median .88) and test−retest reliabilities ranged from .49 to .85 (median .77). − For six Leadership Base scales alternative form reliabilities ranged from .82 to .92 (median .92) and test−retest reliabilities ranged from .74 to .84 (median .78). − For nine Reflections scales alternative form reliabilities ranged from .87 to .94 (median .90) and test−retest reliabilities ranged from .74 to .85 (median .81). − For twenty Sales scales alternative form reliabilities ranged from .86 to .95 (median .91) and test−retest reliabilities ranged from .70 to .85 (median .79). − For twenty−seven Entrecode® Entrepreneurial Potential Scales alternative form reliabilities ranged from .70 to .94 (median .85) and test−tetest reliabilities ranged from .57 to .84 (median .73). Quality of Computer−Generated Reports Report Name: Expert report i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 10 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 2 Report Name: Personal Wave Professional Styles © 2014 BPS Page 19 of 28
  20. 20. i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 4 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 1 Report Name: Line Manager i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 3 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 2 Wave Professional Styles © 2014 BPS Page 20 of 28
  21. 21. Report Name: Entrepreneurial i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 5 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 2 Report Name: Interview Guide i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 15 Wave Professional Styles © 2014 BPS Page 21 of 28
  22. 22. viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 13 Report Name: Summary Development Report i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 18 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 11 Report Name: Types Report i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: Wave Professional Styles © 2014 BPS Page 22 of 28
  23. 23. vii) Length−number of printed pages: 5 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 8 Report Name: Team Roles i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 4 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 5 Report Name: Leadership i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: Wave Professional Styles © 2014 BPS Page 23 of 28
  24. 24. vi) Practicality: vii) Length−number of printed pages: 11 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 3 Report Name: Reflections Report i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 7 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 1 Report Name: Sales Report i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: Wave Professional Styles © 2014 BPS Page 24 of 28
  25. 25. v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 3 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 2 Report Name: Premium Development Overall adequacy of computer−generated reports: i) Scope or coverage: ii) Reliability: iii) Relevance or Validity: iv) Fairness, or freedom from systematic bias: v) Acceptability: vi) Practicality: vii) Length−number of printed pages: 29 viii) Length index (number of pages (including composite and derived scales) divided by number of scales), multiplied by 10 and expressed as an integer: 8 Reviewers' comments on the quality of computer generated reports: Wave can produce 12 computer generated reports: Expert Report , Personal Report, Line Manager Report, Entrepreneurial Report, Types Report, Summary Development Report, Premium Development Report, Leadership Report, Interview Guide, Reflections Report, Sales Report and Team Roles Report. The reports are clearly presented and laid out and make good use of colour. Personal and Line Manager reports, which provide lay feedback, are provided free if the Expert Report is purchased. The other reports, which provide more detail, are available to purchase individually. The Handbook contains an example of each type of report (other than the Premium Development Report). All reports carry a clear ‘health warning’ that the contents are based on the test−taker’s self−report. The Handbook contains a short section of guidance on how to detect and explore potential faking and distortion. The Development Reports provide advice to improve performance at work, under four headers: building strengths, possible overplayed strengths (‘watch fors’, with action points), development tips and managing limitations. However, it is not obvious how this advice is derived from the psychometric data. The Interview Guide provides two to four questions (with follow−ups), derived from the psychometric assessment, on Wave Professional Styles © 2014 BPS Page 25 of 28
  26. 26. each of the Sections of the Wave hierarchy. All of the other reports rely heavily on the presentation of data as profile scores and although each has an explanatory paragraph many users might prefer more by way of a narrative summary account. Adequate training on Wave would seem to be essential if most is to be made of these reports as the selection and interpretation could be seen as being key to their successful use. This is particularly the case for the Expert Report, which presents detailed information on each of the 36 dimensions with little ‘unpacking’ or comment. Although the ‘Handbook’ suggests a degree of training is required to use the reports this is not always made clear in the report itself. In summary the comprehensive range of reports available to the user is a strength of Wave, especially if the user is technically sophisticated enough to undertake the processing of the information provided. FINAL EVALUATION Evaluative report of the test: The version of Wave Professional Styles under review was published in 2005, and the current version of the Handbook was published in 2012. Wave consists of 216 questions, which are organized hierarchically as 4 ‘clusters’, 12 ‘sections’, 36 ‘dimensions’ and 108 ‘facets’, each of which contains 2 questions, one about motive (will do) the other about talent (can do). The short form of the test (Wave Focus Styles) uses a single 2−question facet corresponding to each of the 36 dimensions. The scoring is based on an innovative combination of normative and ipsative methods. The outputs of the test include individual profiles at different levels of the hierarchy, a typology of 16 combinations of people and task types, and analyses of divergence between normative and ipsative scores, and between motives and talents. It is a tribute to the authors that responses to just 216 items can produce so much information. The test is intended to provide information that could be useful for a wide variety of occupational applications at any stage of the employment cycle. If someone is looking for a ‘one stop shop’ for a ‘typical’ assessment of occupational behaviour, Wave attempts to do just that. It is therefore likely to be particularly attractive to the busy test−user who may not have time to select a battery of tests, perhaps from different publishers. However, Wave Styles produces a wealth of output and a considerable investment in time is required to use it to the full. The documentation on the test – a 744 page Handbook – is impressive, though much of the technical data presented would benefit from a more extensive narrative to spell out the implications of the data presented in a multiplicity of well laid−out tables, and the absence of an index in a volume of this size is a serious limitation. The 12 computer−generated reports are similarly attractive on the eye, but again present a wealth of information that would be easier to digest with more textual interpretation. It is not obvious from the Handbook which reports are produced for the short (Focus) version of the test and how, if at all, the Focus and Professional reports differ. Wave is administered and scored electronically. There is no paper version, and no access to the raw scores, leaving the user wholly dependent on the computer−generated reports. Psychometric information in the reports is presented as Sten scores, which are clear and easily comprehensible, but do not relate in a transparent way to the norms presented in the Handbook. Psychometrically, the test has been well constructed. The test has good reliability, and norms are presented for many different well−described occupational groups, although how the standardization sample was recruited is a little unclear. It is disappointing that more evidence of criterion based validity is not present, not because insufficient studies are quoted but rather because of the narrowness of the criteria used. There is a heavy reliance on ratings and ratings mainly on one hierarchical competency framework. Many will be happy with this, but it would have been reassuring if additional criteria were also employed, including more evidence of predictive validity. Having said this there is every reason to believe that Wave will give good results in its intended areas of use. The test is available in a number of languages, but it is not clear whether separate standardizations have been carried out for non−English speaking groups and caution should be exercised if use with different cultures is contemplated. A user’s confidence in the use of Wave will to a large extent depend on the comprehensiveness of the training given by the publisher. A thorough grounding in the nature of the measures available together with the allied performance frameworks appears essential. Equally important is training on the feedback process. Here the ‘Handbook’ provides useful guidance. Wave Professional Styles © 2014 BPS Page 26 of 28
  27. 27. In summary Wave Professional Styles is a good ‘omnibus’ measure with a wide range of applications, but comprehensive training on the underlying model is essential for its safe use. Conclusions: Wave Professional Styles is a well−developed and well−researched test that is suitable for use in all the contexts for which it was designed. However, Wave could benefit from more predictive validation studies against a more extensive set of criteria. The Wave is well presented on−line and easy for the test taker to use. It produces in a single test the information that might be gained from the administration of several other instruments and will be a valuable tool in the kitbag of trained professionals. The computer−based reports are comprehensive and detailed, albeit many of them are hard work to read because they include little by way of narrative to explain the data presented. A note of caution concerns the vast amount of data that is generated from a single test sitting. Safe use of this test will be enhanced by comprehensive training on the models on which the Wave is based and the performance frameworks with which it is associated. Recommendations: Suitable for use in the area(s) of application defined by the distributor, by test users who meet the distributor's specific qualifications requirements • SUMMARY EVALUATION OF THE TEST Content Domains: Personality − State Personality − Trait Motivation Intended or main area(s) of Use: Work and Occupational Counselling, Advice, Guidance, and Career Choice Intended mode of use (conditions under which the instrument was standardised and validated): Controlled unsupervised administration. Control over conditions (timing etc) and some control of identify of the test taker (e.g. tests administered over the Internet but only to known individuals − password restricted access) Test Description: Test Name: Wave Professional Styles Local test distributor / publisher: Saville Consulting UK Ltd Date of Current Review: 2014 Date of Publication of Current Review/Edition: 2005 Constructs Measured: The 108 Facets are hierarchically organized into 36 ‘Dimensions’, which are further grouped into 12 ‘Sections’ (Evaluative, Investigative, Imaginative, Sociable, Impactful, Assertive, Resilient, Flexible, Supportive, Conscientious, Structured and Driven). Wave Professional Styles © 2014 BPS Page 27 of 28
  28. 28. These combine to create 4 ‘Clusters’ (Thought, Influence, Adaptability and Delivery). Administration Mode: Computerised Web−based application − unsupervised/self−assessment Computerised Web−based application − supervised/proctored Response Mode: Computerised Instrument Evaluation: Characteristics Evaluation Quality of Documentation Quality of Materials Norms and reference groups Construct validity Criterion−related validity Reliability−overall Number of Computer−Generated Reports 12 The British Psychological Society © 2014. All rights reserved. No part of this publication may be reproduced or transmitted, in any form or by any means, without permission. Debug mode: **0 (none), 1 (basic), 2 (advanced), 999 (edit) Wave Professional Styles © 2014 BPS Page 28 of 28

×