Loading...
Res No 155-20-15595Agenda Item No:Ba. City Commission Agenda Item Report Meeting Date: November 2, 2020 Submitted by: John Barzola Submitting Department: Police Department Item Type: Resolution Agenda Section: Subject: A Resolution authorizing the City Manager to expend an amount not to exceed $8,100 to Miami Dade College-School of Justice Assessment Center for the Sergeants promotional exam. 3/5 (City Manager-Police Dept.) Suggested Action: Approve Attachments: Memo-Sergeants_Test_2020 (1).docx Reso - Sergeants Test 2020.docx Proposal_South Miami PD 2021.pdf IOS Proposal - South Miami, FL - Promotional Testing for Sergeants - 10.8.20.pdf PSI Cost Estimate.pdf PSI Police Promotional Overview 2020 with Appendices.pdf PSI Public Safety Projects Within Last 5 Years 2020.xlsx 1 CITY OF SOUTH MIAMI OFFICE OF THE CITY MANAGER INTER-OFFICE MEMORANDUM TO:The Honorable Mayor & Members of the City Commission FROM:Shari Kamali, City Manager DATE:November 2, 2020 Agenda Item No.:_____ SUBJECT:A Resolution authorizing the City Manager to expend an amount not to exceed $8,100 to Miami Dade College-School of Justice Assessment Center for the Sergeants promotional exam. BACKGROUND:The Police Department provides professional promotional process for the position of Sergeant. No current eligibility exists. The retirement of Sgt. Lisa King leaves an immediate opening for Sergeant. Three (3) quotes were solicited from professional testing services to provide a professional promotional examination. Miami Dade College-School of Justice Assessment Center provides professional Police Sergeant Promotional process. The Miami Dade College- School of Justice provided a quote $8,100 which was the lowest quote provided for services requested. The South Miami Police Department has used Miami Dade College on 2 previous occasions for the Sergeants Promotional Test and is satisfied with the services received. Quotes Received: Miami Dade College- School of Justice $8,100 Industrial/Organizational Solutions (IOS)$34,210 PSI Services $45,000-$50,000 The Badge exercise is also included and is a job-relevant scenario which is administered orally by a test proctor. Candidate’s responses are recorded and evaluated by 3 assessors. The scenarios vary from critical incident, citizen complaint, personnel issues, employee counseling, disciplinary procedures, use of force, etc. 2 CITY OF SOUTH MIAMI OFFICE OF THE CITY MANAGER INTER-OFFICE MEMORANDUM VENDOR & PROPOSAL AMOUNT: VENDORS AMOUNT MIAMI DADE COLLEGE-SCHOOL OF JUSTICE $8,100 FUND & ACCOUNT:The total expenditure shall be charged to the Police Contractual Services, account number 001-1910-521-3450, which has a balance of $102,006 before this request was made. ATTACHMENTS:Resolution Miami Dade College Proposal IOS Proposal PSI Services Proposal 3 RESOLUTION NO.: _______________1 2 A Resolution authorizing the City Manager to expend an amount not to exceed $8,100 to 3 Miami Dade College-School of Justice Assessment Center for the Sergeants promotional 4 exam.5 6 WHEREAS, the Police Department is in need of providing a Sergeants promotional test to 7 conform to the current Collective Bargaining Agreement; and 8 9 WHEREAS, the Retirement of Sgt. Lisa King has left a promotional opportunity for officers 10 who have three (3) years seniority within the Department at the time of the examination; and11 12 WHEREAS, the City desires to provide a professional promotional examination and 13 opportunity for advancement; and 14 15 WHEREAS, a request for quotes for professional promotional examinations for providing a 16 promotional test for Sergeants revealed that Miami Dade College offered the test for $8,100; and17 18 WHEREAS, the quote in the amount of $8,100 provided by Miami Dade College is for a 19 written exam and badge exercise; and20 21 NOW, THEREFORE, BE IT RESOLVED BY THE MAYOR AND CITY COMMISSION OF THE CITY 22 OF SOUTH MIAMI, FLORIDA THAT:23 24 Section 1.The foregoing recitals are hereby ratified and confirmed as being true and they 25 are incorporated into this resolution by reference as if set forth in full herein.26 27 Section 2:City Manager is authorized to negotiate and enter into an agreement with Miami 28 Dade College- School of Justice for the Sergeants Exam administration. This expenditure will be 29 charged to account # 001-1910-521-3450 Contractual Services.30 31 Section 3. Corrections. Conforming language or technical scrivener-type corrections may 32 be made by the City Attorney for any conforming amendments to be incorporated into the final 33 resolution for signature.34 Section 4.If any section clause, sentence, or phrase of this resolution is for any reason held 35 invalid or unconstitutional by a court of competent jurisdiction, the holding shall not affect the 36 validity of the remaining portions of this resolution.37 38 Section 5. Effective Date. This resolution will become effective immediately upon adoption.39 40 41 PASSED AND ADOPTED this _______ day of _______________, 2020.42 43 ATTEST:APPROVED:44 45 4 _____________________________________________________46 CITY CLERK MAYOR47 48 READ AND APPROVED AS TO FORM COMMISSION VOTE:49 LANGUAGE, LEGALITY AND EXECUTION Mayor Philips50 THEREOF Vice Mayor Welsh 51 Commissioner Gil52 ____________________________Commissioner Liebman 53 CITY ATTORNEY Commissioner Harris54 5 City of South Miami – 2021 Sergeant Promotional Process PROPOSAL South Miami Police Department Police Sergeant Promotional Process October 7, 2020 Prepared By: The Assessment Center Miami Dade College – School of Justice 11380 NW 27th Avenue, Suite 8324 Miami, Florida 33167-3495 P: 305.237.8012 F: 305.237-1807 lguinov@mdc.edu 6 1 | P a g e City of South Miami – 2021 Sergeant Promotional Process TABLE OF CONTENTS Table of Contents PART I – MIAMI DADE COLLEGE ASSESSMENT CENTER DESCRIPTION ......................................................... 3 PART II – SCOPE OF SERVICES ....................................................................................................................... 1 PART V– COST ............................................................................................................................................... 6 7 2 | P a g e City of South Miami – 2021 Sergeant Promotional Process October 7, 2020 South Miami Police Department Chief Rene Landa 6130 Sunset Drive South Miami, FL 33143 Miami Dade College’s Assessment Center presents to the City of South Miami its proposal of our approach to partner with the Police Department for the upcoming promotional process involving the rank of Police Sergeant. We present our methodology to demonstrate that we are fully prepared to meet and exceed all of the personnel selection objectives discussed during our meeting. Our methodology complies with testing guidelines and professional standards set forth by the Uniform Guidelines on Employee Selection Procedures, the Principles for the Validation and Use of Personnel Selection of the Society for Industrial and Organizational Psychology (SIOP), and the Guidelines and Ethical Considerations for Assessment Center Operations. Our objective is to meet all of the Department’s needs in conducting a fair and valid promotional process. Our services will include:  Review of job analyses for the rank of Sergeant  Review of bibliographies (reading lists) currently used as the basis of job knowledge tests and recommendations for revisions  Conduct general orientation session for Sergeant candidates  Development of written examination  Administration of all exercises and tests developed  Recruitment, screening, selection and training of assessors  Oversight of the evaluation process  Preparation and delivery of results  Provision of performance feedback to candidates Additionally, Miami Dade College will assist the Department in addressing any concerns, requests and questions regarding the entire promotional process. Thank you for your time and for the opportunity to provide this service to your Department. If you have any questions please do not hesitate to contact us at 305-237-8012, or via e-mail at LGUINOV@MDC.EDU. Sincerely, Lisa M. Guinovart, M.S. Assessment Center Director 8 3 | P a g e City of South Miami – 2021 Sergeant Promotional Process PART I – MIAMI DADE COLLEGE ASSESSMENT CENTER DESCRIPTION BACKGROUND The School of Justice Assessment Center (Assessment Center) has an unparalleled depth and breadth of experience in public safety assessment ranging from entry level to chief's level positions. The Assessment Center is fully prepared to meet or exceed all of the personnel selection objectives of the communities. The Assessment Center was created in 1981 by Special Response Funds from the Law Enforcement Assistance Administration (LEAA) and for nearly 30 years, has provided comprehensive selection and promotional assessment services to a variety of public safety agencies nationwide. Backed by Mia mi Dade College, an education landmark in South Florida with a long and stable history, the Assessment Center shares in a long-standing tradition of excellence. We have assessed over 22,000 individuals for various positions in public safety and have earned a solid reputation of integrity among public safety agencies. Since 1985, the Assessment Center has assisted city and county managers in external recruitment for chiefs of police and other executive positions such as directors of corrections, finance, p ublic works, and data systems. The Assessment Center has the experience and resources to conduct job analyses, develop and implement job-relevant exercises, evaluate records, and analyze performance data for any public safety position. Testing programs are tailored to individual agencies’ needs and range from content -valid written tests and assessment-based oral boards to full-scale, complex assessment centers that include telecommunications equipment and multimedia. The Assessment Center serves as a model for the development of other assessment centers throughout the United States and internationally. Prompted by the Center’s successful record of accomplishment, agencies including the Federal Bureau of Investigations (FBI) and the London Metropolitan Po lice have benchmarked with our programs to establish similar assessment programs. The Center has also collaborated with law enforcement departments to provide training programs for their personnel to serve as assessors in our record evaluation processes. Maintaining a long standing working relationship with the law enforcement community, the Center has developed and implemented over 170 assessment programs nationwide, primarily in law enforcement testing. The Center has served over 25 agencies in the sta te of Florida, many on a continuing basis. All entry level and promotional assessments for the Miami-Dade Police Department have been conducted by the Assessment Center since 1983. The Center has also served agencies in the states of Virginia, South Carolina, North Carolina, Alabama, and Nebraska. The Assessment Center is staffed by a director, two assessment analysts, five full-time and part-time administrative staff members and two contracted Psychologists. Our team of dedicated professionals, with 9 4 | P a g e City of South Miami – 2021 Sergeant Promotional Process advanced degrees in the behavioral sciences and extensive experience in assessment, strive to develop customized solutions to meet your personnel selection and assessment needs. COMPANY PROFILE (1) Company Contact Information: Miami Dade College, North Campus School of Justice Assessment Center 11380 NW 27th Avenue, Room 8324 Miami, FL 33167 Phone: (305) 237-1476 Fax: (305) 237-1858 (2) Authorized Representative Name, Title, Address, and Phone Number: Raimundo Socorro, Ph.D., School of Justice Director Miami Dade College, North Campus School of Justice 11380 NW 27th Avenue, Room #8310 Miami, FL 33167 Phone: (305) 237-1508 (3) History: Miami Dade College was established in 1959. It is a multi -campus, four-year, state-supported college with eight campuses and numerous outreach centers. Miami Dade College is nationally recognized as one the largest and best colleges in the country. It is governed by a seven -member District Board of Trustees and a college president. The Miami Dade College School of Justice Assessment Center was established in 1981 with Special Response Funds from the Law Enforcement Assistance Administration to provide personnel selection services for public safety. (4) Contract Team: Lisa M. Guinovart, M.S. – Assessment Center Associate Director Armando Falcon, Ph.D. - Assessment Analyst Chantal Ramirez, M.S. – Assessment Analyst (5) Accreditations: Miami Dade College is accredited by the Commission on Colleges of the Southern Association of Colleges and Schools. 10 City of South Miami – 2021 Sergeant Promotional Process 11 City of South Miami – 2021 Sergeant Promotional Process PART II – SCOPE OF SERVICES The City of South Miami Police Department is seeking a qualified consultant to provide the following services in relation to promotional testing for the rank of Police Sergeant:  Review of job analyses for the rank of Sergeant  Review of bibliographies (reading lists) currently used as the basis of job knowledge tests and recommendations for revisions  Conduct a general orientation session for Sergeant candidates  Development of written examination  Development of an assessment exercises (BADGE & Structural Interview)  Administration of all exercises and tests developed  Recruitment, screening, selection and training of assessors  Oversight of the evaluation process  Preparation and delivery of results  Provision of performance feedback to candidates SUBJECT MATTER EXPERT PANEL The development of the assessment exercises will involve a content validity strategy and will comply with professional standards for personnel selection as well as the Department’s applicable rules and regulations and collective bargaining agreements. The validation process will involve the use of subject matter experts (SMEs) from the South Miami Police Department. All SME panels described in this response will meet the following requirements:  SME panel collectively represents the demographics of the employee population with respect to gender, age, race, years of experience, etc.  SMEs must be experienced and active in the target position (e.g. SMEs should not be on probationary status or should not be temporarily assigned to the position). At least one year of job experience in the target rank is required to serve as a SME.  SME panel collectively represents various functional areas and/or shifts of the target rank.  SME panel must include approximately 10% to 20% supervisors of the target rank. SECURITY PLAN Test security is a fundamentally important consideration for all testing programs developed and administered by the Assessment Center. Security is of vital importance to the integrity of the overall system. The methodology used by the Assessment Center involves a promotional testing procedu re that 12 2 | P a g e City of South Miami – 2021 Sergeant Promotional Process minimizes the risk of test exposure. However, there remains an issue of test security. The Assessment Center has many safeguards in place to maintain the security of exercises, checklists, and related test materials. Subject Matter Expert Security Subject Matter Expert (SME) panels are structured to use progressively higher levels of expert review. Job incumbents are used for initial development work (e.g. Job Analyses and Critical Incident Reports). Personnel at or above the respective rank are used for exercise refinement. All SME panel members are required to signify their consent to abide by all Assessment Center procedures by signing security/confidentiality agreements that detail security and confidentiality procedures. All SME panel meetings are conducted in restricted access environments. All SME notes are collected after each meeting. It is the position of the Assessment Center that agencies should communicate their firm commitment to protect and preserve the integrity of systems developed t o facilitate the mobility of personnel up the ranks within their organization. All agencies served by the Assessment Center are strongly encouraged to include in their standard operating policies and procedures a clearly articulated position outlining th e obligation of their agency personnel to maintain the security and confidentiality of testing processes. These test security measures help ensure that all materials related to a specific assessment center are tightly controlled and accounted for at all times, and that SMEs clearly understand the criticality of keeping test materials confidential. Physical Security During the development, administration, and feedback stages of an assessment center, all test exercises and related materials are secured by th e Assessment Center in a restricted-access facility located in the School of Justice. This facility is located within an alarmed area of the School of Justice. Only Assessment Center staff members have access to this facility. During the evaluation stage o f an assessment center, all test exercises and related materials are secured in a restricted-access facility located at the evaluation site. Assessment Center personnel supervise the transport of test exercises and related materials to and from test development and/or evaluation sites. In addition, all of our test materials are duplicated in-house at the Miami Dade College North Campus Copy Center. During duplication, the Copy Center closes its doors to the entire campus to work exclusively on the Center’s job requests. A Center staff member remains physically present at the Copy Center until the duplication is complete. The staff member then transports the documents back to the Center for storage in the vault until test administration. Administration Security Assessment Center personnel supervise all Assessment Center activities. Candidates are not permitted to bring any notes, books, supplies, or communication devices to the testing site. All candidates are given a standard set of supplies for testing pur poses. In order to minimize the exposure of test exercises, stringent measures are applied to the development of test administration schedules. All test components have a maximum exposure time of one day. Where necessary, candidates are sequestered to avoid the likelihood of information sharing that could compromise test security. 13 3 | P a g e City of South Miami – 2021 Sergeant Promotional Process JOB ANALYSIS The Assessment Center will review the current Police Sergeant job analyses to determine the comprehensive range of aptitudes, skills, abilities, and other traits required, in order to identify candidates best suited to perform successfully in the target positions. WRITTEN EXAMINATION DEVELOPMENT The Assessment Center will develop a 110-item multiple-choice written exam for the rank of Sergeant, using source materials selected by the South Miami Police Department. Each item will include an answer key and the appropriate source citation (including page number) for each test item. The written tests will be submitted for review by a SME panel. SMEs will be presented with written test items created by the Assessment Center for review and will decide collectively, under the guidance of a professional Assessment Center staff member, whether items should be included on the exams on the basis of clarity, relevance, and level of difficulty. Items approved by the SME panel will be included written aptitude tests. The written test will include a candidate test booklet and an annotated key. CANDIDATE ORIENTATION SESSION The Assessment Center will facilitate a candidate orientation session for the rank of Sergeant. Candidates will be provided with an overview of the testing process and with information regarding each exercise module and dates, times and locations for each component of the process. Additionally, candidates will receive an orientation/preparation guide with helpful hints. Candidates will also have the opportunity to ask questions at the end of the session. The Assessment Center will provide the venue, handouts and other necessary materials for the orientation. WRITTEN EXAM ADMINISTRATION & EVALUATION Before the test begins, candidates will be required to provide photo ID, sign in, and sign a copy of the test rules. The examination will be timed and once the allotted time frame expires, candidate answer sheets will be collected and sealed for scoring by the Assessment Center. Item appeals will be handled in compliance with collective bargaining agreement requirements. Item appeals will be reviewed by a SME panel under the guidance of Assessment Center professional staff to evaluate the merits of the appeal and make decisions on whether the appeal should be upheld or denied. The Assessment Center will prepare written response to candidates regarding their appeals. Additionally, the Assessment Center will conduct an item analysis to identify the existence of technically flawed items. Points will be awarded for items meeting pre-determined item analysis criteria for exclusion to all candidates unless otherwise requested by the Department. Candidates will be notified regarding their written test scores prior to undergoing the assessment portion of the promotional process. There will be a cut-off score on the written examination as requested and 14 4 | P a g e City of South Miami – 2021 Sergeant Promotional Process determined by the Department. Candidates are required to meet this cut-score score to proceed to the Assessment (BADGE) exercise of the promotional process. The Assessment Center will provide the venue, proctors, and materials necessary for the written exam administration. ASSESSMENT EXERCISE ADMINISTRATION All assessment exercises will be administered individually and separately. To avoid exposure of test materials or the development of multiple or alternate exercise versions, candidates may be held over for extended periods. Prior to exercise administration, candidates will receive a Candidate Test Notification Letter which will include the report dates, times, and locations for each promotional process activity. On test administration days, candidates will be required to go through a sign-in process. As part of this process, candidates will be asked to provide photo identification and will be assigned a candidate number. Lastly, candidates will be required to sign a copy of the test rules and will receive a brief orientatio n. Candidates may be held over for extended periods before and after undergoing an exercise. The Assessment Center will provide the venue, all test materials and supplies, recording equipment and staff necessary for the administration of the assessment exercise. ASSESSMENT EXERCISE EVALUATION The panel of assessors evaluating the candidates shall be at the rank of Sergeant or above, from communities of comparable size and demographics to the South Miami Police Department. Assessors will receive one full day of training by the Assessment Center on the specific portion of the assessment assigned to them. As part of the training, assessors will learn how to take behavioral observation notes and will learn how to use the rating tools and apply the scoring benchmarks. This will enable the trainer to gauge the level of proficiency of the assessors and address any areas needing improvement or clarification. Assessors will work in teams to evaluate candidate records. Each assessor on the panel will first evaluate candidate performance independently. After all scoring is complete, assessors will discuss ratings to reach consensus. Once they have completed their ratings and consensus discussion, evaluation materials produced by the assessors will undergo a rigorous two-part review by the Assessment Center to ensure that the records were evaluated according to the guidelines discussed during training. Records not meeting standards are returned to the assessor team for review and correction. Once a record passes the review process satisfactorily, the record is stored. This process will continue until all records are evaluated and properly reviewed. The Assessment Center will provide materials supplies, media equipment staff, and external assessors. 15 5 | P a g e City of South Miami – 2021 Sergeant Promotional Process RESULTS & TECHNICAL REPORTS The Assessment Center will provide final candidate result packages to the South Miami Police Department. Packages will include, but are not limited to: Statistical summaries including raw scores, means, and standard deviations. The Assessment Center will provide the score report within thirty (30) calendar days after administering the last testing process. Candidates will be given an individualized feedback report including strengths and weaknesses as well as graphs illustrating their performance across exercises and in comparison to group average. 16 City of South Miami – 2021 Sergeant Promotional Process PART V– COST WRITTEN EXAMINATION SERGEANT WRITTEN EXAMINATION DEVELOPMENT Job Analysis Review No Charge Development and preparation of Source Reading List including meetings with Subject Matter Experts (SME) and review of related written exam Source Materials. No Charge Development of Written Examination $2,000.00 SUBTOTAL WRITTEN EXAMINATION DEVELOPMENT: $2,000.00 SERGEANT WRITTEN EXAMINATION ADMINISTRATION Orientation: Preparation of Candidate Test Notification Letters & Candidate Orientation Guide No Charge Execution of 1 comprehensive candidate orientation (with orientation guidebooks) $500.00 Written Exam: Administration Venue, Set Up, & Tear Down No Charge 1 Director (1/2 day) & 1 Assessment Analyst (1 day) $500.00 SUBTOTAL WRITTEN EXAMINATION ADMINISTRATION: $1,000.00 SERGEANT WRITTEN EXAMINATION EVALUATION Written Exam Evaluation: Oversee Item Challenge /Appeals Process $700.00 Conduct Statistical Item Analysis $250.00 Preparation of Final Results & Technical Report $500.00 SUBTOTAL WRITTEN EXAMINATION EVALUATION: $1,450.00 TOTAL SERGEANT WRITTEN EXAMINATION $4,450.00 17 7 | P a g e City of South Miami – 2021 Sergeant Promotional Process SERGEANT ASSESSMENT EXERCISE DEVELOPMENT Subject Matter Experts (SMEs) South Miami Police Department Materials for SME Meetings No Charge Development of 3-Item BADGE Exercise, 3-Item Structured Interview & Scoring Criteria No Charge TOTAL BADGE EXERCISE DEVELOPMENT No Charge SERGEANT ASSESSMENT EXERCISE ADMINISTRATION & EVALUATION (UP TO 15 Candidates) Administration: Candidate Orientation & Test Notification $150.00 Test Administration Venue, Audio Recording Devices, DVDs & Mini DVs No Charge Media Equipment Set Up, Operation, & Staffing No Charge Assessment Center Staff (1 Director, 2 Assessment Analysts (1 day)) $750.00 Evaluation: Recruitment & Selection of Assessors (3 Assessors) No Charge Preparation of Evaluation Materials Including Binding & Duplication of Assessor Training Materials No Charge Assessor Training $300.00 Assessor Meals (Breakfast & Lunch) $500.00 2 Assessment Analysts For Assessor Training and Evaluation $700.00 Final Results: Preparation of Final Results & Technical Report Including Individual Candidate Reports $750.00 TOTAL BADGE EXERCISE ADMINSITRATION & EVALUATION $3,150.00 SERGEANT BADGE EXERCISE FEEDBACK Preparation of Feedback Materials Including Binding & duplication of Training Materials & Candidate Schedules & Notifications No Charge Professional Oversight of Feedback $500.00 TOTAL FEEDBACK $500.00 TOTAL SERGEANT ASSESSMENT CENTER EXERCISES $3,650.00 TOTAL FOR SERGEANT PROMOTIONAL PROCESS $8,100.00 18 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET i Proposal to Provide a Promotional Process for the Rank of Police Sergeant Presented to: The City of South Miami, Florida 19 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET ii RFP Title: Proposal to Provide a Promotional Process for the Rank of Police Sergeant Date of Opening: October 8, 2020 Industrial/Organizational Solutions, Inc. (IOS) is pleased to have the opportunity to present this proposal to the City of South Miami, Florida. This proposal will describe IOS’s plan to provide a Police Sergeant promotional process. We would welcome the opportunity to discuss our proposal with you or to clarify any points about which you may have questions. This proposal contains confidential, proprietary information that IOS considers trade secret. We request that you contact IOS prior to disclosing or sharing the contents of this document with any party, pursuant to a Freedom of Information law request or otherwise. IOS has compiled this proposal independently and without collusion. We truly look forward to the opportunity to serve your agency. Chad C. Legel _________________________________ Chad C. Legel, M.S., President & CEO Proposal Contact: Chad C. Legel, M.S., President & CEO Industrial/Organizational Solutions, Inc. (d.b.a., I/O Solutions, IOS) 1520 Kensington Road – Suite110 Oak Brook, IL 60523 chad@iosolutions.com Phone: 888.784.1290 Fax: 708.410.1558 https://iosolutions.com FEIN: 36-3783421 20 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET iii THIS PROPOSAL CONTAINS TRADE-SECRET INFORMATION: The Freedom of Information Act, 5 U.S.C. 552, generally obligates government agencies to make certain records available to members of the public upon request. FOIA exempts from mandatory disclosure nine categories of information, including documents which contain “trade secrets and commercial or financial information obtained from a person and privileged or confidential.” 5 U.S.C. 552(b)(4). The purpose of subdivision 552(b)(4) is twofold – to protect individuals who disclose information to the government from competitive disadvantage and to preserve the confidentiality of information provided by citizens to the government when the information is of a type that ordinarily would not be released to the public. Audio Technical Services, Ltd. v. United States Department of the Army, 487 F. Supp. 779, 781 (D.C., 1979); Burke Energy Corp. v. United States Department of Energy, 583 F. Supp 507, 510 (D.C. Kan., 1984). In order to be exempt from mandatory disclosure pursuant to subdivision 552(b)(4), information must be (a) relevant to a commercial or financial matter, (b) obtained by the government from a party outside the government and (c) of a confidential or privileged character. National Parks and Conservation Association v. Morton, 498 F.2d 765, 766 (D.C., 1974). IOS considers the section of this proposal title “Scope of Services” to be confidential and trade secret information and should not be disclosed to any external party. 21 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET iv Contents Executive Summary ......................................................................................................................... 1 Company Background ..................................................................................................................... 3 Company Experience and Personnel .............................................................................................. 8 Project Management ................................................................................................................... 147 Project References ...................................................................................................................... 148 Scope of Services ........................................................................................................................ 167 Investment ................................................................................................................................... 186 Appendix A ................................................................................................................................. 189 Disclosures: ............................................................................................................................. 189 22 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 1 Executive Summary This proposal will detail IOS’s plan to provide a promotional process for the rank of Police Sergeant. Following is an overview of the project methodology that will be undertaken to complete this project.  Kick-off Meeting o Meet with client to review methods, discuss expectations/goals, review project challenges/obstacles and establish mutually agreeable timeline.  Job Analysis (may not be necessary if a current analysis exists) o Review existing job analytic information. o Interview and observe incumbents. o Develop and deploy a job analysis questionnaire. o Develop and deploy a linkage questionnaire. o Analyze job analysis outcomes and document findings.  Written Job Knowledge Exam Design o Develop an examination plan/blueprint that identifies core knowledge areas and sources that will be used to assess knowledge areas. o Draft exam questions. o Review exam questions with subject matter experts. o Compile and print final exam. o Identify “minimally qualified” cut-off score. o Design candidate appeal/challenge process/procedure. o Score exams and produce rank-ordered results.  Assessment Center Design o Develop an assessment plan based on discussions with subject matter experts that identifies exercise types and themes that are job-related and will allow for the assessment of the most essential job skills.  Structured oral interview will be included as one of the exercises. o Develop assessment exercises including candidate instructions, rating criteria, assessor/actor scripts, candidate orientation information, etc. o Review and revise exercise content and criteria with subject matter experts as necessary. o Finalize and print assessment materials.  Candidate Orientation Presentation o Develop and deliver a candidate preparation presentation that informs candidates concerning the promotional assessment process, offer preparation insights and strategies and provides information about common mistakes and misconceptions regarding exams and assessments.  Written Exam Administration o Provide test booklets, answer sheets and pencils. o Proctor and oversee the administration of the exam. 23 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 2 o Oversee the candidate appeal/challenge process and review and respond to appeals accordingly. o Score the exam and provide the agency with final results.  Assessment Center Administration o Develop a candidate schedule to accommodate eligible candidates. o Recruit assessors and actors as necessary. o Oversee and manage the administration of the assessment center. o Score the assessment center and create individual candidate developmental feedback reports.  Concluding Meeting o Meet with the agency’s representatives following the project to critique the process and identify any areas for future improvement. As we execute this project plan, there are a few key factors that guide us:  Best Practices: We view it as our responsibility to inform you regarding the best or most appropriate means of accomplishing this project. We will review your current testing process and may make suggestions as to how it can be improved. Often, items like job analysis, cut-off scores, scoring systems, assessment reliability considerations, assessor panels, etc., can be reconsidered to improve a testing process. We will discuss these items with you in an effort to ensure that your process is highly defensible.  Flexibility: While we care to inform you regarding how your process can be improved, we also realize that every department deals with practical constraints (e.g., union agreements, state laws, local ordinances, etc.) that may require a process to be carried out in a particular way. We are very flexible in identifying these needs and working within these constraints.  Disparate Impact: We always attempt to design a selection process that mitigates disparity between protected classes to the greatest extent possible. There are numerous strategies that we can employ related to assessment composition, cut-off score, scoring systems, and other areas that can positively impact the diversity of the promotional list. We will discuss these options during the kick-off meeting to make sure that we chart a mutually desirable path. Because of the complexity and high-stakes nature of this project, there are a number of details that must be finalized based on future discussion and specific information should IOS be awarded this project. The methodology we propose herein is intended as a road map and will be modified accordingly. We look forward to the opportunity to work with you! 24 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 3 Company Background Industrial/Organizational Solutions, Inc. (IOS) was established in 1997 and is incorporated under the laws of the state of Illinois. We are headquartered in Oak Brook, IL (suburban Chicago) and are comprised of approximately 50 employees. IOS specializes in public safety entry-level and promotional selection and the majority of our employees are trained in personnel selection, psychometrics and legal issues associated with selection. Over 30 of our staff hold Masters or Ph.Ds. in Industrial/Organizational Psychology. We are the largest firm in the US dedicated exclusively to public safety personnel selection. IOS’s focus on public safety selection is very intentional. We believe that we provide a valuable and necessary service to public safety agencies. Our two highest goals are to 1) design selection systems that identify the most capable and talented individuals for positions and 2) design selection systems that result in outcomes that are racially and gender diverse by employing selection processes that minimize disparity and adverse impact. We also value the work that public safety agencies do and we are honored to work alongside these agencies. IOS’s expertise in the area of public safety selection has afforded us the opportunity to work with a number of agencies whose selection processes are overseen by a federal court or the US Department of Justice. During these projects, IOS generally works collaboratively with the local agency and the DOJ/court to institute a best practice solution in an environment that has historically dealt with disparate impact outcomes:  Buffalo, NY – IOS worked with the city and federal court to design an entry-level police officer test. IOS also designed police and fire promotional testing process that were reviewed and accepted by the DOJ.  Miami, FL – IOS was selected to design a police lieutenant assessment process and has working with the city and DOJ on multiple occasions to ensure a successful outcome.  Pittsburgh, PA – IOS designed and validated a firefighter testing process that met the requirements of the DOJ and has been successfully deployed multiple times. IOS works with agencies in almost every state in the United States, including 25 of the largest 50 cities by population. We also work with many large county and state law enforcement and fire service agencies. The following client list provides a representative sample of some of the largest agencies for which IOS have provided entry-level or promotional assessment services in the past two years. This list is not exhaustive, but is intended to demonstrate many of the larger agencies with whom we are currently under contract. 25 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 4 ALABAMA  Tuscaloosa FD (Promo)  Tuscaloosa PD (Promo) ARKANSAS  Little Rock FD (Entry) CALIFORNIA  San Diego FD (Promo)  San Diego PD (Promo)  San Francisco FD (Promo)  San Francisco PD (Promo) COLORADO  CO FF Consortium (Entry)  CO State Patrol (Entry, Promo) CONNECTICUT  Bridgeport FD (Entry)  Bridgeport PD (Entry, Promo)  CT Fire Chiefs (Entry)  Hartford PD (Entry) DISTRICT OF COLUMBIA  DC Fire Rescue (Entry) FLORIDA  FL Dept. of Law Enf. (Entry)  Jacksonville Correction (Promo)  Jacksonville PD (Promo)  Miami PD (Promo)  Miami Beach FD (Entry, Promo)  Miami Beach PD (Entry, Promo) GEORGIA  Atlanta PD (Promo)  Augusta FD (Promo)  Cobb FD (Promo)  Cobb SO (Promo)  Columbus PD (Promo)  Forsyth FD (Promo)  Gwinnett CO (Promo)  Gwinnett FD (Promo)  Gwinnett PD (Promo)  Gwinnett SO (Promo) ILLINOIS  Chicago FD (Entry, Promo)  Chicago PD (Entry)  Cook County (Entry, Promo)  IL State Police (Entry, Promo)  Moline PD (Promo)  Naperville FD (Entry)  Naperville PD (Entry)  Rockford FD (Entry, Promo)  Rockford PD (Promo) INDIANA  Indianapolis FD (Entry, Promo)  Indianapolis PD (Entry, Promo) IOWA  Davenport FD (Promo)  Davenport PD (Promo) KANSAS  Kansas City FD (Promo)  Kansas City PD (Promo) MAINE  Portland PD (Entry PO) MARYLAND  Baltimore FD (Promo)  Baltimore PD (Promo)  Howard County PD (Entry)  Montgomery County Corrections (Entry, Promo)  Mont. Cty FD (Entry, Promo)  Montgomery Cty PD (Promo)  Montgomery Cty SO (Promo) MASSACHUSETTS  MA State Police (Promo)  MA Trail Court (Entry, Promo) MICHIGAN  Ann Arbor PD (Promo)  Detroit PD (Promo)  Grand Rapids PD (Promo) MINNESOTA  Minneapolis PD (Promo)  St. Paul PD (Promo) MISSISSIPPI  MS Highway Patrol (Promo) MISSOURI  Springfield FD (Entry, Promo)  Springfield PD (Entry, Promo) NEVADA  Las Vegas FD (Promo) NEW JERSEY  NJ Dept. of Personnel (Entry) NEW MEXICO  Albuquerque FD (Promo)  Albuquerque PD (Promo)  Bernalillo FD (Promo)  Bernalillo CO (Promo)  Los Alamos FD (Entry, Promo)  Los Alamos PD (Entry, Promo) NEW YORK  Buffalo FD (Promo)  Buffalo PD (Entry, Promo)  NY Metro Trans. Auth. (Entry)  NY Dept. Civil Service (Entry) NORTH CAROLINA  Raleigh PD (Promo) OHIO  Cincinnati FD (Entry, Promo)  Cincinnati PD (Entry, Promo) OKLAHOMA  Broken Arrow FD (Promo)  Oklahoma City PD (Entry PAT)  Tulsa FD (Promo)  Tulsa PD (Promo) OREGON  Portland FD (Promo)  Portland PD (Promo) PENNSYLVANIA  Pittsburgh FD (Entry)  Pittsburgh PD (Promo) SOUTH CAROLINA  Columbia PD (Promo) TENNESSEE  Chattanooga FD (Promo)  Chattanooga PD (Promo)  Nashville PD (Promo) TEXAS  Arlington FD (Entry, Promo)  Austin EMS (Promo)  Austin PD (Promo)  Dallas FD (Entry, Promo)  Dallas PD (Promo)  Fort Worth FD (Entry)  Fort Worth PD (Entry)  Harris County Sheriff (Promo)  Houston PD (Promo)  San Antonio FD (Entry, Promo)  San Antonio PD (Entry, Promo) VIRGINIA  Chesapeake PD (Promo)  Chesterfield County PD (Promo)  Suffolk PD (Promo)  Virginia Beach PD (Promo) WISCONSIN  Milwaukee FD (Entry, Promo) 26 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 5 IOS has a reputation as the elite public safety selection consulting firm in the nation. In addition to working with many of the most prominent agencies in the nation, we also publish exams and assessment tools that are routinely considered to be the best in the national marketplace. In 2007, the City of Chicago contracted with an independent consulting firm to evaluate all firefighter exams that were commercially available in an effort to identify the best exam for the City. The available options were narrowed down and then technical documentation was scrutinized. Based on this review, the top three exams were chosen and a local criterion- related validation study was conducted. IOS’s entry-level firefighter exam was then selected as the best option based on validity evidence and its ability to mitigate disparate impact outcomes. The exam was administered to over 20,000 candidates. In 2010, the City of Virginia Beach contracted with another independent consulting firm to conduct a similar review of exams. Again, IOS’s firefighter examination was selected. More recently, in 2013, the City of Chicago engaged in a similar process to identify a police officer exam. IOS’s entry-level law enforcement exam was selected and administered to approximately 16,000 candidates. We are proud of the fact that our products, when independently scrutinized and evaluated by objective, trained experts, are consistently recognized as top notch. IOS’s consultants have experience designing and defending selection processes. In addition to working with the DOJ and the federal courts to design appropriate testing processes, our consultants have also served as expert witnesses and fact witnesses in litigation proceedings. Most notably, IOS developed a testing process for the New Haven, CT Fire Department that was considered in the Supreme Court of the United States in Ricci v. DeStefano. The Court concluded that the testing process was fair and valid and it required that New Haven implement the results of the testing process. IOS’s consultant, Chad Legel, was instrumental in providing evidence/deposition to defend the validity and fairness of the testing process. IOS was founded on the premise that public safety selection processes can be improved. Our experience has taught us that many consulting firms providing public safety testing services do not apply the rigor and expertise needed to properly validate selection tools and ensure their success. IOS is committed to improving the state of public safety selection through a commitment to the science of selection and through education. IOS is a value-driven organization. We are guided by four core values: a commitment to ensuring that our exams and assessments reflect best-practices in the science of selection, a commitment to providing our clients with successful testing solutions and a remarkable consultant-client experience, a commitment to educating our clients as to what we are doing and why and a commitment to putting the nation’s best consulting team at your disposal. These values will be evident to you every step of the way. Commitment to the Science of Selection The development, validation and use of exams and assessments are based on the science of personnel selection and psychometric theory. The science of exam design and validation ensures that the exam can identify qualified candidates and can withstand the rigors of legal scrutiny should the use of the exam ever be challenged. Individuals that develop and implement exams and assessments need to be thoroughly trained in Personnel Selection and Psychometric Theory and have a firm grasp of the field of Industrial Psychology. Too often, testing tools are 27 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 6 not constructed in a manner that ensures their validity or defensibility. Not only is this risky, but it doesn’t provide any assurance that the client is finding the most qualified individuals to hire and promote. In short, when you ignore the science of testing, you risk not identifying those individuals that are going to best contribute to the success of your organization. IOS offers a team of highly qualified and experienced Industrial Psychologists who specialize in designing, validating and implementing testing and assessment processes for public safety organizations. Your agency can be confident in the soundness of our services because we adhere closely to professional guidelines that control the quality and precision of the tools we develop and employ. Our products and services adhere strictly to the following legal and professional guidelines:  Federal Uniform Guidelines on Employee Selection Procedures.  Recent court cases affecting the interpretation of the Uniform guidelines.  Principles for the Validation and Use of Personnel Selection Procedures (Principles), published by Division 14 of the American Psychological Association.  Standards for Education and Psychological Testing published by the American Educational Research Association, and the National Council on Measurement in Education.  Guidelines & Ethical Considerations for Assessment Center Operations published by the International Task Force on Assessment Center Guidelines.  Civil Rights Acts of 1964 and 1991.  The Americans with Disabilities Act.  Standards published by CALEA (Commission on Accreditation for Law Enforcement Agencies).  International Fire Service Training Association (IFSTA).  State laws, local ordinances and contracts. Commitment to the Service of Selection Exams and assessments provide a clear service – they help you make better decisions about who to hire and promote. Testing and assessment choices need to be made in the context of local conditions and values. Collective bargaining agreements, budgets, initiatives to diversify the workforce and other critical factors greatly affect which exams and assessments you select. We realize that in order to best serve our clients interests, we have to understand their condition and needs. We are committed to learning about our clients and working with them to design the most advantageous selection system. A necessary element of this principle is that we are flexible and responsive to our clients. Customer service is a common claim but it is rarely delivered to the proper degree. The principal members of IOS have been successful in the testing and assessment industry by putting our customers first. We deliver on our promises and we will work vigorously to ensure that our client’s interests are well-served. Commitment to Educating our Clients The field of testing and assessment is complicated and constantly evolving. This reality, combined with the fact that many testing conventions do not align with best practice, requires that test users are properly educated about the value and capability of various testing methods and the relative validities and usefulness of these methods. We feel strongly that we must provide both state-of-the-art testing tools and education to our clients about how to best design and deploy their selection processes. We are committed to objectively and 28 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 7 professionally working with test user’s to educate them about the strengths and weaknesses of testing methods and to impart proper expectations about the value and utility of the tools from which they can choose. The principal members of IOS are responsible for establishing an annual meeting to train test users concerning best practices and routinely provide pro bono training seminars for organizations such as the International Personnel Management Association (IPMA), the Commission on the Accreditation of Law Enforcement Agencies (CALEA), the International Personnel Assessment Council (IPAC), the Illinois Public Employer Labor Relations Association (IPELRA), and numerous state police and fire chiefs associations. Commitment to our Employees In order to provide the best products and deliver the highest quality service, you have to employ the best people in your industry. Our employees are the brightest and most capable. You, the client, deserve to feel like you have the most dynamic people and sharpest minds working for you. This will be evident from the first time you meet them until the time they deliver the end product. We aim to foster a working environment that demands excellence, values personal development and rewards dedication and achievement. By doing this, we know that we will best serve our clients and ensure long and successful working relationships. This proposal will detail our approach to managing your project. While we offer what we believe is the appropriate plan to accomplish your objective, we look forward to meeting with you and discussing the project in order to tailor our approach to best serve your interests. We firmly believe that we are the best at what we do. We sincerely hope that you will contact our clients to learn more about us. We look forward to the opportunity to work with you! 29 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 8 Company Experience and Personnel IOS’s team consists of some of the most recognized and experienced leaders in the highly specialized and litigious field of public safety selection. Our consultants are all trained in Industrial/Organizational Psychology and specialize in employment testing and public safety human resources consulting. Our team is sufficiently deep to allow us to proficiently manage numerous projects simultaneously, while never compromising customer service or expertise. Our team is augmented by technical advisors in the areas of law enforcement, fire service and legal issues. Our team is managed and overseen by Chad Legel, M.S. and Brian O’Sullivan, Ph.D. (ABD), who between them have over 35 years of experience in test development and validation. This management teams meets with consultants on a bi-weekly basis to review projects, discuss strategies and address project challenges. This approach ensures that all IOS projects are performed consistently and at the highest possible level. IOS only has one main office from which it conducts all of its work out of. This location is in Oak Brook, IL, which is just outside of Chicago, IL. The following table presents our project team. Team Member Qualifications Project Role Chad Legel, M.S. President and CEO Industrial/Organizational Psychologist Consultant/Project Director Brian J. O’Sullivan, M.S. (ABD) Vice President, Consulting Services Industrial/Organizational Psychologist Consultant/Project Director Mark Tawney, Ph.D. Vice President, Testing Services Industrial/Organizational Psychologist R&D Consultant and Psychometrician Nahren Cama, Ph.D. Manager, Consulting Services Industrial/Organizational Psychologist Consultant Brian Marentette, Ph.D. Manager, Consulting Services Industrial/Organizational Psychologist Consultant Courtney Wierzbicki, Ph.D. Manager, Consulting Services Industrial/Organizational Psychologist Consultant Amy Eitapence, M.A. Manager, Recruitment Services Industrial/Organizational Psychologist Consultant Karen Steiger, M.A. Manager, Content Development M.A. in English/Writing; 19 years’ experience writing and editing public safety exams. Senior Technical Writer 30 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 9 Maxwell Porter, M.S. Manager, Testing Services Industrial/Organizational Psychologist Consultant John Ford, Ph.D. Principal Consultant Industrial/Organizational Psychologist Consultant Irfan Bhanji, M.A. Senior Assessment Consultant Industrial/Organizational Psychologist Consultant Linda Reynaud, M.A. Senior Assessment Consultant Industrial/Organizational Psychologist Consultant Drew Weedfall, Ph.D. Assessment Consultant Industrial/Organizational Psychologist Consultant Christen Dovalina, M.S. Assessment Consultant Industrial/Organizational Psychologist Consultant Samantha Gleaves, M.A. Assessment Consultant Industrial/Organizational Psychologist Consultant Madison Sayler, M.A. Assessment Consultant Industrial/Organizational Psychologist Consultant Veronica Janssen, M.A. Associate Consultant Industrial/Organizational Psychologist Consultant Nick Jochim, M.A. Associate Consultant Industrial/Organizational Psychologist Consultant Maureen Jones, M.A. Assessment Specialist Personnel Selection Specialist Consultant Haley Kilbride, M.A. Assessment Specialist Industrial/Organizational Psychologist Consultant Jacob Wolfarth, M.S. Assessment Specialist Industrial/Organizational Psychologist Consultant Harry Vuong, M.A. Senior Consulting Coordinator Personnel Selection Specialist Consulting Coordinator Allison Johnston, M.A. Consulting Coordinator Personnel Selection Specialist Consulting Coordinator Jaclyn Delagrange Consulting Assistant Personnel Selection Specialist Consulting Assistant 31 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 10 Alex House Consulting Assistant Personnel Selection Specialist Consulting Assistant Bayla Waite Consulting Assistant Personnel Selection Specialist Consulting Assistant Rob Szarek, M.A. Consultant/Sr. Researcher Industrial/Organizational Psychologist Consultant Alexa Bolwin, M.A. Research Analyst Industrial/Organizational Psychologist Research Analyst Taylor DeSantis, M.S. Research Analyst Industrial/Organizational Psychologist Research Analyst Reya Green, M.S. Research Analyst Personnel Selection Specialist Research Analyst Tami Hatzis Content Development Specialist Personnel Selection Specialist Content Specialist Josh DuBois Content Development Assistant Personnel Selection Specialist Content Assistant **We also have a large project support team that allows us to apply a great deal of additional resources to a project if need be. This team includes additional associate consultants, job analysts, technical writers, etc. The following section presents our organizational chart followed by a professional biography for each of our key project team members. 32 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 11 33 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 12 CHAD C. LEGEL, M.S. President & Chief Executive Officer Background Statement: Mr. Legel is the President and a Principal Member of Industrial/Organizational Solutions, Inc., a personnel and human resources consulting firm that specializes in testing and assessment. Mr. Legel has 18 years of experience in the field of public safety testing and assessment. He specializes in developing, validating, deploying and defending public safety testing systems such as assessment centers, promotional job knowledge examinations, physical ability tests, entry- level examinations and oral interviews. Education: Illinois Institute of Technology, Chicago, Illinois Master of Science: Industrial/Organizational Psychology (2000) Coursework completed toward a Ph.D. in same field Cornell College, Mt. Vernon, Iowa Bachelor of Arts, Psychology (1998) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Chicago Industrial/Organizational/Psychology (CIOP) Professional Experience: 2013 to Present Industrial/Organizational Solutions, Inc. President & Chief Executive Officer  Manage the operations of one of the nation’s leading public safety human resources consulting firms  Consult with public safety agencies to develop, validate and defend entry-level and promotional testing/assessment systems  Provided best-practice training to public safety agencies, human resources and civil service personnel 2011 to 2013 Selection Works, LLC President and CEO, Co-Founder  Led recruitment, test publication and consulting divisions  Consulted with police and fire agencies to design entry-level and promotional testing systems  Developed state-of-the-art entry-level examinations for national distribution 34 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 13 2006 to 2011 Industrial/Organizational Solutions, Inc. President and Chief Operations Officer  Managed the recruitment, test publication and consulting divisions of a multi-million dollar human resources consulting firm  Provided training to human resources and civil service personnel nationwide  Provided best-practice training to municipalities  Managed the Fire and Police Testing Summit, a national training seminar 2003 to 2006 Industrial/Organizational Solutions, Inc. Vice-President of Consulting  Managed a public safety consulting team  Validated exams and assessments  Managed the deployment of large scale selection projects  Designed Department of Transportation certification examinations  Created training articles for national trade journals 1999 to 2003 Industrial/Organizational Solutions, LLC Consultant  Developed testing and assessment processes for public safety organizations  Researched and developed standardized, off-the-shelf entry-level and promotional examinations for public safety agencies  Designed sales talent selection tools for private industry 1999 United Airlines Contractor – Flight Attendant Hiring  Interviewed prospective flight attendants  Analyzed employee survey data Summary of Selection Projects: Mr. Legel has a wealth of experience providing selection consulting services to a diverse listing of state, municipal and county public safety agencies. Following is a partial list of the jobs that Mr. Legel has personally managed in the past 19 years: Client Project Title Timeframe Colorado Highway Patrol Major Assessment Center 2017 Hattiesburg, MS Review and Redesign of Civil Service Rules; develop of police and fire promotional testing procedures 2017 Raleigh, NC Police Detective, Sergeant and Captain promotional assessments 2017 San Antonio, TX Police Captain Assessment 2017 35 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 14 Oklahoma City, OK Police Officer Physical Ability Test 2017 Dallas, TX Fire Battalion Chief Assessment 2017 Plano, TX Police Officer Physical Ability Test 2016 Colorado Highway Patrol Patrol captain assessment center 2016 Rockford, IL Recruited applicants, and developed and administered assessment center for the rank of Police Chief. 2016 Colorado Highway Patrol Patrol sergeant assessment center. 2015 St. Paul, MN Police Sergeant written exam and assessment center. 2015 Mesquite, TX Development of entry-level firefighter physical ability test. 2015 San Antonio, TX Development and administered a written exam and assessment center for the police lieutenant and police captain ranks. 2015 St. Paul, MN Development and administered a written exam and assessment center for the police sergeant rank. 2014 San Antonio, TX Development and administered a written exam and assessment center for the police captain rank. 2013 Cobb County, GA Developed and administered written exams and assessment centers for the police sergeant and lieutenant ranks. 2013 Cobb County, GA Developed and administered written exams and assessment centers for the sheriff sergeant and lieutenant ranks. 2013 Des Plaines, IL Developed and administered a fire lieutenant written exam and assessment center. 2013 Augusta, GA Developed and administered written examinations and assessment centers for the ranks of sergeant (engineer), lieutenant, captain and battalion chief. 2012 Davenport, IA Developed assessment centers for the police sergeant and lieutenant ranks. 2012 Lincolnshire, IL Developed and administered a police sergeant assessment center. 2012 Illinois State Police Developed and administered a written examination and assessment center for the ranks of master sergeant and sergeant. 2012 Plano, TX Developed an entry-level firefighter oral interview process. 2012 Des Plaines, IL Developed and administered a police sergeant written exam and assessment center. 2012 Austin, TX Validation of an entry-level firefighter examination process 2011 Baltimore, MD Development of a promotional examination and assessment for police lieutenant 2011 Bridgeport, CT Validation of a selection process for firefighter 2011 Cook County, IL Validation of entry-level corrections officer examination 2011 CSX Transportation Consultation regarding the design of a police officer recruitment and selection process 2011 Omaha, NE Job analysis for police and fire ranks and development of a police lieutenant assessment process 2011 36 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 15 Pittsburgh, PA Validation of a written examination for firefighter 2011 Tulsa, OK Development and administration of police department promotional exams and assessment centers 2011 Austin, TX Development and administration of a written examination and assessment center for fire battalion chief 2010 Baltimore, MD Development of a promotional examination and assessment for police sergeant 2010 Bridgeport, CT Validation of a selection process for police officer 2010 Dell, Inc. Translation and validation of a global sales force assessment 2010 Sidney, OH Development of a selection process for the fire chief position 2010 Baltimore, MD Development of a promotional examination and assessment for police lieutenant 2009 Chicago, IL Validation of a dispatcher examination and development of a test of verbal fluency 2009 Joplin, MO Development of a firefighter physical ability test 2009 Lincoln, NE Development of a firefighter physical ability test 2009 Minneapolis, MN Development of an assessment center process for Deputy Chief 2009 San Antonio, TX Review of firefighter hiring process for compliance with professional and legal requirements and best practices 2009 Sidney, OH Development of a selection process for the police chief position 2009 Woodridge, IL Development of a police officer interview process 2009 Augusta, GA Development of exams and assessment centers for the engineer, lieutenant, captain and battalion chief ranks 2008 Baltimore, MD Development of a promotional testing process for police sergeant 2008 Cobb County, GA Development of exams and assessment centers for the sergeant and lieutenant ranks in the Sheriff’s Department 2008 Gwinnett County, GA Development of a fire engineer exam and pumping assessment 2008 Wisconsin Department of Health, EMS Board Development of an Intermediate EMT recertification examination 2008 Woodridge, IL Development of a police sergeant interview process 2008 Austin, TX Development and administration of a police chief assessment process 2007 Cook County, IL Development of police, corrections and court services promotional examinations 2007 Dallas, TX Oversight and consultation related to fire promotional testing 2007 Jacksonville, FL Development of promotional testing processes for police sergeant and lieutenant 2007 Virginia Department of Motor Vehicles Review, analysis and update of motor vehicle and motorcycle certification examinations 2007 Waukesha County, WI Development of a sheriff deputy physical ability test 2007 Arlington Heights, IL Development of a police sergeant interview process 2006 37 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 16 Chicago, IL Validation of a firefighter examination 2006 Cobb County, GA Development and administration of police sergeant and lieutenant promotional testing processes 2006 Dallas, TX Oversight and consultation related to fire promotional testing 2006 Hartford, WI Development and administration of a police chief assessment process 2006 Kansas Board of Emergency Medical Services Development of first responder, basic EMT and intermediate EMT certification examinations 2006 Crystal Lake, IL Development of a firefighter physical ability test 2005 Denver, CO Development of a police captain assessment center 2005 Massachusetts Port Authority Development of promotional testing processes for fire lieutenant, captain and assistant chief 2005 Reading, MA Development of a police chief assessment process 2005 Arlington Heights, IL Development of a police officer interview process 2004 Florida Department of Law Enforcement Development of a basic abilities test for academy entrance 2003 Naperville, IL Development of a police officer physical ability test 2003 New Haven, CT Development of promotional testing processes for fire lieutenant and captain 2003 Richton Park, IL Development of a police sergeant interview process 2003 Tulsa, OK Development of entry-level examinations and physical ability tests for police officer and firefighter 2003 Westmont, IL Development of a police officer physical ability test 2003 Arlington Heights, IL Development of a firefighter physical ability test 2002 Arlington Heights, IL Development of a firefighter interview process 2002 Frankfort, KY Development of fire department promotional assessments 2002 Gwinnett, GA Development of an engineer pumping practice assessment 2002 Gwinnett, GA Development of a fire captain assessment process 2002 Massachusetts State Police Development of a video-based situational judgment test for state troopers 2002 Revenue Storm, Inc. Development of a sales person skill and personality assessment tool 2002 METRA Police Department Development of promotional processes for police sergeant and lieutenant 2001 Minneapolis, MN Development of a police sergeant assessment process 2001 Naperville, IL Development of a police officer interview process 2001 Norridge, IL Police staffing and organizational structure study 2001 Orange County, FL Development of a firefighter physical ability test 2001 Warwick, RI Delivery of a candidate examination preparation and training session 2001 38 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 17 Gwinnett County, GA Development of a police corporal promotional assessment process 2000 Gwinnett County, GA Development of a sheriff sergeant promotional assessment process 2000 Little Rock, AR Development of a firefighter interview process 2000 Naperville, IL Development of a firefighter physical ability test 2000 Naperville, IL Development of a firefighter interview process 2000 Savannah, GA Transportability of the CPAT 2000 Upper Arlington, IL Development of an entry-level firefighter examination 2000 Upper Arlington, OH Development of an entry-level police officer examination 2000 In addition to these numerous projects, Mr. Legel has accomplished the following:  Led the development of emergency medical technician certification examination processes for the states of Kansas, Wisconsin, Oregon and Illinois  Provided testing/assessment best practices training to the City of Austin TX and the Austin Firefighters Association during collective bargaining negotiations  Evaluated firefighter testing practices for the City of San Antonio  Evaluated and monitored the police and fire promotional testing practices for the City of Dallas, TX  Founded The Police and Fire Testing Summit, an annual meeting, currently in its sixth year, of police and fire executives and human resources directors to provide training on personnel selection topics.  Redesigned the State of Virginia motor vehicle written certification examination process to improve test validity and reliability Training Presentations:  “Intersecting I-O Psychology and Law Enforcement,” panel discussion at the Society for Industrial Organizational Psychology Annual Conference, Orlando FL, April 2017.  “Trends in Public Safety Recruitment,” Illinois City Managers Association Spring Conference, Lombard IL, March 2017  “An Alternative to High-Volume Oral Interviews,” National Public Employer’s Labor Relations Association Annual Training Conference, Savannah, GA, March 2015  “Improving the Conventional Oral Interview,” Public Safety Assessment Forum, College of DuPage, Glen Ellyn IL, July 2012  “The Illinois POWER Test: Considering its Use as a Screening Tool,” Public Safety Assessment Forum, College of DuPage, Glen Ellyn IL, July 2012  “The Firefighter Hiring Act,” Illinois Public Employer’s Labor Relations Association Training Meeting, Arlington Heights, IL, June 2012  “Achieving Racial Diversity in Public Safety Testing,” Washington State Civil Service Meeting, Ellensburg, WA, September 2011 39 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 18  “Best Practices and Considerations in Response to the Illinois Firefighter Hiring Act (Public Act 97-0251),” Illinois Public Employer Labor Relations Association (IPELRA), Hoffman Estates, IL, September 2011  “Improving Firefighter Selection Practices: Diversity, Recruiting, Oral Interviews and Assessment Centers,” Wisconsin Fire Chiefs Education Association Annual Meeting, Green Bay, WI, September 2011  “Understanding Metrics of Fairness: Adverse Impact, Standardized Mean Differences, and Significance Testing,” International Personnel Assessment Council (IPAC), Washington DC, July 2011  “Testing and Assessment Concepts and Best Practices,” Connecticut Consortium of Municipalities, Wallingford, CT, June 2011  “Using Integrity Tests to Improve Efficiency and Mitigate Disparate Impact”, Ohio Association of Chiefs of Police Annual Training Conference, Newark OH, April 2011  “Considering Disparate Impact Beyond Impact Ratios,” Presented at the 2011 International Personnel Assessment Council Conference, Washington D.C., Legel, C. C., O’Sullivan, B. J. & Tawney, M. W. (2011).  “Test Developer’s Perspective on Ricci v. DeStefano”, International Personnel Management Association, Human Resources Central Region Conference, June 2010  “The Ricci Decision and Its Impact on Hiring and Promotions”, Connecticut Conference of Municipalities Annual Conference, Hartford CT, October 2009  “Firefighter Entry-level and Promotional Testing Best Practices,” Austin Labor Relationship and Collective Bargaining Negotiation Session, Austin, TX, September 2009  “Test Validity”, Littler Mendelson P.C. Class Action Strategy Conference, Phoenix AZ, April 2009  “Selecting and Retaining Generation Y Candidates for Public Safety Agencies,” Police and Fire Testing Summit, Oak Brook IL, January 2006  “Development of Reliable Structured Oral Interviews”, CALEA, Burlington VT, November 2000 Publications:  Upgrading Your Police Officer Oral Interviews, Law and Order Magazine, Hendon Publishing, December 2012.  Evaluating an Entry-Level Examination, Law and Order Magazine, Hendon Publishing, December 2005. Litigation:  City of Clarksville Police Department - Promotional process challenge – 2002  Provided deposition testimony  City of New Haven Fire Department - Promotional process challenge – 2004 – 2009, (Ricci v. DeStefano, 129 S. Ct. 2658, 2671, 174 L. Ed. 2d 490 (2009))  Provided deposition testimony  City of Austin Fire Department - Entry-level firefighter selection arbitration – 2011  Testified on behalf of the City of Austin as the test developer/expert 40 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 19 BRIAN J. O’SULLIVAN, Ph.D. (ABD) Vice President of Consulting Background Statement: Mr. O’Sullivan worked as a project manager and consultant for a large public safety testing firm that deals with law enforcement and fire service agencies prior to joining IOS, Inc. Mr. O’Sullivan has successfully led our consulting team as the Director of Consulting at IOS since 2001. He was promoted to his present role as Vice President in 2011. In his ten years with our firm, Brian has managed over 100 projects involving police and fire department assessment centers, promotional test development, structured oral interviews, entry-level exam development, physical-ability test development and performance-evaluation-process development and he personally handles many of our most notable projects. A partial list of Mr. O’Sullivan’s project work follows below:  San Diego, CA Police Department  Atlanta Police Department  Buffalo, NY Police Department  New York City Police Department  Washington, D.C. Metropolitan Police Dept.  Chicago Fire Department  Massachusetts State Police Department  New Jersey State Police Department  Austin, TX Police and Fire Departments  Houston Police Department  Minneapolis Police Department  Forsyth County, GA Sheriff’s Office  Orange County, FL Fire Rescue  North Miami Police Department  Savannah, GA Police Department  Gwinnett County Police Dept.  Gwinnett County Sheriff’s Dept.  Gwinnett County Corrections Department  Huntsville, AL Police Department  Fort Wayne, IN Police Department  Newport News, VA Police Department  Joliet, IL Police Department  Cook County, IL Sheriff’s Department  Jefferson County, KY Sheriff’s Office  Iowa Sheriff’s Association  Denver, CO Fire Department  Racine, WS Fire Department  New Haven, CT Fire Department 41 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 20 Education: Illinois Institute of Technology, Chicago, Illinois Ph.D. (ABD): Industrial/Organizational Psychology (1999-2009) Master of Science: Industrial/Organizational Psychology (1998) Illinois State University, Normal, Illinois Bachelor of Science, Psychology and Communications (1995) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Chicago Industrial/Organizational/Psychology (CIOP) Professional Experience: 2011 to Present Industrial/Organizational Solutions, Inc. Vice President of Consulting 2005 to 2011 Industrial/Organizational Solutions, Inc. Director of Consulting  Responsible for hiring, training and mentoring team of I/O Psychologists in test development/assessment projects.  Manages consulting team to ensure soundness of methodology, quality of products and services.  Manages and serves as the primary technical lead, consultant and day- to-day project manager/client liaison on the firm’s largest consulting initiatives.  Manages all aspects of consulting projects, including work plans, project deliverables, scheduling, communications and client relations.  Consults with agencies, associations, civil service commissions, state peace officer standards and training bodies, union representatives, and other parties regarding all aspects of public safety human resources.  Conducts and reports job analysis, which includes the following tasks: conducting job observations; developing, administering and analyzing job-analysis questionnaires; reporting job-analysis findings; developing and facilitating task/skill-linkage workshops; analyzing linkage data; developing examination plans.  Conducts and reports criterion-related and content-validation studies. Analyzes data from these studies. Also develops technical reports, incorporating job-analysis, criterion-related and content- validation results.  Develops, validates and implements the following: cognitive, personality and situational-judgment tests for entry-level and 42 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 21 promotional positions; structured behavior-based interviews; written and role-play assessment exercises; physical-ability tests.  Acts as an assessor in assessment centers, structured interviews and other assessment components. Assesses candidates on critical competency areas. 2002 to 2005 Industrial/Organizational Solutions, Inc. Project Manager  Responsible for hiring, training and mentoring team of I/O Psychologists in test development/assessment projects.  Manages and serves as the primary technical lead, consultant and day- to-day project manager/client liaison on the firm’s largest consulting initiatives.  Managed consulting team to ensure soundness of methodology, quality of products and services.  Managed all aspects of consulting projects, including contracting, determination of project deliverables, scheduling, communications, client relations, billing and ensuring the technical quality of all work to be completed.  Developed, validated, and implemented the following: cognitive, personality and situational-judgment tests for entry-level and promotional positions; structured behavior-based interviews; written and role-play assessment exercises.  Responsible for all phases of research and test development, including experimental design, data collection and analysis, item-writing and analysis, report-writing, and final presentation.  Conducted job analysis, which included the following tasks: job observations; developed, administered, and analyzed job-analysis questionnaires; reported job analysis findings; developed and facilitated task/skill-linkage workshops; analyzed linkage data; developed examination plans.  Conducted and reported criterion-related and content validation studies. Analyzed data from these studies. Developed technical reports, incorporating job analysis, criterion-related and content- validation results. 1997 to 2001 Stanard and Associates, Inc. Senior Human Resources Consultant  Served as primary project manager and consultant (reporting directly to the vice president) on projects involving entry-level selection and promotional assessment, performance management, employee- attitude survey development, and organizational analysis and development. 43 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 22  Managed all aspects of consulting projects, including contracting, determination of project deliverables, scheduling, communications, client relations, billing and ensuring the technical quality of all work to be completed.  Developed and implemented performance-management processes and 360-degree feedback instruments.  Developed entry-level selection processes and promotional processes in accordance with all local, state and federal guidelines.  Developed employee-attitude and retention surveys, analyzed data and prepared reports for clients.  Developed entry-level tests, promotional examinations and assessments, structured oral assessments/interviews, assessment centers and physical-ability assessments.  Conducted all technical aspects of consulting projects, including all phases of job analysis, product development, test administration, product training, data analysis, scoring, validation and project documentation/technical-report writing.  Advised clients on legal issues related to selection, promotion, evaluation of employee performance and other related human resources matters.  Conducted research on current products, including validation studies, statistical analyses and normative studies. Provided technical support for off-the-shelf products.  Responsible for new product development, including the development and validation of a nationwide correctional officer examination expected to increase company sales revenue by approximately 10-15 percent.  Supervised full-time consultant and was responsible for hiring, training and supervising all interns.  Wrote proposals and presented services to clients. 1999 NCS/Pearson Research Associate  Maintained and revised SRA employment tests, norms and administration manuals. Developed and programmed computerized versions of tests and provided technical support. 1997 The Ball Foundation Test and Selection Specialist  Provided entry-level testing services to local area apprenticeship programs and aided office manager with general human resources functions.  Developed an internal employee handbook. 44 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 23  Conducted job analyses, constructed test batteries, administered examinations, and conducted scoring and reporting activities.  Developed a product catalog and coordinated nationwide distribution.  Conducted marketing and sales activities, including developing marketing pieces and attending relevant trade shows (SHRM, ASTD and APA).  Conducted preliminary research activities for test development. Summary of Selection Projects: Mr. O’Sullivan has a wealth of experience providing selection consulting services to a diverse listing of state, municipal and county public safety agencies. Following is a partial list of the jobs that Mr. O’Sullivan has personally managed in the past 16 years: Contract Title Project Description Timeframe Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2020 Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2020 San Diego Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Police Officer III. 2020 San Diego Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Detective. 2020 San Diego Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2020 San Diego Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2020 Buffalo, NY Police Department Development and administration of written examinations and assessment centers for the rank of Detective. 2019 Buffalo, NY Police Department Development and administration of written examinations and assessment centers for the ranks of Detective Sergeant. 2019 Buffalo, NY Police Department Development and administration of written examinations and assessment centers for the ranks of Lieutenant. 2019 Buffalo, NY Police Department Development and administration of written examinations and assessment centers for the rank of Captain. 2019 Chicago Fire Department Job analysis and development of written examination and assessment center for the rank of Battalion Chief. 2019 Milwaukee Fire Department Development and administration of written examinations and assessment centers for the rank of Lieutenant. 2019 Milwaukee Fire Department Development and administration of written examinations and assessment centers for the rank of Captain. 2019 Minneapolis Police Department Development of assessment center for the rank of Lieutenant. 2019 Austin Police Department Development and administration of assessment centers for the rank of Sergeant. 2019 Austin Police Department Development and administration of assessment centers for the rank of Lieutenant. 2019 Austin Police Department Development and administration of assessment centers for the rank of Commander. 2019 Houston Police Department Job analysis, written examination, and assessment center 2018 45 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 24 development, validation, administration, and scoring for the rank of Sergeant. Houston Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2018 Houston Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Captain. 2018 Pittsburgh, PA Bureau of Police Written examination, and assessment center development, validation, administration, and scoring for the rank of Sergeant. 2018 Minneapolis, MN Police Department Written examination and assessment center development, validation, and administration for the rank of Sergeant. 2018 Indianapolis Metropolitan Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Sergeant. 2018 Indianapolis Metropolitan Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2018 Indianapolis Metropolitan Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Captain. 2018 Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2018 Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2018 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration and scoring for the rank of Police Officer III. 2018 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration and scoring for the rank of Detective. 2018 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration and scoring for the rank of Sergeant. 2018 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2018 New York Civil Service, State Wide Entry Level Test Job analysis, validations, and development of written examination to be used as a state tool. 2015-2018 Pittsburgh, PA Bureau of Police Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Sergeant. 2017 Pittsburgh, PA Bureau of Police Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2017 Minneapolis, MN Police Department Written examination and assessment center development, validation, and administration for the rank of Lieutenant. 2017 Miami, FL Police Department Job analysis and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2017 Buffalo, NY Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Detective. 2017 Buffalo, NY Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank 2017 46 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 25 of Detective Sergeant. Buffalo, NY Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2017 Buffalo, NY Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Captain. 2017 Buffalo, NY Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Inspector. 2017 San Francisco, CA Police Department Job analysis and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2016-2017 Chicago, IL Fire Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Paramedic Field Chief. 2016-2017 Chicago, IL Fire Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Ambulance Commander. 2016-2017 Cincinnati, OH Police Department Assessment center development, validation, administration, and scoring for the rank of Captain. 2016-2017 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Police Officer III. 2016 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Detective. 2016 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Sergeant. 2016 San Diego Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2016 Ann Arbor, MI Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2016 Ann Arbor, MI Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2016 Houston Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2016 Houston Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2016 Houston Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Captain. 2016 Minneapolis, MN Police Department Written examination and assessment center development, validation, and administration for the rank of Sergeant. 2016 Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2016 Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2016 Baltimore City, MD Police Department Job analysis update, written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2015-2016 Baltimore City, MD Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of EMS Battalion Chief. 2015 47 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 26 Baltimore City, MD Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of EMS Captain. 2015 Baltimore City, MD Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of EMS Lieutenant. 2015 Baltimore City, MD Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Suppression Lieutenant. 2015 Baltimore City, MD Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Suppression Captain. 2015 Baltimore City, MD Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Suppression Lieutenant. 2015 Cook County Sheriff's Office Department of Corrections Written examination, validation, and scoring for the ranks of Sergeant and Lieutenant. 2015 Cook County Sheriff's Office Department of Court Services Written examination, validation, and scoring for the ranks of Sergeant and Lieutenant. 2015 Cook County Sheriff's Office Written examination, validation, and scoring for the ranks of Police Officer, Sergeant and Lieutenant. 2015 Ann Arbor, MI Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant. 2015 Ann Arbor, MI Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2015 Minneapolis, MN Police Department Written examinations and assessment center development, validations, and administration for the ranks of Lieutenant. 2015 Milwaukee, WI Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Lieutenant. 2015 Milwaukee, WI Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Captain. 2015 Minneapolis, MN Police Department Written examinations and assessment center development, validation, and administration for the rank of Sergeant. 2014 Arlington, TX Fire Department Job analysis review/update, written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2014 Arlington, TX Fire Department Job analysis review/update, written examination and assessment center development, validation, administration and scoring for the rank of Captain. 2014 St. Louis, MO Police Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Lieutenant. 2013-2014 St. Louis, MO Police Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Captain. 2013-2014 Miami, FL Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2013-2014 Atlanta, GA Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Sergeant. 2013-2014 48 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 27 Atlanta, GA Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Lieutenant. 2013-2014 San Diego, CA Police Department Job analysis written examination and structured oral assessment development, validation, administration and scoring for the rank of Police Officer III. 2013-2014 San Diego, CA Police Department Job analysis written examination and structured oral assessment development, validation, administration and scoring for the rank of Detective. 2013-2014 San Diego, CA Police Department Job analysis written examination and structured oral assessment development, validation, administration and scoring for the rank of Sergeant. 2013-2014 San Diego, CA Police Department Job analysis written examination and structured oral assessment development, validation, administration and scoring for the rank of Lieutenant. 2013-2014 Chicago, IL Police Department Job analysis, written examination, and assessment center development, validation, administration, and scoring for the rank of Sergeant. 2013-2014 Milwaukee, WI Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Captain. 2013 Milwaukee, WI Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of Lieutenant. 2013 Milwaukee, WI Fire Department Job analysis, written examination, and assessment center development, validation, and administration for the rank of HMO. 2013 San Jose, CA Fire Department Written examination and assessment center development, validation, and administration for the rank of Fire Engineer. 2013 San Jose, CA Fire Department Written examination and assessment center development, validation, and administration for the rank of Prevention Inspector. 2013 San Jose, CA Fire Department Written examination and assessment center development, validation, and administration for the rank of Arson Investigator. 2013 Springfield, IL Police and Fire Departments Written Examinations and Oral Interview development and administration for police and fire. 2013 Minneapolis, MN Police Department Job analysis review/update, written examination and assessment center development, validations, administration and scoring for the rank of Lieutenant. 2012-2013 Buffalo Police Department Job analysis written examination and structured oral assessment/assessment center development, validations, administration and scoring for Detective, Detective-Sergeant, Lieutenant, Captain and Inspector. 2011-2012 San Diego, CA Police Department S Job analysis written examination and structured oral assessment development, validations, administration and scoring for police officer III, Detective, Sergeant and Lieutenant. 2011-2012 Atlanta, GA Police Department Sergeant and Lieutenant Promotional Process Job analysis update, written examination and structured oral assessment development, validations, administration and scoring for police Sergeant and Lieutenant. 2011-2012 Tulsa, OK Police Department Job analysis and written examination and assessment center development, validation, administration and scoring for the ranks of Corporal, Sergeant, Captain, Major and Deputy Chief. 2011-2012 Ann Arbor, MI Police Department Written examination and assessment center development, validation, administration and scoring for the ranks of Sergeant and Lieutenant. 2011-2012 49 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 28 Chicago Fire Department, EMS Job analysis and written examination and assessment center development, validation, administration and scoring for the rank of Ambulance Commander and Paramedic Field Chief 2011-2012 Omaha Police and Fire Department Oversight on the development of written examinations and assessment centers, including development, validation, administration and scoring for the rank of Police Sergeant, Lieutenant and Captain and Fire Specialist, Lieutenant, Captain and Battalion Chief. Conduct job analysis for all protective services ranks. Assistance with the implementation of the city's entry-level fire fighter selection process. 2011-2012 Florida Department of Transportation, Motor Carrier Division Written examination development, validation, administration and scoring for the rank of Sergeant. 2011 Cook County Sheriff's Office promotional process Written examination development, validation, administration and scoring for the ranks of Corrections Sergeant and Lieutenant, Court Services Sergeant and Lieutenant and Sheriff's Police Officer (lateral), Sergeant and Lieutenant. 2011 Springfield, IL Police and Fire Departments Test coordination, administration, scoring/banding, analysis and reporting for entry-level positions of police officer and firefighter. 2011 Minneapolis Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Sergeant 2011 Buffalo Police Department Development and validation of a test battery for Police Officer in the BPD. Includes the development of a parallel cognitive examination, the transportability of an integrity inventory and an assessment of personality. Worked with city's legal council, the DOJ (and DOJ's testing expert) and the local courts to implement a suitable entry-level selection process and was able to lift 33-year old court oversight of the process. 2010-2011 Arlington, TX Fire Department Written examination and assessment center development, validation, administration and scoring for the ranks of Lieutenant and Captain. 2010-2011 Minneapolis Police Department Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2010-2011 Illinois State Police (through the State Police Merit Board) Written examination and assessment center development, validation, administration and scoring for the ranks of Sergeant and Master Sergeant. 2010 Houston Police Department Promotional Process Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant (Additional testing from 2009 list). 2010 Massachusetts State Police Written job knowledge development, validation and administration to the rank of Sergeant 2010 Washington D.C. Fire Sergeant, Lieutenant and Captain Promotional Process Job analysis, written examination and assessment center development, validation, administration and scoring for three ranks 2009-2010 Washington D.C. Fire Suppression Battalion Chief Promotional Process Job analysis, written examination and assessment center development, validation, administration and scoring for Fire Suppression Battalion Chief 2009-2010 Washington D.C. EMS Captain Promotional Process Job analysis, written examination and assessment center development, validation, administration and scoring for EMS Captain 2009-2010 City of Chicago- Police Communication Operator I Selection Process Test coordination, administration, scoring, analysis and reporting for entry-level position. 2010 San Diego, CA Police Job analysis, written examination and structured oral assessment 2009-2010 50 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 29 Department Sergeant and Lieutenant Promotional Process development, validation, administration and scoring for police Sergeant and Lieutenant. Los Alamos Fire Department Promotional Process to Assistant Chief and Deputy Chief Assessment center development, validation, training, administration and reporting for the ranks of AC and DC. 2009 Illinois State Police (through the State Police Merit Board) Job analysis to the rank of Trooper. 2009 City of Chicago- Police Communication Operator I Selection Process Job analysis, test development and validation for a test of oral communication clarity. Test coordination, administration, scoring, analysis and reporting for entry-level position. 2009 Atlanta, GA Police Department Sergeant and Lieutenant Promotional Process Job analysis, written examination and structured oral assessment development, validations, administration and scoring for police Sergeant and Lieutenant. 2009 Chicago Fire Department- Entry-Level Fire USERRA Examination Retesting for entry-level fire examination for military candidates per USERRA: Fire entry-level written examination administration and application processing. 2009 Cook County Sheriff's Office promotional process Written examination development and administration for sheriff's police Sergeant and Lieutenant, court services Sergeant and Lieutenant and correctional division Sergeant, Lieutenant and Captain. 2009 Houston Police Department Promotional Process Written examination and assessment center development, validation, administration and scoring for the rank of Lieutenant. 2009 Houston Police Department Promotional Process Written examination and assessment center development, validation, administration and scoring for the rank of Captain. 2009 Washington, D.C. Metropolitan Police Department Development of a job knowledge examinations for the ranks of Sergeant, Lieutenant, and Captain 2008-2009 Massachusetts State Police Written job knowledge and structured oral assessment development, validation and administration to the rank of Captain. 2009 Minneapolis Police Job analysis to the ranks of Sergeant, Lieutenant and Captain for the MPD. 2009 DC Metro Police Development of a job knowledge examination for the ranks of Sergeant, Lieutenant, and Captain. 2008-2009 Milwaukee Fire Department Entry-Level Testing Process Job analysis, validation, administration and scoring on a fire entry- level examination for the city of Milwaukee. Development, validation and assessor training of an entry-level fire structured oral interview. 2008-2009 Austin, TX Police Commander and Lieutenant Promotional Process Developed, validated, and administered an assessment center for the ranks of Lieutenant and Commander. 2008 Buffalo, NY Police Promotional Process Retesting for military candidates: Written examination and assessment center for Lieutenant, Detective and Captain. 2008 Chicago Fire Department- FCOI Entry-Level Testing Entry-level written examination validation, administration, application processing and scoring for Fire Communications Officer I position. 2008 Joplin MO Police Department PAT Development Development and validation of an entry-level police physical ability test for the Joplin Police Department. 2008 Los Alamos Fire Department Promotional Process Development, validation and administration of assessment centers for the ranks of Battalion Chief and Assistant Chief for LAFD. 2008 51 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 30 Lisle, IL Police Department Promotional Process for Sergeant Administration of an OTS written examination and development, validation, administration and scoring of a Sergeant assessment center. 2008 Buffalo, NY Police Promotional Process to the ranks of Detective and Detective Sergeant Job analysis, written examination and structured oral interview (Detective) and structured oral assessment (Detective Sergeant) development, validation, administration and scoring for police Detective and Detective Sergeant. 2008 Buffalo, NY Police Promotional Process to the ranks of Captain and Inspector Job analysis, written examination and assessment center development, validation, administration and scoring for police Captain and Inspector. 2008 Buffalo, NY Police Promotional Process to the rank of Lieutenant Job analysis, written examination and structured oral assessment development, validation, administration and scoring for police Lieutenant. 2008 State of New Jersey, Department of Personnel Retesting of entry-level test made up of three parallel forms (developed in 2005-2006) for positions statewide including law enforcement officer, sheriff’s officer, sheriff’s deputy, corrections officer, state ranger, etc. Scoring and analysis for NJ-LEE 2008 administration 2008 Moline, IL Police Department Promotional Process to Lieutenant Development, validation and administration of a written examination and assessment center to the rank of Lieutenant in the Moline PD. 2007 Gwinnett County, GA Protective Services Job analysis for the ranks of Corporal, Sergeant and Lieutenant in the Police, Sheriff’s and Corrections Department; Job analysis for the ranks of Driver Engineer, Lieutenant and Captain in the Fire Department. 2007 Chicago Fire Department- Entry-Level Fire Written Examination Entry-level written examination validation, administration coordination and scoring-- project consultant role. 2006 Forsyth, GA Sheriff’s Department Promotional written examination and assessment center development, validation and administration to the ranks of Sergeant, Lieutenant & Captain. 2006 Tacoma, WA Police Department Lieutenant promotional examination development. 2006 Houston, TX Police Department Promotional written examination and assessment center development, validation, and administration to the rank of Captain. Development and administration of a promotional job knowledge examination for 41 candidates, as well as the development of an assessment center for 25 candidates. 2005-2006 Austin, TX Police Department Conducted a job analysis and managed the job knowledge examination development for the rank of Corporal. 2005-2006 State of New Jersey, Department of Personnel Re-development of three forms of the NJ-LEE entry-level selection tool. 2005-2006 Minneapolis, MN Police Department Assessment center development, validation, and administration to the rank of Lieutenant. Job analysis revision & assessment center development, validation & administration for 40 candidates. 2005 Austin, TX Police Department Developed, validated, and administered an assessment center for the rank of Sergeant. 2005 Austin, TX Police Department Conducted a job analysis and managed the job knowledge examination development for the rank of Detective. 2005 Orange County, FL Fire Rescue Department Job analysis, assessment center development; situation judgment test development, computer writing assessment development and job knowledge examination development for the rank to Lieutenant. 2005 Massachusetts State Police Written job knowledge and structured oral assessment 2005 52 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 31 development, validation and administration to the rank of Captain. Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Corporal in the Sheriff’s Department. 2005 Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Sergeant in the Sheriff’s Department. 2005 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Lieutenant in the Sheriff’s Department. 2005 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Lieutenant in the Sheriff’s Department. 2005 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Sergeant in the Corrections Department. 2005 Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Corporal in the Police Department. 2005 Fort Lauderdale, FL Fire Department Fire Inspector II promotional examination development. 2005 Fort Lauderdale, FL Fire Department Driver Engineer promotional examination development. 2005 Tacoma, WA Police Department Captain promotional examination development. 2005 Tacoma, WA Police Department Detective promotional examination development. 2005 West Pittston, PA Police Department Entry-level examination administration; PAT administration/consultation & oral board development and administration for 27 candidates in August, 2005 & 9 candidates in October, 2005. 2005 Washington, D.C. Metropolitan Police Department Development of a job knowledge examination for the ranks of Sergeant, Lieutenant, and Captain, for 1015, 196, and 108 candidates, respectively. 2004-2005 Austin, TX Police Department Developed, validated, and administered an assessment center for the ranks of Lieutenant and Commander. 2004-2005 Massachusetts State Police Written job knowledge and structured oral assessment development, validation and administration to the rank of Lieutenant. 2004-2005 Reading, MA Police Department Assessment center development and administration to the rank of Police Chief. 2004-2005 Springfield, MO Police Department Promotional written examination and assessment center development, validation & administration to the ranks of Corporal & Sergeant. 2004-2005 State of New Jersey, Department of Personnel Statewide job analysis, physical ability test development, assessment center development; situation judgment test development, computer writing assessment development and job knowledge examination development. Initial job analysis and entry-level test development and validation of three parallel forms for positions statewide including law enforcement officer, sheriff’s officer, sheriff’s deputy, corrections officer, state ranger, etc. 2004 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Lieutenant in the Sheriff’s Department. 2004 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Communications Officer III & IV in the Communication’s Division of the Police 2004 53 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 32 Department. Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Sergeant in the Sheriff’s Department. 2004 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Sergeant in the Police Department. 2004 St. John’s Island, SC Fire Protection District Entry-level and incumbent standard physical ability test development & validation and fitness program development. 2004 McHenry County, IL Sheriff’s Department Promotional written examination and assessment center development and administration to the rank of Lieutenant, 8 candidates, 7 assessors recruited. 2004 Westchester, IL Fire Department Promotional written examination and assessment center development and administration to the rank of Lieutenant, 5 candidates (3 AC). 2004 Norwalk, CT Police Department Conducted a job analysis, entry-level (NCJOSI) test field-testing, and criterion validation. 2004 Orange County, FL Fire Rescue Department Job analysis, assessment center development; situation judgment test development, computer writing assessment development for the rank to Battalion Chief. Spring 2004 Orange County, FL Fire Rescue Department Job analysis, assessment center development; situation judgment test development, computer writing assessment development and job knowledge examination development for the rank to Battalion Chief. Fall 2004 New York State Police Department Entry-Level written examination validation. Conducted a job analysis, field tested I/O Solutions Entry-Level Exam (NCJOSI), and validated the criterion of the NCJOSI. 2003-2004 Massachusetts State Police Written job knowledge examination development, validation and administration to the rank of Sergeant. 2003-2004 Ozark, AL and Enterprise, AL Fire Departments Entry-level and incumbent standard physical ability test development & validation for two agencies. 2003-2004 Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Corporal in the Police Department. 2003 Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Sergeant in the Sheriff’s Department. 2003 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Sergeant in the Police Department. 2003 Schaumburg, IL Police Department Promotional written examination development and administration to the rank of Lieutenant, 3 candidates. 2003 Westmont, IL Police Department Promotional Job Knowledge Examination Development & Assessment Center Development for 11 sergeant (8 AC) candidates and 4 lieutenant candidates (3 AC). 2003 Clarksville, TN Police Department Promotional process development and validation for the ranks of Sergeant & Lieutenant. 2003 Richton Park, IL Police Department Assessment center development and administration to the rank of Sergeant, 3 candidates, 3 assessors recruited. 2003 New Haven, CT Fire Department Promotional written job knowledge examination and structured oral assessment development, validation & administration to the ranks of Lieutenant and Captain. 2003 Frankfort, KY Fire Department Assessment center development and administration to the ranks of Driver Engineer, Lieutenant, Captain, Battalion Chief & Deputy Chief. 2003 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Lieutenant in the 2002-2003 54 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 33 Police Department. Gwinnett County, GA Protective Services Written job knowledge and practical tactical exercise development, validation and administration to the rank of Driver Engineer in the Fire Department. 2002-2003 North Miami, FL Police Department Promotional written examination and assessment center development, validation & administration to the ranks of Sergeant & Lieutenant. 2002-2003 Matteson, IL Police Department Assessment center development and administration to the rank of Sergeant, 5 candidates, 3 assessors recruited. 2002-2003 Tulsa, OK Police and Fire Department Entry-level police and fire written examination development & validation, structured oral interview development & validation and entry-level physical ability test development & validation. 2002-2003 Massachusetts State Police Entry-level video-based situational judgment test development, validation and administration. 2002 Gwinnett County, GA Protective Services Written job knowledge examination development, validation and administration to the rank of Corporal in the Sheriff’s Department. 2002 Gwinnett County, GA Protective Services Written job knowledge, assessment center development and structured oral assessment development validation and administration to the rank of Sergeant in the Correction’s Department. 2002 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Lieutenant in the Sheriff’s Department. 2002 Gwinnett County, GA Protective Services Written job knowledge development, validation and administration to the rank of Corporal in the Corrections Department. 2002 Gwinnett County, GA Protective Services Written job knowledge and assessment center development, validation and administration to the rank of Captain in the Fire Department. 2002 North Miami, FL Police Department Entry-level police written examination development & validation, B-PAD validation, entry-level physical ability test validation and entry-level selection process review. 2002 Roselle, IL Police Department Managerial assessment for promotion to Deputy Chief. 2002 Gwinnett County, GA Protective Services Job analysis for the ranks of Corporal, Sergeant and Lieutenant in the Police, Sheriff’s and Corrections Department; Job analysis for the ranks of Driver Engineer, Lieutenant and Captain in the Fire Department. 2002 Publications: O’Sullivan, B.J. and Roch, S. (April 1999). The Longitudinal Effects of FOR and Observational Training on Accuracy. Paper presented at the 14th Annual Meeting for the Society for Industrial and Organizational Psychology, Atlanta, GA. Jones, J.A., O’Sullivan, B.J., and Carson, A. (August 1998). Mechanical Aptitude Measurement. Paper presented at the Meeting of the American Psychological Association, San Francisco, CA. 55 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 34 MARK W. TAWNEY, Ph.D. Principal & Vice President of Testing Services Background Statement: Dr. Tawney has more than ten years of experience in public safety testing and assessment and has led the test publication and research and development divisions of two national public safety testing firms. Dr. Tawney is a Principal Member who leads our Testing Services division. Dr. Tawney is a gifted psychometrician and test validation expert. He holds a Ph.D. in Industrial/Organizational Psychology from Illinois Institute of Technology and a Master’s degree in Experimental Psychology from DePaul University. Dr. Tawney’s major area of research and contribution to the field of I/O psychology deals with investigating alternative methodologies to enhance non-cognitive selection tools. His research aims to enhance the predictive quality of these assessment tools such that they can be used in conjunction with more traditional cognitive assessments. Education: Illinois Institute of Technology, Chicago, Illinois Ph.D.: Industrial/Organizational Psychology (2012) Master of Science: Industrial/Organizational Psychology (2007) DePaul University, Chicago, Illinois Master of Science, Experimental Psychology (2005) Coe College, Cedar Rapids, Iowa Bachelor of Science, Psychology & Philosophy (2003) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Chicago Industrial/Organizational/Psychology (CIOP) Professional Experience: 2013 to Present Industrial/Organizational Solutions, Inc. Principle - Vice President of Testing Services 2011 to 2013 Selection Works, LLC. Co-Founder - Vice President of Research and Development 2005 to 2011 Industrial/Organizational Solutions, Inc. Research and Development Manager 56 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 35 Summary of Selection Projects: The following table provides a small sampling of Dr. Tawney’s projects over the past few years. Client Consulting Work Client Project Description Timeframe New York State Department of Civil Service Development of state-wide entry-level selection examination for the State of New York. Project work includes: Examination plan development from job analytic data, development of cognitive and non-cognitive questions, pilot and local field-testing, and criterion-related and content validation strategies for the various components of the examination. Served as Department’s expert to collaborate with the Department of Justice, who had oversight on the project. 2016-Present Milwaukee, WI Fire Department Oversaw the development, validation, and implementation of entry-level selection process which included an exam battery of cognitive ability, personality, and a structured oral interview. Developed methods for deploying "daily" SOI training for 200 plus civilian raters. 2017-Present Cincinnati, OH Fire Department Conducted local transportation validation study and job analysis for an entry-level selection tool for the position of firefighter. Analyzed results for an assessment of impact of minimally qualified cut-off point. 2017-Present Indianapolis, IN Police Department Oversaw the development, validation, and implementation of entry-level selection process which included an exam battery of cognitive ability, integrity and a structured oral assessment - including a video-based writing sample, video-based SJT, and structured oral interview. 2017 - Present Chicago, IL Police Department Developed methodologies and strategies to establish minimally qualified cut-point for the developed examination and analyzed the testing impact on sub-groups. 2017-2018 Cook County Merit Board, IL Customized and validated entry-level selection process for the entry-level positions. Including CTT and DIF analysis for item level impact. Conducted empirical keying methodology to validate a custom keying solution for a measure of integrity. Project yielded customized solution that greatly reduced impact against protected classes, while maintaining prediction of essential criteria. 2017 Edmond, OK Fire Department Oversaw the development, validation, and implementation of entry-level selection process which included an exam battery of cognitive ability, personality, integrity and a structured oral interview. Developed methods for adding structure to final stage Chief's interview - a semi-structured interview. 2017 57 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 36 Indianapolis, IN Fire Department Oversaw the development, validation, and implementation of entry-level selection process which included an exam battery of cognitive ability, personality, integrity and a structured oral interview. 2017 Cincinnati, OH Police Department Conducted local transportation validation study and job analysis for an entry-level selection tool for the position of police officer. Analyzed results for an assessment of impact of minimally qualified cut-off point. 2017 Richmond, VA Fire Department Conducted local transportation validation study and job analysis for an entry-level selection tool for the position of firefighter. Analyzed results for an assessment of impact of minimally qualified cut-off point. 2017 New Jersey Civil Service Commission Customized and re-validated entry-level selection process for the entry- level firefighter position across the state of New Jersey. Including CTT and DIF analysis for item level impact, and removal of poor functioning items. Conducted predictive validation study with selection tool by linking recently hired firefighters' academy and job performance with their examination results. 2017-Present New York Mass Transit Authority Police Department Conducted a state-wide transportability validation study for the large scale administration of an entry-level police examination to approximately 20,000 applicants. 2017 Lubbock, TX Customized an entry-level selection exam and conducted a transportability validation study, while servicing as City’s expert to collaborate with the Department of Justice. 2016-2017 Dallas, TX Police Department Conducted job analysis for all non-executive ranks within the department. 2016 - 2017 Montgomery County. MD Fire and Rescue Department Customized and validated entry-level selection process for the entry-level firefighter position. Including CTT and DIF analysis for item level impact, transportability study and structure oral interview development utilizing a content validation strategy. 2016 Chicago, IL Police Department Customized and validated entry-level selection process for the entry-level police officer position. Including CTT and DIF analysis for item level impact. Conducted empirical keying methodology to validate a custom keying solution for a measure of integrity. 2016 Bridgeport, CT Fire Department Customized and validated entry-level selection exam for the entry-level firefighter position. Including CTT and DIF analysis for item level impact, concurrent validation study and content validation study for a situational judgement test. 2016 58 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 37 Norwalk, CT Fire Department Standardization, cut-off score analysis, and impact statistics of entry-level testing battery which included a multi-staged written exams and structured oral interview. 2016 Buffalo, NY Police Department Development and validation of entry-level selection exam for the entry- level police officers position. 2015-2016 New York State Department of Civil Service Development of state-wide job analysis for the purpose of developing and validation a new entry-level selection process for the State of New York. Project work included: Job analysis interviews and observation, development of job analysis questionnaire and linkage analysis survey and the development of a comprehensive state-wide sampling plan and analysis. Served as Department’s expert to collaborate with the Department of Justice, who had oversight on the project. 2015-2016 Massachusetts Trial Court Development and validation of physical ability test for the position of entry-level court officers for the Massachusetts Trail Court. 2015 Bridgeport, CT Police Department Customized and developed entry-level selection tool to hire for the entry-level police officers for the Bridgeport, CT Police Department which included: local field-testing and criterion-related validation study, content validation sessions and custom scoring of a Situation judgement test with SMEs, item-analysis and DIF analysis on archival examination results, custom cut-score analysis. 2015 New Jersey Civil Service Commission Customized and developed entry-level selection tool to hire for the entry-level firefighter position in the State of New Jersey which included: transportation of evidence for a written exam, local field-testing, content validation sessions and custom scoring of a Situation judgement test with SMEs, field test and validation of a physical ability test, item-analysis and DIF analysis on examination results, custom cut-score analysis. 2015 Houston, TX Fire Department Assessed test and item functioning after initial use by means of conducting DIF analysis and item analysis on applicant data with the goal of improving future test performance. 2015 Florida Department of Financial Service - Firefighter Job Task Analysis Conduct state-wide job analysis for the entry-level firefighter position (paid and volunteer) in the State of Florida which includes: interviews, large scale job analysis questionnaire deployment to over 5000 incumbent firefighters, linkage analysis, and development of technical reports. 2014-2015 Chicago, IL Fire Department Aided in the validation and customization work of an entry-level selection tool to hire entry-level firefighters for the city of Chicago. 2014-2015 59 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 38 Developed methodologies and strategies to establish minimally qualified cut-point for the developed examination and analyzed the testing impact on sub-groups. New Jersey Civil Service Commission Conducted state-wide job analysis for the entry-level firefighter position in the State of New Jersey which included: interviews, large scale job analysis questionnaire deployment to 3000 incumbent firefighters, linkage analysis and test plan development. 2014-2015 Miami, FL Fire Department Conducted job analysis, transportability validation study and cut-off score analysis for an entry-level selection examination. 2014 Massachusetts Trial Court Conducted state-wide job analysis, test customization and validation of the entry-level selection examination for three position within the organization. 2014 Chicago IL Police Department Aided in the validation and customization work of an entry-level selection tool to hire entry-level police officers for the city of Chicago. 2013-2014 Developed methodologies and strategies to establish minimally qualified cut-point for the developed examination and analyzed the testing impact on sub-groups. Pittsburgh PA Fire Department Validated and customized a scoring solution of an off-the-shelf product for use in hiring entry-level firefighters. Conducted DIF analysis, item-analysis and cut-score analysis as well as pareto-optimal weighting analysis between battery components to ensure examination process was unbiased and withstood legal scrutiny. 2013-2014 Pittsburgh PA Fire Department Served as a technical expert for the city as they collaborated with the Department of Justice for the entry-level hiring process. 2013-2014 Pennsylvania State Police Customized and validated off-the-shelf selection tool for use by the state to select individuals to enter into state academies. Conducted DIF analysis, item-analysis and a cut-score analysis to ensure examination was unbiased against sub-groups 2013-2014 Connecticut Emergency Medical Services Developed and validated Emergency Medical Examinations (EMR & EMT) recertification exams for the state of Connecticut. 2013-2014 San Antonio, TX Conducted transportability validation study and job analysis for the development and customization of an entry-level firefighter selection battery including a cognitive ability measure, personality index and integrity inventory. Oversaw the delivery and administration of this selection system. 2009-Present 60 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 39 San Antonio, TX Conducted transportability validation study and job analysis for the development and customization of an entry-level police officer selection battery including a cognitive ability measure, personality index and integrity inventory. Oversaw the delivery and administration of this selection system. 2009-Present Houston, TX Fire Department Developed and validated a customized entry-level selection tool to hire entry-level firefighters. 2012-2013 Assessed test and item functioning after initial use by means of conducting DIF analysis and item analysis on applicant data with the goal of improving future test performance. Illinois State Police & Illinois State Police Merit Board Developed and validated a customized entry-level selection tool to select applicants to enter into academies. 2012 Edmond, OK Fire Department Developed and validated a customized entry-level selection tool to hire entry-level firefighters. 2012 Dell Computers Developed globally administered sales talent assessment tool used by Dell Large Enterprise Unit to aid in a larger organization change initiative. 2009-2011 Managed the IT infrastructure development of a web administration application to allow for global deployment of the assessment tool. Managed assessment tool’s translation into 6 languages and assessed parallelism by means of structural equation modeling. Austin, TX Fire Department Development and validation of entry-level selection process battery, including a cognitive ability measure, personality index, and integrity inventory. 2010-2011 Served as technical expert and legal support for the development, validation, and implementation of entry-level selection process. Seattle Fire Department Designed a testing process and customized scoring solution for firefighters. 2011 Cook County Merit Board, IL Developed and presented county-wide candidate orientation and test preparation program for County’s entry-level criminal justice officer positions. 2009-2011 Bridgeport, CT Police Department Oversaw the development, validation, and implementation of entry-level selection process which included an exam battery of cognitive ability, personality, integrity and a structured oral interview. 2010 61 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 40 Buffalo, NY Police Department Development and validation of a test battery for Police Officer in the BPD. Includes the development of a parallel cognitive examination, the transportability of an integrity inventory and an assessment of personality. Worked with city's legal council, the DOJ (and DOJ's testing expert) and the local courts to implement a suitable entry-level selection process and was able to lift 33-year old court oversight of the process. 2010-2011 Missouri State Highway Patrol Conducted longitudinal study to validate integrity exam for entry-level troopers. 2007-2010 New York State Police Developed, validated and implemented state-wide entry-level selection exam, including the customization of a personality inventory and cognitive ability tool. 2009 Milwaukee Fire Department Conducted job analysis, transportability validation study, and examination customization / development for the entry-level selection process. 2008 Wisconsin Emergency Medical Service Board Served as technical expert for the development of EMT – Intermediate Technician exam. 2008 Developed, implemented and supported state wide web-application system for the delivery of EMT – I-Tech exam program. Florida Department of Law Enforcement Developed, validated and implemented statewide assessment program for entry-level criminal justice officers and managed statewide client service for this program. 2006-Present Criminal Justice Officer Basic Abilities Test - Law Enforcement Officer (CJBAT - LEO) Entry-level examination for the State of Florida's law enforcement positions. Exam measures cognitive ability. 2007 Criminal Justice Officer Basic Abilities Test - Corrections Officer (CJBAT - CO) Entry-level examination for the State of Florida's corrections positions. Exam measures cognitive ability. 2007 Published Assessments Assessment Tool Description Timeframe Law Enforcement Officers Selection Tool (LST) Entry-level examination battery for law enforcement containing cognitive ability, personality, biodata and integrity. 2011-2012 Firefighter Selection Tool (FST) Entry-level examination battery for fire services containing cognitive ability, personality, biodata and integrity. 2011-2012 Situational-Based Styles Assessment - Firefighter (SBSA-FF) Entry-level situational judgment test (SJT) for fire services. 2011-2012 62 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 41 Situational-Based Styles Assessment - Law Enforcement Officer (SBSA-LEO) Entry-level situational judgment test (SJT) for law enforcement. 2011-2012 Dispatcher Selection Tool (DST) Entry-level examination for emergency dispatcher. Assessment consists of a cognitive ability measure. 2011-2012 Sales Talent Indicator Assessment to identify individuals who are likely to be successful in sales who have no previous experience based on a personality and attribute inventory. 2009-2010 Sales Talent Predictor Assessment to identify talented sales staff with two or more years of experience. Assessment battery includes sales competency model and a personality and attribute inventory. 2009-2010 National Criminal Justice Officers Selection Inventory - Integrity (NCJOSI^2) Entry-level examination battery for law enforcement containing cognitive ability and integrity. 2009 Integrity Inventory Developed for the public safety sector, this examination measures the construct of integrity. 2009 National Firefighter Selection Inventory (NFSI) Entry-level examination battery for fire services containing cognitive ability and personality. Personality framework was redesigned from previous version. 2008 Publications:  Choplin, J. M. & Tawney, M. W. (August, 2005). Comparison-Induced Anchoring Effects, Proceedings of the 27th Annual Conference of the Cognitive Science Society, Stresa, Italy.  Legel, C. C., O’Sullivan, B. J. & Tawney, M. W. (2011). Considering Disparate Impact Beyond Impact Ratios. Presented at the 2011 International Personnel Assessment Council Conference, Washington D.C.  Tawney, M. W. (2006). Implicit attitudes, racial bias, and LMX: The impact of latent racism on leader member exchange relationships. Presented at the 27th Annual Industrial Organizational and Organizational Behavior (IOOB) Conference, Fairfax, VA.  Tawney, M. W. (2008). Integrity Testing…the Selection Tool of the Future. Law & Order Magazine, 56(12), 34-38.  Tawney, M. W. and Choplin J. (2004) Comparison-induced anchoring effects. Presented at the Midwest Psychology Association, Chicago, IL  Tawney, M. W., and Deprez-Sims, A-S. (2010). Integrity Testing for Public Safety Selection. Presented at the 118th American Psychological Association (APA) conference, San Diego, CA.  Tawney, M. W., Ishaya, N. and Ayman, R. (2007). Effect of Ratings sources and Leader's Gender on Performance Appraisal. Presented at the 28th Annual Industrial Organizational and Organizational Behavior (IOOB) Conference, Indianapolis, IN.  Tawney, M. W., and Lehn, D. (2003). Individual difference in sensory threshold for self-relevant stimuli as a function of self-consciousness. Presented at Midwest Psychology Association, Chicago, IL. 63 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 42  Tawney, M. W., Mead, A., & Solberg, E. (April, 2008). Empirical keying of personality-oriented data: Development and comparison of methods. Symposium Presented at the 23rd Annual Society for Industrial and Organizational Psychology Conference, San Francisco, CA.  Towler, A. & Tawney, M. W. (March, 2008). Charismatic leadership emergence: Role of work values. Presented at the 20th Annual Association for Psychological Science Conference, Chicago, IL. 64 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 43 Nahren Cama, Ph.D. Manager, Consulting Services Background Statement: Dr. Cama enjoys working with clients to develop and implement customized solutions to fit their needs. She possesses exceptional writing and organizational skills and is experienced communicating with all organizational levels and managing projects to successful completion. She is skilled in building relationships with her clients, gaining their buy-in and trust. At IOS, Dr. Cama has managed all phases of entry-level and promotional selection processes for agencies all across the country. Education: Illinois Institute of Technology Ph.D. in Industrial/Organizational Psychology (2018) Completed Ph.D. comprehensive exams (2007) M.S. in Industrial/Organizational Psychology (2006) Lake Forest College B.A. in Psychology and Business (2003) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2016-present Manager, Consulting Services  Performs the following in addition to the responsibilities of the Senior Consultant: o Leads a team of consultants to ensure delivery of high- quality assessment projects. Ensures that team members are applying best practices of the industry. Actively mentors and teaches team members. o Supervises a team of consultants in order to ensure effective completion of tasks. Conducts annual performance appraisals and directly oversees team members work on a weekly basis. o Manages the hiring, training, and development of new consulting staff. 65 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 44 2014-2016 Senior Consultant, I/O Solutions, Inc.  Consult with public safety agencies to develop, validate and defend entry-level and promotional testing/assessment systems  Conduct job analysis (interviews, observations, and questionnaires) to identify essential tasks, knowledge, skills, abilities and other characteristics relevant to the position  Provide best-practice training to public safety agencies, human resources and civil service personnel  Develop technical reports for processes documenting all aspects of the process and validation evidence to support assessments 2012-2014 Consultant, Daniels Consulting Group  Developed training content for a multinational insurance corporation  Supported the change management efforts of a worldwide manufacturing organization 2008-2008 Assessor – Contract Position, United Airlines  Observed, recorded, and evaluated leadership behaviors during multiple group exercises  Met with Assessor team and United HR representative to finalize ratings and determine who met qualifications  Wrote individual candidate feedback reports to support ratings and overall decisions and delivered individual feedback to candidates 2008-2008 Consultant – Contract Position, HR Alignment, Ltd.  Assisted with the development of a competency based interview selection guide  Researched and recommended books for a leadership competency guide  Edited and reviewed reports and guides 2007-2007 Consultant – Contract Position, Daniels Consulting Group  Responsible for the qualitative and quantitative analysis of data for a healthcare accreditation company undergoing changes in standards affecting their customers.  Created reports to leadership presented survey data, themes, and areas for the organization to take action on to improve efficiencies following the changes. 2008-2012 66 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 45  Developed communication pieces for various stakeholders within a healthcare accreditation company following a change in technology. 2007-2007 Survey Project Manager, The Foresight Group  Created client databases for survey collection  Facilitated day-to-day client and consultant interactions.  Created client reports and presentations 2007-2007 Lecturer – Contract Position, Northeastern Illinois University  Created and presented lectures. Facilitated class discussions. Designed quizzes and exams. Designed in-class activities and projects. 2006-2006 Consultant – Contract Position, Daniels Consulting Group  Designed an employee job satisfaction survey for a fitness club client. Conducted quantitative and qualitative analyses of survey data. Wrote a report detailing survey results and trends.  Developed communication pieces on several topics for new-hires in a healthcare organization; topics ranged from communication skills to phone etiquette. 2004-2006 Organizational Development Associate, Advocate Christ Medical Center  Designed a survey for new-hires to gain better understanding of the factors that attracted them to the organization and position of interest. Helped manage associate satisfaction survey process. Wrote reports communicating survey results for HR and leadership.  Developed an administrator guide for a selection inventory. Revised/updated training guides as needed for various programs.  Responsible for the management of recognition programs. Communicated various aspects of these programs to leadership. Summary of Selection Projects: Client Project Description Timeframe City of San Francisco Developed and administered assessment exercises and content for the police lieutenant position. 2019-2020 Cobb County Sheriff's Office - Sergeant & Lieutenant Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. 2019 67 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 46 Client Project Description Timeframe Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. City of Roswell, Georgia Development of written examination and assessment center for the Roswell Police Department Sergeant and Lieutenant positions. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. 2019 Cook County Sheriff's Office Merit Board Written examination development and administration for sheriff's police Lateral Entry Officer, Sergeant, and Lieutenant, court services Sergeant and Lieutenant and correctional division Sergeant, and Lieutenant. 2019 Cobb County, GA Developed and administered assessment center for the rank of Fire Captain. 2019 IL State Police Merit Board Development of written examination and assessment center for the Illinois State Police Lieutenant and Captain positions. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. Development of technical report. 2019 Monroe County Sheriff’s Office Developed and administered assessment centers for the rank of Road Lieutenant and Jail Sergeant. 2018-2019 Cobb County Sheriff's Office - Sergeant & Lieutenant Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. 2018-2019 City of Roswell, Georgia Developed and administered written examination and assessment center for the rank of Police Sergeant and an assessment center for the rank of Police Lieutenant. 2018-2019 Shelby County Sheriff's Office Conducted a job analysis and pay study for the Criminal Analyst position. 2018 Monroe County Sheriff’s Office Developed and administered assessment centers for the rank of Investigator and Road Sergeant. 2018 Shelby County Sheriff's Office Development of a job description for the Evidence Custodian position. 2018 IL State Police Merit Board Conducted a job analysis for the lieutenant and captain positions (interviews and questionnaires) and developed a technical report. 2018 IL State Police Merit Board Development of written examination and assessment center for the Illinois State Police Sergeant and Master Sergeant positions. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. Development of technical report. 2018 68 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 47 Client Project Description Timeframe Cook County Sheriff's Office Merit Board Written examination development and administration for sheriff's police Lateral Entry Officer, Sergeant, and Lieutenant, court services Sergeant and Lieutenant and correctional division Sergeant, and Lieutenant. 2017-2018 Cobb County, GA Developed and administered assessment center for the rank of Fire Captain. 2017-2018 City of Roswell, Georgia Developed and administered written examination and assessment centers for the rank of Police Sergeant and Lieutenant. 2017-2018 Skokie Fire Department Developed and administered assessment center for the rank of Fire Chief. 2017 Shelby County Sheriff's Office Developed and administered assessment center for the rank of Captain. 2017 Cobb County Sheriff's Office - Sergeant & Lieutenant Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. 2017 City of Roswell, Georgia Developed and administered written examination and assessment center for the rank of Police Sergeant. 2017 IL State Police Merit Board Development of written examination and assessment center for the Illinois State Police Lieutenant and Captain positions. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. Development of technical report. 2017 IL State Police Merit Board Conducted a job analysis for the sergeant and master sergeant positions (interviews and questionnaires) and developed a technical report. 2017 IL State Police Merit Board Created entry-level assessment tools for the trooper position. 2016 IL State Police Merit Board Development of written examination and assessment center for the Illinois State Police Sergeant and Master Sergeant positions. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. Development of technical report. 2016 IL State Police Merit Board Conducted a job analysis for the trooper position (interviews and questionnaires) and developed a technical report. 2016 City of San Francisco Developed and administered assessment exercises and content for the police lieutenant position. 2016 69 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 48 Client Project Description Timeframe City of San Francisco Developed and administered assessment exercises and content for the fire Battalion Chief position. 2016 City of Roswell, Georgia Development of written examination and assessment center for the Roswell Police Department Sergeant and Lieutenant positions. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. 2016 Plano Police Department Development and validation of an entry-level PAT. 2016 Ann Arbor, MI Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for Ann Arbor Police Department sergeant and lieutenant positions. 2016 Miami Police Department Development of parallel assessment center for the Lieutenant rank. 2015 New York State Department of Civil Service Development of job analysis and linkage questionnaires. 2015 State of Washington Review and validation of new items for state driver's license exams. 2015 Las Cruces, NM PAT Development of candidate and proctor guides for an entry-level PAT, and provided technical report. 2015 Chicago Police Department - Sergeant Merit Process Assisted in the development of training and interview materials for the sergeant merit process. 2015 Ann Arbor, MI Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for Ann Arbor Police Department sergeant and lieutenant positions. 2015 Cook County Sheriff's Office Merit Board Written examination development and administration for sheriff's police Lateral Entry Officer, Sergeant, and Lieutenant, court services Sergeant and Lieutenant and correctional division Sergeant, and Lieutenant. 2015 City of Kenosha - Police Chief Assessment Center Development and administration of an assessment center for the Police Chief position in the Kenosha Police Department. 2015 Florida Department of Financial Service - Firefighter Job Task Analysis Conduct state-wide job analysis for the entry-level firefighter position (paid and volunteer) in the State of Florida which includes: interviews, large scale job analysis questionnaire deployment to over 5000 incumbent firefighters, linkage analysis, and development of technical reports. 2014-2015 Cobb County Police Department - Sergeant & Lieutenant Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. 2014-2015 70 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 49 Client Project Description Timeframe Gwinnett County Police Department; Corporal, Sergeant, and Lieutenant Development of written examination and assessment center. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. 2015 Lincoln Fire & Rescue Department - Captain Development of written examination and assessment center. Delivery of candidate orientation. Conduct assessor training and administration of written exam and assessment center. Development of candidate feedback reports. 2014-2015 Cobb County Sheriff's Office - Sergeant & Lieutenant Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. 2014 Orleans Parish Sheriff's Office Selection process for HR Director position 2014 Virginia Beach Fire Department - Captain rank Conduct linkage analysis and report results. Development of written examination and assessment center. Delivered candidate orientation. Conducted assessor training and administered written exam and assessment center. Development of candidate feedback reports. 2012 Prince William County Department of Fire and Rescue- Tech II and Lieutenant ranks Development of written examination and assessment center. Delivered candidate orientation. Conducted assessor training and administered written exam and assessment center. Development of candidate feedback reports. 2012 Lincoln Fire & Rescue Department; Re-validation Re-validation of entry-level physical ability test. 2011-2012 East Point Police Department, Sergeant, Lieutenant, and Captain Promotionals Development of written examinations and structured oral interviews. 2011 Los Alamos Fire Department - Company Officer Development and management of written examination and assessment center. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2011 Los Alamos Fire Department - Fire Chief Developed a structured oral interview with rating criteria. Developed a custom "meet and greet" rating document. 2011 Los Alamos Fire Department - Battalion Chief Development assessment center and interview. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2011 71 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 50 Client Project Description Timeframe Savannah-Chatham Metropolitan Police Department - Make-up Sergeant and Captain Promotional Processes Development of written examinations and assessment centers. Delivered candidate orientations. Conducted assessor training and administered written exams and assessment centers. Development of candidate feedback reports. Developed technical documentation. 2011 Gwinnett County Police Department - Sergeant and Lieutenant Promotional Processes Development of written examinations and assessment centers. Delivered candidate orientations. Conducted assessor training and administered written exams and assessment centers. Development of candidate feedback reports. 2011 Westmont Police Department - Sergeant Rank Development of written examination and assessment center. Delivered candidate orientation. Conducted assessor training and administered written exam and assessment center. Development of candidate feedback reports. 2011 Prince William County Department of Fire and Rescue- Tech II and Captain ranks Conducted a job analysis. Development of written examination and assessment center. Delivered candidate orientation. Conducted assessor training and administered written exam and assessment center. Development of candidate feedback reports. 2011 Broward County Community College - Tech II and Captain processes Validation study 2011 Illinois State Police - Lieutenant and Captain promotional process Development of lieutenant and captain written examinations and assessment centers. 2011 Gwinnett County Police Department Corporal Promotional Process Developed written exam, conducted candidate orientation, administered exam, compiled results. 2010-2011 Evanston Police Department - Sergeant Promotional Process Development of written examination and assessment center. Delivered candidate orientation. Conducted assessor training and administered written exam and assessment center. Development of candidate feedback reports. 2010-2011 Buffalo Police Department - entry-level process Development and management of written exam, validation, development of technical report. 2010-2011 Massachusetts State Police - Captain make-up Promotional Process Development and management of job knowledge written exam and structured oral interview. 2010-2011 City of Bridgeport Developed technical report for Bridgeport Police Department's entry-level selection process. 2010 Lafayette Police Department - Sergeant, Lieutenant, and Captain written exams Development and management of job knowledge written examinations. 2010 72 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 51 Client Project Description Timeframe Los Alamos Fire Department - Driver Engineer Promotional Process Development and management of written exam and practical assessment. Conducted assessor training and managed administration of practical assessment. Developed candidate feedback reports. 2010 Savannah-Chatham Metropolitan Police Department - Sergeant and Lieutenant Promotional Processes Development and management of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training, and managed assessment center administrations. Developed candidate feedback reports. 2010 Cobb County Sheriff's Office - Sergeant and Lieutenant Promotional Processes Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. Developed technical report. 2010 Los Alamos Fire Department - Captain Promotional Process Development and management of written examination and assessment center. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2010 Los Alamos Police Department - Sergeant Promotional Process Development and management of written exam and structured oral interview. Development of candidate feedback reports. 2010 Missouri State Highway Patrol - entry-level SOI development Development of structured oral interview. 2010 Illinois State Police - Sergeant and Master Sergeant Promotional Processes Conducted job analysis update. Development and management of written examinations. Development of technical report. 2010 Massachusetts State Police -Sergeant Rank Managed the development of the sergeant exam. 2010 Los Alamos Fire Department - Battalion Chief Development of assessment center exercises, structured oral interviews and related materials, conduct assessor training, and develop feedback reports. Developed candidate feedback reports. 2010 Gwinnett County Police Department - Sergeant Rank Managed all phases of the promotional process for the Sergeant rank. Coordinated with main contact and SMEs to finalize written exams, and create assessment center exercises. Conducted training for candidates and assessors. Managed the administration of the assessment center. Developed feedback reports. 2010 73 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 52 Client Project Description Timeframe District of Columbia Fire Department Conducted job analysis, managed subject matter experts, developed assessment center exercises, managed administration of assessment center, developed feedback reports. 2010 Gwinnett County Police Department; Corporal, Sergeant, and Lieutenant Promotions Managed all phases of the promotional process for the Corporal, Sergeant, and Lieutenant ranks. Coordinated with main contact and SMEs to finalize written exams, assessment center exercises. Conducted training for candidates and assessors. Managed the administration of the assessment center. Developed feedback reports. 2009 Cobb County Sheriff's Office - Sergeant Promotional Process Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. Developed technical report. 2009 Los Alamos Fire Department - Driver Engineer Promotional Process Development and management of written exam and practical assessment. Conducted assessor training and managed administration of practical assessment. Developed structured oral interview. Developed candidate feedback reports. 2009 East Point Police Department, Sergeant, Lieutenant, and Captain Promotions Managed all phases of the promotional process for Sergeant, Lieutenant, and Captain ranks. Worked with SMEs to determine the exam reading list for the Sergeant and Lieutenant rank and finalized the written exams. Other aspects of this project included: finalizing written exams, developing assessment center exercises and a structured oral interview, creating feedback reports, candidate orientations, assessor training, overseeing administration of the assessments, and development of candidate feedback reports. 2009 Lincoln Fire & Rescue Department; Entry-level PAT development and validation Conducted a physical task job analysis to assess which physical tasks were essential to the job of a firefighter within the Lincoln Fire & Rescue Department. Helped in the development and validation of the physical ability test. Created a candidate guide, proctor guide, and technical report. 2008-2009 Bradley Police Department; Sergeant Promotional Developed a promotional-level oral interview for their sergeant assessment. Provided training to assessors on how to rate candidates during the interview. Facilitated the administration of the oral interviews. 2008-2009 Norwood Park Fire Department; Entry-level firefighter oral interview Developed an entry-level interview and other materials for their entry-level hiring. Provided training to their commissioners on administering the interview. 2008-2009 74 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 53 Client Project Description Timeframe Aurora Fire Department; Captain Assessment Center Development of assessment center exercises. Training of assessors and commissioners. Managed administration of assessment center. 2009 Gwinnett County Sheriff's Office Work with internal SME to finalize reading lists for Corporal, Sergeant, and Lieutenant ranks. Finalize written exams and oversee administration. Development of an assessment center for the Lieutenant rank. 2009 Bradley Police Department; Lieutenant Promotional Development of assessment center exercises. Provide training to assessors and oversee administration of the assessment center. 2009 Los Alamos Fire Department - Deputy Chief & Assistant Chief Development of assessment center exercises and related materials. Developed candidate feedback reports. 2009 Conference Presentations:  Cama, N., and Ayman, R. The Paradox of Communication Technology In The Work- Family Interface. (April, 2020). Presented at the Society for Industrial Organizational Psychologists (SIOP) Conference, Austin, TX.  Ishaya, N., Ayman, R., & Korabik, K. Why So Much Guilt? Investigating How Overload Hurts and Why Control May Help. (May, 2013). Presented at the Society for Industrial Organizational Psychologists (SIOP) Conference, Houston, TX.  Ishaya, N. and Ayman, R. Predicting Work-Family Conflict Via Perceived Involvement And Overload. (August, 2008). Presented at the APA Convention, Boston, MA.  Tawney, M., Ishaya, N. and Ayman, R. (March, 2007). Effect of Ratings sources and Leader's Gender on Performance Appraisal. Presented at the Industrial Organizational and Organizational Behavior (IOOB) Conference, Indianapolis, IN.  Ronen, E., Ishaya, N., and Kozminski, K. (May, 2006). Job Satisfaction Among Healthcare Employees: Does Time of Year Matter? Presented at the Society for Industrial Organizational Psychologists (SIOP) Conference, Dallas, TX.  Velgach, S., Ishaya, N., and Ayman, R. (April, 2005). Multi-Method Approach to Investigate Work-Family Conflict. Presented at the Society for Industrial Organizational Psychologists (SIOP) Conference, Los Angeles, CA. 75 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 54 BRIAN J. MARENTETTE, Ph.D. Manager, Strategic Assessments Background Statement: Dr. Marentette’s primary areas of expertise include applied statistics, psychometrics, job analysis, test development, and test validation. Dr. Marentette has published research in the International Journal of Selection and Assessment and the Organizational Psychology Review and has taught several undergraduate- and graduate-level courses in the area of psychological testing. He is also an active member in the Society for Industrial/Organizational Psychology, frequently presenting research at the annual conference. Dr. Marentette brings a high level of technical knowledge and expertise to his clients in developing and validating situational judgment tests, personality assessments, cognitive ability tests, job simulations, structured oral interviews, job knowledge tests and assessment centers. Education: DePaul University, Chicago, Illinois Ph.D. in Industrial/Organizational Psychology (2015) California State University, Sacramento, California M.A. in Industrial/Organizational Psychology (2009) Michigan State University, East Lansing, Michigan B.A. in Psychology (2005) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2014-Present Manager, Consulting Services, IOS, Inc.  Manages high-stakes assessment processes, often involving Federal oversight by the Department of Justice and other special interest groups.  Consult with public safety agencies to develop, validate and defend entry-level and promotional testing/assessment systems  Provided best-practice training to public safety agencies, human resources and civil service personnel 76 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 55  Collaborates with the R & D team at I/O Solutions to create off-the- shelf tests.  Leads a consulting team by innovating current practices and procedures to stay current with cutting-edge techniques in the industry. Identifies team performance goals that contribute to the organization’s effectiveness.  Supervises a team of consulting staff. Organizes, plans and monitors employees’ work to ensure effective task completion. Conducts annual performance appraisals and directly oversees team member work on an ongoing basis. Recruits, hires and onboards new employees.  Manages the training and development of consulting staff, including the development and delivery of structured training on highly technical concepts, coaching, and ongoing development. 2012-2014 Consultant, IOS, Inc.  Consult with public safety agencies to develop, validate and defend entry-level and promotional testing/assessment systems  Provided best-practice training to public safety agencies, human resources and civil service personnel  Collaborates with the R & D team at I/O Solutions to create off-the- shelf tests. 2011-2012 Teaching Fellow, DePaul University  Full course responsibility for PSY 343 Introduction to Psychological Measurement (class size of 35 upperclassmen).  Topics in the course included test development, validation, adverse impact analysis, item writing, and related statistical analyses. 2008-2011 Doctoral Research Assistant, DePaul University  Designed and implemented research studies; managed project phases, conducted literature reviews, designed methods and procedures, collected and analyzed data.  Prepared and wrote manuscripts and research reports; originated content, added theoretical background, prepared content for publication.  Developed lab tasks that guided students learning of statistical analyses using SAS and SPSS statistical software.  Conducted weekly lab sessions instructing students how to conduct statistical analyses using SAS and SPSS. Provided one-on-one instruction and tutoring. 2006-2008 Consultant, Biddle Consulting Group, Inc.  Developed and validated a variety of personnel assessments for public and private sector agencies, including situational judgment tests, 77 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 56 personality inventories, structured interviews, multi-media situational judgment tests, and job-knowledge written tests.  Conducted job analyses using focus groups, interviews, and online job analysis questionnaires.  Developed and delivered training to organizations on job analysis, personnel selection procedures, and Equal Employment Opportunity compliance.  Served as product manager of off-the-shelf personnel assessment software. Coordinated efforts of software development, marketing and advertising. Designed assessment program and developed multi- media situational judgment test content. 2006 Graduate Assistant (Intern), California State Personnel Board  Planned job analysis and examination development projects, including determining project timelines and participants.  Facilitated subject matter expert meetings for jobs with varying levels of complexity (e.g., entry level, management and doctorate level scientist).  Created job analysis and test development technical reports.  Developed a job analysis database tool in MS Access consisting of over 300 job task statements and 200 KSA statements utilized by SMEs across dozens of state agencies in California. Summary of Selection Projects: Dr. Marentette has a wealth of experience providing selection consulting services to a diverse listing of state, municipal and county public safety agencies. Following is a partial list of the jobs that Dr. Marentette has personally managed in the past several years: Client Project Title Timeframe Massachusetts Trial Court Development of an assistant chief court officer and chief court officer examination 2019-present Massachusetts Trial Court Development of an assistant chief probation officer and chief probation officer examination 2019-present Washington, D.C. Fire & EMS Department Development of captain, lieutenant and sergeant written examinations and assessment centers 2020-present Montgomery County Fire & Rescue Service Development of a master firefighter examination 2020-present Montgomery County Fire & Rescue Service Development of a fire lieutenant examination and assessment center 2020-present Detroit, MI Police Department Development of detective, sergeant, and lieutenant written examinations and assessment centers 2018-2019 Portland, OR Fire & Rescue Department Development of battalion chief, captain and 2018-present 78 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 57 lieutenant assessment centers Grand Rapids, MI Police Department Development of a captain assessment center 2018-present Grand Rapids, MI Police Department Development of a lieutenant examination and assessment center 2018-present Grand Rapids, MI Police Department Development of a sergeant examination and assessment center 2018-present Dallas, TX Fire & Rescue Department Development of fire operations driver, operations lieutenant, captain, battalion chief; fire prevention lieutenant, captain and section chief examinations and assessment centers. 2018-present Montgomery County Fire & Rescue Service Development of a master firefighter examination 2018 Montgomery County Fire & Rescue Service Development of a fire lieutenant examination and assessment center 2018 Massachusetts State Police Development of a lieutenant examination and oral interview 2017 Montgomery County, MD Sheriff's Office Development of a sergeant examination and oral interview 2017 Montgomery County, MD Sheriff's Office Development of a lieutenant examination and oral interview 2017 Massachusetts Trial Court Development of an assistant chief court officer and chief court officer examination 2017 Massachusetts Trial Court Development of an assistant chief probation officer and chief probation officer examination 2017 Montgomery County, MD Police Department Development of master police officer examination 2017 Montgomery County, MD Police Department Development of sergeant examination and assessment center 2017 San Francisco, CA Fire Department Development of lieutenant assessment center 2017 Ann Arbor, MI Police Department Development of sergeant examination and assessment center 2017-present Ann Arbor, MI Police Department Development of lieutenant examination and assessment center 2017-present Grand Rapids, MI Police Department Development of a captain assessment center 2017 Grand Rapids, MI Police Department Development of a lieutenant examination and assessment center 2017 Grand Rapids, MI Police Department Development of a sergeant examination and assessment center 2017 Buffalo, NY Fire Department Development of lieutenant examination and assessment center 2016-2017 Buffalo, NY Fire Department Development of captain examination and assessment center 2016-2017 Buffalo, NY Fire Department Development of battalion chief assessment center 2016-2017 79 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 58 Buffalo, NY Fire Department Development of division chief assessment center 2016-2017 Chicago, IL Fire Department Development of ambulance commander examination and oral assessment 2016-2017 Chicago, IL Fire Department Development of paramedic field chief examination and oral assessment 2016-2017 Bridgeport, CT Police Department Development of a detective examination and assessment center 2015 Bridgeport, CT Police Department Development of a police officer oral interview 2015 Roswell, GA Police Department Development of a lieutenant assessment center 2015 Roswell, GA Police Department Development of a sergeant examination and assessment center 2015 Massachusetts State Police Development of a lieutenant examination and oral interview 2015 Massachusetts Trial Court Development of an assistant chief court officer and chief court officer examination 2015 Massachusetts Trial Court Development of an assistant chief probation officer and chief probation officer examination 2015 Montgomery County, MD Police Department Development of a sergeant examination and validation of assessment center 2015 Montgomery County, MD Police Department Development of a master police officer examination 2015 Montgomery County Fire & Rescue Service Development of a fire captain assessment center 2015 Montgomery County Fire & Rescue Service Development of a master firefighter examination 2014 Montgomery County Fire & Rescue Service Development of a fire lieutenant examination and assessment center 2014 Montgomery County, MD Sheriff's Office Development of a sergeant examination and oral interview 2014 Montgomery County, MD Sheriff's Office Development of a lieutenant examination and oral interview 2014 Arlington, TX Fire Department Development of a captain examination and assessment center 2014 Arlington, TX Fire Department Development of a lieutenant examination and assessment center 2014 Massachusetts State Police Development of a captain examination and oral interview 2014 80 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 59 Miami Beach, FL Fire Department Validation of an entry-level firefighter examination and development of an oral interview 2014 Miami Beach, FL Fire Department Development of a captain examination and oral interview 2014 Miami Beach, FL Fire Department Development of a lieutenant examination and oral interview 2014 Bridgeport, CT Police Department Development of a sergeant examination and assessment center 2014 Round Rock, TX Fire Department Development of a captain examination 2014 Massachusetts Trial Court Validation of an entry-level probation officer examination 2014 Massachusetts Trial Court Validation of an entry-level court officer examination 2014 Roswell, GA Fire Department Development of a fire chief assessment center 2014 Chicago OEMC Implementation of a dispatcher examination process 2014 Grand Rapids, MI Police Department Development of a captain assessment center 2014 Grand Rapids, MI Police Department Development of a lieutenant examination and assessment center 2014 Grand Rapids, MI Police Department Development of a sergeant examination and assessment center 2014 Chattanooga, TN Fire Department Development of a captain examination and assessment center 2014 Chattanooga, TN Fire Department Development of a lieutenant examination and assessment center 2014 Augusta, GA Fire Department Development of a battalion chief examination and assessment center 2014 Augusta, GA Fire Department Development of a captain examination and assessment center 2014 Augusta, GA Fire Department Development of a lieutenant examination and assessment center 2014 Augusta, GA Fire Department Development of a sergeant examination and practical assessment 2014 Baltimore Fire Department Development of a battalion chief examination and assessment center 2014 Chicago Police Department Development of a police sergeant examination and assessment center 2013 81 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 60 Massachusetts State Police Department Development of a police captain examination and structured oral interview 2013 Miami Beach Police Department Development of a police lieutenant examination and assessment center 2013 Miami Beach Police Department Development of a police sergeant examination and assessment center 2013 Montgomery County Department of Corrections and Rehabilitation Development of a corrections lieutenant examination and assessment center 2013 Montgomery County Department of Corrections and Rehabilitation Development of a corrections sergeant examination and assessment center 2013 Montgomery County Fire & Rescue Service Development of an entry level fire fighter exam and structured oral interview 2013 Montgomery County Fire & Rescue Service Development of a fire battalion chief examination and assessment center 2013 Montgomery County Fire & Rescue Service Development of a fire captain examination and assessment center 2013 New York Metropolitan Transit Authority Police Department Implementation of an entry level police officer examination 2013 Roswell, GA Police Department Development of a police chief examination and assessment center 2013 Roswell, GA Police Department Development of a police lieutenant examination and assessment center 2013 Roswell, GA Police Department Development of a police sergeant examination and assessment center 2013 Round Rock, TX Fire Department Development of a fire captain written examination 2013 Round Rock, TX Fire Department Development of an entry level fire fighter written examination 2013 Chicago Fire Department Development of a paramedic field chief written examination and structured oral interview 2012 Chicago Fire Department Development of an ambulance commander written examination and structured oral interview 2012 Chicago Metra Police Department Development of a police sergeant examination and structured oral interview 2012 Chicago Metra Police Department Development of a police lieutenant examination and structured oral interview 2012 82 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 61 Massachusetts State Police Department Development of a police lieutenant examination and structured oral interview 2012 New Jersey Civil Service Commission Job analysis for corrections captain position 2012 New Jersey Civil Service Commission Job analysis for corrections lieutenant position 2012 New Jersey Civil Service Commission Job analysis for corrections sergeant position 2012 Conference Presentations:  Marentette, B. J. (June, 2015). Situational Judgement Tests: An alternative to high-volume structured oral interviews. Presentation at the 7th annual Public Safety Testing Summit, Oak Brook, IL.  Marentette, B. J. (June, 2014). Methods of test score use in public safety selection and promotional testing. Presentation at the 6th annual Public Safety Testing Summit, Oak Brook, IL.  Bell, S. T., Mann, K., Marentette, B. J., Adair, C.., Fisher, D., & Arman, G. (2013, May). Assertiveness and Concern as Predictors of Shared Mental Models and Team Performance. Poster presented at the 25th annual meeting of the Association for Psychological Science, Washington, D.C.  Halpert, J.A., Linnabery, E., Abben, D.R., Marentette, B.J., Lueck, M.D. (2013). Test- Retest Predictive Validities And a Moderator. Poster presentation at the 26th annual conference for the Association for Psychological Science, San Francisco, CA.  Marentette, B. J., Halpert, J. A., Linnabery, E., & Passuello, M. (April, 2012). Situational Judgment Tests as a Form of Realistic Job Preview. Poster presentation at the 27th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.  Fisher, D. M., Marentette, B. J., Adair, C. K. (April, 2012). Team Selection Regarding Personality Composition: Propositions for Improved Validity. Poster presentation at the 27th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.  Bell, S. T., Marentette, B. J., Adair, C.K., Fisher, D. M., Arman, G. & Mann, K. E. (April, 2012). Keeping Tabs on Teammates: Reliance, Team Processes, and Team Performance. Symposium presentation at the 27th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.  Adair, C. K., Arman, G., Quesnell, T. J., Marentette, B. J., Fisher, D. M., Mann, K. E., Bell, S. T. (April, 2012). Team-level personality and its relationship with team processes. Poster presentation at the 27th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.  Marentette, B. J., Bell, S.T., Adair, C. K., Fisher, D., Lewis, D. D., & Gerding, D. (April, 2010). Team Process Measurement: Comparing Team Member and Observer Ratings. Poster presented at the 25th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA. 83 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 62  Adair, C. K., Bell, S. T., Marentette, B. J., Fisher, D., Gerding, D. (April, 2010). An Application of the Punctuated Equilibrium Model to Team Processes. Poster presented at the 25th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.  Haynes, S., Halpert, J., Marentette, B. J., & Lueck, M. (April, 2010). Investigating Pregnancy and Marital Status Discrimination in Employee Performance Appraisals. Interactive poster presented at the 25th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.  Halpert, J., Haynes, S., Lueck, M., Marentette, B. J. (April, 2010). Attitudes about Pregnant Employees: Change over Twenty Years. Interactive poster presented at the 25th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.  Marentette, B. J., Kuang, D. C., & Meyers, L. S. (April, 2008). An Investigation of Influence of Test Taking Fatigue on SJTs. Poster presented at the 23rd annual conference of the Society for Industrial and Organizational Psychology, San Francisco, CA.  Marentette, B. J. & Throckmorton, D. (March, 2007). Using On-Line Survey Tools to Complete Job Analyses. Presented at the 21st Annual Personnel Testing Council of Northern California Conference, Sacramento, CA. Publications:  Marentette, B. J., Meyers, L. S., Hurtz, G. M., & Kuang, D. C. (2012). Order Effects on Situational Judgment Test Items: A Case of Construct-Irrelevant Difficulty. International Journal of Selection and Assessment, 20, 318-332.  Kalchman, M. & Marentette, B. J. (2012). Homework as test preparation: Its promise and efficacy. Current Issues in Middle Level Education, 17, 1-8.  Bell, S. T. & Marentette, B. J. (2011). Team viability for long-term and ongoing organizational teams. Organizational Psychology Review, 1, 275-292. 84 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 63 COURTNEY WIERZBICKI (Nelson), PH.D. Manager, Consulting Services Background Statement: Dr. Wierzbicki (formerly Nelson) is a consultant at Industrial/Organizational Solutions, Inc., a personnel and human resources consulting firm that specializes in testing and assessment. Dr. Wierzbicki has experience in developing, validating, administering and defending assessment centers, promotional job knowledge examinations, entry-level examinations and oral interviews in the public safety sector. Education: The University of Tulsa, Tulsa, Oklahoma Doctor of Philosophy: Industrial/Organizational Psychology (2013) Dissertation: The impact of differential learning on the relationship between training and store level outcomes The University of Tulsa, Tulsa, Oklahoma Master of Arts: Industrial/Organizational Psychology (2010) Saint Louis University, St. Louis, Missouri Bachelor of Arts, Psychology (2008) Certificate in Human Resource Management, John Cook School of Business Affiliations: Society for Industrial/Organizational Psychology (SIOP) Society of Human Resource Management (SHRM) Professional Experience: 2016 to Present Industrial/Organizational Solutions, Inc. Manager  Performs all functions of a consultant, in addition to:  Leads a team of consultants to deliver high-quality assessment projects. Ensures that team members are applying best practices of the industry. Actively mentors and teaches team members.  Supervises a team of consultants in order to ensure effective completion of tasks. Conducts annual performance appraisals and directly oversees team members work on a weekly basis.  Manages the hiring, training, and development of new consulting staff. 85 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 64 2016 Industrial/Organizational Solutions, Inc. Senior Consultant  Independently and simultaneously managed multiple test development and validation projects for promotion testing\ assessment systems  Conducted job analysis (interviews and observations) to identify the essential knowledge, skills, and abilities and other characteristics necessary for performance 2013 to 2015 Industrial/Organizational Solutions, Inc. Consultant  Consulted with public safety agencies to develop, validate and defend entry-level and promotional testing/assessment systems  Provided best-practice training to public safety agencies, human resources and civil service personnel 2011 to 2012 Hogan Assessment Systems Research Associate, Research & Development Department  Constructed selection profiles for client specific jobs using personality and values-based assessments  Conducted validity generalization research for employee selection and development initiatives using meta-analysis, synthetic validity, and transport validity methods  Aligned and mapped client competency models to Hogan competencies  Carried out job analysis procedures including content analysis/job documentation review, interviewing job incumbents, and collection of focus group data 2010 PennWell Corporation Intern, Human Resources  Conducted employee selection interviews and made hiring recommendations to managers  Designed structure oral interviews through job analysis, question generation, and question selection  Interpreted psychological assessments (cognitive ability, personality, and emotional intelligence) 2009 to 2010 Reliant Live Consulting Intern  Compiled multiple resources to develop leadership competencies following 360 performance appraisal 86 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 65  Created and managed a database of leadership development resources Summary of Selection Projects: Client Project Description Timeframe Tulsa, OK Developed and administered an assessment center for the rank of Fire Deputy Chief 2020 San Antonio, TX Developed a written examination for the rank of Police Sergeant. 2020 Tulsa, OK Developed and administered an assessment center for the rank of Police Major. 2019 Tulsa, OK Developed and administered an assessment center for the rank of Police Captain. 2019 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2019 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Sergeant. 2019 Tulsa, OK Developed a written examination for the rank of Fire Equipment Operator. 2019 St. Paul, MN Developed and administered written examination and assessment center for the rank of Police Sergeant. 2019 Forsyth County, GA Developed and administered written examination and assessment center for the rank of Battalion Chief. 2019 Forsyth County, GA Developed and administered written examination and practical assessment for the rank of Fire Apparatus Operator. 2019 Guilford County Sheriff's Office Conducted job analysis for the ranks of Detention Sergeant, Detention Lieutenant, Sworn Sergeant, and Sworn Lieutenant. Developed written examinations for the ranks of Sworn Sergeant and Detention Sergeant. Developed assessment centers for the ranks of Detention Sergeant and Detention Lieutenant 2018 San Antonio, TX Developed a written examination for the rank of Police Sergeant and Captain. 2018 Pinellas County Sheriff's Office, FL Developed written examination for the ranks of Sergeant (Law Enforcement Office and Department of Corrections) 2018 Mississippi Highway Safety Patrol Conducted job analysis for the ranks of Master Sergeant, Lieutenant, and Major. Developed and administered structured oral interviews for the ranks of Master Sergeant, Lieutenant, and Major 2018 Arlington, TX Developed a written examination for the rank of Deputy Fire Marshal. 2018 Arlington, TX Developed a written examination for the rank of Fire Prevention Specialist. 2018 87 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 66 Dallas, TX Developed and administered assessment center for the rank of Police Sergeant. 2018 Forsyth County, GA Developed and administered written examination and assessment center for the rank of Lieutenant. 2018 Forsyth County, GA Developed and administered written examination and practical assessment for the rank of Fire Apparatus Operator. 2018 Arlington, TX Developed a written examination for the ranks of Lieutenant and Captain. 2018 Tulsa, OK Developed and administered written examination and assessment center for the rank of Captain. 2018 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2018 Austin, TX Developed a written examination for the rank of Police Lieutenant. 2018 Austin, TX Developed a written examination for the rank of Police Commander. 2018 Austin, TX Developed a written examination for the rank of Police Sergeant. 2018 Arlington, TX Developed a written examination for the rank of Fire Apparatus Operator. 2018 Tulsa, OK Developed and administered written examination and assessment center for the rank of District Chief. 2018 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Sergeant. 2018 San Antonio, TX Developed a written examination for the rank of Police Detective-Investigator. 2018 Pinellas County Sheriff's Office, FL Developed written examination for the ranks of Sergeant (Law Enforcement Office and Department of Corrections) 2017 Austin, TX Developed a written examination for the rank of Police Corporal - Detective. 2017 Tulsa, OK Developed a written examination for the rank of Fire Equipment Operator. 2017 Forsyth County, GA Developed and administered written examination and assessment center for the rank of Battalion Chief. 2017 St. Paul, MN Developed and administered written examination and assessment center for the rank of Police Sergeant. 2017 St. Paul, MN Developed and administered assessment center for the rank of Police Commander. 2017 Dallas, TX Developed and administered assessment center for the rank of Police Sergeant. 2017 Dallas, TX Developed and administered assessment center for the rank of Police Lieutenant. 2017 88 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 67 San Antonio, TX Developed and administered written examination and assessment center for the rank of Police Sergeant. 2017 Arlington, TX Developed and administered written examination and assessment center for the rank of Fire Captain. 2017 Arlington, TX Developed and administered written examination and assessment center for the rank of Fire Lieutenant. 2017 Tulsa, OK Developed and administered assessment center for the rank of Police Captain. 2017 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2017 Montgomery County, MD Developed and administered assessment center for the rank of Fire Captain. 2017 Montgomery County, MD Developed and administered assessment center for the rank of Fire Battalion Chief. 2017 San Antonio, TX Developed and administered written examination and assessment center for the rank of Police Lieutenant. 2017 San Antonio, TX Developed and administered written examination and assessment center for the rank of Police Captain. 2017 Forsyth County, GA Developed and administered written examination and practical assessment for the rank of Fire Apparatus Operator. 2017 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Sergeant. 2017 San Antonio, TX Developed a written examination for the rank of Police Detective/Investigator. 2016 Tulsa, OK Conducted job analysis for the ranks of Fire Equipment Operator, Captain, District Chief, Assistant Chief, and Deputy Chief. 2016 Montgomery County, MD Developed and administered written examination and assessment center for the ranks of Master Firefighter and Fire Lieutenant. 2016 Chattanooga, TN Developed and administered written examination and assessment center for the rank of Police Sergeant. 2016 Chattanooga, TN Developed and administered written examination and assessment center for the rank of Police Lieutenant. 2016 Chattanooga, TN Developed and administered written examination and assessment center for the rank of Police Captain. 2016 Pinellas County Sheriff's Office, FL Developed written examination for the ranks of Sergeant (Law Enforcement and Department of Corrections) 2016 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2016 Springfield, IL Developed and validated structured oral interview for entry level police officer. 2016 Montgomery County, MD Conducted job analysis for the rank of Fire Assistant Chief. 2016 89 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 68 Forsyth County, GA Developed and administered written examination and assessment center for the rank of Fire Lieutenant. 2016 Montgomery County, MD Conducted job analysis for the rank of Police Captain. 2016 Forsyth County, GA Developed and administered written examination and practical assessment for the rank of Fire Apparatus Operator. 2016 Arlington, TX Developed and administered written examination and assessment center for the rank of Fire Lieutenant. 2016 Arlington, TX Developed and administered written examination and assessment center for the rank of Fire Captain. 2016 Austin, TX Developed and administered assessment center for the rank of Police Commander. 2016 Austin, TX Developed and administered assessment center for the rank of Police Lieutenant. 2016 Austin, TX Developed and administered assessment center for the rank of Police Sergeant. 2016 St. Paul, MN Developed and administered assessment center for the rank of Police Sergeant. 2016 Omaha, NE Developed and administered assessment center for the rank of Fire Battalion Chief. 2016 Tulsa, OK Developed and administered assessment center for the rank of Police Corporal. 2016 Tulsa, OK Developed and administered assessment center for the rank of Police Sergeant. 2016 Rockford, IL Conducted candidate screening and administered assessment center for the rank of Police Chief 2016 Springfield, IL Developed and administered written examination and assessment center for the ranks of Police Lieutenant. 2016 Springfield, IL Developed and administered written examination and assessment center for the ranks of Police Sergeant. 2016 Tulsa, OK Developed and administered assessment center for the rank of Police Major. 2016 Tulsa, OK Developed and administered assessment center for the rank of Police Captain. 2015 Austin, TX Conducted job analysis for the ranks of Police Detective, Corporal, Sergeant, Lieutenant, and Commander. 2015 Buffalo, NY Conducted job analysis for the rank of Police Officer 2015 St. Paul, MN Developed and administered assessment center for the rank of Police Lieutenant. 2015 Pinellas County Sheriff's Office, FL Developed written examination for the ranks of Sergeant (Law Enforcement and Deputy of Corrections) 2015 Forsyth County, GA Developed and administered written examination and assessment center for the rank of Fire Battalion Chief. 2015 90 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 69 San Antonio, TX Developed a written examination for the rank of Police Sergeant. 2015 San Antonio, TX Developed a written examination for the rank of Police Detective. 2015 San Antonio, TX Developed and administered written examination and assessment center for the rank of Police Lieutenant. 2015 San Antonio, TX Developed and administered written examination and assessment center for the rank of Police Captain. 2015 Arlington, TX Conducted job analysis, developed a written examination, and developed a scoring rubric for a practical skills evaluation for the rank of Apparatus Operator 2015 Omaha, NE Developed and administered an in-basket exercise and assessment centers for the rank of Police Captain. 2015 Springfield, IL Developed and validated structured oral interview for entry level firefighter. 2015 Springfield, IL Developed and validated structured oral interview for entry level police officer. 2015 Forsyth County, GA Conducted job analysis, developed and administered written examination and assessment center for the rank of Fire Lieutenant. 2015 Omaha, NE Developed and validated structured oral interview for entry level firefighter. 2015 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Sergeant. 2015 San Jose, CA Developed written examination for the rank of Fire Engineer. 2015 Arlington, TX Developed and administered written examination and assessment center for the rank of Fire Lieutenant. 2015 Arlington, TX Developed and administered written examination and assessment center for the rank of Fire Captain. 2015 Tulsa, OK Developed and administered assessment center for the rank of Deputy Chief of Police. 2015 Springfield, IL Conducted job analysis, developed and administered written examinations and assessment center for the rank of fire battalion chief. 2015 Springfield, IL Conducted job analysis, developed and administered written examinations and assessment center for the rank of Fire Captain. 2015 San Jose, CA Conducted job analysis, developed and administered written examination and assessment center for the rank of Fire Captain. 2014 City of Miami, FL Conducted job analysis for the rank of Firefighter. 2014 San Antonio, TX Developed written examination for the rank of Police Sergeant. 2014 San Antonio, TX Developed written examination for the rank of Police Detective. 2014 Plano, TX Developed written examination for the rank of Assistant Chief. 2014 Plano, TX Developed written examination for the rank of Battalion Chief. 2014 91 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 70 Plano, TX Developed and administered written examination for the rank of Fire Captain. 2014 Plano, TX Developed written examination for the rank of Fire Lieutenant. 2014 Plano, TX Developed written examination for the rank of Fire Apparatus Operator. 2014 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2014 Tulsa, OK Developed and administered assessment center for the rank of Police Captain. 2014 Tulsa, OK Developed and administered assessment center for the rank of Police Major. 2014 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Sergeant. 2014 Pinellas County Sheriff's Office, FL Developed and administered written examination and assessment center for the rank of Sergeant LEO (Law Enforcement Officer). 2014 Pinellas County Sheriff's Office, FL Developed and administered written examination and assessment center for the rank of Sergeant DCB (Detention & Corrections Bureau). 2014 Pinellas County Sheriff's Office, FL Developed and administered assessment center for the rank of Lieutenant LEO (Law Enforcement Officer). 2014 Pinellas County Sheriff's Office, FL Developed and administered assessment center for the rank of Lieutenant DCB (Detention & Corrections Bureau). 2014 DFW Airport Fire and Police Developed technical interview questions for the ranks of Police Sergeant and Lieutenant and Fire Captain and Battalion Chief. 2014 St. Paul, MN Assisted in the development of written examination and assessment center for the rank of Sergeant. 2014 Omaha, NE Developed and administered written examinations and assessment centers for the rank of Fire Captain. 2014 San Jose, CA Developed written examination for the rank of Fire Engineer. 2014 San Jose, CA Conducted job analysis, developed and administered written examinations and assessment center for the rank of fire battalion chief. 2014 Miami Beach, FL Conducted job analysis, developed and administered written examinations and structured oral interview for the rank of Lifeguard II. 2014 Miami Beach, FL Conducted job analysis, developed and administered written examinations and structured oral interview for the rank of Lifeguard Lieutenant. 2014 Omaha, NE Developed and administered assessment center for Assistant Fire Chief. 2014 92 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 71 Omaha, NE Developed and administered written examination and assessment center for police captain. 2013 San Jose, CA Developed and administered written examinations and assessment centers for the rank of Arson Investigator. 2013 San Jose, CA Developed and administered written examinations and assessment centers for the rank of Fire Prevention Inspector. 2013 Forsyth County, GA Conducted job analysis, developed and administered written examinations and assessment center for the rank of fire Battalion Chief. 2013 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2013 DFW Airport Fire and Police Conducted job analysis for the ranks of fire captain and battalion chief, sergeant and lieutenant for police. Linked job analysis tasks and KSAs to off-the-shelf promotional tests. 2013 Milwaukee, WI Assisted in the development and administration of written examinations and assessment centers for the rank of lieutenant. 2013 Milwaukee, WI Conducted job analysis and assisted in the development and administration of written examinations and assessment centers for the rank of fire captain. 2013 Omaha, NE Conducted job analysis and assisted in the development of written examination and assessment center for the rank of paramedic shift supervisor. 2013 Presentations:  Nelson, C.A., Benson, M., Bonilla, D., Foldes, H., Muros, J., Roberston, L., & Zaldivar, K. (April 2015). Stepping into Organizations: Strategies for Talking to the Business. Panel Discussion at the 30th Annual Conference for the Society for Industrial and Organizational Psychology, Philadelphia, PA.  Nelson, C. A., Brummel, B. J., & Wright, C. (April, 2013). The relationship between training scores and store-level outcomes. Poster presented at the 28th Annual Conference for the Society for Industrial and Organizational Psychology, Houston, TX.  Parker, K. N., Brummel, B. J., Nelson, C. A., Hannam, K. A., & Thomason, A. (April, 2013). Examining the curvilinear relationship between income and job satisfaction. Poster presented at the 28th Annual Conference for the Society for Industrial and Organizational Psychology, Houston, TX.  Simonet, D. V., Narayan, A., & Nelson, C. A. (April, 2013). Who is empowered? A dispositional basis to psychological empowerment. Poster presented at the 28th Annual Conference for the Society for Industrial and Organizational Psychology, Houston, TX.  Nelson, C. A., Simonet, D. V., & Narayan, A. (April, 2012). Exploring the proactive realization process of organizational culture via critical incident methodology. In A. Narayan (Co-Chair) & D. Simonet (CoChair), Key to the Lock: Matching Qualitative Methodology to Organizational Questions. Symposium at the 27th Annual Conference for the Society for Industrial and Organizational Psychology, San Diego, CA. 93 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 72  Palmer, A. J., Robertson, L., Nelson, C. A., & Pickering, D. (April, 2012). Employability using personality assessment. Poster presented at the 27th Annual Conference for the Society for Industrial and Organizational Psychology, San Diego, CA.  Simonet, D. V., Narayan, A., & Nelson, C. A. (April, 2012). Mediated relationships between psychological safety and multidimensional psychological empowerment. Poster presented at the 27th Annual Conference for the Society for Industrial and Organizational Psychology, San Diego, CA.  Simonet, D. V., Packell, K. M., Pellebon, C., Nelson, C. A., & Narayan, A. (April, 2011). Eliciting and categorizing organizational member values: Value laddering technique. Poster presented at the 26th Annual Conference for the Society for Industrial and Organizational Psychology, Chicago, IL. Publications: Smit, B., Ryan, L., & Nelson, C. (in press). Does job autonomy increase or decrease negative emotional displays from service workers? A test of competing hypotheses. Journal of Personnel Psychology. Simonet, D. V., Narayan, A., & Nelson, C. (2015). A social-cognitive moderated mediated model of psychological safety and empowerment. Journal of Psychology: Interdisciplinary and Applied, 149, 818-845. Nelson, C. A., Brummel, B. J., Grove, D. F., Jorgenson, N., Gamble, R., & Sen, S. (2010). Measuring creativity in software design. Proceedings of the International Conference on Computational Creativity, 205-214. 94 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 73 Amy Eitapence, M.A. Manager, Recruitment Services Background Statement: Ms. Eitapence is the manager for the recruitment division of I/O Solutions, Inc. (IOS), a personnel and human resources consulting firm that specializes in testing and assessment. Ms. Eitapence has experience in developing, validating, and administering assessment centers, promotional job knowledge examinations, entry-level examinations and oral interviews in the public safety sector. Ms. Eitapence also oversees and serves as a project manager for municipal clients in police officer/firefighter recruitment processes. Education: University at Albany, SUNY Master of Arts, Industrial/Organizational Psychology (2013) University at Oneonta, SUNY Bachelor of Science, Psychology (2010) Affiliations: Society for Industrial/Organizational Psychology (SIOP) American Psychological Association (APA) Professional Experience: 2014 to Present Industrial/Organizational Solutions, Inc. (IOS) Manager, Recruitment Services  Project management for entry-level recruitment, including but not limited to: developing job applications, coordinating the intake of applications and supporting documentation, screening applications using client specific parameters and administering testing  Consult with public safety agencies to develop, validate and defend entry-level and promotional testing/assessment systems. Systems include entry-level and promotional written exams, oral interviews and assessment centers.  Provide best-practice training to public safety agencies, human resources and civil service personnel  Oversee recruitment project managers and administrative staff. 95 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 74 2013 to 2014 Excelsior College Data Analyst  Conduct an array of activities that support psychometric, test administration, and test development functions  Assist in standard setting sessions, using the Angoff method to set cut points for college level examinations  Conduct analyses including, but not limited to: item analysis, dimensionality, speededness, and form equating  Coordinate the production of psychometric technical handbooks for publication 2011-2013 Chequed.com Research Analyst  Talent management solutions for organizations with high-volume hiring needs  Facilitated in the selection procedures for a pre-employment selection and screening  Responsibilities included directly working in test development, validation studies, job analyses, candidate interviews, and job profile benchmarking 2011-2013 NYS Department of Civil Service Intern, Testing Services Assistant  Assisted in test item construction, review, and evaluation  Contributed to appropriate exam comparisons, synthesizing surveys, subject matter expert reviews, and constructing exams Summary of Selection Projects: Client Project Description Timeframe Northbrook Fire Department Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Elmhurst Fire Department Development of assessment center for battalion chief and lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Oak Park Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 96 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 75 Oak Park Fire Department Development and management of written examination for battalion chief. Development of assessment center for battalion chief. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Orland Fire Department Development of entry-level structured oral interview for firefighter. 2019 La Grange Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Burr Ridge Police Department Development of entry-level structured oral interview for police officer. 2019 Arlington Heights Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Wheeling Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. Prepared and managed Merit and Efficiency process. 2019 Plainfield Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Grayslake Police Department Written exam development, administration and scoring for rank of sergeant. Development and management of structured oral interview (SOI) for sergeant. Conducted assessor training and management of SOI. 2019 Elgin Fire Department Written exam development, administration and scoring for rank of lieutenant. 2019 Wilmette Fire Department Written exam development, administration and scoring for rank of lieutenant. 2019 97 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 76 Des Plaines Fire Department Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2019 Berywn Fire Department Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Berywn Police Department Development and management of written examination for sergeant and lieutenant. Development of assessment center for sergeant and lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Oak Park Police Development and management of structured oral interview (SOI) for police chief. Conducted assessor training and management of SOI. 2018 Northbrook Fire Department Development and management of written examination for captain. Development of assessment center for captain. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Long Grove Fire Protection District Written exam development, administration and scoring for rank of lieutenant. 2018 Des Plaines Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Winnetka Police Department Written exam development, administration and scoring for rank of sergeant. 2018 Morton Grove Fire Department Written exam development, administration and scoring for rank of lieutenant. 2018 Woodridge Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 98 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 77 Mundelein Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Lincolnshire-Riverwoods FPD Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Hinsdale Police Department Written exam development, administration and scoring for rank of sergeant. 2018 Lemont Police Department Written exam development, administration and scoring for rank of sergeant. 2018 Downers Grove Police Department Development of structured oral interview for entry-level officers. 2018 Downers Grove Fire Department Development of structured oral interview for entry-level firefighters. 2018 Barrington Fire Department Written exam development, administration and scoring for rank of assistant chief. 2018 Elgin Fire & Police Departments Development of entry-level structured oral interview for both police and fire. 2018 Arlington Heights Police Department Development of assessment center for commander. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Arlington Heights Police Department Written exam development, administration and scoring for rank of sergeant. 2018 La Grange Fire Department Development and management of written examination for lieutenant. Development and management of structured oral interview. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2018 Brookfield Fire Department Development of structured oral interview for rank of captain. Conducted assessor training and management of oral interviews. 2017 Brookfield Police Department Development of structured oral interview for rank of sergeant. Conducted assessor training and management of oral interviews. 2017 Brookfield Police Department Development of structured oral interview for rank of lieutenant. Conducted assessor training and management of oral interviews. 2017 Clarendon Hills Police Department Written exam development, administration and scoring for rank of sergeant. 2017 Darien Police Department Development of structured oral interview for entry-level officers. 2017 99 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 78 Evergreen Park Police Department Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Glenview Police Department Development of structured oral interview for entry-level officers. 2017 Huntley Fire Protection District Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Huntley Fire Protection District Development and administration of structured oral interview for entry-level firefighters. 2017 La Grange Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Lake Bluff Police Department Written exam development, administration and scoring for rank of sergeant. Development of structured oral interview. 2017 Lake Forest Police Department Written exam development, administration and scoring for rank of sergeant. 2017 Mount Prospect Police Department Development of structured oral interview for entry-level officers and lateral hires. Conducted assessor training. 2017 Mount Prospect Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Niles Fire Department Development and management of written examinations for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. 2017 Oak Park Fire Department Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 100 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 79 Oak Park Police Department Development and management of written examinations for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Plainfield Police Department Development and administration of structured oral interview for entry-level officers. 2017 Plano Police Department Development and administration of structured oral interview for entry-level officers. 2017 Rolling Meadows Police Department Written exam development, administration and scoring for rank of sergeant. 2017 Western Springs Police Department Development and management of written examination for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Westmont Police Department Development and management of written examinations for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Wheeling Fire Department Development and management of written examination for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2017 Willowbrook Police Department Development of structured oral interview for entry-level officers. 2017 Wilmette Police Department Written exam development, administration and scoring for rank of lieutenant. 2017 Woodridge Police Department Development of structured oral interview for entry-level officers. 2017 Arlington Heights Police Department Written exam development, administration and scoring for rank of sergeant. 2016 Norwood Park Fire Protection District Development of structured oral interview for entry-level firefighters. 2016 West Chicago Police Department Written exam development, administration and scoring for rank of sergeant. Development of oral interview, conduct assessor training and manage interview administration. 2016 Western Springs Police Department Development of structured oral interview for entry-level officers. Conduct assessor training and manage interview administration. 2016 101 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 80 Berwyn Fire Department Development and management of written examinations for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2015 Berwyn Police Department Development and management of written examinations for sergeant and lieutenant. Development of assessment centers for sergeant and lieutenant. Conducted candidate orientations. Conducted assessor training and managed assessment center administrations. Developed candidate feedback reports. 2015 Downers Grove Fire Department Development and management of written examinations for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2015 Long Grove Fire Department Written exam development, administration and scoring for rank of lieutenant. 2015 Maywood Fire Department Development and administration of oral interviews for entry- level firefighter. 2015 Maywood Police Department Development and administration of oral interviews for entry- level officer. 2015 Moline Fire Department Development and management of written examinations for Battalion Chief. Development of assessment center for Battalion Chief. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2015 Morton Grove Fire Department Written exam development, administration and scoring for rank of sergeant. 2015 Mundelein Police Department Development of oral interview for entry-level officer. Development of oral interview for sergeant. Administration of written examinations for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2015 Norwood Park Fire Protection District Written exam development, administration and scoring for rank of lieutenant. Development of oral interview, conduct assessor training and manage interview administration. 2015 Palatine Fire Department Written exam development, administration and scoring for rank of lieutenant. 2015 Plano Police Department Development of structured oral interview for entry-level officers. 2015 102 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 81 Prospect Heights Fire Protection District Development and management of written examinations for lieutenant. Development of assessment center for lieutenant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2015 Tri-State Fire Protection District Development and administration of oral interviews for entry- level firefighter. 2015 Villa Park Police Department Development and management of written examinations for sergeant. Development of assessment center for sergeant. Conducted candidate orientation. Conducted assessor training and managed assessment center administration. Developed candidate feedback reports. 2015 Wilmette Fire Department Written exam development, administration and scoring for rank of lieutenant. 2015 Woodridge Police Department Development of structured oral interview for entry-level officers. 2015 103 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 82 KAREN STEIGER, M.A. Manager, Content Development Background Statement: Ms. Steiger has over 18 years’ experience at IOS writing and editing entry-level and promotional tests for public safety agencies across the nation. Ms. Steiger developed the item- writing and editing guidelines that have now become standard company practice. She supervises the entire Technical Writing Division and assigns and evaluates the work of other team members. Ms. Steiger acts as a member of the Off-the-Shelf Testing Division, leading efforts to update and maintain our current off-the-shelf testing products as well as develop new examinations. She has written and edited IOS’s entry-level and promotional off-the-shelf exams, including multiple forms of the NCJOSI, NFSI, NCJOSI2, Fundamentals of Law Enforcement promotional exams and Core Concepts of Law Enforcement Management promotional exams, among many others. She continues to perform all technical writing duties as needed—including writing and editing entry-level and promotional tests for public safety agencies, developing and writing study guides, online practice tests and other candidate resources, and editing I/O Solutions’ technical reports and marketing materials. Ms. Steiger has personally worked with nearly all of IOS’s consulting clients, past and present, including the following:  Arlington, TX Fire Department  Atlanta, PD Police Department  Austin, TX Police and Fire Departments  Baltimore, MD Police Department  Buffalo, NY Police Department  Cobb County, GA Sheriff's Office  Cook County, IL Sheriff's Department  East Point, GA Fire and Police Departments  The Florida Department of Law Enforcement  Fort Lauderdale, FL Fire Department  Gwinnett County Corrections, Fire, Police and Sheriff's Departments  Houston, TX Police Department  Massachusetts State Police  Metra Police Department  Portland, ME Fire and Police Departments  Prince William County, VA Fire Department  San Antonio, TX Fire and Police Departments  San Diego, CA Police Department  Shelby County, TN Fire Department  The Southern Nevada Health District  Springfield, IL Fire and Police Departments  St. Lucie County, FL Fire District  Stamford, CT Police Department  Tacoma, WA Police Department  Warwick, RI Fire and Police Departments 104 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 83 Maxwell G. Porter, M.S. Manager, Testing Services Background Statement: Mr. Porter is a Manager in the Research & Development Department, a division of Testing Services at IOS, a personnel and human resources consulting firm that specializes in testing and assessment. He also is currently in the process of completing his PhD in Industrial- Organizational Psychology at Illinois Institute of Technology. Mr. Porter has experience in entry-level as well as promotional testing processes with a specialty in data/statistical analysis, test development and validation, and job analysis procedures. He has worked with clients on a statewide, municipal, and department level. As a manager, Mr. Porter frequently engages in the following employment testing processes:  JAQ development and analysis  Online survey design & implementation  Adverse impact analysis  Statistical significance testing  Criterion-related validation  Content-validation (CVR implementation, Angoff calculation, SME review sessions)  Full-cycle project management  Physical Ability Test (PAT) development, validation, implementation and analysis  Cut score analysis & defensibility  Test score banding (SED)  Classical test theory item-analysis  Legal issues in personnel selection  Differential item functioning (DIF)  Assessment center development, implementation, and analysis  Structured Oral interview (SOI) development and implementation  Validity Transportation  Individual assessment (i.e. personality and managerial/leadership potential assessment) Education: Illinois Institute of Technology, Chicago Doctor of Philosophy, Industrial-Organizational Psychology (2019 expected) Illinois Institute of Technology, Chicago Master of Science, Industrial-Organizational Psychology (2016) Creighton University, Omaha Bachelor of Science, Psychology, Magna cum Laude (2011) Minor: Business Administration 105 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 84 Presentations & Publications: Tawney, M. W., Jacobson, C. J., Porter, M. G., & Szarek, R. W. (2019, April). Reducing disparity and increasing diversity in law enforcement hiring. Poster presented at the Thirty- Fourth Annual Conference of the Society for Industrial and Organizational Psychology, National Harbor, MD. Porter, M. G., Morris, S. B., & De Corte, W. (2017, April). Pareto-Optimal Composites: Predicted vs. Realized Validity-Diversity Tradeoffs. Symposium presented at Thirty- Second Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL. Porter, M. G., & Morris, S. B. (2015, November). An examination of Pareto-optimality to public safety selection data. Presented at the Ideas in Testing Research seminar, Chicago, IL. Professional Experience: April 2019 to Present Industrial/Organizational Solutions, Inc. (IOS) Manager, Testing Services 2017 to April 2019 Industrial/Organizational Solutions, Inc. (IOS) Consultant/Sr. Researcher, Testing Services 2014 to June 2017 Industrial/Organizational Solutions, Inc. (IOS) Research Analyst, Testing Services 2014 Industrial/Organizational Solutions, Inc. (IOS) Research Associate, Testing Services 2013 – 2014 Illinois Institute of Technology (IIT) Graduate Statistics Teaching Assistant Summary of Selection Projects: Client Project Description Timeframe Anaheim, CA Fire Department Job Analysis and & Validity Transportation process for entry-level firefighter position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Developed custom SOI, provided in-depth incumbent assessor training and oversaw SOI administration 2020 Fort Worth, TX Police Department Job Analysis and & Validity Transportation process for entry-level Patrol Officer position. Managed the administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. 2020 106 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 85 Provided written exam for ~600 candidates and delivered comprehensive adverse impact report of test outcomes. Greenville, SC Police Department Job Analysis and & Validity Transportation process for entry-level Patrol Officer position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Developed custom SOI, provided in-depth incumbent assessor training and oversaw SOI administration. Validated (content & criterion) existing custom physical ability test (PAT) simulation. Worked with SMEs to content validate departmental Medical Health Screening process. 2019 Greenville, SC Fire Department Job Analysis and & Validity Transportation process for entry-level firefighter position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Developed custom SOI, provided in-depth incumbent assessor training and oversaw SOI administration. Validated (content & criterion) existing custom physical ability test (PAT) simulation. Worked with SMEs to content validate departmental Medical Health Screening process. 2019 Washington DC, Fire & EMS Job Analysis and & Validity Transportation process for entry-level firefighter position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Provided written exam for over 3,000 candidates and delivered comprehensive adverse impact report of test outcomes. 2019 Fort Worth, TX Fire Department Job Analysis and & Validity Transportation process for entry-level firefighter position. Managed the administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Provided written exam for ~1,300 candidates and delivered comprehensive adverse impact report of test outcomes. 2019 Indianapolis, IN Police Department Written examination scoring. Full SOI development and administration with department SMEs. 2019 Lakewood, CO Police Department Developed custom SOI, implemented video SJT and administered integrity inventory to entry level candidates. Performed statistical score banding of results and delivered comprehensive adverse impact report. 2016-2020 Edmond, OK Fire Department Developed custom SOI, managed onsite rater training, and video recording of candidate responses. Developed new custom Semi- structured Oral Interview (i.e. Chief’s interview) provided in-depth training to fire chiefs. 2018 Indianapolis, IN Fire Department Managed the administration of a written exam to over 1,000 firefighter candidates. Developed fully custom SOI based on job analytic data and SME input. Provided in-depth incumbent assessor training and oversaw SOI administration to over 800 candidates. 2018 Miami Beach, FL Fire Department Provided multiday on-site candidate orientation sessions. Oversaw administration and scoring of written exam for over 500 firefighter applicants 2018 Providence RI, Police Department Developed fully custom SOI based on job analytic data and SME input. Provided in-depth incumbent and community assessor training. 2018 107 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 86 Bridgeport, CT Police Department Oversaw administration of custom written examination. Developed fully custom SOI based on job analytic data and SME input. Provided in-depth incumbent assessor training and oversaw SOI administration to over 250 candidates. 2018 Indianapolis, IN Police Department Job Analysis and & Validity Transportation process for entry-level police officer position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Developed custom SOI, implemented custom video written exercise, and custom video SJT. 2017-2018 Cobb County, GA Fire Department Job Analysis and & Validity Transportation process for entry-level firefighter position. Delivered online JAQ survey. Analyzed and re- validated previously developed PAT. Provided new cut score recommendation based on analysis of archival incumbent data. 2017-2018 Miami Beach, FL Police Department Developed fully custom entry-level written exercise. Scripted, filmed, and directed the onsite creation of six video scenarios. Developed objective criteria/rating scales based on assessment of candidate recall and written ability. 2017-2020 Edmond, OK Fire Department Job Analysis and & Validity Transportation process for entry-level firefighter position. Delivered online JAQ survey. Administered battery of entry-level exams. Developed custom SOI, managed onsite rater training, and video recording of candidate responses. Developed custom Semi-structured Oral Interview (i.e. Chief’s interview) provided in-depth training to fire chiefs. 2017 New York Metropolitan Transit Authority (MTA) Job Analysis and & Validity Transportation process for entry-level police officer position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. 2017 Bridgeport, CT Fire Department Criterion-related validation of past test data. Situation Judgement content validation. Written examination item-analysis and administration. Custom SOI development and administration 2016 Indianapolis, IN Police Department Criterion-related validity transportation analysis conducted. On-site job analysis visits. Written examination scoring. Full SOI development and administration with department SMEs. 2016-2017 Massachusetts Trial Court, MA Development, validation (content & criterion), field-testing, and administration of custom physical ability test (PAT) simulation 2015-2016 Florida Department of Law Enforcement (FDLE) CJBAT exam cut-score re-analysis and modification. Classical and Differential Item Functioning (DIF) item analyses performed. Adverse impact analysis and significance testing. 2015-2016 Florida Division of State Fire Marshal, Bureau of Fire Standards & Training Conducted statewide job analysis for firefighter position. Included: job analysis interviews, observations, JAQ survey development, administration and analysis. Identified critical KSAOs & Tasks and analyzed survey data across variety of key demographic variables. 2015 New York Civil Service Commission Conducted statewide job analysis for police officer position: Included on-site observations, interviews and JAQ content reviews 2015 New Jersey Civil Service Commission Conducted statewide job analysis for firefighter position including: on-site interviews, observations, JAQ administration, analysis, and written exam field testing. 2015 Bridgeport, CT Police Department Creation, validation, and scoring of custom written exam process. 2015 108 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 87 JOHN M. FORD, PH.D. Principal Consultant Background Statement: Dr. Ford is a Principal Consultant at Industrial/Organizational Solutions, Inc., a personnel and human resources consulting firm that specializes in testing and assessment. Dr. Ford has experience as the principal consultant and project manager for entry-level and promotional assessment products and services, including standardized written tests, job analyses, test validation and transportability studies, oral boards/structured interviews, physical ability tests, promotional written tests, promotional assessment centers, litigation support, performance evaluations, and recruitment surveys. In addition to the development, validation, and administration of assessment processes, Dr. Ford also has extensive experience in providing the following consulting services:  Assisting agencies with preparing RFPs that are specifically targeted to the organizations needs and/or getting sole source approval.  Providing expert consultation for both technically literate and lay client contacts regarding legal requirements and issues (e.g., EEOC guidelines, testing options, test results, validation, adverse impact, cut scores, accommodations for candidates with disabilities, etc.).  Reviewing new legislation and local requirements/civil service guidelines to ensure client compliance.  Educating and working with civil service commissions, human resource agencies, public safety departments, and government officials to ensure the legal defensibility of their selection systems and to gain organizational, political, and public acceptance for their processes.  Meeting with interested parties (e.g., unions and employee groups, community organizations, minority groups, media) to address concerns and gain buy- in for projects.  Conducting statistical analyses (item analyses, scoring, adverse impact, repeat test taker analyses, reliability analyses, statistical validation, transportability analyses, fairness analyses, etc.) and creating eligibility lists.  Analyzing, interpreting, and presenting project results to client representatives, government officials, civil service commissions, and community representatives.  Providing litigation support (e.g., conducting statistical analyses, reviewing reports, participating in strategy meetings with attorneys, preparing supporting materials) for expert witness testimony and court cases. Education: Louisiana State University, Baton Rouge, Louisiana Doctor of Philosophy: Industrial/Organizational Psychology (2002) 109 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 88 California State University, Bakersfield, Bakersfield, California Master of Arts: Psychology (1998) Weber State University Bachelor of Science, Psychology (1996) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2016 to Present Industrial/Organizational Solutions, Inc. Principal Consultant  Consult with public safety agencies to develop, validate, administer, and defend promotional testing/assessment systems 2001 to 2015 CWH Research, Inc. Director of Consulting Services  Management and supervision of a staff that provides entry-level and promotional testing products, services, consultation, and client support to public agencies across the United States and Canada. Summary of Selection Projects: Client Project Description Timeframe Tuscaloosa Fire Rescue Development of assessment center for Fire Marshal. Conduct assessor training and manage assessment center. Manage candidate feedback reports. 2019 San Diego Fire-Rescue Department Management of USERRA assessment center for Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2019 Columbus Fire & Emergency Medical Services Management of written examinations for Sergeant, Lieutenant, and Captain. Development of assessment centers for Sergeant, Lieutenant, Captain, and Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2019 Bexar County Sheriff's Office Management of written examinations for Detention Corporal, Detention Captain, Detention Sergeant, Law Enforcement Entry Test, Law Enforcement Investigator, and Law Enforcement Sergeant. 2019 Albuquerque Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2019 110 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 89 Metropolitan Nashville Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2018-2019 Tuscaloosa Fire & Rescue Service Management of written examinations for Fire Apparatus Operator, Lieutenant, and Captain. Development of practical examination (Aerial Test, Pump Test, Rodeo Test) for Fire Apparatus Operator. Development of assessment centers for Lieutenant and Captain. Development of oral interview for Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage practical examination, assessment center, and oral interview administrations. Manage candidate feedback reports. 2018-2019 Metropolitan Nashville Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant and Lieutenant. Conduct candidate orientation. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2018-2019 Albuquerque Fire Rescue Management of written examinations for Paramedic Driver and Suppression Driver. 2018-2019 Cobb County Fire & Emergency Services Management of written examinations for Engineer and Lieutenant. Development of assessment centers for Engineer and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2018 Bexar County Sheriff's Office Management of written examinations for Detention Corporal, Detention Sergeant, Detention Lieutenant, Law Enforcement Investigator, Law Enforcement Sergeant, Law Enforcement Lieutenant, and Law Enforcement Transfer. 2018 San Diego Fire-Rescue Department Management of written examinations for Captain and Battalion Chief. Development of assessment centers for Captain and Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2017-2018 Tuscaloosa Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2017-2018 Albuquerque Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2017-2018 Albuquerque Fire Rescue Management of written examinations for Paramedic Driver, Suppression Driver, Paramedic Lieutenant, Suppression Lieutenant, Captain, and Battalion Chief. Development of assessment centers for Captain and Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback 2017-2018 111 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 90 reports. Columbus Fire & Emergency Medical Services Management of written examinations for Sergeant, Lieutenant, and Captain. Development of assessment centers for Sergeant, Lieutenant, Captain, and Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2017 Metropolitan Nashville Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2017 Albuquerque Fire Department Development and administration of job analysis for Paramedic Driver, Suppression Driver, Paramedic Lieutenant, Suppression Lieutenant, Captain, and Commander. 2017 Kansas City, Kansas Police Department Management of written examination for Captain. Development of assessment center for Captain. Conduct candidate orientation. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. 2017 Cobb County Fire & Emergency Services Management of written examinations for Engineer and Lieutenant. Development of assessment centers for Engineer and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2017 Tuscaloosa Fire & Rescue Service Management of written examinations for Fire Apparatus Operator, Lieutenant, and Captain. Development of practical examination (Aerial Test, Pump Test, Rodeo Test) for Fire Apparatus Operator. Development of assessment centers for Lieutenant and Captain. Development of oral interview for Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage practical examination, assessment center, and oral interview administrations. Manage candidate feedback reports. 2017 Albuquerque Police Department Management of written examination for Sergeant and Lieutenant. Development of assessment center for Sergeant and Lieutenant. Conduct candidate orientation. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. 2017 Albuquerque Fire Department Management of written examination for Driver. 2017 Bexar County Sheriff's Office Management of written examinations for Law Enforcement Deputy Sheriff, Law Enforcement Transfer Exam, Law Enforcement Lieutenant, Law Enforcement Captain, Detention Corporal, Detention Sergeant, Detention Lieutenant, and Detention Captain. 2017 Bexar County Sheriff's Office Management of written examinations for Law Enforcement Investigator and Sergeant. 2016 Metropolitan Nashville Police Department Management of written examination for Sergeant Development of assessment center for Sergeant. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. 2016 Albuquerque Police Department Management of written examination for Sergeant. Development of assessment center for Sergeant. Conduct candidate 2016 112 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 91 orientation. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. Albuquerque Fire Department Management of written examinations for Driver, Lieutenant, Captain, and Commander. Development of assessment centers for Captain and Commander. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2016 University of Alabama Police Department Management of written examinations for Sergeant, Lieutenant, and Captain. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2016 Cobb County Fire & Emergency Services Management of written examinations for Engineer and Lieutenant. Development of assessment centers for Engineer and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2016 San Diego Fire-Rescue Department Management of written examinations for Captain and Battalion Chief. Development of assessment centers for Captain and Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2016 Kansas City, Kansas Police Department Management of written examinations for Sergeant, Detective, and Captain. Development of assessment centers for Sergeant, Detective, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2015-2016 Tuscaloosa Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2015-2016 Columbus Fire & Emergency Medical Services Management of written examinations for Sergeant, Lieutenant, and Captain. Development of assessment centers for Sergeant, Lieutenant, Captain, and Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2015 Metropolitan Nashville Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2015 Cobb County Fire & Emergency Services Development of structured interview for Captain. Conduct assessor training and manage structured interview administration. Manage candidate feedback reports. 2015 Cobb County Fire & Emergency Services Management of written examinations for Engineer and Lieutenant. Development of assessment centers for Engineer and Lieutenant. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2015 Tuscaloosa Fire & Rescue Service Management of written examinations for Fire Apparatus Operator, Lieutenant, and Captain. Development of practical examination (Aerial Test, Pump Test, Rodeo Test) for Fire 2014-2015 113 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 92 Apparatus Operator. Development of assessment centers for Lieutenant and Captain. Development of oral interview for Battalion Chief. Conduct candidate orientations. Conduct assessor training and manage practical examination, assessment center, and oral interview administrations. Manage candidate feedback reports. Fairfax County Police Department Development and validation of Physical Ability Test for entry- level Police Officer and entry-level Animal Control Officer positions. 2010-2015 University of Alabama Police Department Management of written examinations for Sergeant, Lieutenant, and Captain. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2014 Metropolitan Nashville Police Department Management of written examination for Sergeant. Development of assessment center for Sergeant. Conduct candidate orientation. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. 2014 Tuscaloosa Fire & Rescue Service Development of oral interview for EMS Chief. Conduct assessor training and manage oral interview administration. Manage candidate feedback reports. 2014 Casa Grande Police Department Management of written examination for Sergeant. Development of assessment center for Sergeant. Conduct candidate orientation. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. 2014 Charleston Fire Department Scoring, data analysis, and evaluating structured oral interview for entry-level Firefighter. 2014 Charleston Fire Department Development, validation, and scoring of structured oral interview for entry-level Firefighter. 2014 Rhode Island State Police Department Conduct assessor training for structured oral interview for entry-level Trooper position. 2014 Rockford Police Department Management of written examination for Sergeant. Development of assessment center for Lieutenant Conduct candidate orientation. Conduct assessor training and manage assessment center administration. Manage candidate feedback reports. 2014 Albuquerque Fire Department Conduct orientation for Captain assessment center. 2014 Arvada Fire Protection District Development, validation, and scoring of oral interview for entry- level Firefighter. 2014 Tuscaloosa Police Department Management of written examinations for Sergeant and Lieutenant. Development of assessment centers for Sergeant, Lieutenant, and Captain. Conduct candidate orientations. Conduct assessor training and manage assessment center administrations. Manage candidate feedback reports. 2013-2014 Presentations:  Ford, J.M., Hornick, C.W., & Fox, K. (2014, July). Adverse Impact: A Primer on What You Need to Know. Workshop given at the Thirty-Eighth Annual Conference of the International Personnel Assessment Council, Denver, CO.  Ford, J.M., Hornick, C.W., & Fox, K. (2014, July). Public Safety Hiring: A Tutorial on Improving Your Selection Process From the RFP to the First Day of the Academy. Presentation given at the Thirty-Eighth Annual Conference of the International Personnel Assessment Council, Denver, CO. 114 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 93  Igou, F., Feuquay, J., Blair, M., Hornick, C.W., & Ford, J. M. (2011, July). To band or not to band: Is that the question? Presentation given at the Thirty-fifth Annual Conference of the International Personnel Assessment Council, Washington D.C.  Ford, J.M., Hornick, C.W., & Carnahan, T.J. (2008, June). Bridging the bureaucratic morass: Successfully navigating common obstacles to successful assessment and hiring in the public sector. Presentation given at the Thirty-Second Annual Conference of the International Public Management Association Assessment Council, Oakland, CA.  Ford, J.M., & Blair, M.D. (2008, June). It all begins with the job analysis: Using job analyses to identify talent and drive organizational success. Workshop given at the Thirty-Second Annual Conference of the International Public Management Association Assessment Council, Oakland, CA.  Carnahan, T.J., Hornick, C.W., & Ford, J.F. (2008, June). The first shall be last and the last shall be first: Does presentation order and scheduling impact candidate scores in assessment centers? Presentation given at the Thirty-Second Annual Conference of the International Public Management Association Assessment Council, Oakland, CA.  Ford, J.M. (2007, June). Moving beyond “eeny, meeny, miny, moe”: What factors should guide the evaluation of selection tests. Presentation given at the Thirty-first Annual Conference of the International Public Management Association Assessment Council, St. Louis, MO.  Blair, M.D., & Ford, J.M. (2007, June). Adverse Impact: Pitfalls, pollyanna, and practical advice for practitioners. Workshop given at the Thirty-first Annual Conference of the International Public Management Association Assessment Council, St. Louis, Mo.  Blair, M.D., & Ford, J.M. (2006). It all begins with the job analysis: Expanding the job domain to minimize adverse impact in selection. PTC Quarterly, 2, 10-11.  Ford, J.M., & Blair, M.D. (September, 2006). Adverse impact: Misconceptions, facades, and pitfalls. What practitioners need to know today. Presentation given at the Western Regional Intergovernmental Personnel Assessment Council 2006 Meeting, Berkeley, CA.  Ford, J.M., & Blair, M.D. (2006, June). How do you rate what you don’t know: The impact of familiarity between raters and ratees on performance evaluations. Presentation given at the Thirtieth Annual Conference of the International Public Management Association for Human Resources Assessment Council, Las Vegas, NV.  Ford, J.M., & Blair, M.D. (2006, June). Approaches to addressing adverse impact: Opportunities, facades, and pitfalls. Presentation given at the Thirtieth Annual Conference of the International Public Management Association for Human Resources Assessment Council, Las Vegas, NV.  Blair, M.D., & Ford, J.M. (2006, June). Factors influencing applicant performance when retaking employment exams. Presentation given at the Thirtieth Annual Conference of the International Public Management Association for Human Resources Assessment Council, Las Vegas, NV.  Blair, M.D., & Ford, J.M. (2006, June). The next generation in firefighter selection: A new model for a new generation. Presentation given at the Thirtieth Annual Conference of the International Public Management Association for Human Resources Assessment Council, Las Vegas, NV.  Ford, J.M., Hornick, C.W., & Killian, J.H. (2005, June). Do you see what I see?: Are different rater sources evaluating the same thing? Presentation given at the Twenty-ninth Annual 115 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 94 Conference of the International Public Management Association for Human Resources Assessment Council, Orlando, FL.  Ford, J.M., & Greguras, G.J. (2003, April). Examining the influence of organizational politics on multisource feedback systems. Paper presented at the Eighteenth Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.  Ford, J.M., & Greguras, G. J. (2002, April). Examining the influence of supervisor- subordinate agreement on LMX relationships. Paper presented at the Seventeenth Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Canada.  Greguras, G. J., Ford, J., & Brutus, S. (2001, May). Manager attention to and use of multi- source feedback. In M. P. Born & M. E. G. van der Velde (Co-chairs), Managerial behavior and assessment. Symposium conducted at the Tenth Annual Conference of the European Congress on Work and Organizational Psychology, Prague, Czech Republic.  Ford, J. M. & Greguras, G. J. (2001, April). An examination of the multidimensionality of supervisor perceptions of leader-member exchange. Paper presented at the Sixteenth Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.  Greguras, G. J., Ford, J. M., & Brutus, S. (2000, April). Which rater source matters most? A policy capturing study. Paper presented at the Fifteenth Annual Conference of the Society for Industrial and Organizational Psychology, New Orleans, LA. Publications: Greguras, G.J., & Ford, J.M. (2006). An examination of the multidimensionality of supervisor and subordinate perceptions of leader-member exchange. Journal of Occupational and Organizational Psychology, 79, 433-465. Greguras, G. J., Ford, J.M., & Brutus, S. (2003). Manager attention to multisource feedback. Journal of Management Development, 22, 345-361. 116 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 95 Irfan Bhanji, M.A. Senior Assessment Consultant Background Statement: Irfan joined IOS in 2012 as an associate consultant. He has managed all phases of promotional selection processes for agencies across the country. He delivers excellence to clients in terms of customer service, professionalism, expertise and overall client satisfaction. Education: Elmhurst College M.A. in Industrial/Organizational Psychology (2011) Northern Illinois University B.S. in Psychology (2009) Affiliations: Society for Industrial/Organizational Psychology (SIOP) The Chicago Industrial/Organizational Psychologists (CIOP) Professional Experience: 2019-Present Senior Assessment Consultant, I/O Solutions, Inc.  Manages consulting projects, including, determination of project deliverables, scheduling, communications, client relations, and ensuring the technical quality of all work to be completed  Develops and administers assessment centers and assessor training  Conducts subject matter expert meetings to review test content and items 2014-Present Assessment Consultant, I/O Solutions, Inc.  Manages consulting projects, including, determination of project deliverables, scheduling, communications, client relations, and ensuring the technical quality of all work to be completed  Develops and administers assessment centers and assessor training  Conducts subject matter expert meetings to review test content and items 117 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 96 2012-2014 Associate Consultant, I/O Solutions, Inc.  Manages and assists project managers on consulting projects, including, determination of project deliverables, scheduling, communications, client relations, and ensuring the technical quality of all work to be completed  Develops and administers assessment centers and assessor training  Conducts subject matter expert meetings to review test content and items 2010-2010 HR Intern – YWCA of Metropolitan Chicago  Developed, administered, and analyzed annual employee survey  Facilitated focus groups to gather employee feedback on the organizational environment of the agency  Presented survey results and action plan to senior management team  Communicated with a variety of departments and stakeholders at agency Summary of Selection Projects: Client Project Description Timeframe Baltimore City Fire Department Development of Job Knowledge Assessment Center for the rank of Suppression Captain and Lieutenant 2019 Baltimore City Fire Department Development of Job Knowledge Assessment Center for the rank of EMS Captain and Lieutenant 2019 Kansas City, Kansas PD Developed written exam and assessment center for the rank of Police Sergeant and Detective 2019 Kansas City, Kansas FD Developed written exam and assessment center for the rank of Fire Captain 2019 Kansas City, Kansas FD Conducted Job-Analysis for the ranks of Fire Captain and Battalion Chief 2019 Guilford Sheriff's Office Developed assessment center for the rank of Sworn Sergeant, Sworn Lieutenant, Detention Sergeant and Detention Lieutenant 2019 Cincinnati, OH Developed assessment center for the rank of Police Captain 2019 Cincinnati, OH Developed written exam and assessment center for the rank of Fire District Chief 2019 Cincinnati, OH Developed assessment center for the rank of Fire Captain 2018 Richfield, MN Developed assessment center for the rank of Police Sergeant & Lieutenant 2018 Guilford Sheriff's Office Developed assessment center for the rank of Sworn Sergeant, Sworn Lieutenant, Detention Sergeant and Detention Lieutenant 2018 118 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 97 August, GA Fire Department Job knowledge written examination and assessment center development for the rank of Sergeant, Lieutenant, and Captain 2018 Kansas City, Kansas PD Conducted Job-Analysis for the ranks of Police Sergeant, Detective and Captain 2018 Indianapolis, IN Conducted Job-Analysis for the ranks of Police Sergeant, Lieutenant, and Captain 2017 Omaha, NE Job knowledge written examination and assessment center development for the rank of Police Sergeant. 2017 Baltimore City Fire Department Development of Job Knowledge Assessment Center for the rank of EMS Captain and Lieutenant 2017 Baltimore City Fire Department Development of Job Knowledge Assessment Center for the rank of Suppression Captain and Lieutenant 2017 Cincinnati, OH Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Fire Lieutenant 2017 Baltimore City Fire Department Development of Job Knowledge Written Exam & Practicum for Emergency Vehicle Driver and Pump Operator 2016 St. Charles, MO Development of Job Knowledge Written Exam & Assessment Center for the rank of Police Sergeant 2016 Baltimore City Fire Department Development of Job Knowledge Assessment Center for the rank of Suppression Battalion Chief 2016 Omaha, NE Development of Assessment Center for the rank of Fire Chief 2016 Northbrook, IL Job knowledge written examination and assessment center development for the rank of Fire Captain. 2016 Cobb County, GA Conducted Job-Analysis and developed assessment center for the rank of Battalion Chief and Deputy Chief 2016 Omaha, NE Job knowledge written examination and assessment center development for the rank of Police Lieutenant. 2016 Montgomery Sheriff Job knowledge written examination and assessment center development for the rank Sergeant and Lieutenant. 2016 Elmhurst Fire Department Job knowledge written examination and assessment center development for the rank of Fire Lieutenant and Battalion Chief. 2016 IL State University Civil Service Oral Interview development for the rank of Police Officer. 2015 IL State University Civil Service Assessment center development for the rank of Police Sergeant. 2015 Prince William County Department of Fire & Rescue Job knowledge written examination and assessment center development for the ranks of Lieutenant, Captain, and Technician II. 2015 Baltimore City Fire Department Conducted Job-Analysis and developed Job knowledge written examination for the rank of EMS Lieutenant 2015 119 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 98 Baltimore City Fire Department Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of EMS Captain. 2015 Baltimore City Fire Department Conducted Job-Analysis and developed Job knowledge written examination for the rank of Suppression Captain and Lieutenant. 2015 Baltimore City Fire Department Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of EMS Battalion Chief. 2015 Baltimore City Fire Department Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Sergeant and Lieutenant. 2015 Omaha, NE Physical Ability Test (PAT) development and validation for the rank of Police Officer. 2015 Omaha, NE Job knowledge written examination and assessment center development for the rank of Fire Apparatus Engineer. 2015 Omaha, NE Job knowledge written examination and assessment center development for the rank of Police Sergeant. 2015 Omaha, NE Assessment center development for the rank of Deputy Police Chief. 2015 Woodridge, IL PD Job knowledge written examination and assessment center development for the rank of Sergeant. 2015 Cedar Rapids, PD Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Sergeant and Lieutenant. 2015 Prince William County Department of Fire & Rescue Job knowledge written examination and assessment center development for the ranks of Lieutenant, Captain, and Technician II. 2014 St. Louis PD Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Lieutenant and Captain. 2014 Omaha, NE Assessment center development for the rank of Fire Chief. 2014 Omaha, NE Job knowledge written examination and assessment center development for the rank of Fire Battalion. 2014 Omaha, NE Job knowledge written examination and assessment center development for the ranks of Police Lieutenant. 2014 Omaha, NE Assessment center development for the rank Paramedic Supervisor. 2014 Hoffman Estates, IL PD Job knowledge written examination and assessment center development for the ranks of Sergeant and Lieutenant 2014 Westmont, IL PD Job knowledge written examination and assessment center development for the rank of Sergeant. 2014 Cook County Forest Preserve PD Job knowledge written examination and assessment center development for the rank of Sergeant and Deputy Commander. 2014 120 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 99 Darien, IL PD Job knowledge written examination and assessment center development for the rank of Sergeant. 2014 Prince William County Department of Fire & Rescue Job knowledge written examination and assessment center development for the ranks of Lieutenant, Captain, and Technician II. 2013 Omaha, NE Job knowledge written examination and assessment center development for the rank of Police Sergeant. 2013 Omaha, NE Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Assistant Fire Marshal. 2013 Omaha, NE Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Paramedic Supervisor. 2013 Omaha, NE Job knowledge written examination and assessment center development for the rank of Fire Apparatus Engineer. 2013 Prince William County Department of Fire & Rescue Job knowledge written examination and assessment center development for the ranks of Lieutenant, Captain, and Technician II. 2012 Omaha, NE Conducted Job-Analysis and developed Job knowledge written examination and assessment center for the rank of Assistant Fire Chief. 2012 Darien, IL PD Job knowledge written examination and assessment center development for the rank of Sergeant. 2012 121 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 100 Linda H. Reynaud, M.A. Senior Assessment Consultant Ms. Reynaud is a consultant at Industrial/Organizational Solutions, Inc., a personnel and human resources consulting firm that specializes in testing and assessment. Ms. Reynaud has a variety of experiences in the private and public sectors in developing, validating, and administering selection processes. Ms. Reynaud also has extensive experience in job analyses and technical report writing. EDUCATION George Mason University M.A. Industrial/Organizational Psychology, 1988 James Madison University B.S. Psychology (cum laude, with distinction), 1987 Professional Experience 2012-Present Assessment Consultant, I/O Solutions, Inc.  Consult with public safety agencies to design, develop and validate promotional testing/assessment systems  Conduct job analysis to identify the essential knowledge, skills, and abilities and other characteristics necessary for performance  Write technical reports for assessment processes, documenting all aspects of the development, administration, results and validation evidence to support assessments 2000-2011 Consultant, e-Selex.com  Served as project manager, designing and developing online selection systems for large private industry clients  Conducted validation analyses and authored validation reports  Wrote proposals, technical specifications documents, and technical reports 1997-2000 Consultant, SHL, Inc.  Consulted with Fortune 500 clients to develop and validate selection tools  Managed the design and development of online hiring systems  Completed large scale job analyses  Wrote technical reports to document projects 1994-1997 Testing and Assessment Analyst, Circuit City Stores, Inc.  Developed and implemented assessment centers for store management positions  Developed Competency Models, utilized for recruitment, hiring, promotion and performance management systems 122 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 101  Created a 360 degree feedback instrument, training, and policies  Designed interview guides for store positions  Completed test validation analyses and wrote validation reports  Developed standard operating procedures for store selection  Led cross-functional project to improve sales associate productivity and customer service 1992-1997 Adjunct Instructor, University of Richmond 1993-1994 Organizational & HR Development Manager, Virginia Department of Rehabilitative Services  Facilitated organizational development programs  Led cross-functional team in the development of success factors for agency staff  Conducted training needs assessment and developed agency’s annual HR development plan  Authored training proposal to obtain federal in-service training grant funds; managed grant funds 1990-1993 Selection & Assessment Specialist, Altria  Developed and administered assessment centers for sales manager position and middle manager positions  Developed 360 feedback instrument and process  Designed and implemented assessment processes for various purposes (e.g., placement in re-organization, associate development, training needs analysis)  Participated in the design of a performance appraisal system  Participated in the development of a Management Competency Model  Conducted job analyses and validation studies on selection instruments  Authored panel interview guides and test administration manuals 1988-1990 Research Associate, Human Resources Research Organization (HumRRO)  Served as research analyst under lead researchers  Authored technical reports, prepared communication materials, conducted job analysis interviews 1988 Research Associate (Intern), Morris & McDaniel, Inc  Wrote items for police, security and fire job knowledge exams 123 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 102 Summary of IOS Selection Projects: Client Project Description Timeframe Austin-Travis County EMS Job knowledge written examination development for the rank of Clinical Specialist-Field and Clinical Specialist-Communications. Job knowledge written examination development and assessment center development and administration for the rank of Captain- Communications and Captain-Field. 2019 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Commander, Captain and Engineer in the Bernalillo Fire Department. 2019 Chattanooga Fire Department Job analyses and job knowledge written examination development and assessment center development and administration for the ranks of Lieutenant, Captain and Battalion Chief. 2019 Coral Gables Fire Department Job knowledge written examination development for the rank of Captain in Coral Gables Fire Department. 2019 Jacksonville Sheriff's Office - Corrections Job knowledge written examination development and assessment center development and administration for the ranks of Corrections Sergeant and Lieutenant in the Jacksonville Sheriff's Office. 2019 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Battalion Chief in the Los Alamos Fire Department. 2019 Austin-Travis County EMS Job knowledge written examination development for the ranks of Medic II Communications, Medic II Field and Commander Communications. Job knowledge written examination development and assessment center development and administration for the ranks of Commander Communications, Captain Field and Captain Communications. 2018 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Commander and Lieutenant in the Bernalillo Fire Department. 2018 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, Battalion Chief and Assistant Fire Chief in the Carrollton Fire Department. 2018 Chattanooga Police Department Assessment center development and administration for the rank of Captain in the Chattanooga Police Department. 2018 Coral Gables Fire Department Job knowledge written examination development for the ranks of Lieutenant and Commander in Coral Gables Fire Department. Coordinated entry-level testing. 2018 Jacksonville Sheriff's Office Job knowledge written examination development and assessment center development and administration for the ranks Police Sergeant and Lieutenant in the Jacksonville Sheriff's Office. Job analyses for the ranks of Corrections Sergeant and Lieutenant in the Jacksonville Sheriff's Department. 2018 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Battalion Chief in the Los Alamos Fire Department. 2018 124 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 103 Los Alamos Police Department Job knowledge written examination development and assessment center development and administration for the rank of Sergeant in the Los Alamos Police Department. 2018 Austin-Travis County EMS Job knowledge written examination development for the ranks of Medic II Communications, Medic II Field, and Captain Field. Structured Oral Interview development for rank of Medic I. 2017 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Commander and Captain in the Bernalillo Fire Department. Practical assessment development and administration for the rank of Engineer. 2017 Bernalillo County Metropolitan Detention Center Job knowledge written examination development and assessment center development and administration for the ranks of Sergeant and Lieutenant. 2017 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, and Battalion Chief in the Carrollton Fire Department. 2017 Chattanooga Fire Department Assessment center development and administration for the ranks of Lieutenant and captain in the Chattanooga Fire Department. 2017 Coral Gables Fire Department Job knowledge written examination development for the rank of Captain in Coral Gables Fire Department. Coordinated entry-level testing. 2017 Jacksonville Sheriff's Office Assessment center development for military make-up assessments for the ranks of Police Sergeant and Lieutenant in the Jacksonville Sheriff's Office. 2017 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Battalion Chief in the Los Alamos Fire Department. . 2017 Austin-Travis County EMS Job knowledge written examination development and assessment center administration and development for the ranks of Captain Field, Commander Field, Captain Communications, and Commander Communications. Job knowledge written examination development for the ranks of Medic II Communications and Medic I Field. Structured Oral Interview development for rank of Medic I. 2016 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Commander and Lieutenant in the Bernalillo Fire Department. 2016 Bernalillo County Metropolitan Detention Center Job knowledge written examination development and assessment center development and administration for the ranks of Sergeant and Lieutenant. 2016 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, and Battalion Chief in the Carrollton Fire Department. 2016 Coral Gables Fire Department Job knowledge written examination development for the ranks of Lieutenant and Battalion Chief in Coral Gables Fire Department. 2016 125 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 104 Jacksonville Sheriff's Office Assessment center development and administration for the ranks of Sergeant and Lieutenant in the Jacksonville Sheriff's Office. 2016 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Battalion Chief in the Los Alamos Fire Department. Entry level firefighter testing coordination. 2016 Los Alamos Police Department Job knowledge written examination development and assessment center development and administration for the rank of Sergeant in the Los Alamos Police Department. 2016 Savannah Police Department Job knowledge written examination development and assessment center development and administration for the rank of Sergeant in the Savannah Police Department. 2016 Austin Fire Department Job knowledge written examination development for the ranks of Fire Specialist, Lieutenant, Captain, and Battalion Chief in the Austin Fire Department. 2015 Austin-Travis County EMS Job knowledge written examination development for the ranks of Medic II Communications, Captain Communications, Commander Communications, and Medic II Field. Structured Oral Interview development for Medic I. 2015 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Captain and Engineer in the Bernalillo Fire Department. 2015 Bernalillo County Metropolitan Detention Center Job knowledge written examination development and assessment center development and administration for the ranks of Sergeant and Lieutenant. 2015 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, and Battalion Chief in the Carrollton Fire Department. 2015 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Battalion Chief in the Los Alamos Fire Department. 2015 Montgomery County Department of Corrections and Rehabilitation Job knowledge written examination development and assessment center development and administration for the ranks of Sergeant and Lieutenant. 2015 Edmond, OK Structured oral interview development for the rank of Chief - Fire Prevention. 2015 Massachusetts Trial Court Written exam, situation judgement test and essay exam development for the ranks of ACPO and CPO in Probation and Family and Juvenile Courts. 2015 Austin Fire Department Job knowledge written examination development for the ranks of Fire Specialist, Lieutenant, Captain, Battalion Chief and Division Chief in the Austin Fire Department. Job analysis for rank of Division Chief. 2014 126 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 105 Austin-Travis County EMS Job knowledge written examination development for the ranks of Medic II Communications, Captain Communications, Commander Communications, Medic II Field, Captain Field, and Commander Field. Assessment center development and administration for Captain Field and Commander Field in the Austin-Travis County EMS department. Structured Oral Interview development for entry- level Medic. 2014 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Captain and Commander in the Bernalillo Fire Department. 2014 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, and Battalion Chief in the Carrollton Fire Department. 2014 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Battalion Chief in the Los Alamos Fire Department. Completed entry-level firefighter testing. 2014 Los Alamos Police Department Job knowledge written examination development and assessment center development and administration for the rank of Sergeant. 2014 Lafayette (LA) Sheriff's Office Job knowledge written examination development for first-line supervisors in the sheriff’s department. 2013 Austin Fire Department Job knowledge written examination development for the ranks of Fire Specialist, Lieutenant, Captain, and Battalion Chief in the Austin Fire Department. 2013 Austin-Travis County EMS Job analyses and job knowledge written examination development for the ranks of Medic II Communications, Captain Communications, Commander Communications, Medic II Field, Captain Field, Commander Field and Division Chief. Job analyses and entry-level examination development for the ranks of Medic I Communications and Medic I Field. 2013 Bernalillo County Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Captain and Commander in the Bernalillo Fire Department. 2013 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, and Battalion Chief in the Carrollton Fire Department. 2013 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Captain and Battalion Chief in the Los Alamos Fire Department. 2013 Carrollton Fire Department Job knowledge written examination development for the ranks of Apparatus Operator, Sub Apparatus Operator, Captain, and Battalion Chief in the Carrollton Fire Department. 2012 127 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 106 Los Alamos Fire Department Job knowledge written examination development and assessment center development and administration for the ranks of Driver Engineer, Captain, and Deputy Chief in the Los Alamos Fire Department. Completed entry-level firefighter testing. 2012 Odessa Fire Department Job knowledge written examination development for the ranks of Field Training Officer, Detective, Lieutenant, Sergeant, Corporal and Police Officer. 2012 Lafayette (LA) Sheriff's Office Job knowledge written examination development for the rank of Dispatch Supervisor in the sheriff’s department. 2012 128 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 107 Andrew Weedfall, Ph.D. Assessment Consultant Background Statement: Dr. Weedfall is a consultant at Industrial/Organizational Solutions, Inc., a personnel and human resources consulting firm that specializes in testing and assessment. Dr. Weedfall has experience with job analysis, the development, validation, and administration of job knowledge examinations and assessment centers, and project management. Education: North Carolina State University – Raleigh, NC Doctor of Philosophy: Industrial/Organizational Psychology (2019) Dissertation: Individual Differences in Perceptions of Cyber Incivility: Mitigating the Negative Effects of Incivility with Job Crafting North Carolina State University – Raleigh, NC Master of Science: Industrial/Organizational Psychology (2016) West Virginia University – Morgantown, WV Bachelor of Arts, Psychology (2013) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2019 to Present Industrial/Organizational Solutions, Inc. Consultant  Consults with public safety agencies to develop, validate, and defend promotional testing and assessment systems  Conducts job analysis (interviews and observations) to identify the essential knowledge, skills, abilities, and other characteristics necessary for performance  Provides best-practice training to public safety agencies, human resources, and civil service personnel  Develops technical reports for processes by documenting all aspects of the process and validation evidence to support assessments 129 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 108 2014 to 2018 The Friday Institute for Educational Innovation Graduate Research Assistant  Planned, conducted, and delivered reports of, statistical tests and quantitative data analysis  Leveraged data analysis expertise to provide recommendations to clients regarding the best approach to assess and evaluate their programs.  Designed reports to meet client needs by presenting data and results through non-technical language and effective visuals to inform the future directions of their programs.  Used psychometrics expertise to advise on and conduct a scale validation study and co-authored a published manuscript  Independently managed and met goals and expectations for concurrent projects. Summary of Selection Projects: Client Project Description Timeframe Grand Rapids Police Department - Grand Rapids, MI Assisted with development and administration of the assessment center for the ranks of Sergeant, Lieutenant, and Captain 2019-2020 Kalamazoo Department of Public Safety - Kalamazoo, MI Developed written examination for the rank of Detective 2019-2020 Pasadena Police Department - Pasadena, TX Developed written examinations for the rank of Sergeant and Lieutenant 2019-2020 Buffalo Fire Department - Buffalo, NY Assisted with the development and administration of the assessment center for the rank of Captain 2019 Bernalillo County Fire Department - Bernalillo County, NM Assisted with implementation of the assessment center for the rank of Engineer 2019 Guilford County Sheriff's Office - Guilford County, NC Assisted with the development and administration of the assessment center for the ranks of Sworn Sergeant and Sworn Lieutenant 2019 Kalamazoo Department of Public Safety - Kalamazoo, MI Developed written examination and wrote a technical report for the rank of Sergeant 2019 Augusta Fire Department - Augusta, GA Assisted with implementation of the assessment center for the ranks of Sergeant and Lieutenant 2019 Raleigh Police Department - Raleigh, NC Assisted with implementation of the assessment center for the rank of Detective 2019 Raleigh Police Department - Raleigh, NC Assisted with implementation of the assessment center for the ranks of Sergeant and Lieutenant 2019 Austin Police Department - Austin, TX Assisted with implementation of the assessment center for the rank of Lieutenant 2019 Baltimore Police Department - Baltimore, MD Assisted with implementation of the assessment center for the rank of Sergeant 2019 130 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 109 Springfield Fire Department - Springfield, MO Conducted job analysis interviews and observations for the rank of Firefighter 2019 Cincinnati Fire Department - Cincinnati, OH Assisted with implementation of the assessment center for the rank of District Chief 2019 Tulsa Police Department - Tulsa, OK Assisted with implementation of the assessment center for the rank of Captain 2019 Publications and Presentations:  Jones, M. G., Ennes, M., Weedfall, A., Chesnutt, K., Cayton, E. (In Press). The development and validation of a measure of science capital, habitus, and future science interests. Research in Science Education.  Oakley, S. L., Weedfall, A., Pond III, S. B., (April, 2016). Linking Core Self-Evaluations to Organizational Citizenship: A Risk Orientation Perspective. Poster presented at the 31st Annual Conference for the Society for Industrial and Organizational Psychology, Anaheim, CA.  Oakley, S. L., Weedfall, A., Pond III, S. B., (April, 2016). Enhancing Attitudes Toward Diversity Training: Framing, Format, and Goal Orientation. Poster presented at the 31st Annual Conference for the Society for Industrial and Organizational Psychology, Anaheim, CA. 131 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 110 Christen Dovalina, M.S. Assessment Consultant Background Statement: Ms. Dovalina joined IOS in 2015. She is familiar with best practices in selection test development and validation, including job analysis, test plan development, criterion-related validity analysis, adverse impact analysis, and utility analysis. Education: Texas A&M University M.S. in Industrial/Organizational Psychology (2014) Bradley University MBA – Management Concentration (2010) Bradley University B.S. in Psychology (2006) Affiliations: Society for Industrial/Organizational Psychology (SIOP) American Psychological Association (APA) Professional Experience: 2018 to Present Assessment Consultant, I/O Solutions, Inc.  Consults with public safety agencies to develop, validate, and defend entry-level and promotional testing/assessment systems  Independently and simultaneously manages multiple assessment development and validation projects  Conducts job analysis (interviews and observations) to identify the essential knowledge, skills, abilities, and other characteristics necessary for performance  Provides best-practice training to public safety agencies, human resources and civil service personnel  Develops technical reports for processes, documenting all aspects of the process and validation evidence to support assessments 2015 to 2018 Assessment Specialist, I/O Solutions, Inc.  Managed and assisted project managers on consulting projects, including the development, validation, and defense of entry-level and promotional testing/assessment systems  Developed and administered assessments centers, including 132 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 111 assessor training  Conducted subject matter expert meetings to review and validate exam content  Developed technical reports for processes, documenting all aspects of the process and validation evidence to support assessments Summary of Selection Projects: Client Project Description Timeframe Massachusetts State Police Department Developed Sergeant written examination (MMU). 2019-2020 Arlington, TX Fire Department Developed written examinations for the ranks of Apparatus Operator, Captain, and Lieutenant. 2019-2020 Austin, TX Police Department Developed Detective/Corporal written examination and wrote written exam report. 2019-2020 DFW International Airport Police Services Developed Sergeant and Lieutenant written examinations and specialty SOI questions 2019 Round Rock, TX Fire Department Developed Captain and Battalion Chief written examination. 2019 Montgomery County, MD Department of Correction and Rehabilitation Developed Sergeant and Lieutenant written examinations. Developed and administered Sergeant and Lieutenant assessment centers. 2019 Gwinnett County, GA Police Department Developed and administered Communications Officer III, Communications Officer IV, and Communications Shift Supervisor assessment centers. 2019 Moline, IL Fire Department Developed and administered Captain assessment center. 2019 Chattanooga, TN Police Department Developed Sergeant and Lieutenant written examinations and assessment centers. 2019 Gwinnett County, GA Sheriff's Office Developed Corporal, Sergeant, and Lieutenant written examinations. Developed and administered Lieutenant assessment center. 2019 Gwinnett County, GA Corrections Department Developed Corporal, Sergeant, and Lieutenant written examinations. Developed and administered Sergeant and Lieutenant assessment center. 2019 Austin, TX Police Department Developed Lieutenant written examination. 2019 Jacksonville Sheriff's Office Police Department Assisted with development of Sergeant and Lieutenant assessment center exercises and assessment center administration. 2019 Jacksonville Sheriff's Office Corrections Department Assisted with development of Sergeant and Lieutenant assessment center exercises and assessment center administration. 2019 133 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 112 Massachusetts State Police Department Developed Sergeant written examination. 2019 Austin-Travis County, TX EMS Assisted with development of Commander Field assessment center. Administered Captain Communications assessment center. 2019 Gwinnett County, GA Police Department Developed and administered Corporal, Sergeant, and Lieutenant written examinations. Developed and Administered Sergeant and Lieutenant assessment center. 2019 Arlington, TX Fire Department Developed written examinations for the ranks of Fire Prevention Specialist, Deputy Fire Marshal, Fire Investigator Bomb Technician, Apparatus Operator, Captain, and Lieutenant. 2018 - 2019 Austin, TX Police Department Assisted with development and administration of Sergeant and Commander written examinations and assessment centers. 2018-2019 Moline, IL Police Department Developed and administered Sergeant and Lieutenant written examinations and Lieutenant assessment center. 2018 Plano, IL Police Department Administered Sergeant assessment center. 2018 Moline, IL Fire Department Developed Lieutenant written examination and developed and administered assessment center. 2018 Gwinnett County, GA Police Department Developed and administered Communications Officer III, Communications Officer IV, and Communications Shift Supervisor assessment centers. 2018 Montgomery County, MD Fire and Rescue Service Administered Lieutenant assessment center. 2018 Montgomery County, MD Sheriff's Office Administered Sergeant and Lieutenant structured oral interviews. 2018 Jacksonville Sheriff's Office Police Department Developed Sergeant and Lieutenant assessment center exercises and assisted with assessment center administration. 2018 Jacksonville Sheriff's Office Corrections Department Conducted job analysis for the ranks of Sergeant and Lieutenant. 2018 Chattanooga, TN Police Department Administered Captain assessment center. 2018 Illinois State Police Administered Master Sergeant assessment center. 2018 Austin-Travis County, TX EMS Administered Captain Communications assessment center. 2018 Plano, TX Fire Department Developed written examinations for the ranks of Deputy Chief, Lieutenant, Fire Engineer, and Captain. 2018 Montgomery County, MD Department of Correction and Rehabilitation Administered Lieutenant structured oral assessment. 2018 Arvada, CO Fire Department Administered Paramedic assessment center. 2018 134 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 113 Gwinnett County, GA Sheriff's Office Developed Corporal, Sergeant, and Lieutenant written examinations. 2018 Gwinnett County, GA Corrections Department Developed Corporal, Sergeant, and Lieutenant written examinations. Developed and administered Sergeant and Lieutenant assessment center. 2018 DFW International Airport Police Services Developed Sergeant and Lieutenant written examinations and specialty SOI questions, conducted assessor/administrator training, and wrote technical report. 2017 - 2018 Moline, IL Fire Department Developed and administered Battalion Chief assessment center. 2017 - 2018 Plano, TX Fire Department Developed Captain written examination. 2017 - 2018 Austin, TX Police Department Developed Detective/Corporal written examination and wrote written exam report. 2017 Houston, TX Police Department Developed Captain and Sergeant make-up assessment center exercises. 2017 Dallas, TX Police Department Developed Sergeant and Lieutenant assessment center exercises and wrote Lieutenant technical report. 2017 Jacksonville Sheriff's Office Police Department Conducted job analysis for the ranks of Sergeant and Lieutenant. 2017 Raleigh, NC Police Department Conducted job analysis interviews for ranks of Detective, Sergeant, and Lieutenant. 2017 Arlington, TX Fire Department Developed Lieutenant and Captain assessment center exercises and administered assessment center. 2017 Gwinnett County, GA Sheriff's Office Developed and administered Corporal, Sergeant, and Lieutenant written examinations. Developed and administered Lieutenant assessment center. 2017 Gwinnett County, GA Police Department Developed and administered Corporal, Sergeant, and Lieutenant written examinations. Developed and Administered Sergeant and Lieutenant assessment center. 2017 Buffalo, NY Police Department Conducted job analysis interviews for the ranks of Detective, Detective Sergeant, and Lieutenant. 2017 Round Rock, TX Fire Department Developed Driver Engineer and Lieutenant written examination. 2017 Plano, TX Fire Department Developed Captain, Driver Engineer, Lieutenant, Deputy Chief, and Battalion Chief written examinations. 2017 Moline, IL Police Department Developed and administered Sergeant and Lieutenant written examinations and Lieutenant assessment center. 2017 Gwinnett County, GA Police Department Developed and administered Communications Officer III, Communications Officer IV, and Communications Shift Supervisor assessment centers. 2017 Rockford, IL Police Department Developed Lieutenant structured oral interview. 2017 135 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 114 Tulsa, OK Fire Department Conducted job analysis interviews, observations, and JAQ analyses for ranks of Fire Equipment Operator, Captain, District Chief, Assistant Chief, and Deputy Chief. Wrote job analysis technical report. 2016 - 2017 Chicago, IL Fire Department Conducted job analysis for Ambulance Commander and Paramedic Field Chief and wrote job analysis technical report. Assisted with development of written examination and structured oral interview. 2016 - 2017 Montgomery County, MD Fire and Rescue Service Wrote promotional process technical report for the ranks of Master Firefighter and Lieutenant. 2016 Bridgeport, CT Police Department Wrote promotional process technical report for the rank of Captain. 2016 Austin, TX Police Department Wrote job analysis technical report for rank of Commander. 2016 Arlington, TX Fire Department Wrote job analysis technical report for the ranks of Lieutenant and Captain. 2016 Montgomery County, MD Police Department Wrote job analysis technical report for the rank of Captain. 2016 Chattanooga, TN Police Department Wrote job analysis technical report for the ranks of Sergeant, Lieutenant, and Captain 2016 Chattanooga, TN Police Department Developed Sergeant, Lieutenant, and Captain assessment center exercises and administered assessment centers. 2016 Moline, IL Police Department Developed and administered Captain assessment center. 2016 Lakewood, CO Police Department Conducted Police Officer Structured Oral Interview assessor training. 2016 Los Alamos, NM Police Department Developed Sergeant assessment center exercises and administered assessment center. 2016 Des Plaines, IL Fire Department Developed Lieutenant assessment center exercises. 2016 Norwalk, CT Fire Department Developed and administered firefighter structured oral interview. 2016 Providence, RI Police Department Conducted assessor training for entry level SOI. 2016 Savannah-Chatham, GA Metropolitan Police Department Developed Sergeant assessment center exercises and administered assessment center. 2016 Cook County, IL Sheriff's Office Conducted job analysis interviews for the ranks of Corrections Sergeant and Lieutenant, Court Services Sergeant and Lieutenant, and Sheriff's Police Officer, Sergeant, and Lieutenant. 2015 Springfield, MO Police Department Developed Sergeant assessment center exercises and administered assessment center. 2015 136 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 115 Montgomery County, MD Fire and Rescue Service Developed Captain assessment center exercises and administered assessment center. 2015 Des Plaines, IL Police Department Developed Sergeant assessment center exercises and administered assessment center. 2015 Bridgeport, CT Police Department Conducted job analysis for Captain. Developed assessment center exercises and administered assessment center. 2015 Illinois State Police Developed Lieutenant and Captain assessment center exercises and administered assessment center. 2015 Gwinnett County, GA Sheriff's Department Conducted job analysis interviews and wrote technical report for the ranks of Corporal, Sergeant and Lieutenant. 2015 Publications:  Berry, C. M., Barratt, C. L., Dovalina, C. L., & Zhao, P. (2014). Can racial/ethnic subgroup criterion-to-test standard deviation ratios account for the conflicting differential validity and differential prediction evidence for cognitive ability tests? Journal of Occupational and Organizational Psychology, 87, 208 – 220. doi:10.1111/joop.12036 Conference Presentations:  Asenuga, O. A., Dovalina, C. L., & Berry, C. M. (2014, May). Other-Ratings of Counterproductive Work Behaviors: A Meta-analysis. Poster presented at the 2014 annual conference of the Society for Industrial and Organizational Psychology, Honolulu, HI.  Barratt, C. L., Dovalina, C. L., Zhao, P., Lomeli, L., & Berry, C. M. (2013, April). Frequency versus time-lost: Not proxies for voluntary and involuntary absence. Poster presented at the 2013 annual conference of the Society for Industrial and Organizational Psychology, Houston, TX.  Dovalina, C. L., Berry, C. M., & Sackett, P. R. (2013, April). Self-efficacy: Incremental validity beyond ability and implications for differential prediction. Poster presented at the 2013 annual conference of the Society for Industrial and Organizational Psychology, Houston, TX.  Barratt, C. L., Dovalina, C. L., Zhao, P., & Berry, C. M. (2012, April). Effects of subgroups’ variances on cognitive ability test differential validity. Poster presented at the annual meeting of the Society for Industrial and Organizational Psychology, San Diego, CA.  Dovalina, C. L. & Berry, C. M. (2012, April). Race differences in within-persons grade variability. Poster presented at the annual meeting of the Society for Industrial and Organizational Psychology, San Diego, CA. 137 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 116 Samantha Gleaves, M.A. Assessment Consultant Background Statement: Samantha joined IOS in 2015. Ms. Gleaves has experience in developing, validating and administering assessment centers and promotional job knowledge written exams. Ms. Gleaves communicates well with clients throughout all phases of the projects she manages. She also has a great deal of experience in ensuring the logistics for assessment centers and job knowledge written examinations are managed well. Education: Roosevelt University M.A. in Industrial/Organizational Psychology (2016) North Central College B.A. in Psychology (2014) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2019-Present Assessment Consultant, I/O Solutions, Inc.  Develops, validates, and implements all components of promotional processes.  Coordinates with clients and other vendors any and all logistics related to the administration of promotional processes (i.e., securing facility space).  Manages and conducts job analyses.  Develops and administers assessment center exercises.  Manages job knowledge written examinations.  Uses SPSS and Excel to analyze data gathered from written examinations and assessment centers.  Develops technical reports for all phases of assessment processes.  Communicates with clients to ensure all aspects of the process are complete and meet their requirements. 138 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 117 2018-2019 Senior Assessment Specialist, I/O Solutions, Inc.  Worked closely with the Vice President of Consulting Services to develop, validate, and implement all components of promotional processes for large public safety clients.  Managed and conducted job analyses.  Developed and administered assessment center exercises.  Managed job knowledge written examinations.  Used SPSS and Excel to analyze data gathered from written examinations and assessment centers.  Developed technical reports for all phases of assessment processes.  Communicated with clients to ensure all aspects of the process are complete and meet their requirements. 2016-2018 Assessment Specialist, I/O Solutions, Inc.  Worked directly with consultants and consulting managers to develop, validate and implement assessment centers for jobs in the public safety industry.  Managed and conducted job analyses.  Developed and administered assessment center exercises.  Managed job knowledge written examinations.  Used SPSS and Excel to analyze data gathered from written examinations and assessment centers.  Developed technical reports for all phases of assessment processes.  Communicated with clients to ensure all aspects of the process are complete and meet their requirements. 2015-2016 Consulting Assistant, I/O Solutions, Inc.  Worked with consultants to provide support for all phases of promotional processes.  Proof read, printed, and assembled all assessment-related documents to prepare for the administration of assessment processes.  Assisted in the administration of job knowledge written examinations and assessment centers.  Entered data and compiled feedback reports for assessment candidates after the administration of the promotional process. 139 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 118 Summary of Selection Projects: Client Project Description Timeframe Miami Beach, FL Police Department Job analysis for the ranks of Sergeant and Lieutenant. 2020 Springfield, MO Police Department Development of written examination and assessment centers for the ranks of Corporal and Lieutenant. 2020 Mississippi State Highway Patrol Development and administration of written examination, assessment center and internal interview for the rank of Master Sergeant. 2020 Pittsburgh Bureau of Police Development and administration of written examination and assessment center for the rank of Sergeant. 2020 Buffalo, NY Police Department Development and administration of written examinations and assessment centers for the ranks of Detective, Detective Sergeant, Lieutenant and Captain 2019 Springfield, MO Police Department Administration of assessment center for the rank of Sergeant. 2019 Chicago Fire Department Job analysis and development of written examination and assessment center for the rank of Battalion Chief. 2019 Milwaukee Fire Department Development of written examinations and assessment centers for the ranks of Lieutenant and Captain. 2019 Edmond, OK Fire Department Development of written examinations for the ranks of Lieutenant, Training Captain, and EMS Captain. 2019 Minneapolis Police Department Development of assessment center for the rank of Lieutenant. 2019 Montgomery County, MD Police Department Administration of assessment center for the rank of Sergeant. 2019 Tulsa Police Department Administration of assessment center for the rank of Sergeant. 2019 San Antonio Police Department Assisted in the administration of the assessment center for the rank of Lieutenant. 2019 Mississippi State Highway Patrol Development and administration of assessment center and internal interview for the rank of Master Sergeant. 2019 Austin Police Department Development and administration of assessment centers for the ranks of Sergeant, Lieutenant and Commander. 2019 Tulsa Police Department Administration of assessment center for the rank of Corporal. 2018 Orland Park Fire Prevention District Administration of driving practical assessment for the rank of Lieutenant. 2018 Mississippi State Highway Patrol Administration of assessment center for the rank of Master Sergeant. 2018 Minneapolis Police Department Development and administration of written examination for the rank of Sergeant. Development of assessment center for the rank of Sergeant. 2018 140 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 119 Indianapolis Fire Department Administration of written examination and assessment center for the ranks of Lieutenant, Captain, and Battalion Chief. 2018 Pittsburgh Bureau of Police Development and administration of written examination and assessment center for the rank of Sergeant. 2018 Arlington Heights Fire Department Development and administration of written examination and assessment center for the rank of Lieutenant. 2018 Indianapolis Metropolitan Police Department Development of written examinations and assessment centers for the ranks of Sergeant, Lieutenant, and Captain. 2018 San Diego Police Department Development and administration of written examinations and assessment centers for the ranks of Police Officer III, Detective, Sergeant, and Lieutenant. 2018 Atlanta Police Department Development and administration of written examinations and assessment centers for the ranks of Sergeant and Lieutenant. 2018 Houston Police Department Development and administration of written examinations and assessment centers for the ranks of Sergeant, Lieutenant, and Captain. 2018 Edmond, OK Fire Department Development of a written examination for the rank of Fire Prevention Captain. 2018 Miami, FL Police Department Development of assessment center exercises for the Lieutenant promotional process. Assisted in the administration of the assessment center. 2017 Buffalo, NY Police Department Job analysis, written examinations, and assessment centers for the ranks of Detective, Detective Sergeant, Lieutenant, Captain and Inspector. 2017 Cincinnati, OH Police Department Assessment center administration for the rank of Captain. 2017 Garwood, NJ Police Department Development of written examination for the rank of Sergeant. 2017 East Brunswick, NJ Police Department Development of written examination for the rank of Sergeant. 2017 Greeley, CO Fire Department Development of written exam and written exercise for the rank of Lieutenant. 2017 Arvada Fire Protection District Development and administration of assessment center for the rank of Paramedic. 2017 Houston Police Department Development and administration of written examinations and assessment centers for the ranks of Sergeant, Lieutenant, and Captain. 2016 Atlanta Police Department Administration of written examinations for the ranks of Sergeant and Lieutenant. 2016 Dallas, TX Police Department Job analysis interviews and questionnaires for the ranks of Sergeant and Lieutenant. 2016 141 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 120 New Jersey Transit Police Department Development of written examinations for the ranks of Sergeant and Lieutenant. 2016 Portland, ME Fire Department Development of written examinations for the ranks of Lieutenant and Captain. 2016 Wooster, OH Police Department Development of a written examination for the rank of Sergeant. 2016 Edmond, OK Fire Department Development of a bank of structured oral interview questions for the rank of Deputy Chief of Operations 2016 142 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 121 Madison Sayler, M.A. Assessment Consultant Background Statement: Madison Sayler is a consultant at Industrial/Organizational Solutions, Inc., a personnel and human resources consulting firm that specializes in testing and assessment. Madison has experience with job analysis, the development, validation, and administration of job knowledge examinations and assessment centers, and project management. Education: Roosevelt University – Chicago, IL Doctor of Philosophy: Industrial/Organizational Psychology (Expected December, 2020) Dissertation: The Right Fit: The Influence of Person-Environment Fit, Frustration, and Reactance on Counterproductive Work Behaviors Roosevelt University – Chicago, IL Master of Arts: Industrial/Organizational Psychology (2018) Nebraska Wesleyan University – Lincoln, NE Bachelor of Science: Psychology (2012) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2020 to Present Industrial/Organizational Solutions, Inc. Consultant  Consults with public safety agencies to develop, validate, and defend promotional testing and assessment systems.  Conducts job analysis (interviews and observations) to identify the essential knowledge, skills, abilities, and other characteristics necessary for performance.  Provides best-practice training to public safety agencies, human resources, and civil service personnel.  Develops technical reports for processes by documenting all aspects of the process and validation evidence to support assessments. 143 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 122 2018 to 2020 Chicago Transit Authority Talent Assessment and Analytics Specialist  Served as internal consultant to departments across the Authority to identify their talent needs; create and deliver valid, effective, and legally defensible selection plans.  Conducted job analyses, developed test and interview content, wrote technical reports and delivered results to clients.  Relied on psychometric expertise to develop custom assessments, set cut scores and conduct adverse impact analyses.  Applied strong verbal presentation, collaboration, and interpersonal skills to create and deliver presentations using non- technical language, to meet clients’ needs.  Successfully managed and met goals for multiple large scale projects. Summary of Selection Projects: Client Project Description Timeframe Broken Arrow Fire Department – Broken Arrow, OK Assisted with administration of the assessment center for the rank of Lieutenant. 2020 DC Fire and Emergency Medical Services – Washington, DC. Assisted with administration of the assessment center for the ranks of Sergeant, Lieutenant, and Captain. 2020 Montgomery County Sherriff’s Office, Montgomery County, MD Assisted with development and administration of the assessment center for the ranks of Sergeant and Lieutenant. 2020 Publications and Presentations:  Paul, Elisabeth, Sayler, M., Kampe, R., Hamilton, J. P., (May, 2019). T66. Functional Connectivity Between Extrastriate Body Area and Default-Mode Network Predicts Depersonalization Symptoms in Major Depression: Findings from a Multi-Network Comparison. Poster presented at the 74th Annual Meeting the Society of Biological Psychiatry, Chicago, IL.  Paul, Elisabeth, Sayler, M., Kampe, R., Hamilton, J. P., (March, 2019). Functional Connectivity Between Extrastriate Body Area and Default Mode Network Predicts Depersonalization Symptoms in Major Depression: Findings from an A Priori Specified Multinetwork Comparison. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging.  Bergamino, M., Sayler, M., Yeh, H-W., Hamilton, J. P., (June, 2017). Statistical Differences in The White Matter Tracts in Subjects with Depression by Using Different Skeletonized Voxel- Wise Analysis Approaches and DTI Fitting Procedures. Brain Research.  Bergamino, M., Pasternak, O., Sayler, M., Hamilton, J. P., (November, 2015). Applying A Free-Water Correction to Diffusion Imaging Data Uncovers Stress-Related Neural Pathology in Depression. NeuroImage: Clinical. 144 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 123  Hamilton, J. P., Sayler, M., Fogelman, P., Gotlib, I., (February, 2015). Depressive Rumination, the Default-Mode Network, and the Dark Matter of Clinical Neuroscience. Biological Psychiatry.  Driver, D., Gogtay, N., Greenstein, D., Gochman, P. A., Sayler, M., Rapoport, J., (October, 2013). Premorbid Impairments in Childhood-Onset Schizophrenia. Paper presented at the 60th Annual Meeting of American Academy of Child and Adolescent Psychiatry, Orlando, FL.  Driver, D., Greenstein, D., Sayler, M., Rapoport, J., Gogtay, N., (April, 2013). Premorbid Speech/Language, Motor, Academic and Social Impairments in Childhood-Onset Schizophrenia. Paper presented at the 68th Annual Scientific Meeting of the Society of Biological Psychiatry, San Francisco, CA. 145 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 124 Veronica Jenssen, M.A. Senior Assessment Specialist Background Statement: Veronica joined IOS in 2016. She is familiar with best practices in selection test development and validation. Her areas of expertise include, job analyses, data analysis, development of job knowledge exams and assessment center exercises, validation of selection processes, and technical report writing. Education: University of Tulsa M.A. in Industrial/Organizational Psychology (2016) Colorado State University B.S. in Psychology with a concentration in Industrial/Organizational Psychology (2013) Colorado State University B.A. in Journalism/Technical Writing with a concentration in Public Relations (2013) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2016-Present Assessment Specialist, I/O Solutions, Inc.  Work directly with consultants to develop, validate and implement assessment centers for jobs in the public safety industry.  Collaborate with others to conduct job analyses  Develop structured oral interviews  Use SPSS and Excel to analyze data gathered from job analyses and assessment centers  Implement and maintain test security on site at assessment centers  Develop and edit technical reports  Work with clients to development job-knowledge exams 2014-2016 International Research Intern, Hogan Assessment Systems  Assemble and maintain item translations, equivalence analyses and international norms for Hogan’s personality assessments  Conduct job analyses for clients 146 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 125  Write and edit abstracts, short technical reports, and long technical reports documenting the evidence and recommendations for selection processes for international clients  Establish customized personality scale cut-scores for international client’s selection solutions based on criterion- related validation evidence and validity generalization evidence  Aid research consultants in preparation and presentation of study results to clients Summary of Selection Projects: Client Project Description Timeframe Buffalo, NY Fire Department Compilation of promotional process technical report for the rank of Captain 2020 Monroe County, NY Sheriff's Office Development of Court Security Sergeant assessment center 2020 Carrollton, TX Fire Department Development of written exams for various ranks 2019-2020 Clarksville, TN Police Department Development of annual promotional process for Captain, Lieutenant, and Sergeant rank 2019 - 2020 Davenport, IA Police and Fire Development of written exams for Sergeant and Corporal (police) and fire engineer 2019 - 2020 New Jersey Chiefs of Police Development of written exams for various New Jersey Departments 2019-2020 Roanoke County, VA Police Department Development of Police Officer II and Sergeant written examination 2016 - 2020 Greeley, CO Fire Department Development of Battalion Chief written assessment 2019 Monroe County, NY Sheriff's Office Development of Investigator Sergeant assessment center 2019 Racine, WI Police Department Development of lieutenant assessment center 2019 Broken Arrow Fire Department Development of captain assessment center 2019 Bridgeport, CT Police Department Compilation of promotional process technical report for the rank of sergeant 2019 Montgomery County, OH Sheriff's Office Development of written exam for Sergeant Rank 2018-2019 Pueblo, CO Police Department Development of Lieutenant and Sergeant written examination 2017 - 2019 147 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 126 Rockford, IL Police Department Development of Sergeant and Police Investigator written examination 2017 - 2019 Indianapolis, IN Fire Department Compilation of promotional process technical report for the ranks of Lieutenant, Captain, and Battalion Chief 2018 Broken Arrow, OK Fire Department Job analysis for the ranks of captain and battalion chief 2018 Illinois State Police Development of the Master Sergeant SOI exercise 2018 Guilford County, NC Police Department Development of the Sworn Sergeant SOI exercise 2018 Mississippi Highway Safety Patrol Compilation of Technical Report for the Ranks of Master Sergeant, Lieutenant and Captain 2018 Warwick, RI County Police Department Development of Lieutenant and Captain written examination 2016 - 2018 Council Bluffs, IA Fire Department Development of Fire Engineer and Fire Marshal written examination 2017 - 2018 Illinois State Police Compilation of Technical Report for the Ranks of Trooper, Sergeant & Master Sergeant, and Lieutenant and Captain 2016 - 2018 Providence, RI Police Department Development of Lieutenant written examination 2017 Cranston, RI Police Department Development of Captain and Lieutenant written examination 2017 Tri-Lakes Monument Fire Department Development of Lieutenant written examination 2017 San Francisco Fire Department Compilation of Technical Report for the rank of Battalion Chief 2017 Omaha, NE Police Department Development of structured oral interview questions for the rank of Sergeant 2017 San Francisco Police Department Compilation of Technical Report for the rank of Lieutenant 2016-2017 Los Alamos Fire Department Development of Structured Oral Interview for Fire Captain and Battalion Chief 2016 148 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 127 Nick Jochim, M.A. Assessment Specialist Background Statement: Nick Jochim joined IOS in February of 2018. He completed his Master’s Degree in Industrial/Organizational Psychology from Elmhurst College in May 2019. Nick Jochim has experience with job analysis, test development and validation, assessment administration and recruitment. Education: Elmhurst College M.A. in Industrial/Organizational Psychology (2019) Elmhurst College B.A. in Psychology (2017) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2019-Present Assessment Specialist, I/O Solutions, Inc.  Works directly with consultants and consulting managers to develop, validate and implement promotional processes for public safety clients.  Facilitates item review meetings with clients  Manages and conducts job analyses to gather information about position(s) of interest  Manages written exam projects  Uses SPSS and Excel to generate results for various reports 2018-2019 Consulting Coordinator, I/O Solutions, Inc.  Manages recruitment of assessors for I/O Solutions’ assessment processes  Coordinates with assessors on travel arrangements, expenses and logistics  Reviews and proofs documents related to assessments  Print and prepare assessment and testing materials 149 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 128 Summary of Selection Projects: Client Project Description Timeframe Tulsa, OK Developed and administered an assessment center for the rank of Fire Deputy Chief 2020 San Antonio, TX Developed a written examination for the rank of Police Sergeant. 2020 Colorado State Patrol Developed and administered an assessment center for the rank of Major 2020 Tulsa, OK Developed and administered an assessment center for the rank of Police Major. 2019 Tulsa, OK Developed and administered an assessment center for the rank of Police Captain. 2019 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Corporal. 2019 Tulsa, OK Developed and administered written examination and assessment center for the rank of Police Sergeant. 2019 Tulsa, OK Developed a written examination for the rank of Fire Equipment Operator. 2019 St. Paul, MN Developed and administered written examination and assessment center for the rank of Police Sergeant. 2019 Forsyth County, GA Developed and administered written examination and assessment center for the rank of Battalion Chief. 2019 Forsyth County, GA Developed and administered written examination and practical assessment for the rank of Fire Apparatus Operator. 2019 Colorado State Patrol Developed and administered an assessment center for the rank of Captain 2019 Guilford County Sheriff's Office Conducted job analysis for the ranks of Detention Sergeant, Detention Lieutenant, Sworn Sergeant, and Sworn Lieutenant. Developed written examinations for the ranks of Sworn Sergeant and Detention Sergeant. Developed assessment centers for the ranks of Detention Sergeant and Detention Lieutenant 2018 San Antonio, TX Developed a written examination for the rank of Police Sergeant and Captain. 2018 Pinellas County Sheriff's Office, FL Developed written examination for the ranks of Sergeant (Law Enforcement Office and Department of Corrections) 2018 150 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 129 Haley Kilbride, M.A. Assessment Specialist Background Statement: Haley joined IOS in 2017. She obtained her Master’s Degree in Industrial/Organizational Psychology from Elmhurst College. Haley has experience in the job analysis process and item writing. She also is experienced in developing, validating and administering assessment centers and job knowledge exams. Education: Elmhurst College Master of Arts: Industrial/Organizational Psychology (2019) North Central College Bachelor of Science, Human Resource Management (2017) Minor in Psychology and Spanish Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: May 2019 to Present Assessment Specialist, I/O Solutions, Inc.  Works with department subject matter experts to develop assessment content  Works directly with consultants and consulting managers to develop, validate and implement assessment centers for jobs in the public safety industry.  Facilitates item review meetings with clients  Travels to various client locations throughout the US to assist with assessment center administration  Communicated with clients and third party vendors to ensure all aspects of the process are complete and meet their requirements.  Manages and conducts job analyses to gather information about position(s) of interest  Manages written exam projects independently  Administers exams/assessments and analyzes resulting data  Uses SPSS and Excel to generate results for various reports 151 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 130  Develops documents for promotional testing and recordkeeping (e.g., administrative documents, technical reports, responses to exam appeals, etc.) 2017 to May 2019 Consulting Assistant, I/O Solutions, Inc.  Worked with consultants to provide support for all phases of promotional processes.  Proof read, printed, and assembled all assessment-related documents to prepare for the administration of assessment processes.  Assisted in the administration of job knowledge written examinations and assessment centers.  Entered data and compiled feedback reports for assessment candidates after the administration of the promotional process. Summary of Selection Projects: Client Project Description Timeframe Atlanta Police Department Written examination and assessment center development, validation, administration and scoring for the ranks of Sergeant and Lieutenant. 2020 San Diego Police Department Written examination and assessment center development, validation, administration and scoring for the ranks of Sergeant, Lieutenant, Detective and Police Officer III. 2020 Chicago, IL Fire Department Written examination scoring and analysis and assessment center development for the rank of Battalion Chief. 2019 Bellevue, NE Fire Department Development of written examination for the ranks of Fire Captain and Battalion Chief 2019 Baltimore, MA Police Department Conducted a job analysis for the ranks of Sergeant and Lieutenant. Developed and administered assessment center for the rank of Lieutenant. 2019 Shelby County Sheriff's Office Developed and administered assessment center for the rank of Captain. 2019 Detroit, MI Police Department Administration of assessment center for the rank of Detective. 2019 Harris County, TX Sheriff's Office Administration of assessment center for the rank of Captain 2019 Raleigh, NC Police Department Development and management of written examination for the ranks of Sergeant and Detective. Development and management of the assessment center for the ranks of Sergeant, Lieutenant, and Detective 2019 152 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 131 Milwaukee, WI Fire Department Development and administration of assessment center for the ranks of Captain and Lieutenant. 2019 Buffalo, NY Police Department Written examination validation and assessment center development for the ranks of Detective, Detective Sergeant, Lieutenant, and Captain. 2019 San Francisco, CA Fire Department Job analysis and assessment center development and validation for the rank of Lieutenant. 2019 Buffalo, NY Police Department Development of assessment centers for the ranks of Detective, Detective Sergeant, Lieutenant and Captain 2019 Chicago Fire Department Development of written examination and assessment center for the rank of Battalion Chief. 2019 Austin Police Department Development of assessment centers for the ranks of Sergeant, Lieutenant and Commander. 2019 Cincinnati, OH Fire Department Administration of assessment center for the rank of Fire Captain. 2019 Monroe County Sheriff’s Office Developed assessment centers for the rank of Road Lieutenant and Jail Sergeant. 2018 City of Roswell, Georgia Developed the written examination and assessment center for the rank of Police Sergeant and an assessment center for the rank of Police Lieutenant. 2018 Atlanta Police Department Development and administration of written examinations and assessment centers for the ranks of Sergeant and Lieutenant. Development of candidate feedback reports. 2018 Houston Police Department Administration of assessment center for the rank of Lieutenant. 2018 IL State Police Merit Board Conducted job analysis interviews for the ranks of Sergeant and Master Sergeant. Development of candidate feedback reports. 2018 Shelby County Sheriff's Office Conducted a job analysis and pay study for the Criminal Analyst position. 2018 Minneapolis, MN Police Department Administration of assessment center for the rank of Sergeant. 2018 Indianapolis, IN Fire Department Administration of assessment center for the ranks of Fire Lieutenant, Captain, and Battalion Chief. 2018 Cobb County, GA Developed and administered assessment center for the rank of Fire Captain. 2017 Cook County Sheriff's Office Merit Board Written examination development for court services Sergeant and Lieutenant and correctional division Sergeant, and Lieutenant. 2017 153 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 132 Jacob Wolfarth, M.S. Assessment Specialist Background Statement: Jacob Wolfarth joined IOS in January of 2019. In his graduate studies and professional career, Mr. Wolfarth has primarily focused on assessment and selection, especially as it relates to public safety. Mr. Wolfarth has experience with job analysis, test development and validation, assessment administration, adverse impact analysis, and technical report writing. Education: Indiana University – Purdue University of Indianapolis M.S. in Industrial/Organizational Psychology (2018) Butler University B.A. in Music (2016) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2019-Present Assessment Specialist, I/O Solutions, Inc.  Works closely with Dr. Brian Marentette, a Manager of Consulting Services, to develop, validate, and implement all components of promotional processes for large public safety clients.  Manages and conducts job analyses.  Develops and administers assessment center exercises.  Uses SPSS, R, and Excel to analyze data gathered from written examinations and assessment centers.  Develops technical reports for all phases of assessment processes. July-September, 2018 Research Consultant, TechPoint  Worked closely with the CEO and the Director of Talent to design and execute a job analysis project  Led the execution of the Jobs in Tech Project, one of the first projects to synthesize jobs in technology, outlining the necessary tasks and KSA’s to succeed 154 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 133 Summary of Selection Projects: Mr. Wolfarth has a wealth of experience providing selection consulting services to a diverse listing of state, municipal and county public safety agencies. Following is a partial list of the jobs that Mr. Wolfarth has been involved with in the past year. Client Project Description Timeframe Dallas Fire Department Assessment center for the rank of Lieutenant 2020 Portland Fire Department Job analysis, written exam, and assessment center for the rank of Lieutenant 2020 Grand Rapids Police Department Assessment center for the ranks of Sergeant, Lieutenant, and Captain 2020 Ann Arbor Police Department Assessment center for the ranks of Sergeant and Lieutenant 2020 Chattanooga Fire Department Assessment center for the ranks of Battalion Chief, Captain, and Lieutenant 2019 Moline Fire Department Assessment center for the rank of Captain 2019 Gwinnett County Fire Department Assessment center for the rank of Lieutenant 2019 Chattanooga Police Department Assessment center for the ranks of Sergeant and Lieutenant 2019 Montgomery County Fire and Rescue Service Assessment center for the ranks of Captain and Battalion Chief 2019 Dallas Fire Department Assessment center for the ranks of Battalion Chief and Captain 2019 Ann Arbor Police Department Assessment center for the ranks of Sergeant and Lieutenant 2019 Detroit Police Department Structured oral interview and written examination for the ranks of Sergeant, Detective, and Lieutenant 2019 Grand Rapids Police Department Assessment center for the ranks of Sergeant, Lieutenant, and Captain 2019 Portland Fire Department Job analysis and assessment center for the rank of Captain 2019 Cobb County Sheriff’s Office Assessment center for the ranks of Sergeant and Lieutenant 2019 Virginia Beach Police Department Assessment center for the rank of Sergeant. 2019 Columbia Police Department Job analysis for the ranks of Corporal, Sergeant, and Lieutenant. 2019 155 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 134 Harry Vuong, M.A. Senior Consulting Coordinator Background Statement: Harry joined IOS in 2017. Mr. Vuong has experience in recruiting assessors and coordinating their travel, expenses, and logistics for assessment centers. Mr. Vuong also has experience in administering assessment centers and promotional job knowledge written exams. Mr. Vuong communicates well with project managers and assessors throughout all phases of the projects he is part of. Education: The Chicago School of Professional Psychology M.A. in Industrial/Organizational Psychology (2013) Loyola University Chicago B.A. in Psychology (2009) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2019-Present Senior Consulting Coordinator, I/O Solutions, Inc.  Manage recruitment of assessors (raters) for I/O Solutions’ assessment processes for external clients.  Coordinate with assessors on travel arrangements, expenses and logistics.  Review and proof documents related to assessments.  Create/modify administrative materials related to assessment administration.  Print and prepare assessment and testing materials.  Travel and assist with administration of on-site assessment center coordination and activities.  Assist with entering and proofing critical assessment data. 2017-2019 Consulting Coordinator, I/O Solutions, Inc.  Manage recruitment of assessors (raters) for I/O Solutions’ assessment processes for external clients. 156 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 135  Coordinate with assessors on travel arrangements, expenses and logistics.  Review and proof documents related to assessments.  Create/modify administrative materials related to assessment administration.  Print and prepare assessment and testing materials.  Travel and assist with administration of on-site assessment center coordination and activities.  Assist with entering and proofing critical assessment data Summary of Selection Projects: Client Project Description Timeframe Albuquerque Police Department Manage the recruitment and coordination of assessors of an Assessment Centers for the rank of Lieutenant. Assisted in administration of the assessment center. 2019 San Diego Fire-Rescue Department Manage the recruitment and coordination of assessors for an USERRA Assessment Centers for the rank of Battalion Chief. 2019 Ann Arbor Police Department Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Sergeant and Lieutenant. Assisted in administration of both assessment centers. 2019 Dallas Fire-Rescue Department Manage the recruitment and coordination of assessors for an Assessment Centers for the rank of Captain. 2019 Columbus Fire & Emergency Medical Services Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Sergeant, Lieutenant, Captain, and Battalion Chief. Assisted in administration of both assessment centers. 2019 Detroit, MI Police Department Manage the recruitment and coordination of assessors for an Assessment Center for the ranks of detective, sergeant, and lieutenant. Assisted in administration of the assessment center. 2019 Tuscaloosa, AL Fire Rescue Service Administration of a Practical exam for the rank of Fire Apparatus Operator. 2019 Metropolitan Nashville Police Department Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Sergeant and Lieutenant. 2019 Grand Rapids Police Department Manage the recruitment and coordination of assessors for Assessment Centers for the rank of Sergeant, Lieutenant, Captain. Assisted in administration of assessment center. 2019 Forsyth County, GA Fire Department Administration of assessment center for the rank of Lieutenant. 2019 Albuquerque Fire Rescue Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Captain and Battalion Chief. Assisted in administration of both assessment centers. 2018 Albuquerque Police Department Manage the recruitment and coordination of assessors for three Assessment Centers for the rank of Sergeant (2) and Lieutenant. Assisted in administration of all three assessment centers. 2018 San Diego Fire-Rescue Department Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Captain and Battalion Chief. Assisted in administration of both assessment centers. 2018 157 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 136 Ann Arbor Police Department Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Sergeant and Lieutenant. Assisted in administration of both assessment centers. 2018 Dallas Fire-Rescue Department Manage the recruitment and coordination of assessors for three Assessment Centers for the rank of Section Chief, Captain, and Lieutenant. 2018 Buffalo Fire Department Manage the recruitment and coordination of assessors for three Assessment Centers for the rank of Battalion Chief, Captain, and Lieutenant. Assisted in administration of assessment centers. 2017 Grand Rapids Police Department Manage the recruitment and coordination of assessors for Assessment Centers for the rank of Sergeant, Lieutenant, Captain. Assisted in administration of assessment center. 2017 Broken Arrow, OK Fire Department Manage the recruitment and coordination of assessors for an Assessment Center for the rank of EMS Training Major. 2017 Dallas Fire-Rescue Department Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Captain and Lieutenant. Assisted in administration of assessment center. 2017 Dallas Police Department Manage the recruitment and coordination of assessors for an Assessment Center for the rank of Sergeant. 2017 Metropolitan Nashville Police Department Manage the recruitment and coordination of assessors for three Assessment Centers for the rank of Captain, Lieutenant, and Sergeant. 2017 Columbus, GA Fire & EMS Manage the recruitment and coordination of assessors for four Assessment Centers for the rank of Battalion Chief, Captain, Lieutenant, and Sergeant. 2017 Indianapolis Metropolitan Police Department Job analysis interviews for the ranks of Sergeant and Lieutenant. 2017 Albuquerque Police Department Manage the recruitment and coordination of assessors for two Assessment Centers for the rank of Sergeant and Lieutenant. Assisted in administration of assessment centers. 2017 Baltimore City Fire Department Manage the recruitment and coordination of assessors for three Assessment Centers for the rank of EMS Battalion Chief, EMS Captain, and EMS Lieutenant. 2017 158 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 137 Robert W. Szarek, M.A. Consultant/Sr. Researcher, Testing Services Background Statement: Robert has worked in the Research & Development (also known as Testing Services) department of IOS since obtaining his Master’s Degree in Industrial-Organizational Psychology from Elmhurst College in 2012. Over the course of his tenure at IOS, Robert engaged in numerous client projects leveraging skills in statistical analyses to aid in test customization, validation and technical report composition. Of particular interest to Robert is the development and advancement of job analysis related techniques, including researching innovative ways to analyze and present job analysis data to clients. Most recently, Robert has undertaken additional internal projects related to the development of feedback reports for candidates testing for various agencies, in addition to the development of customized scoring dashboards for real time data analytics. Robert also engages in the internal development of new off the shelf examinations and products. Some specific areas of statistical analyses that Robert has previously engaged in with clients on a statewide and municipal level include:  Adverse impact analysis  Statistical significance testing  Criterion-related validation  Content-validation (CVR implementation, Angoff calculation, SME review sessions)  Cut score analysis & defensibility  Classical test theory item-analysis  Legal issues in personnel selection  Differential item functioning (DIF)  Validity Transportation Robert also possesses a keen interest in the following areas of the field:  Open Source “R” Software (Packages: ggplot2, tidyverse, haven, xlsx, dplyr, psych, shiny)  Introductory Python skills  SPSS Syntax Script Writing  Structured Oral interview (SOI) development, implementation and scoring Education: Elmhurst College, Elmhurst, IL Master of Arts, Industrial-Organizational Psychology (Graduated in 2012) Northeastern Illinois University, Chicago, IL Bachelor of Science, Psychology (Graduated in 2010) Minor: Philosophy Professional Experience: 159 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 138 June 2017 to Present Industrial/Organizational Solutions, Inc. (IOS) Consultant/ Sr. Researcher, Testing Services 2012 to June 2017 Industrial/Organizational Solutions, Inc. (IOS) Research Analyst, Testing Services Summary of Selection Projects: Client Project Description Timeframe Washington State Patrol Exam validation with entry level troopers 2017 Peoria, IL Administration and on-site scoring of entry level police officer examination 2016 - Current College of DuPage Online Consortium testing for entry level firefighters 2016 - Current Cincinnati, OH Entry level firefighter and police officer testing and scoring 2017 - Current Dallas, TX Job analysis study of four promotional ranks in the police department 2016 - 2017 Houston, TX On site scoring of promotional police testing processes 2016 Kalamazoo, MI Entry Level Candidate Training Sessions 2016 - Current Rhode Island State Police Written Examination Scoring, and Structured Oral Interview development, scoring 2017 - Current Milwaukee, WI Entry Level Firefighter Written Examination scoring and Structured Oral Interview development, administration and scoring 2017 - Current Charleston, SC Structured Oral Interview Training and Scoring 2017 - Current Springfield, IL Police Department Written Examination Scoring and Structured Oral Interview Development 2017 - Current Springfield, IL Fire Department Written Examination Scoring and Structured Oral Interview Development 2017 - Current San Antonio, TX Fire Department Development of entry-level written examination for 2,600 candidates participating in the 2016 SAFD examination process 2015 - Current Montgomery County, DOCR Development of video-based entry-level testing process 2019 - Current Montgomery County, Maryland Custom entry-level written examination and structured oral interview procedure for the fire department. 2016 - Current New York Civil Service Commission Engage in the analysis and technical report compilation of the statewide job analysis study performed in 2015. 2016 - Current San Antonio, TX Police Department Development of online entry-level written examination for the police department including standardization. 2015 - Current Massachusetts Trial Court, MA Compilation of technical report 2015-2016 St. Andrews, SC Archival data analysis on all prior testing cycles to recommend revised procedures for employee selection. 2015 160 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 139 New Jersey Civil Service Commission Conducted statewide job analysis for firefighter position including: on-site interviews, observations, JAQ administration, analysis, and written exam field testing. Compiled all relevant technical documentation outlining scope of work performed. 2015 - Current FDLE, Florida Oversee the day to day operation of our online testing platform (PS3) for numerous testing campuses across the state. 2012 - Current Illinois State Police (ISP) Entry Level SOI Development. Overseeing of uploading promotional examinations to online testing system. 2014 - Current 161 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 140 Alexa Bolwin, M.A. Research Analyst, Testing Services Background Statement: Alexa joined IOS in October of 2015. She obtained her Master’s Degree in Industrial/Organizational Psychology from Elmhurst College in 2016. As an analyst, Alexa often uses various statistical analyses to aid in test customization, validation, and technical report composition. As an analyst, Alexa also frequently engages in the following employment testing processes:  Job analysis  Adverse impact analysis  Statistical significance testing  Criterion-related validation  Content-validation (CVR implementation, Angoff calculation, SME review sessions)  Classical test theory item-analysis  Validity transportation  Cut score analysis Alexa also possesses an interest in the following areas of the field:  Survey design and implementation  Open source “R” software Education: Elmhurst College, Elmhurst, IL Master of Arts, Industrial-Organizational Psychology (2016) University of Illinois at Chicago, Chicago, IL Bachelor of Arts, Psychology; Criminology, Law, & Justice (2014) Professional Experience: March 2019- Industrial/Organizational Solutions, Inc. (IOS) Present Research Analyst September 2016- Industrial/Organizational Solutions, Inc. (IOS) March 2019 Testing Services Coordinator June 2016- Industrial/Organizational Solutions, Inc. (IOS) September 2016 Research Assistant 162 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 141 October 2015- Industrial/Organizational Solutions, Inc. (IOS) June 2016 Consulting Assistant Summary of Selection Projects: Client Project Description Timeframe Caledonia, WI Fire Department Oversaw local content validation survey administration, data analysis, and reporting for off-the-shelf promotional examinations. 2018-2020 Gainesville, GA Fire Department Waterford, MI Fire Department Sylvania Township, OH Fire Department Dawson County, GA Fire Department North Port, AL Fire Department South Haven, MI Fire Department Monroe, OH Fire Department Pflugerville, TX Fire Department Pendleton, OR Fire Department Burton, OH Fire Department Fayetteville, AR Fire Department South Chicago Heights, IL Fire Department Anchorage, AK Police Department West Des Moines, IA Fire Department Sandusky, OH Police Department Milton, GA Fire Department LaGrange Park, IL Police Department 163 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 142 Milton, GA Fire Department Oversaw local content validation survey administration, data analysis, and reporting for off-the-shelf promotional examinations. 2018-2020 DeSoto County, MS Police Department Mount Vernon, OH Police Department Princeton, IL Fire Department Gladstone, MO Fire Department North Branford, CT Police Department Topsfield, MA Fire Department Papillion, NE Police Department West University Place, TX Fire Department Florence, AL Fire Department Southaven, MS Police Department Clinton, IA Fire Department Dayton, VA Police Department Barboursville, WV Police Department Reading, PA Fire Department Clayton County, GA Fire Department Reading, PA (Fire EMS Fire Department) Robinson, IL Police Department Lebanon, OH Police Department Windsor Severance, CO Fire Rescue 164 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 143 Massachusetts Trial Court Oversaw online examination registration for entry-level and promotional examination processes. 2017-2020 San Antonio Police and Fire Department Assisted with writing custom cognitive ability items semi- annually. 2016-2020 New York State Department of Civil Service Assisted with custom situation-based item writing process. Assisted with development of items measuring cognitive ability. Assisted with development of and managed the administration of an online performance evaluation tool. Assisted with validity transportation process for four entry-level positions in the State of New York; delivered online survey; analyzed survey results and compiled report. 2016-2019 Massachusetts Trial Court Assisted with development of and administered online performance evaluation tool. 2019 Austin-Travis County EMS Assisted with development of custom structured oral interview scenarios and criteria for Medic I Field and Medic I Communications positions. 2019 Creve Coeur, MO Developed semi-custom examinations for the ranks of EMS Captain and Fire Captain. 2019 Sylvania Township, OH Developed a semi-custom examination for the rank of Lieutenant. 2019 New Jersey State Association of Chiefs of Police Developed statewide promotional examinations for the ranks of First Line Supervisor and Middle Manager. 2019 Public Safety Testing, WA Conducted adverse impact analyses to examine potential disparate impact and subsequent risk for litigation pertaining to the agency's use of three entry-level examinations. Compiled technical report for analysis of each of the three examinations. 2016 165 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 144 Taylor DeSantis, M.S. Research Analyst, Testing Services Background Statement: Taylor joined IOS in July of 2019. He obtained his Master’s Degree in Industrial/Organizational Psychology from Baruch College in 2017. As an analyst, Taylor often uses various statistical analyses to aid in test customization, validation, and technical report composition. Education: Baruch College, Weissman School of Arts and Sciences, New York, NY Master of Science: Industrial/Organizational Psychology (2017) University of Albany, Albany, NY Bachelor of Arts, Psychology (2014) Affiliations: Society for Industrial/Organizational Psychology (SIOP) Professional Experience: 2019 to Present Industrial/Organizational Solutions, Inc. Research Analysist  Conducts job analysis (interviews and observations) to identify the essential knowledge, skills, and abilities and other characteristics necessary for performance  Creates and administers surveys for validity transportation  Runs adverse impact analyses  Performs statistical significance testing  Conducts criterion-related validation studies  Analyzes validity transportation results 2017 to 2019 Department of Citywide Administration Services Tests and Measurement Specialist  Conducted job analysis (interviews and observations) to identify the essential knowledge, skills, and abilities and other characteristics necessary for performance  Oversight of item creation for multiple-choice tests, including item writing, revisions of material, test review and exam administration  Created education and experience-based exams  Determined merit of qualifications for education and experienced-based exams for initial exam and for protests 166 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 145  Oversaw test validation boards comprised of subject matter experts  Headed research for the development of new multi-title test process Summary of Selection Projects: Client Project Description Timeframe Fort Worth, TX Police Department Job Analysis and & Validity Transportation process for entry-level Patrol Officer position. Managed the administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Provided written exam for ~600 candidates and delivered comprehensive adverse impact report of test outcomes. 2020 Peoria, IL Fire Department Administration of entry-level fire fighter examination 2020 Greenville, SC Police Department Job Analysis and & Validity Transportation process for entry-level Patrol Officer position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Developed custom SOI, provided in-depth incumbent assessor training and oversaw SOI administration. Validated (content & criterion) existing custom physical ability test (PAT) simulation. Worked with SMEs to content validate departmental Medical Health Screening process. 2019 Greenville, SC Fire Department Job Analysis and & Validity Transportation process for entry-level firefighter position. Managed the conduction of on-site job analysis visits as well as administration of an online JAQ survey. Analyzed JAQ data and compiled transportability report. Developed custom SOI, provided in-depth incumbent assessor training and oversaw SOI administration. Validated (content & criterion) existing custom physical ability test (PAT) simulation. Worked with SMEs to content validate departmental Medical Health Screening process. 2019 Cook County Sheriff's Office Job Analysis and & Validity Transportation process for entry-level Patrol Officer position. Analyzed JAQ data and compiled transportability report. 2019 San Antonio Police and Fire Department Assisted with writing custom cognitive ability items. 2019 Elk Grove Village, IL Fire Department Data clean-up, analyzation of data, and scoring of written exams. 2019 - 2020 LaGrange, IL Fire Department Wilmette, IL Police Department 167 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 146 Grayslake, IL Fire Department Elmwood Township, MI Fire Department Geneva, IL Fire Department Elmhurst, IL Police Department Des Plaines, IL Fire Department Arlington Heights, IL Police Department Hinsdale, IL Fire Department Wheaton, IL Fire Department 168 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 147 Project Management IOS’s project management system is simple and effective. Our corporation employs a large team of Industrial/Organizational Psychologists who serve as project managers/ consultants. From this team, a single individual is identified to manage the day-to-day responsibilities associated with a given project/client. While there is one point of contact in order to simplify the communication process between client and consultant, there is also a team of individuals available to support the project and multiple consultants who are familiar with the project status. Following is an overview of the tenets of our project management system: 1. Sole Point-of-Contact: A single, primary consultant/project director is designated to head a project. This ensures a consistent source of information for the client and simple, effective lines of communication. This individual will be available by email, office phone and cell phone on a regular basis. A secondary contact person will also be designated should the agency have an urgent need to contact someone at IOS to discuss the project. 2. Team Support: While a single person is managing the project, that person is supported by a large team of consultants, technical writers, project assistants, job analysts, and administrative staff to ensure that the necessary manpower is always available to accomplish the job at hand. The project manager will leverage this assistance to accomplish the proposed project plan according to the stated schedule. The project director is responsible for quality control measures and for transfer of all project deliverables. IOS maintains one of the largest professional staffs in the public safety selection consulting industry, allowing us to meet your agencies needs with a great deal of flexibility and responsiveness. 3. Team Leader Oversight: The project director is supervised by the Team Leader, who ensures that all timelines are met and all project works meets our high standards. The Team Leader regularly meets with project director to discuss project progress and strategies. We constantly analyze and evaluate our methodologies in a ‘group think’ setting to ensure an efficient, effective and state-of-the-art solution. 4. Planning and Scheduling System: The project director will be the main client contact and will conduct weekly planning and coordination meetings with the entire project time to ensure that work is efficiently allocated and project managers/coordinators and technical writing staff are on schedule and producing work that follows our methodology and meets our high standards. 169 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 148 Project References Law Enforcement and Fire Service Promotional Selection Processes Client Atlanta, GA Police Department Contact Michael O’Connor Major Address/Phone/E-mail 226 Peachtree Street SW Atlanta, GA 30303 404.546.7942 MOconnor@atlantaga.gov Contract Title Police Promotional Written Exams and Oral Assessments Contract Period 2009, 2011-2012, 2013-2014, 2016, 2017-2018 - Present Description:  Conducted job analyses in 2009 and 2016, with reviews/updates in 2011, 2013 and 2017  Developed written exams for Sergeant and Lieutenant  Developed and administered oral assessments for Sergeant and Lieutenant  Recruited and trained 24 assessors  Coordinated and secured examination sites and all lodging for assessors No post-test litigation or complaints filed. Client Bridgeport, CT Police Department Contact Lisa Mastronunzio Civil Service Examiner Address/Phone/E-mail 45 Lyon Terrace Bridgeport, CT 06604 203.576.7109 Lisa.Mastronunzio@Bridgeportct.gov Contract Title Development of Police Detective Written Exam and Development and Administration of Police Detective Assessment Center Contract Period January 2020 – Present Contract Title Development of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period December 2018 – February 2019 Contract Title Development and Administration of Police Captain Assessment Center Contract Period September 2015 – November 2015 Contract Title Development of Police Detective Written Exam and Development and Administration of Police Detective Assessment Center Contract Period December 2014 – April 2015 Contract Title Development of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period February 2014 – July 2014 Contract Title Development of Police Lieutenant Written Exam and Development and Administration of Police Lieutenant Assessment Center Contract Period March 2013 – December 2013 Client City of Broken Arrow, OK Contact Jeremy Moore Fire Chief Address/Phone/E-mail 1101 N. 6th Street Broken Arrow, OK 74012 918.259.8360 x6355 JKMoore@brokenarrowok.gov Contact Mark Steward Assistant Fire Chief Address/Phone/E-mail 1101 N. 6th Street Broken Arrow, OK 74012 170 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 149 918.259.8360 x6361 JSteward@brokenarrowok.gov Contract Title Development and Administration of Fire Captain and Battalion Chief Written Exams and Assessment Centers Contract Period January 2019 – July 2019 Contract Title Development and Administration of Fire Lieutenant, Fire Investigator and OPS Training Officer Written Exams and Assessment Centers Contract Period March 2018 – February 2019 Contract Title Development and Administration of Fire Lieutenant, Deputy Chief and EMS Major Written Exams and Assessment Centers Contract Period February 2017 – January 2018 Client Buffalo, NY Fire Department Contact G.G.(Gladys) Herndon-Hill Commissioner Address/Phone/E-mail 65 Niagara Square City Hall - Room 1001 Buffalo, NY 14202 716.851.9613 CS15@ch.ci.buffalo.ny.us Contract Title Development of a Fire Captain Written Exam and Development and Administration of a Fire Captain Assessment Center Contract Period August 2019 – January 2020 Contract Title Development of Fire Lieutenant and Captain Written Exams and Development and Administration of Fire Lieutenant, Captain, Battalion Chief and Division Chief Assessment Centers Contract Period January 2017 – February 2018 Client Buffalo, NY Police Department Contact G.G. (Gladys) Herndon-Hill Commissioner Address/Phone/E-mail 65 Niagara Square City Hall - Room 1001 Buffalo, NY 14202 716.851.9613 CS15@ch.ci.buffalo.ny.us Contract Title Development of Police Detective, Detective Sergeant, Lieutenant and Captain Written Exams and Development and Administration of Police Detective, Detective Sergeant, Lieutenant and Captain Assessment Centers Contract Period September 2019 – February 2020 Contract Title Development of Police Inspector, Detective, Detective Sergeant, Lieutenant and Captain Written Exams and Development and Administration of Police Inspector, Detective, Detective Sergeant, Lieutenant and Captain Assessment Centers Contract Period January 2017 – November 2017 Contract Title Development of Police Inspector, Detective, Detective Sergeant, Lieutenant and Captain Written Exams and Development and Administration of Police Inspector, Detective, Detective Sergeant, Lieutenant and Captain Assessment Centers Contract Period December 2011 – August 2012 Contract Title Development of Police Inspector and Captain Written Exams and Development and Administration of Police Inspector and Captain Assessment Centers Contract Period September 2007 – September 2008 Contract Title Development of Police Detective, Detective Sergeant and Lieutenant Written Exams and Development and Administration of Police Detective, Detective Sergeant and Lieutenant Structured Oral Assessments Contract Period September 2007 – September 2008 171 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 150 Chattanooga, TN Fire Department Contact Iris Neal Human Resources Generalist Address/Phone/E-mail Fire Administration Building 910 Wisdom Street Chattanooga, TN 37406 423.643.7207 ineal@chattanooga.gov Contract Title Development and Administration of Fire Lieutenant and Captain Written Examinations and Development and Administration Fire Lieutenant, Captain and Battalion Chief Assessment Centers Contract Period December 2018 – May 2019 Contract Title Development and Administration of Fire Lieutenant and Captain Assessment Centers Contract Period April 2017 – June 2017 Contract Title Development and Administration of Fire Lieutenant and Captain Assessment Centers Contract Period July 2014 – December 2015 Client Chattanooga, TN Police Department Contact Shea Jefferson, MPS Human Resources Generalist Address/Phone/E-mail 101 East 11th Street - Suite 201 Chattanooga, TN 37402 423.643.7214 sjefferson@chattanooga.gov Contract Title Development and Administration of Police Sergeant and Lieutenant Written Examinations and Development and Administration of Police Sergeant, Lieutenant and Captain Assessment Centers Contract Period June 2019 – November 2019 Contract Title Development and Administration of Police Captain Assessment Center Contract Period March 2018 – July 2018 Contract Title Development and Administration of Police Sergeant and Lieutenant Written Examinations and Development and Administration of Police Sergeant, Lieutenant and Captain Assessment Centers Contract Period April 2016 – November 2016 Client City of Chicago, IL Contact Jill May, M.S. Testing Specialist Address/Phone/ E-mail 121 N. LaSalle, 1102 Chicago, IL 60602 312.744.1240 Jill.May@cityofchicago.org Contract Title Development and Administration of Fire Battalion Chief Promotional Examination and Development and Administration of Fire Battalion Chief Oral Examination Contract Period August 2019 – Present Description:  Developing and validating a written job-knowledge examination for each position  Developing and validating a structured oral interview process for each position Contract Title EMS Promotional examinations for Ambulance Commander and Paramedic Field Chief Contract Period February 2016 – September 2017 Description:  Developing and validating a written job-knowledge examination for each position  Developing and validating a structured oral interview process for each position Contract Title Development and Administration of Police Sergeant Written Examination and Written Assessment Contract Period August 2012 – June 2014 Description:  Developed and validated a written job-knowledge examination for the Sergeant rank  Developed and validated a written assessment for the Sergeant rank 172 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 151 Contract Title EMS Promotional examinations for Ambulance Commander and Paramedic Field Chief Contract Period May 2011 – August 2013 Description:  Developed and validated a written job-knowledge examination for each position  Developed and validated a structured oral interview process for each position Contract Title EMS Promotional examinations for Ambulance Commander and Paramedic Field Chief Contract Period January 2006 – December 2006 Description:  Developed and validated a written job-knowledge examination for each position  Developed and validated a structured oral interview process for each position Contract Title Provision of Entry-Level Exams – Police and Fire Contract Period Police: 2013/2014, 2016 - Present Contract Period Fire: 2006, 2012, 2013, 2014 Client City of Cincinnati, OH Contact Latisha Hazell Division Manager, Workforce Management Address/Phone/E-mail 805 Central Avenue – Ste. 200 Cincinnati, OH 45202 513.352.1507 latisha.hazell@cincinnati-oh.gov Contract Title Development and Administration of Police Lieutenant Written Examination and Assessment Center Contract Period June 2019 - Present Contract Title Development and Administration of Police Sergeant Written Examination and Situational Judgment Test Contract Period December 2019 – Present Contract Title Development and Administration of Police Captain Assessment Center Contract Period November 2018 – January 2019 Contract Title Development and Administration of Police Lieutenant Assessment Center Contract Period February 2018 – May 2018 Contract Title Development and Administration of Police Sergeant Written Exercise Contract Period February 2018 – May 2018 Contract Title Development and Administration of Police Captain Assessment Center Contract Period January 2017 – June 2017 Contract Title Development and Administration of Fire Lieutenant Written Examination and Assessment Center Contract Period January 2017 – August 2017 Client Cobb County, GA Sheriff’s Office Contact Shirene Douglas, PHR, SHRM-CP Human Resources Manager Address/Phone/ E-mail Cobb County Government 100 Cherokee Street, 2nd Floor Marietta, GA 30090 770.528.2538 shirene.douglas@cobbcounty.org Contract Title Sergeant and Lieutenant Promotional Processes Contract Period 2007-2018 Description:  Developed and administered job knowledge written exams for Sergeant and Lieutenant  Conducted candidate orientations  Developed and administered assessment centers for Sergeant and Lieutenant  Trained assessors  Developed candidate feedback reports No post-test litigation or complaints filed. 173 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 152 Client Forsyth County, GA Fire Department Contact Jason Shivers Division Chief – Technical Services Address/Phone/E-mail 3520 Settingdown Road Cumming, GA 30028 678.513.5898 RJShivers@forsythco.com Contract Title Development and Administration of Fire Apparatus Operator, Lieutenant and Captain Written Exams and Development and Administration of Fire Lieutenant and Captain Assessment Centers Contract Period January 2020 – Present Contract Title Development and Administration of Fire Apparatus Operator and Battalion Chief Written Exams and Development and Administration of Fire Apparatus Operator and Battalion Chief Assessment Centers Contract Period September 2019 – February 2020 Contract Title Development and Administration of Fire Apparatus Operator and Lieutenant Written Exams and Development and Administration of Fire Apparatus Operator and Lieutenant Assessment Centers Contract Period March 2018 – May 2019 Contract Title Development and Administration of Fire Apparatus Operator and Battalion Chief Written Exams and Development and Administration of Fire Apparatus Operator and Battalion Chief Assessment Centers Contract Period January 2017 – November 2017 Contract Title Development and Administration of Fire Apparatus Operator and Lieutenant Written Exams and Development and Administration of Fire Apparatus Operator and Lieutenant Assessment Centers Contract Period January 2016 – July 2016 Contract Title Development and Administration of Fire Apparatus Operator, Lieutenant and Captain Written Exams and Development and Administration of Fire Lieutenant and Captain Assessment Centers Contract Period March 2015 – December 2015 Contract Title Development and Administration of Fire Battalion Chief Written Exam and Development and Administration of Fire Battalion Chief Assessment Center Contract Period July 2013 – October 2013 Client Gwinnett County, GA Contact (Main) Erica Queen Address/Phone/E-mail Human Resources 75 Langley Drive Lawrenceville, GA 30046 770.822.7949 Erica.Queen@gwinnettcounty.com Contact (Police) Sharon Morris Address/Phone/E-mail Human Resources 75 Langley Drive Lawrenceville, GA 30046 770.822.7930 Sharon.Morris@gwinnettcounty.com Contact (Corrections) Sharon Morris Address/Phone/E-mail Human Resources 75 Langley Drive Lawrenceville, GA 30046 770.822.7930 Sharon.Morris@gwinnettcounty.com Contact (Sheriff) Adrienne McAllister Address/Phone/E-mail Human Resources 75 Langley Drive Lawrenceville, GA 30046 770.822.7914 Adrienne.McAllister@gwinnettcounty.com 174 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 153 Contact (Fire) Donna Pratt HR Associate III Address/Phone/E-mail Human Resources 75 Langley Drive Lawrenceville, GA 30046 770.822.7901 Donna.Pratt@gwinnettcounty.com Contact (Fire) Michelle Saunders HR Associate III Address/Phone/E-mail Human Resources 75 Langley Drive Lawrenceville, GA 30046 770.822.7925 Michelle.Saunders@gwinnettcounty.com Contract Title Police, Sheriff, Corrections and Fire Promotional Exams and Assessment Centers Contract Period June 2002 – Present Description:  Conduct job analysis for 12 ranks: Corporal, Sergeant and Lieutenant within the Police, Sheriff and Corrections departments and Driver Engineer, Lieutenant and Captain in the Fire Department  Develop written examinations, situational judgment tests, structured oral assessments and assessment centers as follows: o Police Corporal: 12 cycles o Police Sergeant: 13 cycles o Police Lieutenant: 12 cycles o Sheriff Corporal: 16 cycles o Sheriff Sergeant: 18 cycles o Sheriff Lieutenant: 17 cycles o Corrections Corporal: 9 cycles o Corrections Sergeant: 11 cycles o Corrections Lieutenant: 9 cycles o Fire Engineer: 12 cycles o Fire Lieutenant: 11 cycles o Fire Captain: 6 cycles  Helped develop and maintain the candidate appeal and review process associated with protective service examinations This contract has been renewed/re-awarded five times beyond the initial contract. Client Houston, TX Police Department Contact Pete Lopez Assistant Chief Address/Phone/E-mail 1200 Travis Street – 18th Floor Houston, TX 77002 713.308.1877 pedro.lopez@houstonpolice.org Contract Title Development and Administration of Police Sergeant, Lieutenant and Commander Written Examinations and Development and Administration of Police Sergeant, Lieutenant and Commander Assessment Centers Contract Period January 2020 - Present Contract Title Development and Administration of Police Sergeant, Lieutenant and Captain Written Examinations and Development and Administration of Police Sergeant, Lieutenant and Captain Assessment Centers Contract Period March 2018 – March 2019 Contract Title Development and Administration of Police Sergeant, Lieutenant and Captain Written Examinations and Development and Administration of Police Sergeant, Lieutenant and Captain Assessment Centers Contract Period June 2016 – December 2016 Client City of Indianapolis, IN Contact (Police and Fire) Bruce Henry, IPMA-SCP IMPD Human Resource Address/Phone/E-mail 200 E. Washington Street – Ste. 1541 Indianapolis, IN 46204 175 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 154 Manager 317.327.2596 Bruce.Henry@indy.gov Contact (Fire) Kris Harrison Human Resources Manager Address/Phone/E-mail Indianapolis Fire Department 955 Fort Wayne Avenue Indianapolis, IN 46202 317.327.6775 Kristin.Harrison@indy.gov Contract Title Development and Administration of Police Sergeant, Lieutenant and Captain Written Examinations and Assessment Centers Contract Period February 2020 - Present Contract Title Development and Administration of Fire Lieutenant, Captain and Battalion Chief Written Examinations and Assessment Centers Contract Period March 2018 – October 2018 Contract Title Development and Administration of Police Sergeant, Lieutenant and Captain Written Examinations and Assessment Centers Contract Period January 2018 – August 2018 Client Jacksonville, FL Sheriff’s Office Contact Bart W. Laird, PHR, MSHRM Manager of Personnel Services Address/Phone/ E-mail Dept. of Personnel & Professional Services 501 E. Bay Street, Suite 303 Jacksonville, FL 32202 904.630.5899 Bart.Laird@jaxsheriff.org Contract Title Development and Administration of Police Sergeant and Lieutenant Written Examinations and Assessment Centers Contract Period February 2020 – Present Contract Title Development and Administration of Corrections Sergeant and Lieutenant Written Examinations and Assessment Centers Contract Period January 2018 – September 2019 Contract Title Development and Administration of Police Sergeant and Lieutenant Written Examinations and Assessment Centers Contract Period September 2017 – January 2019 Contract Title Development and Administration of Police Sergeant and Lieutenant Assessment Centers Contract Period April 2016 – January 2017 Client City of Las Vegas, NV Contact Jacob Tilford Human Resources Analyst Address/Phone/E-mail 500 N. Casino Center Blvd. Las Vegas, NV 89101 702.229.4879 jtilford@LasVegasNevada.gov Contract Title Development and Administration of a Fire Engineer Written Examination and Practical Examination Contract Period February 2019 – July 2019 Contract Title Development and Administration of Fire Captain and Battalion Chief Written Examinations and Assessment Centers Contract Period June 2018 – November 2018 Contract Title Development and Administration of EMS Field Coordinator and Fire Engineer Written Examinations and Development and Administration of Fire Engineer Practical Examination Contract Period January 2017 – January 2018 Contract Title Development and Administration of Fire Captain and Battalion Chief Written Examinations and Assessment Centers Contract Period May 2016 – November 2016 Contract Title Development and Administration of EMS Coordinator Written Examination Contract Period May 2015 – October 2015 176 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 155 Client Los Alamos, NM Contact Denise Cassel Employee Services Manager Address/Phone/E-mail Los Alamos County Human Resources 1000 Central Avenue – 2nd Floor Los Alamos, NM 87544 505.662.8047 denise.cassel@lacnm.us Contract Title Police and Fire Promotional Processes Contract Period 2010 – Present Description:  Conduct job analysis for 8 ranks: Driver Engineer, Captain (previously Company Officer), Division Chief (Admin)/Battalion Chief (Operations), Deputy Chief and Assistant Chief within the Fire Department and Sergeant, Lieutenant and Captain within the Police Department.  Develop written examinations, practical examinations, structured oral interviews, assessment centers and written exercises as follows: o Fire Driver Engineer: 11 cycles o Fire Captain: 7 cycles o Fire Division/Battalion Chief: 10 cycles o Fire Deputy Chief: 4 cycles o Fire Assistant Chief: 2 cycles o Fire Chief: 1 cycle o Police Sergeant: 6 cycles o Police Lieutenant: 2 cycles o Police Captain: 2 cycles Client Massachusetts State Police Department Contact Michelle Small Chief Administrative Officer Address/Phone/E-mail 470 Worcester Road Framingham, MA 01702 508.820.2366 michelle.small@state.ma.us Contract Title Development of Police Sergeant Written Exam Contract Period December 2019 - Present Contract Title Development of Police Sergeant Written Exam Contract Period January 2019 – June 2019 Contract Title Development of Police Lieutenant Written Exam and Structured Oral Interview Contract Period May 2018 – January 2019 Contract Title Development of Police Captain Written Exam and Structured Oral Interview Contract Period March 2017 – January 2018 Contract Title Development of Police Sergeant Written Exam Contract Period June 2016 - November 2016 Contract Title Development of Police Lieutenant Written Exam and Structured Oral Interview Contract Period December 2014 – August 2015 Contract Title Development of Police Sergeant Written Exam Contract Period January 2014 – June 2014 Contract Title Development of Police Captain Written Exam and Structured Oral Interview Contract Period April 2013 – January 2014 Contract Title Development of Police Lieutenant Written Exam and Structured Oral Interview Contract Period March 2012 – August 2012 Contract Title Development of Police Captain Written Exam Structured Oral Interview Contract Period September 2010 – April 2011 Contract Title Development of Police Lieutenant Written Exam and Oral Interview Contract Period August 2010 – February 2011 Contract Title Development of Police Sergeant Written Exam Contract Period December 2009 – April 2010 177 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 156 Contract Title Development of Police Captain Written Exam and Structured Oral Interview Contract Period March 2009 – November 2009 Contract Title Development of Police Lieutenant Written Exam and Structured Oral Interview Contract Period September 2007 – May 2008 Contract Title Development of Police Sergeant Written Exam Contract Period November 2006 – March 2007 Contract Title Development of Police Captain Written Exam and Structured Oral Interview Contract Period November 2005 – March 2006 Contract Title Development of Police Lieutenant Written Exam and Structured Oral Interview Contract Period September 2004 – March 2005 Contract Title Development of Police Sergeant Written Exam Contract Period May 2003 – January 2004 Minneapolis, MN Police Department Contact Bill Champa Human Resource Manager Address/Phone/E-mail 250 S. 4th Street – Room 100 Minneapolis, MN 55415 612.275.2039 Bill.Champa@minneapolismn.gov Contract Title Development and Administration of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period February 2020 - Present Contract Title Development and Administration of Police Lieutenant Assessment Center Contract Period April 2019 – August 2019 Contract Title Development and Administration of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period February 2018 – March 2019 Contract Title Development and Administration of Police Lieutenant Written Exam and Development and Administration of Police Lieutenant Assessment Center Contract Period March 2017 – October 2017 Contract Title Development and Administration of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period January 2016 – June 2016 Contract Title Development and Administration of Police Lieutenant Written Exam and Development and Administration of Police Lieutenant Assessment Center Contract Period March 2015 – August 2015 Contract Title Development and Administration of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period January 2014 – June 2014 Contract Title Development and Administration of Police Lieutenant Written Exam and Development and Administration of Police Lieutenant Assessment Center Contract Period January 2013 – November 2013 Contract Title Development and Administration of Police Sergeant Written Exam and Development and Administration of Police Sergeant Assessment Center Contract Period January 2011 – June 2011 (Reawarded for 2012-13) Contract Title Development and Administration of Police Lieutenant Written Exam and Development and Administration of Police Lieutenant Assessment Center Contract Period June 2010 – December 2010 Contract Title Police Sergeant and Lieutenant Job Analysis Contract Period January 2009 – July 2009 Contract Title Development and Administration of Police Captain Assessment Center Contract Period February 2008 – May 2008 Contract Title Development and Administration of Police Sergeant Assessment Center Contract Period March 2006 – June 2006 178 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 157 Contract Title Development and Administration of Police Lieutenant Assessment Center Contract Period August 2005 – October 2005 Contract Title Development and Administration of Police Sergeant Assessment Center Contract Period August 2001 – January 2002 Client Montgomery County, MD Department of Corrections Contact Jacquelyn D. Anderson, PHR Human Resource Specialist Address/Phone/ E-mail 101 Monroe Street – 7th Floor Rockville, MD 20850 240.777.5023 jacquelyn.anderson@montgomerycountymd.gov Contract Title Development of Sergeant and Lieutenant Written Exams and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period September 2019 – January 2020 Contract Title Development of Lieutenant Written Exam and Development and Administration of Lieutenant Assessment Center Contract Period March 2018 – December 2018 Contract Title Development of Sergeant and Lieutenant Written Exams and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period January 2017 – March 2018 Contract Title Development of Sergeant and Lieutenant Written Exams and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period June 2015 – February 2016 Contract Title Development of Sergeant and Lieutenant Written Exams and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period July 2013 – February 2014 Client Montgomery County, MD Fire Rescue Service Contact Jacquelyn D. Anderson, PHR Human Resource Specialist Address/Phone/ E-mail 101 Monroe Street – 7th Floor Rockville, MD 20850 240.777.5023 jacquelyn.anderson@montgomerycountymd.gov Contract Title Development and Administration of Fire Captain and Battalion Chief Assessment Centers Contract Period April 2019 – March 2020 Contract Title Development of Master Firefighter and Lieutenant Written Examinations and Development and Administration of Lieutenant Assessment Center Contract Period March 2018 – December 2018 Contract Title Development and Administration of Fire Captain and Battalion Chief Assessment Centers Contract Period February 2017 – February 2018 Contract Title Development of Master Firefighter and Lieutenant Written Examinations and Development and Administration of Lieutenant Assessment Center Contract Period July 2016 – January 2018 Contract Title Completed Assistant Chief Job Analysis Contract Period September 2015 – October 2016 Contract Title Development and Administration of Fire Captain and Battalion Chief Assessment Centers Contract Period April 2015 – March 2016 Contract Title Development of Master Firefighter and Lieutenant Written Examinations and Development and Administration of Lieutenant Assessment Center Contract Period July 2014 – December 2014 Contract Title Development and Administration of Fire Captain and Battalion Chief Assessment Centers Contract Period February 2013 – January 2014 179 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 158 Client Montgomery County, MD Police Department Contact Debra Marshall Human Resources Address/Phone/ E-mail 101 Monroe Street – 7th Floor Rockville, MD 20850 240.777.5042 debra.marshall@montgomerycountymd.gov Contract Title Development of Master Police Officer and Sergeant Written Exams and Development and Administration of Police Sergeant Assessment Center Contract Period February 2019 – November 2019 Contract Title Development of Master Police Officer and Sergeant Written Exams and Development and Administration of Police Sergeant Assessment Center Contract Period March 2017 – March 2018 Contract Title Completed Police Captain Job Analysis Contract Period May 2016 – August 2016 Contract Title Development of Master Police Officer and Sergeant Written Exams and Development and Administration of Police Sergeant Assessment Center Contract Period February 2015 – April 2016 Client Montgomery County, MD Sheriff’s Office Contact Debra Marshall Human Resources Address/Phone/ E-mail 101 Monroe Street – 7th Floor Rockville, MD 20850 240.777.5042 debra.marshall@montgomerycountymd.gov Contract Title Development of Sergeant and Lieutenant Written Examinations and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period January 2020 – Present Contract Title Development of Sergeant and Lieutenant Written Examinations and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period January 2018 – December 2018 Contract Title Development of Sergeant and Lieutenant Written Examinations and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period February 2016 – October 2016 Contract Title Development of Sergeant and Lieutenant Written Examinations and Development and Administration of Sergeant and Lieutenant Assessment Centers Contract Period January 2014 – January 2015 Client City of Portland, OR Contact Ryan Gillespie Division Chief Address/Phone/ E-mail 4800 NE 122nd Avenue Portland, OR 97230 503.758.2991 Ryan.Gillespie@portlandoregon.gov Contract Title Development and Administration of Fire Lieutenant Assessment Center Contract Period January 2020 – Present Contract Title Development and Administration of Fire Captain Assessment Center Contract Period January 2019 – November 2019 Contract Title Development and Administration of Fire Battalion Chief Assessment Center Contract Period July 2018 – July 2019 Client City of Raleigh, NC Contact Timothy Tomczak Captain Address/Phone/E-mail 6716 Six Forks Road Raleigh, NC 27615 919.996.1231 Timothy.Tomczak@raleighnc.gov Contract Title Development and Administration of Fire Lieutenant Assessment Center 180 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 159 Contract Period January 2020 – Present Contract Title Development and Administration of Fire Captain Assessment Center Contract Period January 2019 – November 2019 Contract Title Development and Administration of Fire Battalion Chief Assessment Center Contract Period July 2018 – July 2019 Client City of San Antonio, TX Contact Sarah Bilger, PHR Senior HR Analyst Address/Phone/E-mail Department of Human Resources 111 Soledad, Suite 100 San Antonio, TX 78205 210.207.8719 Sarah.Bilger@sanantonio.gov Contract Title Development of Police Detective and Sergeant Written Examinations Contract Period February 2020 - Present Contract Title Development of Police Detective, Lieutenant and Captain Written Examinations and Development and Administration of Police Lieutenant and Captain Assessment Centers Contract Period August 2018 – March 2019 Contract Title Development of Police Detective and Sergeant Written Examinations Contract Period November 2017 – February 2018 Contract Title Development of Police Sergeant, Lieutenant and Captain Written Examinations and Development and Administration of Police Lieutenant and Captain Assessment Centers Contract Period January 2017 – October 2017 Contract Title Development of Police Detective and Sergeant Written Examinations Contract Period July 2016 – January 2017 Contract Title Development of Police Detective, Sergeant, Lieutenant and Captain Written Examinations and Development and Administration of Police Lieutenant and Captain Assessment Centers Contract Period January 2015 – December 2015 Contract Title Development of Police Detective and Sergeant Written Examinations Contract Period April 2014 – September 2014 Contract Title Development of Police Detective, Sergeant, Lieutenant and Captain Written Examinations and Development and Administration of Police Lieutenant and Captain Assessment Centers Contract Period June 2012 – September 2013 Client San Diego, CA Police Department Contact Salvador Gonzalez Sr. Personnel Analyst Address/Phone/E-mail 1200 Third Ave - Suite 300 San Diego, CA 92101 619.236.7734 SalvadorG@sandiego.gov Contract Title Development of Sergeant, Lieutenant, Detective and POIII Exams and Structured Oral Assessments Contract Period November 2019 - Present Description:  Completed a job analysis updates for the ranks of Sergeant, Lieutenant, Detective and POIII  Developed written exams for all ranks  Delivered a candidate orientation presentation  Developed structured oral assessments for all ranks Contract Title Development of Sergeant, Lieutenant, Detective and POIII Exams and Structured Oral Assessments Contract Period December 2017 – July 2018 Description:  Completed a job analysis for the ranks of Sergeant, Lieutenant, Detective and POIII  Developed written exams for all ranks  Delivered a candidate orientation presentation 181 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 160  Developed structured oral assessments for all ranks Contract Title Development of Sergeant, Lieutenant, Detective and POIII Exams and Structured Oral Assessments Contract Period January 2016 – June 2016 Description:  Developed written exams for Sergeant, Lieutenant, Detective and POIII  Delivered a candidate orientation presentation  Developed structured oral assessments for all ranks Contract Title Development of Sergeant, Lieutenant, Detective and POIII Exams and Structured Oral Assessments Contract Period December 2013 – June 2014 Description:  Developed written exams for Sergeant, Lieutenant, Detective and POIII  Delivered a candidate orientation presentation  Developed structured oral assessments for all ranks  Trained 20 assessors and administered assessment centers for 117 Sergeant, 47 Lieutenant, 87 Detective and 4 POIII candidates. Contract Title Development of Sergeant, Lieutenant, Detective and POIII Exams and Structured Oral Assessments Contract Period January 2012 – May 2012 Description:  Completed a job analysis for the ranks of Sergeant, Lieutenant, Detective and POIII  Developed written exams for all ranks  Delivered a candidate orientation presentation  Developed structured oral assessments for all ranks  Trained 20 assessors and administered assessment centers for 117 Sergeant, 44 Lieutenant, 80 Detective and 4 POIII candidates. Contract Title Development of Sergeant and Lieutenant Exams and Structured Oral Assessments Contract Period October 2009 – February 2010 Description:  Developed written exams for sergeant and lieutenant  Delivered a candidate orientation presentation  Developed sergeant and lieutenant structured oral assessments  Trained 24 assessors and administered assessment centers for 116 sergeant and 45 lieutenant candidates Contract Title Development of Sergeant and Lieutenant Exams and Assessment Centers Contract Period April 2007 – December 2007 Description:  Developed written exams for sergeant and lieutenant  Delivered a candidate orientation presentation  Developed sergeant and lieutenant assessment centers  Trained 25 assessors and administered assessment centers for 125 sergeant and 50 lieutenant candidates No post-test litigation or complaints filed. Client City of San Francisco, CA Contact Dave Johnson Public Safety Examination Manager Address/Phone/E-mail Department of Human Resources One South Van Ness Ave - 4th Floor San Francisco, CA 94103 415.557.4871 dave.johnson@sfgov.org Contract Title Development and Administration of Police Captain Assessment Center Contract Period February 2020 - Present Contract Title Development and Administration of Police Lieutenant Assessment Center Contract Period July 2019 – February 2020 182 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 161 Contract Title Development and Administration of Fire Lieutenant and Battalion Chief Assessment Centers Contract Period August 2016 - March 2018 Contract Title Development and Administration of Police Lieutenant Assessment Center Contract Period November 2015 – January 2017 Client City of St. Paul, MN Contact Dan Ferralez Human Resources Consultant Address/Phone/ E-mail Office of Human Resources 25 West 4th Street – 200 City Hall Annex Saint Paul, MN 55102 651.266.6596 dan.ferralez@ci.stpaul.mn.us Contract Title Development and Administration of Police Commander Assessment Center Contract Period December 2019 - Present Contract Title Development and Administration of Police Sergeant Written Examination and Assessment Center Contract Period May 2019 – February 2020 Contract Title Development and Administration of Police Sergeant Written Examination and Assessment Center Contract Period September 2017 – February 2018 Contract Title Development and Administration of Police Commander Assessment Center Contract Period July 2017 – January 2018 Contract Title Development and Administration of Police Sergeant Written Examination and Assessment Center Contract Period March 2016 – September 2016 Contract Title Development and Administration of Police Commander Assessment Center Contract Period June 2015 – December 2015 Contract Title Development and Administration of Police Sergeant Written Examination and Assessment Center Contract Period December 2012 – June 2014 Client City of Tulsa, OK Contact Marty Drobinko Testing Coordinator Address/Phone/E-mail 175 East 2nd Street – Suite 1450 Tulsa, OK 74103 918.596.1069 MDROBINKO@cityoftulsa.org Contract Title Fire Equipment Operator, Captain, District Chief, Assistant Chief and Deputy Chief Contract Period 2017 - Present Description:  Developed and administered written exams for Fire Equipment Operator  Developed and administered written exams and assessment centers for Fire Captain  Developed and administered written exams and assessment centers for District Chief  Developed and administered assessment center for Deputy Chief Contract Title Police Sergeant, Corporal, Captain and Major Contract Period 2011 - Present Description:  Developed and administered written exam and assessment center for Police Corporal  Developed and administered written exam and assessment center for Police Sergeant  Developed and administered written exam and assessment center for Police Lieutenant  Developed and administered written exam and assessment center for Police Captain  Developed and administered assessment center for Police Major  Developed and administered assessment center for Police Deputy Chief 183 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 162 Client City of Virginia Beach, VA Contact Miriam K. Bryant Human Resources Coordinator Address/Phone/E-mail Municipal Center Building 18 – Rm 134 2424 Courthouse Drive Virginia Beach, VA 23456 757.385.4663 mbryant@vbgov.com Contract Title Development and administration of Master Police Officer Written Examination Contract Period October 2019 – January 2020 Contract Title Development and administration of Police Sergeant Written Examination and Development and Administration of Police Sergeant, Lieutenant and Captain Assessment Centers Contract Period February 2019 – October 2019 Contract Title Development and administration of Master Police Officer Written Examination Contract Period October 2018 – February 2019 Contract Title Development and administration of Master Police Officer Written Examination Contract Period July 2017 – December 2017 Contract Title Development and administration of Police Lieutenant Written Examination and Assessment Center Contract Period February 2017 – October 2017 Contract Title Development and administration of Fire Captain and a Fire Battalion Chief Promotional Processes Contract Period January 2017 – October 2017 Description:  Developed written exam for Fire Captain  Developed and administered assessment centers for both ranks Contract Title Development and administration of Master Police Officer Written Examination Contract Period September 2016 – December 2016 Contract Title Development and administration of Police Sergeant Written Examination and Assessment Center Contract Period July 2016 – February 2017 Contract Title Development and administration of Fire Captain and a Fire Battalion Chief Promotional Processes Contract Period March 2016 – December 2016 Description:  Developed written exam for Fire Captain  Developed and administered assessment centers for both ranks Contract Title Development and administration of Master Police Officer Written Examination Contract Period September 2015 – December 2015 Contract Title Development and administration of Police Lieutenant Assessment Center Contract Period June 2015 – December 2015 Contract Title Development and administration of Fire Captain and a Fire Battalion Chief Promotional Processes Contract Period January 2015 – December 2015 Description:  Developed written exam for Fire Captain  Developed and administered assessment centers for both ranks Contract Title Development and administration of Police Sergeant Written Examination and Assessment Center Contract Period July 2014 – June 2015 Contract Title Development and administration of Fire Captain Promotional Process Contract Period January 2014 – September 2014 Description:  Developed written exam for Fire Captain  Developed and administered an assessment center 184 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 163 Contract Title Development and administration of Master Police Officer Written Examination Contract Period October 2013 – April 2014 Contract Title Development and administration of Fire Captain and a Fire Battalion Chief Promotional Processes Contract Period August 2013 – December 2013 Description:  Developed written exam for Fire Captain  Developed and administered assessment centers for both ranks Contract Title Development and administration of Police Lieutenant Assessment Center Contract Period May 2013 – November 2013 Contract Title Development and administration of Police Sergeant Written Examination and Assessment Center Contract Period December 2012 – May 2013 Contract Title Development and administration of Fire Captain and a Fire Battalion Chief Promotional Processes Contract Period August 2012 – February 2013 Description:  Conducted job analysis for the rank of Battalion Chief and linkage analysis for Captain  Developed written exam for Fire Captain  Developed and administered assessment centers for both ranks Law Enforcement and Fire Service Entry-Level Selection Processes Client Bridgeport, CT Police and Fire Departments Contact Lisa M. Mastronunzio, SHRM-SCP A/Personnel Assistant II Address/Phone/E-mail Civil Service Commission City of Bridgeport 45 Lyon Terrace Bridgeport, CT 06604 203.576.7109 lisa.mastronunzio@bridgeportct.gov Contract Title Development, validation and administration of entry-level written examinations for both the Police Officer and Firefighter positions Contract Period 2010 – Present Description:  Worked with members of the Civil Service Commission to provide examination services as well as a structured oral interview process for police officers and firefighters.  Conducted onsite orientations for candidates and collected data for validation studies.  Trained the supervisors to provide the data on job incumbents.  Developed a structured oral interview and conducted a large-scale training session for the assessors in this process.  Assisted the department in the mandated candidate challenge process.  Delivered a candidate orientation presentation Client City of Chicago, IL Contact Jill May, M.S. Testing Specialist Address/Phone/ E-mail 121 N. LaSalle, 1102 Chicago, IL 60602 312.744.1240 Jill.May@cityofchicago.org Contract Title Entry-level Police Officer Exam Contract Period January 2020 – February 2020 Contract Title Entry-level Police Officer Exam Contract Period June 2019 – August 2019 185 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 164 Contract Title Entry-level Police Officer Exam Contract Period April 2018 - June 2018 Contract Title Entry-level Police Officer Exam Contract Period October 2017 – January 2018 Contract Title Entry-level Police Officer Exam Contract Period February 2017 – May 2017 Contract Title Entry-level Police Officer Exam Contract Period November 2015 – May 2016 Contract Title Entry-level Firefighter Exam Contract Period October 2014 – March 2015 Contract Title Entry-level Police Officer Exam Contract Period November 2013 – February 2014 Contract Title Entry-level Firefighter Exam Contract Period January 2006 – December 2006 (Military make-up examination sessions in 2009 and 2010), December 2014 Description:  Develop and validate a cognitive/personality written exam  Administer exam to over 24,000 candidates  Work with City to establish optimal method of test use Process resulted in no adverse impact against protected classes – no ensuing litigation Client City of Indianapolis, IN Contact Bruce Henry, IPMA-SCP IMPD Human Resource Manager Address/Phone/E-mail 200 E. Washington Street – Ste. 1541 Indianapolis, IN 46204 317.327.2596 Bruce.Henry@indy.gov Contract Title Entry-level Firefighter Exam Process Contract Period November 2019 - Present Contract Title Entry-level Police Officer Exam Process Contract Period August 2019 Contract Title Entry-level Police Officer Exam Process Contract Period February 2019 – April 2019 Contract Title Entry-level Police Officer Exam Process Contract Period September 2018 – December 2018 Contract Title Entry-level Firefighter Exam Process Contract Period July 2018 – December 2018 Contract Title Entry-level Police Officer Exam Process Contract Period June 2017 – August 2018 Description:  Conducted a job analysis of the police officer position  Developed a structured oral interview for the police officer position  Administered both the written examination and the structured interview Contract Title Entry-level Firefighter Exam Administration and Structured Oral Interview Development and Administration Contract Period June 2016 – December 2016 Contract Title Entry-level Police Officer Exam Process Contract Period March 2016 – June 2016 Description:  Conducted a validity transportability study for the police officer entrance examination  Administered the written examination to over 400 candidates Contract Title Entry-level Firefighter Exam Administration and Structured Oral Interview Development and Administration Contract Period February 2015 – March 2015 186 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 165 Contract Title Entry-level Police Officer Exam Administration Contract Period October 2014 Contract Title Entry-level Police Officer Exam Administration and Structured Oral Assessment Development and Administration Contract Period March 2014 – June 2014 Contract Title Entry-level Police Officer Exam Administration and Structured Oral Assessment Development and Administration Contract Period October 2013 – January 2014 Contract Title Entry-level Firefighter Exam Process Contract Period June 2011 – June 2012 Description:  Conducted a job analysis of the Firefighter position  Conducted a validity transportability study for the firefighter entrance examination  Developed a structured oral interview for the firefighter position  Administered both the written examination and the structured interview to over 800 candidates Client New Jersey Department of Personnel (NJ DOP) Contact Michael M. Szuter Fire Testing Supervisor Address/Phone/E-mail NJ Civil Service Commission Trenton, NJ 08625 609.292.4735 michael.szuter@csc.state.nj.us Contract Title Entry-Level Civil Service Firefighter Written Examinations Contract Period 2014 - 2018 Description:  Conducted a Statewide job analysis of firefighter and developed entry-level exams to be used for Statewide Civil Service Selection  Administration to over 14,000 candidates resulted in NO adverse impact against protected classes of individuals. Contract Title Entry-level Civil Service Law Enforcement Officer Written Exams Promotional Civil Service Correctional Officer Exams and Assessments Contract Period January 2004 – 2008 Description:  Conducted a Statewide job analysis of law enforcement positions and developed entry-level exams to be used for Statewide Civil Service Selection  Administration to over 20,000 candidates resulted in NO adverse impact against protected classes of individuals. Client City of Pittsburgh, PA Contact Paula Kellerman Acting Assistant Director Address/Phone/E-mail 414 Grant Street Pittsburgh, PA 15219 paula.kellerman@pittsburghpa.gov Contract Title Entry-level Firefighter Exam Administration Contract Period 2014, 2016, 2018 Contract Title Entry-level Firefighter Exam Contract Period October 2012 – February 2014 Description:  Conducted a job analysis of the Firefighter position  Conducted a validity transportability study for the firefighter entrance examination  Conducted a criterion-related validation study for the firefighter entrance examination  Developed candidate training curriculum  Implemented process and worked through federal oversight of entire validation and administration Contract Title Entry-level Firefighter Exam 187 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 166 Contract Period October 2011 – July 2012 Description:  Conducted a job analysis of the Firefighter position  Conducted a validity transportability study for the firefighter entrance examination  Conducted a criterion-related validation study for the firefighter entrance examination  Developed candidate training curriculum  Implemented process and worked through federal oversight of entire validation and administration Client City of Virginia Beach, VA Contact Monica Kopin Staffing and Compensation Coordinator Address/Phone/E-mail 2424 Courthouse Drive Virginia Beach, VA 23456 mkopin@vbgov.com Contract Title Validation and Implementation of Firefighter Entrance Examination and Development and Administration of Firefighter Structured Oral Interview Contract Period 2010 - 2018 911 Operations Client City of Chicago, IL Contact Jill May, M.S. Testing Specialist Address/Phone/E-mail 121 N. LaSalle, 1102 Chicago, IL 60602 312.744.1240 Jill.May@cityofchicago.org Contract Title Fire Communications Officer I & Police Communications Officer I and II Contract Period 2008 – 2014 Description:  Implemented the National Public Safety Dispatcher Selection Inventory (NPSDSI) and developed a custom assessment of oral communication clarity.  Administered the exams for each cycle of candidates (up to 150 candidates per cycle) No post-test litigation or complaints filed. 188 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 167 Scope of Services ndustrial/Organizational Solutions, Inc. (IOS) is honored to have the opportunity to respond to your Request for Proposals. This section of the proposal will detail our plan to provide a promotional process for the rank of Police Sergeant for the City of South Miami Police Department. Kick-off Meeting We enter into a project with an expectation of implementing a specific best-practice methodology; however, we understand that every client has unique constraints and conditions that must be carefully considered in determining the proper assessment process design. Local laws, rules and procedures may cause us to alter our course. Also, there are often conventions that affect testing practices that cannot be easily changed or adapted such as Civil Service rules, collective bargaining agreements, etc. This is why we place a high value on conducting an initial kick-off meeting to learn about our client’s particular situation and to consider the specific goals and expectations. This meeting is critical to ensure that we deliver a successful and beneficial service. The Project Director will first meet with your project manager and other designated personnel to discuss and determine specific objectives, personnel roles and project timelines. Job Analysis Job analysis is the thorough investigation of the elements of a job. Generally, a job analysis is more extensive than a job description, which seeks to describe a job in summary/highlight form. The job analysis seeks to identify all job tasks that make up a job and the knowledge, skills abilities and personality characteristics (KSAPs) that are required to perform such job tasks. The job analysis is the foundation piece of a selection process because it serves to define the knowledge, skills and abilities that can be assessed by selection tools. In short, the job analysis is the basis by which we can demonstrate evidence of content validity. A job analysis does not need to be performed each time a selection process is developed. For this reason, it is useful to review the job analysis and determine its age and relevance. Generally, if the job analysis is less than five years old, it is reasonable to conduct a review of the essential tasks, knowledge areas, skills and ability to ensure they are still current and appropriate. If so, the existing job analysis can be used to develop selection tools and IOS will carefully document the job analysis review process to create a record of the review and updating process. To begin the job analysis, IOS will interview a stratified random sample of job incumbents for the target position. The sample will be stratified to ensure that a representative number of I 189 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 168 minorities and females are interviewed. Incumbents will be questioned about critical tasks, daily routines, essential job knowledge, critical skills and abilities, job challenges, critical job incidents, etc. In addition to interviewing incumbents, we will observe them doing their jobs. IOS will also interview a sample of supervisors to gain an additional perspective concerning job duties and responsibilities. Based on information obtained during the interviews/observations, IOS will develop a comprehensive job analysis questionnaire (JAQ). The JAQ contains a list of all potential tasks and KSAPs that are associated with the job. This questionnaire will be completed by a sample of incumbents to gain information about the nature of the job. Incumbents will rate each task or KSAP as to its importance (how important successful completion of the task is to the job) and frequency (how often the task is performed). Specific scales will be used to make these ratings. Prior to its administration, IOS will request that a group of subject matter experts (SMEs) who supervise the target position review the JAQ to ensure that it accounts for all job tasks and KSAPs, uses familiar language and is organized in a meaningful way. After obtaining feedback and making any necessary revisions, IOS will coordinate the administration of the JAQ. The JAQ will be administered to a large sample of incumbents. Incumbents will receive the JAQ along with a computer-scannable answer sheet on which ratings will be collected. We will ask the department to coordinate the completion of the JAQs and direct incumbents to turn the JAQ into a supervisor within approximately two weeks. At the end of this period, all JAQs will be returned to IOS for analysis. JAQ data gathered from incumbents will be analyzed to identify those tasks and KSAPs that are essential to the target job. This will be done by combining the “importance” and “frequency” ratings. Generally, the ratings given to any task or KSAP will be weighted using the following formula: .666(importance rating) + .333(frequency rating). This formula will yield an “essentiality” rating. All those tasks and KSAPs with a high enough essentiality rating will be considered essential job tasks and will be used to define the target job. Using the list of essential job tasks and KSAPs, IOS will engage a process to link essential tasks to the KSAPs that are required to accomplish these tasks. This linkage process will ensure that the job tasks are related to specific, measurable knowledge areas and KSAPs that can be assessed using conventional tests and assessments. Essential KSAPs will later be used as the basis for developing written exams, structured assessments, assessment centers or other types of selection tools. The KSAPs will define the measurable dimensions of each selection tool. The job analysis process will yield a comprehensive list of essential job tasks and KSAPs. A report will be completed to outline the methodology used during the job analysis and to summarize the outcomes and ratings of the job analysis process. This report will document the following methodological elements:  Job observation/interview sampling and outcomes  Job analysis questionnaire development and review 190 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 169  Job analysis questionnaire sampling, distribution and analysis  Task-KSAP linkage  Essential task and KSAP results Promotional Job Knowledge Examination Development and Validation The promotional job knowledge examination is a written tool that is specifically designed to assess essential, job-related knowledge. To this end, exam questions will be drawn from relevant, job-related sources of knowledge (i.e., text books, policy, rules, law, etc.). The job knowledge exam will be supported by content validity evidence. This evidence will be established through a thorough linkage of essential knowledge areas identified in the job analysis to the measureable dimensions of the exam, and through subject matter expert involvement in the development of the exam plan and review of exam items. Exam Plan and Reference List Development The first step in the development of a written job knowledge exam is the identification of the knowledge areas that should be assessed by the exam and the source, or reference materials from which this knowledge can be drawn. An IOS consultant will review the target job description/job analysis to identify those knowledge areas that should be assessed by the written exam. IOS will provide a recommendation as to the knowledge areas that should be assessed. At this point, IOS can work with department subject matter experts to identify appropriate sources to assess the suggested knowledge areas, or the department can independently recommend sources that assess the suggested knowledge areas. The following tables identify common knowledge areas that are assessed by public safety written examinations: General Category Law Enforcement Exams Technical Knowledge Field Operations Criminal Investigation CompSTAT Community Policing Ethics in Policing Multicultural Law Enforcement Management Knowledge Supervision Administration Leadership/Motivation Internal Sources Policy, Procedure, or Operating Guidelines Collective bargaining agreement State Law 191 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 170 Once sources are selected from which to draft examination questions, it is necessary to create an exam plan that identifies the percentage of items that will be drawn from each source/knowledge area, the specific content that will be focused on, and the information that will be provided to candidates in the reference/study list. A subject matter expert (SME) meeting will be arranged to review these recommendations and finalize the selection of knowledge areas and sources/references. During this Exam Plan Development Meeting, SMEs will be asked to assist in identifying the relative importance of the knowledge areas that will be assessed by the exam. These ratings will be used to establish the number of items that will be drafted to assess specific knowledge areas. In addition, SMEs will review reference/source materials and assist in identifying the chapters and sections that are most relevant for the target position. This step will ensure the test is as job-related as possible and will allow candidates to focus specifically on essential information contained in study materials. In short, this meeting will allow department subject matter experts to ensure that the exam plan and the candidate study/reference list meet their expectations. Following the Exam Plan Development Meeting, IOS will generate the final exam study list that will contain a list of sources that will be used to create the exam and the specific chapters/sections that candidates should review. This list will be submitted to the department for distribution to eligible candidates. IOS will also produce a candidate preparation document that provides helpful strategies for study and preparing for the written examination. This document can be distributed to candidates or posted on the agency website. *Note: The department may wish to develop the reference list on their own. In this case, IOS will be charged with ensuring that essential knowledge areas are being assessed via the chosen reference list. IOS may also still need to work with SMEs to identify the specific chapters and sections of source material from which to draw exam items. Question Development and Review IOS technical writers have advanced degrees in writing and are extensively trained, mentored and led by Ms. Karen Steiger, M.S., our Senior Technical Writer/Manager of Technical Writing, who has been with IOS for over a decade and has devoted her professional career to writing public safety examinations that measure relevant job-related content areas. Ms. Steiger has personally created over 200 promotional examinations for public safety agencies and has overseen the development of countless others developed by her staff. Further, our technical writers are responsible for responding to candidate challenges and upholding the reliability of the exams. For anyone who has watched candidates sit through a 2 ½ hour long, 100 item, written exam and worked through a challenge process and air their concerns regarding each specific item, it becomes abundantly clear that the critical role of written exam item development is the backbone of a successful and fair promotional process. Mr. Legel’s insistence on maintaining a highly educated and experienced team of technical exam writers is why IOS has an unblemished litigation history. Using the detailed examination plan that was developed, IOS team of highly-trained technical writers will draft exam questions from the identified source materials. Our technical writers take particular care in identifying the most important topics and concepts from source material 192 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 171 and drafting questions that are highly job-related. In addition, the technical writers will draft questions of varying degrees of difficulty to ensure that the test is able to discriminate between candidates’ levels of job knowledge. Approximately 140 questions will be drafted based on the examination plan. These questions will later be reviewed and distilled down to the best 110 questions. Each question will be drafted in multiple-choice format. Once test questions are developed, an IOS consultant will review each item to ensure that it is job-related, in-line with the exam plan and is free from any apparent bias. The development of good test items is a critical element to the success of any job-knowledge exam. For this reason, we take care to make sure the following criteria are met:  Focus on essential information: Our item writers rely on the job analysis information gathered by our consultants and base the content of an exam on the test blueprint (or exam plan) that has been developed with SME input and I/O psychology best practices. The item writing team combs selected sources to identify truly job-related, important information from which to draft questions. We try to focus on “action,” when possible (i.e., what should an employee do in a given situation, based on policies and procedures, state and federal law, and current management theory).  Multiple-choice test items: “Scan-form”-style testing is efficient and easy to score, but it has a potential weakness – if candidates can identify the correct response through deductive reasoning, then they are demonstrating deductive reasoning ability rather than demonstrating the desired knowledge. This is why it is critical that answer options be well constructed and that there are enough answer options such that the candidate does not a stand a reasonable chance of guessing the right answer. o True/false formats are simply insufficient. o All tests should have at least four answer options. We create variety by developing both fill-in-the-blank items and “situational” items (or vignettes), asking candidates to put themselves in the position of the employees described in the question.  Appropriate difficulty: We ensure that questions developed for a job-knowledge exam are of the appropriate difficulty for the rank or position being tested. Our item writers carefully search through source material for the information most directly related to the job in question. o We avoid “tricky” items that would confuse those who are minimally qualified for the position or rank in question. o After the items have been developed, we rely on SME input when possible to determine that the content of the items is appropriate for the group of candidates being tested. o Another way that we ensure appropriate difficulty is to ensure that answer options are similar to each other in style, structure and length.  One correct response: We carefully craft questions such that there is one correct response and that the remaining responses are incorrect. We consider accuracy to be 193 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 172 absolutely crucial in our job-knowledge exams and thoroughly review items both internally (and with the client, when possible).  Avoidance of negatively worded stems/response options: Negatively worded questions and answer options are confusing. In order to ensure that we are measuring knowledge in a straightforward manner, we try to avoid the use of negatively worded items.  Avoidance of overly complex stems/response options: Again, we want questions to be simple and easy to understand. For this reason, we avoid confusing or overly lengthy questions while ensuring that the item remains accurate. Our item writers are trained to create clear and focused question stems and distracters.  Avoidance of “all of the above” or “both c and d”-style answer options: This style of questioning is generally overused and tends to reduce the psychometric quality of a test.  Cultural and gender sensitivity: Our item writers are trained to use appropriate terminology and language when referring to members of any protected group, referring to their issues and beliefs with respect. When sensitive or controversial subjects must be discussed in our exams, these topics are described in an objective manner. We take care to ensure that personnel are described in gender-neutral, ethnically diverse terms IOS will work with the department to assemble an SME panel to review test questions. The purpose of this review is to ensure that test questions are of high quality and to collect SME rating to support the validity of the test and to enable the determination of an a ppropriate test cut-off score. SMEs will be asked to review each item and offer any pertinent feedback/suggestions. In addition, SMEs will provide a Content Validity Ratio (CVR) Rating and an Angoff Rating. The CVR rating will provide evidence of the degree to which the content of the test mirrors critical content of the job. The Angoff Rating will identify the difficulty of each test item, thus indicating the overall difficulty of the exam. The Angoff process allows for the selection of a cut-off score that equates to minimally qualified levels of job knowledge. While this review is optional, the data collected during this process is highly valuable to the defensibility of the written exam. We recognize that agencies are often required to use a pre-determined cut-off score. In these cases, the Angoff process is still appropriate but must be used in a different manner. As a matter of legal defensibility, it is necessary to demonstrate that a cut-off score, even if mandated, equates to minimally qualified job knowledge levels. In a case where an agency is required to use a 70 percent cut-off score, for example, IOS will work toward selecting test items whose composite Angoff score is approximately 70 percent. In this manner, the cut-off score will be in-line with the test’s difficulty level. Data from the item review meeting will be used to select the 110 test questions that will be included in the written exam. These questions will be professionally compiled into a test booklet. Questions from the same source material will be grouped together under a title 194 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 173 heading such that candidates are aware of where the item was drafted from as they take the test. IOS will produce all necessary test booklets and scannable answer sheets. Exam Scoring, Results Generation and Analysis IOS will create an exam specification for the written exam that identifies the correct answer for each item, the source citation from which the question and correct answer are drawn and the knowledge area that the question assesses. This document will be used to generate a computer-based scoring program that will be used to score the exam. Each completed exam answer sheet will be scanned and processed by an optically scoring machine. Data from this machine will be manually reviewed for accuracy and scored using a computer-generated scoring program. The project consultant will conduct an item analysis process to ensure that each item of the test functions in a reasonable manner. Specifically, IOS will investigate the following:  Adverse impact ratio  Individual item data: item-total correlations, mean (p-value), standard deviation, frequency analysis to check for item with multiple correct responses, etc. If any items appear to be problematic (a large proportion of candidates get the item incorrect or there appear to be multiple correct responses based on candidate data), IOS will engage a discussion with the department and make a recommendation for resolving any issues. Final test scores will then be calculated. Exam results will be presented in a database format such as Microsoft Excel® and submitted to the department per the department’s specifications. Test Challenge/Appeal Process It is commonplace to conclude an examination by allowing candidates to participate in a challenge/appeal process where candidates can scrutinize test items and draft appeals to the validity or accuracy of items. IOS generally handles appeals processes in one of two ways. The first option is preferable, but we work with the department to determine the best solution based on standard procedures, convention, circumstances, etc. Following are the two options for challenge processes: 1. Immediately following the administration of the written exam, candidates are provided with an answer key, their test booklet and a copy of their answer sheet. Candidates are allowed to manually score their exam while being proctored by an IOS representative. Candidates are instructed that this score is preliminary and that tests will be carefully scored at a later time. Following this scoring period, candidates are instructed that they will be afforded the opportunity to file appeals to test items if they so choose. As candidates have been able to review the questions (in the test booklet) and make a determination of which items they answered correctly, they can reasonably determine whether or not they wish to file any appeals at that time. Candidates will be allow a fixed period of time (usually one hour) to draft written appeal on appeal sheets provided by IOS. Candidates will be allowed access to the source materials as well as an exam specification that indicated a citation for each test item. IOS will collect all appeals and 195 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 174 prepare a written response to these appeals. Appeal responses will be turned over to the agency with a recommended action for each item. IOS will engage a discussion with the agency to determine if any changes should be made to the scoring of test items. Once a determination is made, final scores will be calculated and submitted to the agency. 2. Within a few days of the test, candidates will be allotted a structured time period to review the test booklet, copy of their answer sheet, exam specification, answer key and test source materials. Candidates will be allowed to draft appeals which will be submitted, reviewed and responded to in the manner described above. The agency will likely be responsible for monitoring candidates during these structured time periods. Assessment Center Development and Validation An IOS assessment center is a battery of job simulations/exercises geared toward measuring essential management and supervisory skills of promotional candidates. Common exercises include one-on-one role-plays, group presentations, tactical simulations, in-basket/in-box simulations, and written problem solving exercises. An IOS assessment center exercise puts the promotional candidate in a situation that is similar to a function he/she will perform on the job. Assessors, who all have industry-specific management experience, rate the candidate on essential management and supervisory skills and abilities that will determine the candidate’s success on the job. The assessors use specific rating criteria that were developed by experts in the industry. The result of the IOS assessment center is a list of candidates in rank order according to who is most likely to perform at the highest level in the target position. The assessment process also yields an abundance of feedback from the assessors about candidate performance that can be used for developmental purposes. A comprehensive IOS assessment center will incorporate the following components:  Assessors who are experts in their industry and are specifically trained to assess candidate performance  Identification of specific skill and ability dimensions with associated behavioral criteria  A standardized administration schedule  Relevant, simulation-style exercises IOS’ assessment centers are simply the best way to predict how candidates will react to/handle specific job situations. The IOS assessment center method also allows for a broad range of skill assessments. We are able to use a variety of simulations that assess many different skills and abilities. This is a desirable feature as your goal in promotions is to select the most well- rounded candidate. Candidates are also more prone to feel that IOS assessment centers are fair evaluations of their promotional suitability as it is readily apparent that the assessment center is a valid and comprehensive way of making a promotional decision. Finally, an IOS assessment center is also a developmental tool. The results of an assessment center can be tailored to assist an individual candidate toward improving performance in a specific area. Data from a large number of candidates can also be used to help agencies identify training needs. 196 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 175 The assessment center will be supported by content validity evidence. This evidence will be established through the clear linkage of the job description/job analysis to the skills and abilities that will be assessed in the assessment center and by the linkage of essential job tasks to the simulation exercises that are selected. Furthermore, SMEs will assist in reviewing each exercise and rating criteria to ensure local validity. Exercise Selection and Development IOS will first work to identify the skills and abilities that should be assessed by the assessment center by reviewing a current job description/job analysis for the target position. Based on this review, IOS will provide a recommendation concerning the skills and abilities that should be assessed and the types of exercises that should be used to assess these skills and abilities. In order to refine this recommendation, a consultant will meet with department subject matter experts (SMEs) who hold the target rank and/or supervisor the target rank to identify job duties and critical scenarios that would constitute valuable assessment exercises. IOS will use these ideas and suggestions to craft exercises that are locally specific. SMEs will also review skills and abilities that will be assessed to ensure that the assessment is comprehensive and addresses any concerns of the department. While the specific content of the assessment center will be determined during the course of SME meeting, the following tables provide information about the types of exercises that can be created and common skills and abilities that are assessed by our assessment centers. IOS will incorporate the structured oral interview as an exercise in the assessment center. One-on-One Role-Play Scenarios Exercise Subordinate counseling (problem employee) Description The candidate plays the role of a supervisor who must meet with a subordinate employee (role- player) that has one of the following types of problems: a performance deficiency, a recent conflict with a citizen, personal problems or difficulty getting along with coworkers. The candidate will be given introductory information about the subordinate and the problem and will be allowed a brief period of time to prepare for the meeting. The candidate will then meet with the subordinate and attempt to deal with the issue at hand to the best of his/her ability. The candidate will likely meet resistance from the subordinate that will require him/her to exercise supervisory skills. The successful candidate will mitigate the problem and devise an appropriate solution. This exercise can be coupled with the citizen meeting exercise. This paring would require the candidate to gather specific information about a citizen conflict from the citizen and then use this information to deal effectively with the subordinate. Themes This exercise can deal with counseling subordinates on any of the following issues: performance deficiencies, a run-in with a citizen, employee dislike his/her job, depression, marital problems, drug/alcohol issues, or inability to get along with coworkers. Time There is a 10 to 15 minute preparation period prior to the 15 to 20 minute exercise. Resources This exercise requires one skilled role-player/actor to play the role of the subordinate and three qualified assessors. This exercise will occur in a small meeting room. Dimensions Interpersonal skills, problem analysis/problem solving, judgment and reasoning, decision-making ability, management and supervisory skills, composure, conflict resolution, oral communication. Exercise Citizen meeting Description The candidate plays the role of a supervisory officer that is meeting with a citizen. The meeting allows the citizen (role-player) to address a concern that he/she has about the department in general or more specifically about one of the candidate’s subordinates. The goal of the meeting is to work effectively with the citizen to solve his/her problem or collect information toward 197 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 176 bettering the department. The candidate will be given a preparation period to review the task at hand and to devise a game-plan. The candidate will then meet with the citizen. The citizen will follow a specific script that will cause the candidate to have to adapt his/her approach and to think on his/her feet. The successful candidate will gather necessary information, instill confidence in the citizen and set up a system to gauge progress and follow-up with the citizen. This exercise can be used as a precursor to the subordinate counseling role-play. This pairing would require the candidate to use this meeting to gather facts in order to guide the direction of the meeting with the subordinate. Themes The topic of the meeting can be: inappropriate treatment of the citizen by a department member (typically the candidate’s subordinate), the citizen’s specific concerns about recognized deficiencies in the operations of the department, or the collection of feedback from the citizen about proactive improvements that could be made to increase levels of customer service. Time There is a 10 to 15 minute preparation period prior to a 15 to 20 minute interaction. Resources This exercise requires one skilled role-player/actor to play the role of the subordinate and three qualified assessors. This exercise will occur in a small meeting room. Dimensions Interpersonal skills, problem analysis/problem solving, judgment and reasoning, decision-making ability, conflict resolution, public relations skill, oral communication. Exercise Subordinate performance review Description The candidate plays the role of a supervisor that is tasked with presenting a performance appraisal to a subordinate. Prior to the exercise the candidate will be provided with the subordinate’s personnel file. The subordinate will typically have a number of performance deficiencies that have not improved over time. The role-player, playing the subordinate role, will be evasive and, at times, combative about apparent performance weaknesses. The candidate’s objective will be to analyze the subordinate’s personnel file in order to successfully identify and work through the subordinate’s deficiencies. The candidate will have to demonstrate a large degree of assertiveness as he/she counsels the subordinate regarding deficiencies and identifies plans for improvement. The ideal candidate will exercise management skills while maintaining composure in a difficult situation. Themes The subordinate typically has a few areas of performance deficiency that must be addressed in the course of the performance appraisal. These may include: tardiness, improper completion of paperwork, poor attitude, and complaints from coworkers. Time There will be a 15 to 20 minute preparation period for the candidate to review the subordinate’s personnel file and devise a structure for the performance appraisal meeting. The meeting will then last approximately 15 to 20 minutes. Resources This exercise requires one skilled role-player/actor to play the role of the subordinate and three qualified assessors. This exercise will occur in a small meeting room. Dimensions Management and supervisory skills, judgment and reasoning, decision-making ability, planning and organization, interpersonal skills, conflict resolution, oral communication. Presentation-Style Role-Plays Exercise Training presentation Description The candidate will be tasked with preparing a training presentation to be delivered to a group of subordinates. If the candidate is a police promotional candidate, the training session will typically be a roll-call training brief. In the case of a fire promotional candidate, the training will be similar to in-service training. In either case, the candidate will be provided information about a topic that the subordinates must be trained on. The topic will be job related and will be something that the candidate can become familiar with in a short period of time. After a brief time to organize a training session and possibly create visual aids, the candidate will present the lesson to a group of three subordinates. The subordinates will either be role-players or the assessors. During the course of the training the candidate may interact with the role-players, or he/she may require the role-players to participate. The role-players will also be free to ask questions and respond as would be conventional in this type of setting. The successful candidate will deliver a well organized and thoughtful training presentation in a manner that is practical and engages the audience. Supervision skills and interpersonal ability will be critical factors as the 198 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 177 candidate may have to deal with unruly subordinates. Themes Police supervisory themes include: use of force training, pursuit training, community initiatives training and policy training. Department-specific topics can also be used to further customize the process. Time There will be a 15 to 20 minute preparation period for the candidate to review the training material and outline the training agenda. The training session will then last approximately 15 to 20 minutes. Resources This exercise requires three qualified assessors and possibly one or two skilled role-players. The role-players will play the role of subordinates and will have a more interactive role than that of the assessors’. The use of role-plays will make the exercise more realistic and increases the quality of the assessor’s ratings as they are not burdened with interacting during the exercise. Use of assessors as role-players is, however, a less expensive alternative. A dry erase board or flip chart will be provided to the candidate to enable him/her to better structure the training. This exercise will occur in a medium sized meeting room with chairs arranged in a classroom style setting. Dimensions Planning and organization, oral communication, interpersonal skills, management and supervisory skills. Exercise Community presentation Description The candidate is instructed that he/she must deliver a presentation about the department to a group of citizen who are interested in learning more about the department. The group of citizens will be played by either the assessors alone, or the assessors accompanied by additional role-players. The presentation is usually made before a group of citizens or at a city counsel- type meeting. The candidate is typically informed that the chief was scheduled to make this presentation, but he/she was called out of town and has asked the candidate to fill in for him/her. The candidate will be given a brief period of time to organize his/her thoughts and outline the presentation. There are no materials given to the candidate to assist in the development of the presentation. The candidate must develop and deliver a presentation on the specified topic based solely on his/her understanding of the department. The candidate will be able to use visual aids that he/she creates using flipcharts or dry-erase boards. The successful candidate will develop a well-organized and meaningful presentation that provides the audience with valuable information about how the department is operating within their community. Following the presentation the audience will ask questions related to the candidate’s topic. His/her ability to answer questions in a professional and responsive manner will be part of the assessment. Themes The presentation can be related to: department operations, response capabilities, the services provided by the department, the goals/mission of the department, or a recent improvement made within the department. Time There will be a 15 to 20 minute preparation period for the candidate to review and to prepare an outline for his/her presentation. The meeting will then last approximately 15 to 20 minutes. Resources This exercise requires three qualified assessors. The assessors will serve as assessors but will play the role of citizens. They will ask questions of the candidate at the end of the presentation. Additional role-players may be used in this exercise, but it is not necessary. A dry erase board or flip chart will be provided to the candidate to enable him/her to better structure the training. This exercise will occur in a medium sized meeting room with chairs arranged in a classroom style setting. Dimensions Planning and organization, oral communication, interpersonal skills, management and supervisory skills, leadership. Written Exercises Exercise Written problem-solving exercise Description This exercise requires the candidates to compose a written narrative related to an important departmental issue. The goal of this exercise is to outline the thought process that is used in solving a problem or reasoning through a difficult issue. This exercise is useful for evaluating basic problem solving skills, judgment and reasoning, and written communication. This exercise 199 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 178 can also be administered to many candidates at the same time and does not require the use of role-players. Themes Following are optional topics that can be used:  The candidate must identify critical issues facing the department and how he/she would handle these issues.  The candidate is faced with a difficult personnel issue (subordinate performance deficiency or sexual harassment) and must outline in a narrative how he/she would handle this issue.  The candidate will be asked to make suggestions for improving and better implementing a current departmental policy. The policy will be selected by the agency. Time The candidate will be allowed one to two hours, depending on the complexity of the questions, to complete this exercise. Resources A large classroom will be necessary to administer this exercise to a large number of candidates. Dimensions Judgment and reasoning, written communication, planning and organization, management and supervisory skills. Exercise Incident report proof-reading exercise Description This exercise uses copies of your agency’s incident report to assess the candidate’s ability to proof read an incident report for serious errors. A sample report will be created containing errors in documenting critical information and general grammatical and writing errors. The candidate will be asked to proofread the document, identify errors and suggest changes for improving the report. Themes Any standardized report can be used for this exercise. Time The candidate will be allowed 15 to 20 minutes to complete this assignment. Resources A large classroom will be necessary to administer this exercise to a large number of candidates. Dimensions Problem analysis/problem-solving, written communication. In-Basket Exercise Exercise In-basket/In-box Description The candidate is told that he/she was recently promoted and has inherited the former incumbent’s in-basket. It is imperative that he/she go through the in-basket and prioritize and respond to as many items as possible. There are approximately 10 to 20 memos, letters, email messages, phone messages, and other correspondence of varying importance and urgency that deal with personnel issues, scheduling, citizen concerns, planning and development, departmental issues, etc. The successful candidate will properly prioritize the items and provide appropriate written responses for how the issue at hand would be dealt with. In some cases the candidate may have to write a letter and in others simply detail an action-plan that he/she would follow to deal with the situation. There may be an additional follow-up period when assessors question the candidate regarding his/her chosen course of action. Themes There are a variety of in-basket items that are part of this exercise; however, depending on the role of supervisors in your department, more items of the following variety can be incorporated into the exercise in greater quantity: employee misconduct issues, citizen complaints, service- oriented requests, employee counseling issues, or scheduling/administrative issues. Time There is a 10 to 15 minute instruction period prior to the start of the exercise and, depending on the number of in-basket items, this exercise will last one to two hours. Resources The in-basket exercise is rated by two or three assessors following the administration period. This exercise requires a classroom-style room for administration to multiple candidates. Dimensions Planning and organizing, problem analysis/problem solving, judgment and reasoning, decision- making ability, management and supervisory skills, written communication. Situational Structured Oral Assessment Exercise Situational oral assessment Description The situational oral assessment is a panel-style interview that requires the candidate to consider 200 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 179 how he/she would respond to a variety of situations that he/she might encounter on the job. During the interview assessors will pose a variety of job-related questions to the candidate and will rate the candidate’s response using behavioral rating criteria. The questions can cover a variety of job-related scenarios or interpersonal situations. This exercise is not intended to be a role-play, but certain questions can be converted into role-play scenarios to maintain the “simulation” feel of assessment centers. Themes The questions contained in the oral assessment can cover three to four of the following topics: dealing with a problem employee, controlling a tactical situation, managing operations of the agency, or improving ones own skills. Time This exercise typically lasts for 20 to 30 minutes. Resources A small meeting room will be required for this exercise. Dimensions Depending on the questions used, any performance dimension can be assessed in this format. Performance Dimensions The assessors that are used to rate candidate performance are guided by specific performance dimensions with behaviorally anchored criteria. These criteria allow the assessor to look for certain behaviors, actions and thought processes. This method of rating performance ensures consistent and fair ratings. IOS has constructed a comprehensive performance model through years of experience developing and conducting assessment centers for public safety agencies of all sizes. Following is a comprehensive list of the performance dimensions that can be assessed. The behaviorally anchored criteria that accompany these dimensions are specific to the exercises that are used in the assessment center. This information is provided to demonstrate the model that IOS commonly uses. We expect that the performance model will be modified based on the job description for the target rank. General Management Dimensions Composure: The ability to work under stress and perform in unpleasant or traumatic circumstances. The ability to remain calm in stressful situations. The ability to handle stressful situations appropriately, remaining poised and professional. The ability to maintain emotional control in stressful situations. Conflict Resolution: The ability to de-escalate tense situations and quiet potential disturbances. The ability to mediate interpersonal or physical conflict between individuals. Decision-Making Ability: The ability to understand when a decision must be made and the willingness to make a decision. The ability to quickly and efficiently determine an appropriate course of action to target a particular situation. Honesty and Integrity: The ability to act in an honest and fair manner. Willingness to accept responsibility for actions when things go wrong. The ability to display a high degree of integrity and professionalism in action and word. Interpersonal Skill: The ability to listen to others and be considerate of the concerns of others. Ability to successfully and appropriately handle interpersonal interactions with others to gain trust, respect and mutual understanding. The ability to act with tact and diplomacy in dealings with the public - keeping public-relations concerns in mind when interacting with citizens, other agencies and the general public. The ability to counsel, support and be empathetic toward others. Ability to maintain positive work relations with others and interact with people of diverse backgrounds. The ability to use appropriate nonverbal cues to better convey intention of message. Judgment and Reasoning: The ability to accurately perceive the important elements of a situation, evaluate the situation and determine plausible courses of action that would bring about a desirable result. The ability to reason through a particular problem and decipher a logical course of action. The ability to use common sense and intelligence in handling day-to-day activities, problems and decisions. Leadership: The ability to set a proper tone for the functioning of the organization. Ability to serve as a role model in behavior, practice and word. The ability to motivate the organization as a whole to achieve greater efficiency and a higher level of purpose. Management and Supervisory Skills: The ability to direct and guide personnel in the accomplishment of goals 201 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 180 and tasks. (Includes skill in monitoring activities and evaluating results. Expectations and standards are clearly communicated, and consequences exist for insufficient performance.) The ability to assert proper authority and be assertive while maintaining the trust and respect of subordinates. The ability to motivate and counsel employees to greater performance. Skill in prioritizing and delegating the work of others. Oral Communication: The ability to speak in a clear, concise, understandable and appropriate manner. The ability to deliver a message in such a way that others clearly understand its meaning. Appropriate use of nonverbal cues such as good eye contact, posture, etc. Planning and Organization: Skill in establishing a course of action for oneself and others to accomplish goals. Skill in properly planning assignments, work duties and tasks; appropriate allocation of resources to ensure efficient, timely and smooth department operations. The ability to anticipate future needs and plan for the allocation of resources to meet those needs. Problem Analysis/Problem Solving: The ability to quickly perceive problem areas, identify options and the potential impact of various solutions. The ability to determine courses of action or solutions to problems. The ability to implement targeted solutions. Public Relations Skill: The ability to deal with the public in such a way as to protect the image and reputation of the department while providing the public or individuals with necessary or requested information. The ability to deal prudently with information such that the integrity of individuals or the department as a whole is not compromised. Self-Motivation: The ability to remain motivated to perform one’s job despite difficult circumstances, distractions or monotonous work. The ability to take charge when necessary and conduct tasks without being told to do so. The ability to work without supervision and accomplish tasks without prodding. Teamwork Orientation: The ability to work with others in a positive, goal-oriented manner. Willingness and ability to accept a particular role in a team and selflessly carry out that role. Willingness to actively support command staff policies and decisions. Written Communication: The ability to convey written messages in a clear, concise and easy-to-read format. The ability to clearly express ideas in writing to convey intended meaning. The ability to use correct spelling, grammar, syntax and proper subject-verb agreement. Exercise Review IOS will develop the selected assessment exercises in their entirety, including candidate instructions, role-player scripts (if applicable), assessor rating guidelines, scoring scales, etc. Once exercise development is complete, IOS will work with the department to convene an SME meeting to review each exercise and its rating criteria. The purpose of this review will be to ensure that the details of each exercise are specific to the organization and operations of the department and to ensure that rating criteria are consistent with performance expectations of the department. During this review SMEs will also provide input to help define minimally acceptable performance on each exercise. This input will assist in the calibration of rating scales and will be critical during the training of assessors. Assessment Center Scale and Scoring All assessment center scores will be compiled in scoring booklets used by assessors during the course of the assessment process. These booklets will contain all behavioral rating criteria for each exercise as well as the scoring scale that will be used by assessors. The scale normally used by IOS is a 7-point scale that defines ‘7’ as minimally qualified (adequate). The minimally qualified designation creates an effective cut-off score. During assessor training, assessors are informed that any score below ‘7’ is considered less-than-qualified (poor). Therefore, the assessment process has a built-in cut-off score that assessors are keenly aware of and use to distinguish qualified candidates from unqualified. 202 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 181 4 5 6 7 8 9 10 Unacceptable Very Poor Poor Adequate Good Very Good Exceptional IOS’s personnel enter all score data collected from the assessment center process into a database. This process is performed once and then independently verified to ensure accuracy. Then, scoring equations are applied to the data set and subsequently, independently evaluated for accuracy. Obviously, it is of paramount importance that scores be correctly calculated and assigned. Given the vast number of data points collected in an assessment center, we take great care in compiling and calculating scores. Candidate Orientation Preparedness plays a large role in creating fair promotional processes. Because many candidates that will compete in an assessment center have previously experienced assessment centers, it is helpful to provide an orientation process that allows first-time participants to gain the insight that seasoned candidates have obtained through experience. Further, assessment centers from year to year are often conducted by different consulting firms that often conduct assessment centers differently. It is important to clearly communicate to candidates how the assessment center will be conducted and how they will be evaluated. By supplying candidates with an opportunity to learn about and prepare for the assessment center, we will increase the reliability of candidate scores and reduce the risk of group score differences in the assessment center process. IOS advocates the uses of a candidate orientation to create transparency and to provide helpful information to candidates. Specifically, we recommend that a live orientation presentation be delivered. This presentation can be video-recorded for presentation to candidates who were not able to attend the original orientation. Should additional sessions be necessary, these can be conducted based on the need of the individual agency. The following are common components of our candidate orientation guide:  A general introduction to assessment centers  A review of the assessment center development process  Review of the types of exercises that can be included in the assessment center  An in-depth discussion regarding the nuances of individual exercise types  Review of the performance dimensions assessed  Review of the scoring system used for the assessment center  A discussion concerning assessors and assessor training  Advice for preparing for assessment centers  A brief discussion of common mistakes and misconceptions related to assessment centers (these will not cover specific performance expectations)  A review of the rules governing the assessment center  Question and answer session Assessor/Role-player Recruitment Assessors are recruited based on a recruitment plan that is accepted by the department/agency. This plan include the target geographic (national vs. state), the acceptable rank/level of 203 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 182 assessors, gender and race diversity goals, personal qualifications, and any other criteria valued by the department. Following are the common rules that IOS imposes on assessor recruitment:  All assessors must be one rank/level above the rank for which the assessment center is being administered.  All assessors must possess at least two years tenure at the minimally required rank.  All assessors should be from a similarly sized agency that is organized in a substantially similar manner.  Racially diverse assessors will be sought to participate in all assessments. Our standard goal will be that one third of all assessors be classified as racial minorities.  We will seek maximum gender diversity in assessor recruitment. Role-players will also often be required during the assessment center process. IOS prefers to use professional actors to perform this function. Many consultants tend to use law enforcement of fire service personnel to serve as role-players. While we understand the convenience associated with this practice, we disagree with the appropriateness. Professional actors are far superior for a number of reasons:  Consistency/standardization: A professional actor is able to play the same role over and over again without altering presentation. This is a difficult task that requires training and endurance, but it is absolutely essential to the standardization and reliability of the process that each candidate experience the same scenario.  Role-fidelity: Assessment roles are scripted to accomplish a very specific goal. Professional actors are able to aptly model the role that was intended. Often, amateur actors will morph the role to a persona that better suits their personality. This practice can be detrimental to the assessment process as the primary performance dimensions may not be assessed as reliably in this instance.  Realism: Simply put, professional actors do a much better job of making the scenario ‘real’ for the candidate. Given the challenge of role-playing, most candidates find it difficult to treat a scenario as ‘real.’ A gifted actor will help draw the candidate into the process by making the exchange high-fidelity.  Reliability: Ultimately, the superior performance of professional actors, increase the accuracy of candidate performance/scores and give us more confidence that we have fairly and accurately assessed the true ability level of the candidate. IOS will recruit role-players from professional acting guilds and talent organizations. We have established partnerships with talent agencies in various part of the country from which we recruit actors that have been used in past assessment centers. Where it is cost prohibitive to bring actors in from distant locations, we will contact local acting guilds to recruit role-players. Assessor Training IOS will conduct in-depth assessor- and role-player-training workshops to prepare each assessor and role-player to participate in and score the assessment-center exercises. We will require approximately six to eight hours to train the assessors and role-players. This training is generally conducted the day prior to the administration of the assessment center to candidates. 204 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 183 IOS will provide multiple consultants to facilitate training multiple panels at the same time. Following is an outline of the training that is provided: Role-Players: During this time role-players will be briefed on their characters and the requirements of the roles. Role-players will be allowed ample time to ask questions concerning their roles and to practice their roles with IOS’ consultants. Standardization and consistency of presentation will be reinforced during the training session. The culmination of this training will be a mock assessment center that will combine the assessors and role-players. Assessors: Prior to the administration of the assessment centers, assessors will receive an in- depth training session that focuses on a thorough description of the target rank and its responsibilities within the department, the assessor’s role, the dimensions that are measured for the exercise with which the assessor will be involved, careful note-taking, behavioral observation, scoring, common rating errors, reliability of assessment, providing candidate feedback comments and specific assessor duties. Ideally, a representative of the department can be present to answer any specific questions that may arise. This training will also allow ample time for assessor questions and for practice with mock candidates. This training session will end with a mock assessment center that involves the role-players. Assessment Administration IOS oversees the administration of the assessment center process. We send sufficient personnel to conduct all essential functions of orienting candidates, moving candidates throughout the assessment process, answering candidate questions, etc. We may request minor administrative support from the agency to assist in candidate sign-in or to monitor candidates. Prior to mapping out a final assessment administration plan, IOS will meet with agency personnel to discuss administration options. There are several competing interests in any assessment center administration process including: assessment length, the type of exercises utilized, test security and other related factors. Depending on how your agency views these competing interests, a decision must be made regarding how to implement your assessment center administration. IOS has implemented assessments using each of the methods described below on numerous occasions and is intimately familiar with the nuisances of each method. There are typically two contrasting methods—a sequestering method and a traditional model. Traditional Model of Assessment Administration In order to have the most robust and convenient testing process, it is usually ideal to use one assessor panel to rate each exercise (every candidate is seen by the same assessors per exercise). That said, this requires multiple days of administration, requires the use of signed confidentiality agreements and involves an element of “trust” in regard to your candidate population. This typifies the “traditional” model of assessment administration. Since we can typically process a maximum of 16-18 candidates per day in a given rating panel, the number of days is then determined by the number of candidates divided by this number (i.e., 15 or 18). For example, assuming that there are 48 candidates scheduled to participate in an assessment center, with three exercises and three assessors per rating panel. This example would require 205 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 184 three rating panels (nine total assessors if using three per panel) and three (3) days of assessment administration and an additional day for training. While this system allows for many advantages (i.e., fewer assessors required, less staffing burden on the agency, greater standardization of rating panels, etc.), the one major concern is that this system allows for a potential breach of security should a candidate from day one inform other candidates in subsequent days of the content of the assessment. Certainly, this concern can be preempted by requiring candidate confidentiality agreements, but these are not a perfect protection. Therefore, we seek to inform agencies of the strengths and weakness of various models and select the system that best meets the needs and concerns of the agency. An alternate option is the use of a sequestered assessment model. Sequestered Assessment Administration Model(s) In a sequestered assessment, all candidates compete in a given exercise in a single day. A sequestering approach may utilize an a.m. arrival group and a p.m. arrival group. All candidates are sequestered in the process for 4-6 hours to ensure there are no security leaks. There are numerous variations on this approach. In the most typical version of this approach, all candidates are processed through all exercises in a single day. This approach becomes challenging with a large number of candidates but is entirely workable. For example, if there are 48 candidates scheduled for an assessment and two live exercises (with a third written exercise), a city would require six (6) rating panels to run the assessment and a large number of rooms. All exercises could be administered in a single day and the two live exercises could be evaluated on the first day. The written exercise could then be evaluated on a second day. If a third live exercise was introduced instead of a written exercise, nine (9) rating panels would be required in this example, assuming that 16 candidates could be processed by a panel in a single day. Another variation on this approach is to require multiple days of participation by candidates. IOS has utilized this process with the Houston Police Department, the Atlanta Police Department, Washington D.C. Fire & EMS and the Buffalo Police Department. In this approach, candidate sequestering is again utilized, though candidates are required to attend the assessment across two or three days. This method can cut down on the number of rating panels needed as all candidates do not need to be processed in a given exercise in a single day. The City of Houston employed this model with a 50-candidate lieutenant assessment center in 2010. IOS required participation by candidates across three consecutive days. A sequestering process was used within each day to ensure absolute test security. Candidates were “held” within the assessment for 3-6 hours each day. This process was accomplished in three days, while a fourth day was reserved for training. The disadvantage of this approach is that it is not as convenient for those competing in this process. The advantage of this approach is that a sequestered process can be utilized with only three rating panels. A third variation employs the use of videotaping. While other vendors use this method as a primary assessment administration method, we have utilized this method on occasion to address a concern or solve a logistical issue involving the administration (or when required by contract). An advantage to this model is that the videotaping methodology allows the 206 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 185 administrator to process more candidates in a day (upwards of 25 per room vs. 15-18) and allows for fewer rating panels to evaluate the exercise. With this method, the administration can be conducted as a sequestered process, with the ratings occurring over a longer period of time. A disadvantage to this model is that candidates far prefer interaction with live assessors than giving a presentation to a video-camera and they are less likely to rate the process as highly job-related. In some instances, we have combined a videotaping approach with a multiple-day sequestering approach to gain further efficiencies. In summary, IOS and our staff are experts in assessment center implementation. We will evaluate your situation and your preferences and recommend an appropriate approach. We have a great deal of experience with each approach and can work effectively with all methods. IOS will produce all exercise materials needed for the administration of the assessment center and will develop an administration schedule that allows for the efficient processing of all eligible candidates through the assessment center in the most secure manner possible. IOS will work with the department to ensure that the administration schedule is effective based on personnel staffing and manpower concerns and any specific concerns that the department may have about test security. While IOS will oversee the administration of the assessment center, candidates will actually be rated by assessors from outside agencies that hold a rank at or above the level of this target process. One of IOS primary jobs is to monitor assessor performance and to ensure standardized and reliable ratings are being obtained. Candidate Feedback IOS will be responsible for providing candidates with a written feedback report. This report will provide information about the candidates’ performance on each assessment center exercise, each dimension and on the assessment overall. The report will also highlight assessor observations. Finally, the report will provide information about candidate strengths and developmental needs. During the assessment center, assessors will be required to document the candidates’ primary strengths and developmental needs for each assessment exercise. During assessor training, assessors will be instructed as to what information to include as feedback. Assessors will also be encouraged to highlight points for each assessment exercise that increased the candidates’ performance for that scenario or decreased the candidates’ performance. All comments will be screened to ensure that assessors are identifying behaviors that will be understandable to the candidate. This screening will also ensure that there are no inappropriate comments included in the candidates’ feedback reports. Candidates will also be offered normative information so that they can compare their performance with that of the rest of their peer group. This information will assist the candidates in understanding the scale and how other candidates performed as a group. 207 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 186 Investment The following tables outline the costs associated with the project plan described in this proposal. 208 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 187 Project Step Consultant Hrs. Consulting Assoc. Hrs. Tech. Wrtr. Hrs Admin. Hrs.Cost Job Analysis 12 27 0 0 $4,500 Incumbent interviews.3 9 Job analysis questionnaire development and admin.4 8 Analysis and technical reporting.5 10 Written Job Knowledge Examination 13 14 59 0 $7,785 Work with SMEs to develop an examination plan that is linked to the job analysis, and select appropriate knowledge sources.2 Draft 140 multiple choice questions based on exam plan.2 8 48 Review questions with SMEs to assess job-relatedness and difficulty. Identify final 110 questions and establish appropriate cut-off score.3 5 Administer exam.5 Score exam.1 3 Assist in designing/coordinating appeals process and respond to candidate appeals.3 6 Assessment Center Development 63 43 0 8 $16,005 Design three assessment center exercises based on input from SMEs, including candidate preparation materials, scripts, and rating criteria/guidelines.32 Review exercises with SMEs to refine exercise details and ensure accuracy of criteria.3 Develop efficient assessment schedule and coordinate needs for assessment site.2 Recruit assessors and coordinate travel logistics (estimated 7-9 assessors to create 3 panels)9 Administer the assessment center (estimated 30 candidates: 3 days, 2 staff)24 24 Compile assessment scores and conduct quality controls.1 4 4 Develop candidate feedback reports.1 6 4 Candidate Orientation Presentation 6 0 0 0 $1,170 Prepare and deliver a candidate preparation/orientation presentation.6 Project Expenses $4,750 Consultant travel-related costs Administrative costs (shipping, freight, printing) TOTAL PROJECT INVESTMENT $34,210 Project Notes Job analysis is reccomended but may not be necessary if a current job analysis exists. Client will be responsible for providing a suitable site at which to conduct exams and assessments. Client will be responsible for assessor related costs. South Miami, Florida Sergeant Promotional Process Police Sergeant (Estimated 30 candidates) $4,550 $200 209 Copyright © 2020, Industrial/Organizational Solutions, Inc. CONFIDENTIAL – TRADE SECRET 188 Schedule of Hourly Rates - IOS 2020 Category Rate Consultant/Industrial Psychologist $195/hour Consulting Associate $80/hour Technical Writer $70/hour Administrative Assistant $35/hour Details Regarding Proposed Pricing: The cost information provided is based on the facts and figures that were provided in the request for proposals. Should the scope of services need to be changed/modified, or the candidate numbers increase, IOS will calculate any additional project costs, based on our published hourly rates, and submit a change order to the agency prior to engaging in additional work. The proposed costs are fixed-firm and, unless the scope of work is modified, will be invoiced in total. IOS proposes the following invoice schedule: 1/3 upon contract execution, 1/3 upon development of the written examination, and 1/3 upon completion of the project. IOS will provide counsel, deposition and expert testimony services for the purpose of defending the development, validation and administration of selection processes in which we participate. Following are the fees associated with these services: Service Fee Expert counsel, deposition or expert testimony provided by a Ph.D. level Industrial/Organizational Psychologist. $250/hour Administrative services related to litigation support. $25/hour 210 Appendix A Disclosures: No selectmen or other officer, employee or person who is payable in whole or in part from the City currently has any direct or indirect personal interest in IOS. IOS, nor any of our employees, have been named as a defendant in any litigation brought as a results of any contract operations for operations and maintenance. IOS have never been terminated, fired or replaced on a project other than though contracts that have been terminated due to completion. While we have served as experts, we have never been successfully challenged in litigation. 211 212 213 CONFIDENTIALITY STATEMENT The data included in this overview shall not be duplicated, used or disclosed, in whole or in part, for any purpose except for use in this review, and as mandated by applicable law. The data subject to this restriction are contained on every sheet of this overview and related Appendix material marked with the following footer: PSI – Confidential -Not for External Distribution . Prepared by: PSI Services LLC 611 N. Brand Blvd; 10th Floor Glendale, CA 91203 www.psionline.com LAW ENFORCEMENT PROMOTION EXAMINATION SERVICES 214 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution i TABLE OF CONTENTS I. SELECTION PROCESS OVERVIEW ............................................................. 1 A. Job Analysis ............................................................................................. 1 B. Test Plan Specification ............................................................................. 3 1. Technical Knowledge Examination ...................................................... 3 2. Written Work Sample/In-Basket Exercise ............................................ 4 3. Oral Work Sample Test ....................................................................... 4 4. Internal Review Board ......................................................................... 6 C. Test Development and Review................................................................. 6 D. Candidate Preparation ............................................................................. 7 E. Test Administration and Assessor Training .............................................. 8 F. Candidate Appeal Process ....................................................................... 9 G. Assessment Material Security ................................................................ 10 H. Scoring and List Production ................................................................... 10 I. Candidate Score Reports ....................................................................... 11 J. Assessment Review and Adverse Impact Reduction ............................. 11 K. Validation Strategy ................................................................................. 12 L. Project Report ......................................................................................... 13 M. Concluding Statement ............................................................................. 13 215 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution ii LIST OF APPENDICES Appendix A: Sample Technical Knowledge Examination Reading List Excerpt Appendix B: Sample Technical Knowledge Test Items Appendix C: Sample Essay Style In-Basket Exercise Item Appendix D: Sample Oral Work Sample Exercises Appendix E: Sample Candidate Preparation Guide Table of Contents Appendix F: Sample Oral Administration Schedule Appendix G: Sample Candidate Score Report 216 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 1 I. SELECTION PROCESS OVERVIEW PSI strongly advocates the use of multiple-testing modalities and instruments (i.e., the classic definition of an assessment center) for promotion testing. The use of multiple testing modalities most closely replicates the requirements of the job and reduces the impact traditional knowledge examinations may have on members of protected classes. PSI's customized promotional tests are linked directly to the requirements of the job (i.e., they are content -valid). The precise exercises developed are dependent on client need. However, PSI advocates the use of work simulation exercises (written and oral) and other measures (e.g., Internal Performance Review Boards) in addition to traditional tests of technical knowledge. A. Job Analysis To achieve a goal, one must first define the goal. Before designing a testing process to identify effective performers, we must first understand “effective performance” [in terms of the tasks to be performed and the knowledge, skills, abilities and personal characteristics (KSAP’s) needed to perform those tasks]. Therefore, from a bottom-line perspective, the job analysis is the “due diligence” process and helps ensure that the testing system in which you invest is going to produce the results you expect (i.e., effective performers). From the best practice and legal compliance standpoints, and to support a content validation strategy, it is also critical to conduct a thorough job analysis before developing any personnel decision-making process. Linking test content to job content ensures you are making decisions on job-relevant characteristics and satisfies compliance requirements dictated by various legal statutes and guidelines (e.g., ADA, Civil Rights Act of 1991, Uniform Guidelines, APA Standards, SIOP Principles). PSI's approach to promotional testing involves a content-oriented strategy of test validation. A content-oriented strategy rests upon a careful and detailed job analysis of the position, in which essential functions (e.g., important and frequently performed tasks) and the knowledge, skills, abilities and personal characteristics (KSAP’s) that are prerequisites to task performance are delineated. This information enables us to:  determine essential functions and develop a job description,  identify the knowledge, ability and personal characteristic areas critical to job performance,  determine precise specifications for each test component, and  specify the weights to be allocated for each test component. At the onset of the job analysis, copies of previous job analysis data, job descriptions, training materials and knowledge sources are gathered. The specific job analysis steps are as follows: 1) A Preliminary Task List is developed for the position. Job descriptions previously developed for the position (if available) serve as the starting point for the current task list, but PSI's extensive library of police supervisory and managerial task lists would also be referenced. 217 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 2 2) Interviews are conducted for the position with a representative sample of incumbents and supervisors to refine the population of job tasks and contextual factors. Information concerning the requisite knowledge sources (including internal and external sources), abilities and personal characteristics is collected at this time. We typically interview approximately 10 incumbents and 1-2 supervisors. Based on these interviews, initial modifications are made to the preliminary task list. 3) Task list panel reviews are conducted with small, representative groups of incumbents in 2-hour sessions. The preliminary task list is presented for review after being modified by PSI based on the interviews. Any modifications identified based on the panel reviews are incorporated to create the final task list. 4) Separate questionnaires measuring task, knowledge, ability and personal characteristic requirements are distributed to representative samples of job incumbents via internal mail and returned to PSI for analysis. The following types of data are collected: Task Survey ▪ Do you perform this task? If so… ▪ How frequently do you perform this task? ▪ How important is this task to successful job performance? ▪ Is this task required upon promotion, or taught on the job? Ability/Personal Characteristic Survey ▪ How important is the ability/personal characteristic for successful job performance? ▪ For which of the job duties is this ability/personal characteristic important? Knowledge Survey ▪ Do you use this knowledge source on the job? ▪ How important is this knowledge source to successful job performance? ▪ Does the information contained in this knowledge source need to be memorized for successful job performance? ▪ For which of the job duties is each overall knowledge source important? 5) Data collected from the interviews, and questionnaires are combined and analyzed to identify the following: ▪ Essential job functions (e.g., tasks that are performed by 50% or more respondents and are at least moderately important) ▪ Essential knowledge, abilities and personal characteristics ▪ Assessment content (i.e., test specification) ▪ Assessment component weighting 218 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 3 For illustrative purposes, some of the abilities and personal characteristics repeatedly identified as essential to public safety professionals are provided in the table below. Supervisory/Managerial Abilities Personal Characteristics/Competencies ▪ Oral Expression ▪ Adapting & Coping ▪ Written Expression ▪ Communication ▪ Interpersonal Relations ▪ Decision-Making ▪ Information Analysis ▪ Initiative & Independence ▪ Judgment and Decision-Making ▪ Interpersonal Interactions ▪ Behavioral Flexibility ▪ Leadership ▪ Resource Management ▪ Professional Development ▪ Planning and Organizing B. Test Plan Specification Measuring factors related to job performance is the goal of every testing process. Tests should measure important characteristics and do so in proportion to their relative importance. Based on job analysis data, we specify the most important job duties and tasks, as well as the most important underlying knowledge, abilities and personal characteristics. For example, information regarding knowledge sources serves as the basis of the test plan from which the technical knowledge tests, items and weights are derived. More specifically, only knowledge sources identified as important during the job analysis are used to create the technical knowledge examination. Furthermore, the number of items allocated to the various sections of the knowledge sources are based on the importance of the knowledge source as documented in the job analysis. Therefore, there is no question concerning the representativeness of the test items in relation to the job analysis, and, ultimately, to the job. The link runs directly from the job, as represented by the task statements, to the task duties, to the knowledge analysis, and finally to the test format, or plan, and subsequent test items. A similar approach is used to develop any assessment components (e.g., oral work samples) that are used. A reading list is developed for the technical knowledge examination, outlining specific chapters or sections of each knowledge source to be studied, and specifying whether the information will be tested in an open or closed book format. An illustrative excerpt from a reading list for a Police Sergeant Examination appears in Appendix A. As indicated above, the information we gather helps us not only in the development of the reading list itself, but also in the development of the test plan specifying what specific sections of each source should be covered and how the information should be covered (i.e., in a closed- or open-book framework). 1. Technical Knowledge Examination. Understanding and correctly applying relevant laws, statutes, policies and procedures are critical to effective performance. Depending upon the information to be used, an incumbent must either recall the information from memory or be able to access and apply the information from a source (e.g., manual, book). To accurately simulate the 219 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 4 work environment, our technical knowledge examinations typically consist of both an open- and closed-book section. The contents of each section are based on the job analysis results. Our closed-book tests usually consist of traditional, multiple-choice items with four response alternatives each. These items are designed to assess candidates’ understanding of job-related technical knowledge that the job analysis indicates must be memorized, so no reference materials are available for use during this test. Closed-book items typically require candidates to apply department policies and procedures as well as general supervisory knowledge contained in textbooks (see Appendix B for written test question examples). Items on this test are drawn only from reference materials identified on the reading list as closed-book. Our open-book tests usually consist of traditional, multiple-choice items with four response alternatives each. These items are designed to assess candidates’ understanding of job-related technical knowledge that the job analysis indicates need not be memorized but can be looked up when needed. Open-book items require candidates to quickly access the relevant information from sources they would routinely have access to on the job (see Appendix B for written test question examples). Therefore, all reference materials included on the reading list are available for use during the open-book test. However, items on this test are only drawn from reference materials identified on the reading list as open-book. Our Technical Knowledge Examinations usually consist of between 100 to 125 items. The distribution of closed-book and open-book items is based on the results of the job analysis. Typically, about half are closed-book and half are open-book. Approximately 1.5 minutes is allotted per closed-book item and 3 minutes are allotted per open-book item. 2. Written Work Sample/In-Basket Exercise. Prioritizing and responding to daily events, issues and incidents is an integral part of supervisory positions. This exercise presents candidates with various situations and/or forms and paperwork taken directly from important and frequently occurring tasks for the position. Candidates may be presented with a description of an incident or crime scene with accompanying diagrams and asked how the incident should be handled. Other items may relate to the preparation of an annual budget, revisions or additions to department policies and other reports required of incumbents (see Appendix C for a sample item). The precise format of the exercise (e.g., in-basket, scenarios) and the content of the materials (e.g., memo topics, tasks) depend upon the job analysis results. These exercises, however, frequently involve both types of activities (i.e., in-basket and scenario-based items). For smaller administrations and upper level positions (e.g., Captain/Commander) with increased importance of written communication skills, we suggest that this exercise be developed such that candidates provide their answers in an essay or written format for evaluation by a panel of assessors. For essay style evaluations, we develop scoring standards which are reviewed, if possible, by a panel of internal Agency personnel. 3. Oral Work Sample Test. The oral work sample exercises simulate a subset of the job’s core responsibilities (e.g., subordinate conference, incident supervision, group meeting). During each exercise, candidates are assessed by a panel of trained raters (using behaviorally anchored rating scales) on the underlying supervisory/managerial abilities required to successfully perform these tasks. The final oral work sample score represents a combination of scores from 220 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 5 multiple raters, exercises and abilities. The specific format and content of the oral exercises is based upon the results of the job analysis. Oral work sample exercises that are considered, depending on the results of the job analysis, include: a. Performance Discussion (Subordinate Conference) Exercise. Effectively managing a subordinate’s performance is a critical aspect of any supervisory position. The Performance Discussion Exercise requires candidates to interact with a subordinate whose performance is sub-standard or who has displayed disciplinary problems. The candidate assumes the role of a newly promoted incumbent and is presented with background information concerning the subordinate’s performance and job record prior to meeting with the subordinate. Each candidate is required to conduct a short meeting (around 15 minutes) with the subordinate (played by an assessor) in order to address the performance or discipline issue. To ensure consistency across candidates, the subordinate role-player is provided with a script containing standardized responses. It is the candidate’s responsibility to manage the interaction to an effective conclusion. An example of a Performance Discussion Exercise is provided in Appendix D. b. Incident Supervision Exercise. Directing the effective resolution of critical events in the field to preserve public safety, minimizing the risks to personnel and ultimately enhancing public confidence is central to a Department’s success. This exercise presents candidates with a short incident brief (similar to information contained in an initial response call) approximately 10 minutes before the exercise begins. Once the exercise starts, candidates typically have 7 to 10 minutes to present their initial strategy for handling the incident (e.g., resources requested, tactics used, agencies contacted) and bringing it to a successful resolution. Depending upon administrative details, assessors may then provide candidates with an update describing the progression at the scene that the candidates must incorporate into their response. To complete the exercise, assessors may ask all candidates a series of standardized follow-up questions for approximately five minutes. Total exercise time is typically about 15 minutes. An example of an incident supervision scenario is provided in Appendix D. c. Department/Community Relations (Group Meeting) Exercise. Developing and maintaining constructive professional relationships is increasingly important in today’s public safety environment. This exercise requires candidates to interact with a group of community members, peers or colleagues (portrayed by assessors) concerning a subject of significance (e.g., service quality, response time, training deficiencies, events). Candidates are responsible for working effectively with the group members to achieve resolution to the issue or agreement to a course of action (as the situation dictates). This exercise is usually conducted in about 15 minutes. An example of a group meeting scenario is included in Ap pendix D. d. Structured Interview Exercise. This type of exercise requires candidates to respond to a series (around 5 or 6) of job related questions. Two types of questions may be included in the exercise: behavioral and situational. The behavioral questions ask candidates to describe how they have handled specific situations in the past. The situational questions ask candidates to describe how they would handle situations that may arise in the future. Candidates are given approximately 15 to 20 minutes to respond to the interview questions. Sample interview questions are included in Appendix D. 221 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 6 4. Internal Review Board Test. Candidates’ work history, experience, past performance, and job-related training and development, as they relate to potential success in the next higher rank, can be rich sources of information. The problem with such data is that they are not typically organized or evaluated in a structured or standardized manner. The internal review board process provides the necessary structure and standardization. Using a pre-defined format/outline, candidates provide job-related information in the form of a written personnel packet and, if desired, a face-to-face presentation. For example, candidates may be asked to document and/or describe their accomplishments in their current rank; the specific skills, abilities, and knowledge they possess that enhance their readiness for promotion and other aspects of their performance history and capabilities. Trained internal assessors review the personnel packet (approximately 15 minutes), listen to the presentation made by each candidate (approximately 15 minutes), and may pose a series of follow-up questions to each candidate. Assessors are then typically given 20-30 minutes to make independent ratings on behaviorally anchored rating scales covering such aspects of performance history and personal characteristics as Motivation, Attention to Duty, Corporate Citizenship, Supervisory Readiness based on the information contained in the personnel packet and orally presented by the candidate. C. Test Development and Review Technical Knowledge Examination items, Written Work Sample scenario based or in- basket items, Oral Work Sample scenarios and the content of the Internal Review Board are developed by PSI based on test plan specifications derived through the job analysis and consultation with clients. Over the past 25 years, PSI has developed a detailed set of guidelines for the development and review of test items and work sample scenarios/content. All items and work sample scenarios reviewed by PSI undergo a series of steps to ensure:  Technical accuracy  Cultural fairness  Appropriate reading level  Job relevance (links to job analysis)  Organizational appropriateness First, each technical knowledge examination item is reviewed to ensure that the correct item construction has been used, it has been keyed correctly and professional testing procedures have been employed. Three PSI members conduct this review to ensure that all critical issues are identified and all necessary modifications are made. In addition, an educational psychologist who specializes in the evaluation of tests for cultural fairness, reviews all technical knowledge examination items and work sample scenarios for wording and characteristics that might pose a problem for minority or female candidates. Third, a reading level analysis of all items and scenarios is conducted using an accepted readability program. Examination items and scenarios are targeted to be at a level that is equal to or slightly lower than the average reading level of the materials used on the job. By targeting materials at these levels, we restrict the effect that reading speed and comprehension might have on test scores. This ensures that the relevant KSAs, and not reading ability, are measured by test components. Finally, if permitted, a review of all items and scenarios is conducted with internal subject matter experts (SMEs) to address issues of organizational appropriateness (job relevance). Although PSI develops its own links between the 222 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 7 test items/scenarios prepared and the job analysis results via the test plan specifications, the advantage of the participation of internal SMEs in the review process is obvious. Each item and scenario is examined by SMEs on those aspects that might raise objections before being used. In addition, SMES can help establish links between the test content and job domain, with particular attention to discrepancies between policy and actual practice that may only be apparent to SMEs. D. Candidate Preparation Just as world-class athletes work with personal trainers to prepare for competition, today’s public safety professionals invest large amounts of personal time and energy preparing for promotional assessments. Providing information regarding the content and logistics of each examination component helps ensure that all candidates are on equal footing when they take the examination, thereby representing an effective step toward the reduction of adverse impact. Agencies that acknowledge the commitment of their candidates and recognize the criticality of ensuring equal access to information provide “comprehensive training assistance” to their members and often see an increase not only in the caliber of candidates, but also in the general force. Formal candidate preparation programs minimize superficial performance differences, such as mistakes made because of being unfamiliar with a type of test, careless errors, or improperly allocating study time based on rumor and/or “others’ experiences.” Our recommended program includes a description of the strategies that candidates can use to develop their KSA’s to enhance examination (and job) performance as well as the following:  KSA’s assessed  Types of questions asked  Approximate number of test questions  Approximate length of the test(s)  Practice items and exercises  Suggested responses This information is typically presented in a self-paced Candidate Preparation Guide (see sample Table of Contents of a Candidate Preparation Guide in Appendix E). When feasible, we also recommend additional on-site orientation or tutorial programs. There are several forms, ranging in depth that these on-site programs can take.  The most extensive tutorial program consists of classroom-style training sessions to be offered on-site to test candidates. PSI staff train a cadre of agency trainers to facilitate the tutorial program. This version of the tutorial program consists of two, 2.5 hour sessions. The trainers are trained during a day-long session.  If preferred, PSI can develop a condensed version of the classroom style sessions, to be administered in a single session (4 hours). The information covered in this style of session is similar to that covered in Option 1, but is presented in a single session, which may, or may not, be more convenient for attendees. 223 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 8  Finally, PSI can provide a general structured orientation session in which administrative details of the examination process are covered and questions are answered. For example, the orientation process covers the types of tests to be administered, the rules for administration and all applicable guidelines regarding the process. PSI also ensures that all candidates are aware of the stand-alone study guide and reading list. E. Test Administration and Assessor Training Efficiency, security and standardization are the primary test administration goals PSI will help you achieve. PSI staff will work closely with your Department to develop the most efficient administrative system for the promotional process. Whenever possible, we recommend that all candidates complete all phases of the testing process and we suggest administering the written examination to all candidates in a single testing session. This approach maximizes test security. We can also work with your Department to develop a multi-session approach, where two testing sessions will overlap slightly, thereby ensuring that candidates from the first session do not interact with candidates from the second session. For Oral Work Sample Tests, we suggest that a three-member panel of trained assessors evaluate each exercise. Most agencies use external SMEs on a reciprocal-exchange basis to conduct oral work sample simulations. On occasion, some agencies may include one or more internal SMEs on their panels. Regardless of whether all external SMEs, or a combination of external and internal SMEs, are used on the Oral Work Sample panels, the gender and racial make-up of each assessment panel should match the make-up of your Department, or over- represent protected classes. For Written Work Samples, we suggest that an independent panel of two assessors be used to evaluate the responses to the written essay-style work sample. Finally, for the Internal Review Board, we suggest that a panel consisting of 3-4 internal SMEs at the level of Lieutenant or Captain and higher, and representative of your Department with respect to race and gender, be convened to conduct the assessments. The number of panels needed is a direct function of the number of candidates being assessed and the length of the assessment process (number of days). Ideally, one panel would be convened for each exercise/test enabling each panel to evaluate every candidate on a particular exercise (e.g., Panel 1 always rates the performance discussion, Panel 2 always rates the incident supervision, etc.). This is the model that is followed when a relatively small number of candidates is involved. In cases where greater candidate numbers are involved, time and resource constraints often necessitate the formation of multiple “parallel” panels. We have devised training techniques that minimize potential board differences whenever multiple boards are required. We can also examine resulting data and standardize scores as necessary across boards to account for possible board effects. A sample oral administration schedule is provided as Appendix F. This sample is based on each board processing 10 candidates per day. When necessary, we have processed as many as 15 or 16 candidates through each board per day (by shortening exercise and evaluation time, and lengthening assessment days) to minimize the number of assessment days required. 224 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 9 Administrations are immediately preceded by training conducted by PSI staff. The assessors will be trained by PSI personnel on the following areas:  Your department’s culture, structure and job requirements (tasks and KSAO’s)  Assessment logistics and procedures  Oral exercises and related materials  Attributes measured including rating scales and standards to be used (including rating mock candidates to develop common frame of reference)  Correct observation and evaluation of behavior  Recording of notes and of scores  Possible biases/errors raters can make  Use of the results  Candidate preparation activities With respect to the distribution of responsibilities for administrations, the following assumptions are usually made: PSI would: The CLIENT would: Train test monitors Locate and secure all testing and training facilities Have a representative on site for the first day of written and oral testing Locate and secure all test monitors  1 monitor per 25 candidates (written)  2-4 monitors for Oral Board Produce all test materials and instructions Locate and secure all assessors and related costs (travel, food)  3 assessors per panel  no. of panels per exercise depends on the no. of candidates  2 assessors for written work sample Train all assessors Notify candidates of testing schedule(s) Develop administrative guidelines and train administrators This arrangement maximizes efficiency and minimizes costs for your Department. However, PSI frequently assumes responsibility for assessor acquisitions and travel related expenses as well as for making all assessor travel related arrangements (e.g., flights, lodging, per diem). Therefore, if your Department is interested in alternate arrangements, PSI would be happy to discuss these. F. Candidate Appeal Process Perceived fairness of the testing process is often enhanced when candidates are given the opportunity to submit questions regarding the process. At the same time, clear guidelines sho uld be established and activities should be conducted in a controlled environment to minimize security risks and maximize integrity. We recommend that candidates review their examination materials in 225 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 10 a supervised location and submit appeals to questions prior to receiving their scores. Appeals regarding content should be limited to assessment instruments with pre-defined correct and incorrect answers (i.e., multiple choice style tests). For components relying upon assessor evaluation, candidates may appeal matters of procedure (e.g., session length, study time) but not individual assessor ratings. Once appeals are collected, PSI staff review the appeals, provide relevant reference material, summarize any statistical data and formulate a recommendation. Your Department, in consultation with PSI, would render final dispositions on all appeals. G. Assessment Material Security We fully recognize the security concerns faced by public safety agencies involved in testing and follow a prescribed set of steps to preserve the integrity of our testing processes. Specifically, the following steps are taken:  All client project members sign a confidentiality agreement.  Electronic storage drives (and files) are password and firewall protected.  Hard copy assessment materials are maintained in locked storage rooms at PSI.  Assessment materials are personally transported in secure containers whenever possible and tracked when sent with a courier.  Test booklets are numbered and counted before, during, and following administration.  Test booklet numbers are recorded on candidate answer sheets.  Monitors are trained on appropriate test material accounting procedures.  Candidates are prohibited from using electronic communication devices at test sites. In short, our processes and past experience indicates that we can ensure the security of the assessment materials. H. Scoring and List Production Accurately scoring test instruments, combining data across assessments and producing final eligibility lists is of utmost importance. Our team of data analysts follow standard procedure to ensure accuracy of scores and report on results. PSI data verification procedures are extensive and include the following steps:  Agencies are asked to ship data via a secure courier (e.g., Federal Express) or to have a staff member transport the data by vehicle when possible. When test data are shipped, we strongly urge agencies to duplicate and temporarily retain all answer sheet copies prior to shipment in the event of any shipping problems.  Once received, the candidate count is verified to ensure all answer sheets are accounted for before being processed. We also request that all test materials used in the administration be retained until this accounting process is complete. 226 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 11  Prior to scanning the data, ALL answer sheets are visually inspected for potential problems (e.g., ripped answer sheets, missing background information, etc.) that can cause scanning or scoring problems if not rectified.  Answer sheets are then scanned using NCS OpScan 5 double-sided scanners. We use the scanner to scan each answer sheet twice and compare results. Discrepancies between the two scans are examined manually and the value actually appearing on the scan sheet is selected (discrepancies are infrequent, and, when they occur, are caused by light marks and erasures).  Once the data is scanned and cleaned, the data is processed by SPSS (a sophisticated statistical scoring and analysis system). Scores are generated using programs submitted to SPSS that have been checked and double-checked.  Once scores are generated by SPSS, several scores at the low end of the score range, high end of the score range, and in the middle of the score range are hand-scored to confirm the accuracy of the automated scoring process and resulting list. The standard timeframe for this service is two weeks to ensure adequate time to conduct the necessary quality assurance steps. I. Candidate Score Reports Summarizing candidates’ examination scores and dimension-level performance, and providing general developmental comments are important parts of the assessment process. Career-focused candidates can use this information to modify their performance in future testing situations and more importantly, on-the-job. The candidate score reports include:  a detailed explanation of the process used to derive candidates’ scores,  a summary of the performance of all candidates so that candidates can get a sense of how they performed in comparison to the overall testing group,  a score report page providing candidates with their overall scores, test component scores, and scores on elements within each test (e.g., Oral exercises and dimensions),  feedback regarding written test items and associated sources that the candidate answered incorrectly on the technical knowledge examination, and  narrative feedback and developmental suggestions. Sample elements of a score report appear in Appendix G. Score reports are prepared for each candidate. The score reports are provided to your Department for distribution to candidates within two weeks of delivering the final eligibility list. J. Assessment Review and Adverse Impact Reduction Technical accuracy, maximizing job and organizational relevance and reducing the chance of adverse impact are three goals of our test review process. In addition to technical reviews to ensure that correct item/scenario construction has been used, multiple-choice items have been 227 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 12 keyed correctly, and professional testing procedures have been employed, the following mechanisms are employed to maximize test fairness and minimize adverse impact:  Cultural Bias Review: All items/scenarios used in our assessments are thoroughly reviewed by a psychologist with expertise in the development of culture fair tests for wording and characteristics that might pose a problem for minority or female candidates.  Reading Level Analysis: A reading level analysis of all examination items/scenarios is conducted using an accepted readability program. Assessment items and text are targeted to be at a level that is equal to or slightly lower than the reading level of the materials typically used in law enforcement departments. By targeting materials at these levels, we are able to restrict the effect that reading speed and comprehension may have on test scores. This ensures that the relevant KSA’s and not reading ability are being measured by the test.  Organizational Review: All items and scenarios are reviewed by internal SMEs, if permitted, for technical accuracy and appropriateness for use, and to ensure the parallel nature of Oral exercises. The advantage of such a process is obvious. First, the integrity of the test will only be enhanced since internal SMEs have been involved in the review of test items/scenarios. In addition, each item/scenario is examined on those aspects that might raise objections before being used.  Generous Time Limits for Tests: Generous time limits are used on all test components to reduce the impact of reading ability on test scores. As a matter of practice, we develop time limits that would allow all, or in the case of severe outliers, virtually all, of the candidates to complete the testing process.  Candidate Preparation: We use a candidate preparation guide, reading list, and orientation program to delineate exactly which subject matter areas need to be studied and provide suggestions of ways in which the candidates’ preparation time might best be utilized. These materials include tips designed to minimize differences in "test wiseness" that may exist between candidate groups, sample items and exercises and suggested responses to those items/exercises.  Alternate Test Formats: We suggest the introduction of alternate selection devices beyond traditional multiple-choice technical knowledge tests to include oral simulations and an internal review board which are not only job-related, but tend to have less impact on minority groups.  Proportional or Over-representation of Protected Class Groups: We seek to ensure proportional or over-representation of protected class groups in all data collection (e.g., job analysis) and test administration (assessment panel composition) processes. K. Validation Strategy Making employment decisions based upon job-related criteria is critical from both the organizational (“Is this the best person for the job?”) and legal standpoints (“have we complied with relevant statutes and guidelines?”). All of the documentation and links necessary to establish a strong content validity argument are present in the process, and documented in a final project 228 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 13 report. From a content-validity standpoint, we use job analysis data to link test activities to job requirements. The documentation and links include:  Identifying the activities (duties and tasks) performed by incumbents and the requirements (knowledge, abilities, and personal characteristics) needed to perform them  Determining the relative criticality to job performance of the various job activities and requirements (knowledge, abilities, personal characteristics)  Developing test plan specifications that assess critical activities, knowledge areas, ability areas and personal characteristics  Using the test plan specifications directly to develop the final examination components Criterion-related validation is often not feasible for promotional examinations given issues surrounding test security. L. Project Report Documentation of project activities and related data ensures that your Department understands the process used, has sufficient information to address potential challenges and is in compliance with professional standards. A final validation report can be prepared to describe and document the development, administration and validation of the examination. The project report would document all project activities and would be structured around the legal requirements of Title VII, the Uniform Guidelines and other relevant authorities, and professional testing practices. In projects similar to this one, clients have used our project reports as a first line of defense against legal challenges to test validity. As an alternative, project documentation can be maintained at no cost until such time as a formal report may be required. M. Concluding Statement We develop solutions that identify the best candidates for the position while gaining candidate trust and minimizing adverse impact. Since 1972, the identification of top performers through job-related comprehensive assessments has been our legacy. Developing effective human resource management systems based on best practice, research, and the law is our hallmark. We offer the breadth of services (e.g., competency modeling, internet-based recruiting, managerial development, performance management) of an established consulting firm and the depth of knowledge of highly specialized and trained public safety staff. We have been recognized as experts both in our profession and in various Federal District Court cases. Our research identifies us as leaders in the field of testing and validation. We have never failed to complete a contract. 229 Appendix A 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 1 Police Reading List for the Examination for Promotion to the Position of Sergeant (***THIS IS ONLY AN EXCERPT – IT IS NOT A COMPLETE LIST OF SOURCE SECTIONS ***) Police Directive Manual Directive 1: Law Enforcement Role and Authority * CLOSED Directive 2: Jurisdiction and Mutual Aid * CLOSED Directive 15: Planning and Research, Goals and Objectives, and Crime Analysis OPEN * Directive 16: Allocation and Distribution of Personnel and Personnel Alternatives * CLOSED Directive 25: Grievance Procedure OPEN * Directive 26: Disciplinary Procedures OPEN * Directive 35: Performance Evaluation OPEN * Directive 41: Patrol * CLOSED Directive 42: Criminal Investigations * CLOSED Directive 43: Vice, Drugs, and Organized Crime * CLOSED Directive 45: Crime Prevention and Community Involvement * CLOSED Directive 46: Critical Incidents, Special Operations, and Homeland Security * CLOSED Directive 54: Public Information * CLOSED Directive 55: Victim -Witness Assistance * CLOSED Directive 71: Processing and Temporary Detention * CLOSED Directive 81: Communications * CLOSED Directive 83: Collection and Preservation of Evidence * CLOSED Agreement with Fraternal Order of Police Grievance Procedure OPEN * Disciplinary Action OPEN * Internal Investigation OPEN * Holidays with Pay OPEN * Sick Leave * CLOSED Hours of Duty - Overtime Compensation OPEN * Safety and Health * CLOSED 230 Appendix A 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 2 Title 11 – Crimes and Criminal Procedure, Criminal and Traffic Law Manual Chapter 2. General Provisions Concerning Offenses OPEN * Chapter 5. Specific Offenses OPEN * Chapter 19. Arrest and Commitment; Fresh Pursuit * CLOSED Chapter 23. Search and Seizure * CLOSED Chapter 35. W itnesses and Evidence OPEN * Title 16 – Health and Safety, Criminal and Traffic Law Manual Chapter 9. Abuse of Children OPEN * Chapter 47. Uniform Controlled Substances Act OPEN * Chapter 51. The Department of Health and Social Services OPEN * Chapter 67. Authority of Fire Departments and Fire Police Within the State * CLOSED Title 21 – Motor Vehicles, Criminal and Traffic Law Manual Chapter 7. Enforcement; Arrest, Bail and Appeal * CLOSED Chapter 21. Registration of Vehicles * CLOSED Chapter 27. Driver’s License OPEN * Chapter 28. Habitual Offenders OPEN * Chapter 41. Rules of the Road OPEN * Chapter 67. Theft, Unauthorized Use and Damage to Motor Vehicles * CLOSED Chapter 68. Registration and Control of Off-Highway Vehicles * CLOSED Chapter 69. Removal of Motor Vehicles from Public Highways by Police * CLOSED 231 Appendix B 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 1 SAMPLE TECHNICAL KNOWLEDGE EXAMINATION CLOSED-BOOK TEST ITEMS 1. Officers may resort to the lawful use of firearms when they reasonably believe that it is necessary to: A. defend themselves against physical force. B. defend a third person against physical force. C. prevent the escape from custody of a person attempting to escape by the use of a deadly weapon. D. prevent the escape from custody of a person who has committed a felony, whether or not it involved the use of a deadly weapon. 2. When weighing the issues of probable cause in domestic violence incidents, officers should keep in mind that: A. the standards for determining probable cause in these cases are more stringent than in other criminal actions. B. in the absence of contradictory evidence, the victim's willingness to sign a complaint constitutes probable cause for an arrest. C. in cases of marital co-ownership, charges can be placed solely for property destruction only when damage to the victim's property exceeds 500 dollars. D. in cases of mutual combat in which one cannot determine the primary aggressor, it is preferable to charge neither party than to charge both. 3. If, after a suspect in custody has been given Miranda Warnings, the suspect elects to remain silent and does not wish to consult with an attorney, officers may: A. only re-approach the suspect if the suspect initiates further communication. B. re-approach the suspect after waiting a reasonable period of time and re-advising the suspect of his/her Miranda Warnings. C. not initiate any further contact with the suspect. D. not initiate any further contact with the suspect until the suspect has had an opportunity to confer with an attorney. 4. While monitoring your Department radio, you notice that one of your officers has failed to acknowledge an assignment from the Dispatcher. As the officer's immediate supervisor, you are responsible for: A. locating the officer and conducting a preliminary check of the working condition of the officer’s radio. B. personally handling the radio assignment and then initiating disciplinary action against the officer, if appropriate. C. checking on the welfare of the officer and then advising the Dispatcher of the situation. D. preparing a memorandum to the Section Commander, summarizing the assignment and reasons why the officer failed to respond. 232 Appendix B 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 2 5. A person is guilty of burglary in the first degree if the person: A. is homeless and forcibly enters an apartment building to sleep in the basement. B. lives in a rooming house and enters another resident’s occupied, unlocked room to steal a wristwatch. C. enters a store during business hours, hides until the store closes, and then steals a diamond necklace. D. forcibly enters a bank through the roof at 3:00 a.m. to steal cash from the vault. SAMPLE TECHNICAL KNOLWEDGE EXAMINATION OPEN-BOOK TEST ITEMS 1. One of your officers fails to qualify with the Department-issued handgun. As a supervisor, it is your responsibility to: A. ensure that the officer is detailed to the Firearms Training Section until the officer completes the qualification course. B. confiscate the officer’s ID folder and badge, and place the officer on restricted duty, until the officer qualifies. C. immediately forward the officer’s ID folder and badge to the Material Supply Section, after placing the officer on suspension status, until the officer qualifies. D. provide remedial handgun qualification training for the officer until the officer qualifies. 2. A supervisor has identified a member suspected of abusing sick leave. The first step the supervisor should take is to inform the member that he/she has established a pattern of suspected sick leave abuse and to: A. give the member a chance to explain. B. give the member a reasonable period within which to demonstrate some improvement in attendance. C. notify the member that a medical excuse will be required for further absences. D. warn the member that formal charges will be filed if any more unexcused absences are taken. 3. During a scheduled road check, a probationary officer asks you about the proper usage of dealer plates. You would be correct in advising him that dealer plates may: A. be used by the purchaser of a vehicle for a period of up to seven days after taking possession of the vehicle which will be registered in an adjoining state. B. be used for the purpose of delivering a vehicle to a location for the installation of security equipment. C. not be used for the purpose of operating a vehicle after it has been repossessed and is being delivered. D. not be used for the purpose of testing a vehicle for various systems, functions or features. 233 Appendix B 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 3 4. A 16-year-old male was suspected by officers of being involved in the sale of narcotic drugs. One day, an officer stops the youth on a minor traffic violation and requests permission to search the vehicle he is operating. The youth agrees. Upon searching the vehicle, the officer finds a quantity of cocaine. Under these circumstances,: A. the search was bad, because juveniles are not considered by the court to be capable of consenting to any search. B. the cocaine will be inadmissible, because a youth cannot consent to such a search without a parent or guardian being present. C. the court hearing the matter will consider the totality of the circumstances in determining the voluntariness of the youth's consent to the search. D. the evidence seized must be deemed inadmissible unless the youth's parent or guardian gives post facto authorization for the search. 234 Appendix C 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Dis tribution 1 SAMPLE ESSAY STYLE IN-BASKET EXERCISE ITEM The following is an example of the types of items that can be included in the Written In- Basket Background Information Packet. This is only one example of a variety of documents that can be included. Phone Call Message Date: Wednesday, June 5, 2002, 0925 hours To: Captain Howitt Fm: Inspector Beckett “I received a phone call this morning from the leader of a national animal rights group who was complaining because the group’s demonstration at the Capitol was interrupted yesterday by teenagers throwing eggs and yelling obscenities. The animal rights group is planning a demonstration again at 12:00 pm today and does not want to have the same problem. Please handle the situation.” The following is an example of the types of questions that may be included in the Written In- Basket Response Booklet. This is only one example of a variety of questions that may be included. “What actions would you take with regard to the phone call message in your in -basket? Describe the actions as fully as possible.” 235 Appendix D 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 1 SAMPLE ORAL WORK SAMPLE PERFORMANCE DISCUSSION EXERCISE You are a newly promoted sergeant assigned to a particular division for the past two months. During this time, you have gained a good understanding of your job and responsibilities, as well as the basic capabilities of the subordinate officers under your supervision. You feel that most of your subordinates are hard-working, and overall you are pleased with the way things are going. However, some officers who have worked with one particular officer, Officer Mickelson, have come to you suggesting that Officer Mickelson is "just not officer material." Officer Mickelson has been on the department for 9 months. Other officers have complained about a number of problems including: failure to back them up properly, inability to communicate with citizens and suspects, and inadequate knowledge of policies and procedures (especially those dealing with evidence handling at the scene of a crime). The detective unit has also reported to you that Mickelson has displayed some evidence handling deficiencies. This morning you compared the Tape Loggers and the Radio Logs and found that Officer Mickelson did appear to have slow response times when backing up other officers. One particular incident occurred last week when Officer Mickelson took ten minutes to respond to a burglary call even though Mickelson was only three blocks away. By the time Officer Mickelson had arrived, the suspect had already fled the scene. Records also indicated that Officer Mickelson's fingerprints were found on some articles which Mickelson was supposed to package at the scene of yet another recent incident. The officers also report that they have put up with this behavior for the past nine months and are tired of it. They claim that they made complaints to the previous sergeant but that no action was taken. You feel that this situation must be brought under control before it gets any worse. You have decided to hold a private meeting with Officer Mickelson to discuss the situation and work toward a resolution. One member of the board will assume the role of Officer Mickelson. He/she will be introduced to you when you enter the room. Remember, you have 15 minutes to complete this exercise. It is up to you to keep track of the time. 236 Appendix D 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 2 SAMPLE INCIDENT SUPERVISION EXERCISE Instructions When you appear before the board you will be asked to provide an assessment of the incident presented below and an indication of what you would be thinking as you respond to the scene. You should discuss your options for responding and identify any additional resources you would need. After you complete your initial response, the assessors will present you with an update that may affect your initial assessment of the incident. Keep in mind that if you have not completed your initial response within 5 minutes, the assessors will interrupt you to present you with the update. You will have the remaining time to respond to this update. Your entire response must be completed within the 10- minute time period. Incident You are a newly promoted sergeant in the Police Department. It is Wednesday morning and you are at your desk responding to emails and monitoring the radio. At approximately 10:00 a.m., the dispatcher advises you that a suspicious package has been found at the foreign embassy. A janitor found the package while preparing a meeting room for a meeting that is to begin at noon. He called the police as soon as he found the package hidden in a plant situated next to the conference table. Several high level embassy officials are scheduled to attend the meeting. You advise the dispatcher that you are responding to the scene. While en route to the scene, the dispatcher informs you that Officer Quinn and Officer Sanchez have arrived on the scene and that Officer Quinn has confirmed the existence and location of the suspicious package. 237 Appendix D 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 3 SAMPLE CANDIDATE HANDOUT DURING INCIDENT COMMAND EXERCISE Instructions Based on the information presented below, provide us with an updated assessment of the situation and an indication of how you would proceed at this point. Again, discuss your options for responding and identify the additional information or resources you may need. Finally, identify the reporting/notifications that will be necessary and the training/equipment issues that will require follow- up, once the scene has been stabilized. Incident Update Upon your arrival to the scene you receive an update from the dispatcher that the Fire Department has just been notified and should arrive at the scene within several minutes. In the meantime, rumors have been spreading about a bomb in the building. The officers have been trying to keep embassy employees calm; however, the employees have now started to panic and are fleeing the building. One employee tells you that the Embassy has received similar threats over the past few weeks. In addition, you notice that a large crowd of curious by- standers has started to gather around the building. As you take command of the scene, a local news reporter approaches you. Take a little time (e.g., 30 - 60 seconds) to think about the situation and then respond. 238 Appendix D 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 4 SAMPLE ORAL WORK SAMPLE GROUP MEETING EXERCISE You are a newly promoted sergeant, assigned as a supervisor in a patrol division. During the past few weeks, you have been receiving numerous complaints from the business merchants in Hills Plaza regarding the lack of a police presence in the area of their businesses. One of their main concerns is the Wilks Bar and Grill, a liquor licensed establishment that has been a source of problems for years. In particular, the merchants are complaining about the loud music, under-age drinking and open drug dealing outside of the location. According to the merchants, potential customers are afraid to frequent the business area, particularly at night. Fights are a common occurrence, and the business community is extremely upset over a perceived lack of support from the Police Department. You have done some research into the problem and discovered that your officers have made some arrests in the area, but the problem never seems to diminish. When your officers do respond they disperse loiterers and, when appropriate, make arrests. However, as your officers have told you, they can't be everywhere at once. In an attempt to address the issue, you assigned an extra police car, when one was available, to patrol the Hills Plaza business area. The merchants appreciate the occasional extra car, but they see it as a temporary fix. What they actually want is a permanent foot patrol officer assigned to their area. You are now beginning to hear rumors that the members of the Hills Plaza Business Association are prepared to take their complaints to the Mayor. In an effort to address their concerns, you inform your Lieutenant that you would like to have a meeting with the Hills Plaza Business Association. The Lieutenant agrees that such a meeting would be constructive, but warns you that current personnel levels are strained, and it is highly unlikely that any additional personnel can be assigned to the area. Nevertheless, the Lieutenant gives you the go ahead for the meeting. You are now about to enter a meeting of the Hills Plaza Business Association. Three members of the Oral Board will assume the roles of members of the Association. They will introduce themselves to you when you enter the room. You should be prepared to answer their questions, evaluate their suggestions and respond to their objections. Remember that you have ten minutes to complete this exercise, and that it is your responsibility to keep track of time. 239 Appendix D 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 5 SAMPLE ORAL WORK SAMPLE STRUCTURED INTERVIEW EXERCISE QUESTIONS The following is an example of the types of questions that can be included in the Structured Interview Exercise. Behavioral-Type Questions 1. Describe a recent situation when you had to evaluate information. What factors did you include in your evaluation?  Follow-up to #1: What other information would have been useful to guide your analyses? 2. Describe a difficult decision that you had to make recently. What information did you seek before you made your decision?  Follow-up to #2: What was the result of your decision? Situational-Type Questions 1. Assume that as a newly promoted Sergeant you receive information from a reliable source that one of your employees is stealing from the Department. How would you handle this situation?  Follow-up to #1: What factors might influence your decision? 2. Assume that as a newly promoted Sergeant you have been assigned to meet with a business owner about your community policing efforts in the area. How would you handle this meeting?  Follow-up to #2: What would you do to prepare for any obstacles that you may encounter? 240 Appendix E 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not fo r External Distribution 1 SAMPLE CANDIDATE PREPARATION GUIDE TABLE OF CONTENTS INTRODUCTION ............................................................................................................................ iii DESCRIPTION OF EXAM COMPONENTS ................................................................................... 1 I. Description of Multiple-Choice Tests ................................................................................. 1 A. Date, Time and Location .............................................................................................. 1 B. Content ......................................................................................................................... 1 C. Administrative Logistics ............................................................................................... 2 II. Description of Oral Work Sample Test .............................................................................. 3 A. Date, Time and Location .............................................................................................. 3 B. Content ......................................................................................................................... 3 C. Administrative Logistics ............................................................................................... 4 D. Evaluation .................................................................................................................... 5 III. Description of Internal Review Board ............................................................................... 7 A. Date, Time and Location .............................................................................................. 7 B. Content ......................................................................................................................... 7 C. Administrative Logistics ............................................................................................... 7 D. Evaluation .................................................................................................................... 7 PREPARATION STRATEGIES FOR EXAM COMPONENTS ..................................................... 10 I. Preparation Strategies for Multiple-Choice Tests ........................................................... 10 A. How to Study .............................................................................................................. 11 B. General Multiple-Choice Test-Taking Strategies ....................................................... 15 C. Error Analysis ............................................................................................................. 18 II. General Oral Work Sample Test-Taking Suggestions.................................................... 20 III. Specific Preparation Strategies for the Oral Work Sample Test .................................... 21 A. Oral Expression .......................................................................................................... 21 B. Information Analysis ................................................................................................... 22 C. Judgment & Decision Making .................................................................................... 23 D. Interpersonal Relations .............................................................................................. 26 E. Planning & Organizing ............................................................................................... 26 F. Resource Management .............................................................................................. 27 241 Appendix E 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not fo r External Distribution 2 TABLE OF CONTENTS (cont.) SAMPLE ITEMS AND EXERCISES ............................................................................................ 28 I. Sample Closed- and Open-Book Test Items .................................................................. 29 A. Sample Closed-Book Test Items: Departmental Sources ......................................... 29 B. Sample Closed-Book Test Items: Textbook Sources ................................................ 30 C. Sample Closed-Book Test Items: Legal Sources ...................................................... 31 D. Sample Open-Book Test Items: Departmental Sources ........................................... 32 E. Sample Open-Book Test Items: Legal Sources ........................................................ 34 F. Answers to Sample Closed- and Open-Book Items .................................................. 35 II. Sample Oral Work Sample Problem ............................................................................... 36 A. Sample Problem: Subordinate Conference Exercise ................................................ 36 B. Sample Problem: Possible Responses ..................................................................... 37 CONCLUSION ........................................................................................................................ 40 242 Appendix F 611 N. Brand Blvd. 10 th Floor; Glendale, C A 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 1 SAMPLE ORAL ADMINISTRATION SCHEDULE Candidate Report (15)* Prep (35)* Break (5)* IC (15)* Break (5)* SC (15)* 1 7:45-8:00 8:00-8:35 8:35-8:40 8:40-8:55 8:55-9:00 9:00-9:15 Evaluate (25)* 8:55-9:20 9:15-9:40 2 8:25-8:40 8:40-9:15 9:15-9:20 9:20-9:35 9:35-9:40 9:40-9:55 Evaluate 9:35-10:00 9:55-10:20 3 9:05-9:20 9:20-9:55 9:55-10:00 10:00-10:15 10:15-10:20 10:20-10:35 Evaluate 10:15-10:40 10:35-11:00 15-Minute Break 10:40-10:55 11:00-11:15 4 10:00-10:15 10:15-10:50 10:50-10:55 10:55-11:10 11:10-11:15 11:15-11:30 Evaluate 11:10-11:35 11:30-11:55 5 10:40-10:55 10:55-11:30 11:30-11:35 11:35-11:50 11:50-11:55 11:55-12:10 Evaluate 11:50-12:15 12:10-12:35 60-Minute Lunch Break 12:15-1:15 12:35-1:35 6 12:20-12:35 12:35-1:10 1:10-1:15 1:15-1:30 1:30-1:35 1:35-1:50 Evaluate 1:30-1:55 1:50-2:15 7 1:00-1:15 1:15-1:50 1:50-1:55 1:55-2:10 2:10-2:15 2:15-2:30 Evaluate 2:10-2:35 2:30-2:55 15-Minute Break 2:35-2:50 2:55-3:10 8 1:55-2:10 2:10-2:45 2:45-2:50 2:50-3:05 3:05-3:10 3:10-3:25 Evaluate 3:05-3:30 3:25-3:50 9 2:35-2:50 2:50-3:25 3:25-3:30 3:30-3:45 3:45-3:50 3:50-4:05 Evaluate 3:45-4:10 4:05-4:30 10 3:15-3:30 3:30-4:05 4:05-4:10 4:10-4:25 4:25-4:30 4:30-4:45 Evaluate 4:25-4:50 4:45-5:10 Note - Time (in minutes) for each procedure is presented in parentheses 243 Appendix G 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 1 SAMPLE POLICE PROMOTIONAL EXAMINATION CANDIDATE SCORE REPORT – PART I I. Your Individual Test Component Scores Test Component Component Score t-Score Technical Knowledge Test 52 56.67986 Oral Assessment Center: Scenario-Based Exercises 30.83333 58.53392 Oral Assessment Center: Structured Interview 63.00000 62.02771 II. Your Overall Scores and Score Band Overall Scores and Score Band Composite Score 59.38058 Overall Test Score 88.97485 Seniority Points 5.78904 Final Score 94.76389 Score Band B III. Your Scenario-Based Exercise Competency Breakdow ns Competency Scenario-Based Exercises Scenario-Based Exercise Average Incident Supervision Subordinate Performance Oral Communication 8.00000 7.66667 7.83333 Interpersonal Interactions 8.00000 7.66667 7.83333 Analyzing and Deciding 8.33333 6.66667 7.50000 Managing Activities 8.66667 6.66667 7.66667 Total Scenario-Based Exercise Score: 30.83333 Note: Across all candidates, the Scenario-Based Exercise Com petency Ratings (within exercises and averages) range from 1.00000 to 9.00000. 244 Appendix G 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 2 SAMPLE POLICE PROMOTIONAL EXAMINATION CANDIDATE SCORE REPORT – PART I IV. Your Structured Interview Competency Breakdowns Competency Structured Interview Average Oral Communication 9.00000 Interpersonal Interactions 9.00000 Analyzing and Deciding 9.00000 Managing Activities 9.00000 Adaptability 9.00000 Accountability 9.00000 Initiative and Independence 9.00000 Total Structured Interview Score: 63.00000 Note: Across all candidates, the Structured Interview Com petency Ratings range from 1 .00000 to 9.00000. 245 Appendix G 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 3 Incident Command Feedback Report The assessors have provided you with the f ollowing feedback to help you interpret your scores on this exercise and continue to improve your performance in these areas. S = Area of Strength: As you review these statements, precede them with the phrase, “Your performance effectiveness was enhanced by the following behaviors:” NI = Area of Needed Improvement: As you review these statements, precede them with the phrase, “Your performance ef fectiveness would have been enhanced by attending to the following behaviors:” Oral Communication S You expressed your thoughts clearly and were easy to understand. S You provided sufficient detail in your response. S You provided a solid rationale to support your viewpoints. NI Presenting information in a more organized and logical sequence. Interpersonal Interactions S You kept appropriate individuals informed of relevant issues and updates to problems. S You demonstrated an appropriate sensitivity to, and concern for, others’ feelings and opinions. NI Seeking more input and discussion from others when making decisions. Analyzing and Deciding S You effectively identified the critical elements/problems presented. S You took steps to verify the accuracy of information/problems presented. S You considered and evaluated a variety of solutions before choosing the best one. Managing Activities S You managed your time effectively during the exercise to address all of the necessary activities. S You provided detailed information regarding the specific steps to be taken to implement your plan. S You identified all of the available resources needed to carry out your plan. S You retained the appropriate amount of responsibility for problem resolution. NI Defining more specific performance expectations for subordinates. S=Area of Strength, NI=Area of Needed Improvement 246 Appendix G 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 4 Subordinate Performance Feedback Report The assessors have provided you with the f ollowing feedback to help you interpret your scores on this exercise and continue to improve your performance in these areas. S = Area of Strength: As you review these statements, precede them with the phrase, “Your performance effectiveness was enhanced by the following behaviors:” NI = Area of Needed Improvement: As you review these statements, precede them with the phrase, “Your performance ef fectiveness would have been enhanced by attending to the following behaviors:” Oral Communication S You expressed your thoughts clearly and were easy to understand. S You expressed your thoughts in a concise manner. S You provided a solid rationale to support your viewpoints. S You provided your responses at the appropriate pace. Interpersonal Interactions S You kept appropriate individuals informed of relevant issues and updates to problems. S You demonstrated a willingness to help individuals tackle problems. S You demonstrated a clear understanding of how your actions affected others. Analyzing and Deciding S You effectively identified the critical elements/problems presented. S You took steps to verify the accuracy of information/problems presented. NI Developing solutions that more appropriately meet the demands of the situation. Managing Activities S You managed your time effectively during the exercise to address all of the necessary activities. NI Identifying more specific goals and/or objectives to be accomplished. NI Identifying resources better suited to perform each activity. S=Area of Strength, NI=Area of Needed Improvement 247 Appendix G 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 5 Structured Interview Feedback Report The assessors have provided you with the f ollowing feedback to help you interpret your scores on this exercise and continue to improve your performance in these areas. S = Area of Strength: As you review these statements, precede them with the phrase, “Your performance effectiveness was enhanced by the following behaviors:” NI = Area of Needed Improvement: As you review these statements, precede them with the phrase, “Your performance ef fectiveness would have been enhanced by attending to the following behaviors:” Oral Communication S You expressed your thoughts clearly and were easy to understand. S You expressed your thoughts in a concise manner. S You provided sufficient detail in your response. S You presented information in an organized and logical sequence. Interpersonal Interactions S You took steps to develop/maintain constructive working relationships. S You displayed respect for others’ values and perspectives that differed from your own. S You demonstrated a willingness to help individuals tackle problems. S You demonstrated a clear understanding of how your actions affected others. Analyzing and Deciding S You effectively identified the critical elements/problems presented. S You took steps to verify the accuracy of information/problems presented. S You generated practical solutions for resolving problems. S Your solutions met the demands of the situation. Managing Activities S You managed your time effectively during the exercise to address all of the necessary activities. S You identified specific goals and/or objectives to be accomplished. S You identified all of the available resources needed to carry out your plan. S You provided detailed explanations of assigned tasks to individuals. S=Area of Strength, NI=Area of Needed Improvement 248 Appendix G 611 N. Brand Blvd. 10 th Floor; Glendale, CA 91203 • www.psionline.com • (800) 367 -1565 PSI – Confidential -Not for External Distribution 6 C. Developmental Suggestions for the Oral Assessment Center To maximize your promotion exam experience, we suggest that you take some concrete steps to improve in each area identified in your individualized score report (Part I). Your candidate preparation guide presents some practical suggestions f or developing in each of the competencies assessed. In this document, additional suggestions are provided for the four competencies assessed in all three Oral Assessment Center exercises. Based on your individual performance, we encourage you to focus on the competencies and self-inventory questions within this document that are of particular relevance to the competencies where you may have achieved lower scores. These suggestions are not exhaustive and are provided only as a guide for further development. W e encourage you to engage in whatever additional activities or programs you have at your disposal to further develop your skills and abilities. ORAL COMMUNIC ATION DEFINITION: T his dim ension involves speak ing fluently b y expressing ideas and opinions in a clear, concise, and organized m anner. This dim ension also involves the abilit y to prom ote and/or defend ideas b y providing supporting information and presenting thoughts in a convincing and logical m anner. OVERVIEW: Developing ease in oral communication is an im portant and difficult skill to m aster. Presenting inform ation in an organized and logical m anner is a critical aspect of this dim ension. Outlining inform ation before speak ing is an excellent wa y to ensure that all of the im portant issues are covered. In addition, clearl y enunciating your words will greatly enhance your oral comm unication sk ills. W henever possible, practice your oral comm unication skills. Practice allows you to becom e m ore relaxed and effective when comm unicating orally with others. To build skills within the competency of Oral Communication, ask yourself such questions as: ▪ Do m y thoughts flow in a logical m anner? ▪ Have I expressed m y thoughts in a com plete, yet concise m anner? ▪ Have I used proper grammar when speak ing? ▪ Have I presented inform ation in a m anner that is appropriate for m y audience? ▪ Do I clearly enunc iate words? ▪ Do I speak at an appropriate volum e? ▪ Have I provided m y response at an appropriate pace, neither rushing it nor drawing it out? ▪ Am I able to rem ain calm and com posed when speak ing with others and/or giving presentations? ▪ Have I provided enough detail so that others will know what I am trying to conve y? ▪ Have I provided a clear and logical rationale to support m y viewpoints? ▪ Have I avoided contradicting m yself? ▪ Have I avoided distracting verbal m annerism s (e.g., frequent pauses, repeated use of “um”)? ▪ Have I avoided distracting phys ical m annerism s (e.g., lack of eye contact, pacing, tapping/clicking pen)? ▪ Does m y response address the issues/questions presented in the exercise? ▪ Have I summarized k ey points to em phasize their im portance? 249 PSI Services LLC Public Safety Clients and Positions Within Past Five (5) years* Client Positions Allentown, PA Police Officer Arlington County, VA Police Officer Chicago, IL Police Officer Cleveland, OH Police Officer Delaware State Police Trooper Greenville Hospital System Security Officer Massachusetts, Commonwealth of Police Officer and Trooper (Statewide) Massachusetts, Commonwealth of Department of Corrections Officer (Statewide) Middletown, DE Seasonal and Recruit Police Officers Milwaukee, WI Police Officer and Police Aide New Castle County, DE Police Officer New Jersey Department of Personnel Corrections Officer and Law Enforcement Officer (Statewide - 13 job titles) New York State Police Trooper Norfolk, VA Police Officer Oklahoma City, OK Police Officer Omaha, NE Police Officer Overland Park, KS Police Officer Pennsylvania State Police Cadet and Liquor Enforcement Officer Pittsburgh, PA Police Officer Port St. Lucie, FL Police Officer St. Louis University Security Officer St. Louis, MO Police Officer Springfield, IL Police Officer Suffolk County, NY Police Officer Wilmington, DE Patrol Officer Allentown, PA Firefighter Buffalo, NY Firefighter Cleveland, OH Firefighter Massachusetts, Commonwealth of Firefighter (Statewide) Norfolk, VA Firefighter Omaha, NE Firefighter St. Paul, MN Firefighter Springfield, IL Firefighter FIRE ENTRY POLICE AND CORRECTIONS ENTRY Note. All entry police and corrections clients have used (or, in the case of new clients, will use) some form of the Law Enforcement Aptitude Battery. All entry fire clients use the comparable Fire Service Aptitude Battery. ©2019 PSI Services LLC: Confidential Not for External Distribution Page 1250 PSI Services LLC Public Safety Clients and Positions Within Past Five (5) years* Client Positions Abington Township, PA Sergeant and Lieutenant Arkansas State Police Sergeant and Lieutenant Arlington County, VA Sergeant Boston, MA Detective, Sergeant, Lieutenant and Captain Charlotte-Mecklenburg, NC Sergeant, Lieutenant and Captain Chicago, IL Evidence Technician, Sergeant and Detective Cleveland, OH Sergeant, Lieutenant and Captain Delaware State Police Sergeant, Lieutenant and Captain Harvard University Police Sergeant and Lieutenant Honolulu, HI Chief Massachusetts, Commonwealth of Sergeant, Lieutenant, Captain, Deputy Chief and Chief (Statewide & Brookline) Massachusetts, Commonwealth of Department of Corrections Captain (Statewide) Metropolitan Transportation Authority, NY Sergeant and Lieutenant Miami, FL Sergeant and Captain Milwaukee, WI Detective, Sergeant and Lieutenant Montclair, NJ Sergeant, Lieutenant and Captain New Castle County, DE Sergeant and Lieutenant New York State Police Sergeant and Lieutenant Pennsylvania State Police Corporal and Sergeant Oklahoma City, OK Lieutenant and Captain Omaha, NE Sergeant, Lieutenant, Captain and Deputy Chief St. Louis, MO Sergeant, Lieutenant and Captain Wilmington, DE Sergeant and Lieutenant Cleveland, OH Lieutenant and Captain Massachusetts, Commonwealth of Lieutenant, Captain, District Chief, Deputy Chief and Chief (Statewide and Boston) Miami, FL Lieutenant, Captain and Chief Fire Officer Omaha, NE Apparatus Engineer, Captain, Assistant Fire Marshall, Battalion Chief, and Assistant Chief FIRE PROMOTION POLICE AND CORRECTIONS PROMOTION ©2019 PSI Services LLC: Confidential Not for External Distribution Page 2251 PSI Services LLC Public Safety Clients and Positions Within Past Five (5) years* San Antonio, TX Fire Engineer, Lieutenant, Captain and District Chief St. Louis, MO Captain and Battalion Chief *A full project list from 1981 to the present can be provided upon request. ©2019 PSI Services LLC: Confidential Not for External Distribution Page 3252