Holmes+-+Proposal



Prepared for Far West Laboratory Research & Development by San Carlos Educational Consulting Evaluation Proposal for Determining Instructional Purposes

J. Raphael Holmes EDTECH 505 Dr. Perkins 10/4/11

Far West Laboratory for Educational Research and Development (FWR) has requested evaluation proposals for its training program Determining Instructional Purposes (DIP). This document is an evaluation proposal by San Carlos Education Consulting (SCEC).

Determining Instructional Purposes The DIP program is a set of three training units designed to help school administrators and educational administration graduate students learn how to plan school programs successfully. A client coordinator implements the training, and this coordinator does not need specific content knowledge beyond what is gained from previous participation in the program or examination of the course materials. FWL estimates that the training time for the individual units is 10-18 hours, and the program is flexible regarding the way this time is divided. The three units are Setting Goals, Analyzing Problems, and Deriving Objectives. Along with the Coordinator’s Handbook, the units are available only in print form, and they range in length from 155 to 259 pages. As they are designed to be self-contained, the units can be used individually or in combination with one another, according to client preferences. Each unit is sub-divided into four to six modules, which contain readings, practice activities led by the coordinator, and suggested feedback. Many activities entail group work and are grounded in realistic invented scenarios. The cost of the Coordinator’s Handbook is $4.50 per copy, and each unit is available for $8.95 per copy, or $24.95 for a set of three.

Evaluation Method The evaluation will be comprised of the following components: • Expert evaluation of content- Content will be evaluated by one in-house subject matter expert (SME) together with two outside experts. They will provide feedback on the accuracy and quality of the content, for individual modules, individual units, and combinations of units. Our in-house expert will give feedback in the form of a written report, and the two other experts will be independently interviewed after they are given the opportunity to familiarize themselves with the materials. Besides helping FWR decide whether to disseminate the DIP program, this component of our evaluation can help fulfill the secondary purpose of the evaluation, acting as a review and guide for interested school administrators.

• Trial runs- SCEC will arrange for multiple trial runs of DIP components, in either a Bay Area graduate school or school district setting. Each trial run will be observed by a SCEC consultant, and if possible, each trial run will include one full unit of the DIP program. The total target observation time is 40 hours, with 6-14 trainees per trial. Trials will be followed by surveys to the trainee participants regarding their appraisal of the module(s) in terms of format, appeal, and applicability, and surveys will also gauge their interest in continuing to participate in the program. Quantitative, Likert scale components of surveys will be used to describe overall reactions and opinions, while SCEC staff will compile responses to qualitative questions into meaningful overall trends. SCEC will also conduct guided telephone interviews with trial run coordinators to determine their opinions of the program. The guiding focus of information gathering will be to determine whether trainees feel they have benefitted, and if so, in what tangible forms they see these benefits taking shape. The primary purpose of these trials is to give FWR information to inform their decision regarding deployment of the program, but this may also result in material useful to communication with potential clients.

• Attitudes of potential clients - SCEC will take advantage of its nationwide connections to K-12 educators and graduate educational administration instructors to distribute electronic surveys and receive completed surveys from a minimum of 40 potential clients. These instruments will include items assessing the need for a program like DIP, the expected presentation modality, and degree of fit with current practices in school training or graduate school instruction. Survey data will again be both quantitative and qualitative.

• Additional fact finding- SCEC will access and mine publically available data to learn more about the numbers of potential trainees in school districts across the country. SCEC will also contact 5-12 school districts with which it has a close relationship to get general numbers for training budgets in the past five years. Task Schedule The task schedule below assumes a start date of 11/14/2011, leading to completion on 4/16/2012. Dates can be moved forwards or backwards depending on FWR’s desired starting or completion date. Private website detailing progress will be made available to FWR and updated biweekly.

Task Responsible Party Completion Date • Meet with FWR representatives to discuss EPD and proposed evaluation, making any requested modifications. SCEC/FWR 11/14 • Submit to FWR all proposed data collection instruments, including surveys, interview protocols, and framework for SME evaluation. SCEC 11/28 • Provide SCEC with feedback on data collection instruments. FWR 12/12 • Submit finalized data collection instruments, profile of targeted survey respondents, and details for scheduled trial runs. SCEC 12/19 • Submit in-house SME evaluation and school budget information from select districts. SCEC 1/20 • Complete interviews with outside SMEs, complete electronic surveying of potential clients. Submit summary of results. SCEC 2/10 • Videoconference with FWR representatives to discuss SME evaluation and summary of potential client survey data, along with data from any completed trial runs. SCEC/FWR 2/19 • Complete all trial runs and finish collecting associated survey and interview data. SCEC 3/30 • Submit final report to FWR. SCEC 4/16

Bidder Qualifications San Carlos Educational Consulting has been providing evaluation and guidance to educational institutions since 2002. Our clients include schools, school districts, post-secondary institutions, and creators of educational and training resources. Two of our most recent projects were an evaluation of the NSF-funded high school pre-service physics teacher training camp (PPTTC) and a large-scale evaluation of the practicum component of San Francisco State University’s nurse practitioner degree program. We have ample experience evaluating instructional packages, and we are eager to furnish references from our many satisfied clients. Lead Project Personnel Dr. Cynthia Transler is our chief evaluator and the founder of SCEC. She will lead the authoring data collection instruments and will act as the primary point of contact with FWR. She holds an Ed.D. degree from Boise State University, and her work has been published in numerous educational journals. Dr. Transler has authored a highly regarded textbook on educational evaluation, now in its 4th edition. J. Raphael Holmes is our in-house subject matter expert on school administration training, and he will also be in charge of observing trial runs. He has been working with SCEC since 2005. He holds an MS in educational administration from Drexel University, and he has worked as an educational administration instructor at Rutgers University. Budget and Payment The proposed budget of $36,570 is on the following page. Required DIP materials have been tentatively priced at half the sale price detailed by FWR in the RFP, but this is subject to recalculation at FWR’s discretion. $12,000 of budget is to be submitted as payment by 12/12 and the completion of task 3, with the next $12,000 installment paid prior to the 2/19 videoconference a nd the remainder paid upon submission of completed evaluation proposal. Proposed Budget

Personnel

Dr. Cynthia Transler: 40 days at $475/day J. Raphael Holmes: 30 days at $275/day Other project staff: 40 days at $200/day $19,000 $8,250 $8,000

Total personnel expenses $35,250 $35,250 Travel and accommodations

3-day round trip for two: San Francisco to Portland Accommodations and per diem in Portland Mileage for field testing $500 $380 $80

Total travel and accommodations expenses $960 $960 ICT and supplies

Electronic survey tools DIP materials (5 coord. hdbks, 20 full sets of units) $100 $260*

Total ICT and supplies expenses $360 $360 Total budget

$36,570


 * As previously noted, DIP materials have been tentatively priced at half the sale price detailed by FWR, subject to recalculation at FWR’s discretion.