Fidelity of Implementation of Research-Based Instructional Strategies (RBIS) in Engineering Science Courses


Increasing attention is being paid to improvement in undergraduate science, technology, engineering, and mathematics (STEM) education through increased adoption of research-based instructional strategies (RBIS), but high-quality measures of faculty instructional practice do not exist to monitor progress.


American Society for Engineering Education (ASEE). (2009). Creating a culture for schol-
arly and systematic innovation in engineering education: Ensuring U.S. engineering has the
right people with the right talent for a global society. Washington, DC: Author.
American Society for Engineering Education (ASEE). (2012). Innovation with impact.
Washington, DC: Author.
Anderson, S. A. (1997). Understanding teacher change: Revisiting the concerns based
adoption model. Curriculum Inquiry, 27(3), 331–367.
Association of American Universities (AAU). (2011). Undergraduate STEM Initiative.
Retrieved from
Barkley, E. F., Cross, K. P., & Major, C. H. (2005). Collaborative learning techniques:
A handbook for college faculty. San Francisco, CA: Jossey-Bass.
Bay View Alliance. (n.d.) Retrieved from
Bickman, L., Manuel, R., Brown, J. L., Jones, S. M., Flay, B. R., Li, K.-K., ... Massetti,
G. (2009). Approaches to measuring implementation fidelity in school-based program
evaluations. Journal of Research on Character Education, 7(2), 75.
Blakely, C. H., Payer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman,
D. B., & Emshoff, J. G. (1987). The fidelity-adaptation debate: Implications for the
implementation of public sector social programs. American Journal of Community Psychology,
15(3), 253–268.
Borrego, M., Froyd, J. E., & Hall, T. S. (2010). Diffusion of engineering education inno-
vations: A survey of awareness and adoption rates in U.S. engineering departments
Journal of Engineering Education, 99(3), 185–207.
Business-Higher Education Forum. (n.d.) Retrieved from
Byerley, A. (2001). Using multimedia and “active learning” techniques to “ energize” an intro-
ductory engineering thermodynamics class. Paper presented at the 31st ASEE/IEEE
Frontiers in Education Conference, Reno, NV.
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A concep-
tual framework for implementation fidelity. Implementation Science, 2(40), 1–9. DOI:

Cashman, E., & Eschenbach, E. (2003). Active learning with Web technology - Just in time!
Paper presented at the 33rd ASEE/IEEE Frontiers in Education Conference, Boulder,
Century, J., Rudnick, M., & Freeman, C. (2010). A framework for measuring fidelity of
implementation: A foundation for shared language and accumulation of knowledge.
American Journal of Evaluation, 31(2), 199–218.
Coller, B. D. (2008). An experiment in hands-on learning in engineering mechanics:
Statics. International Journal of Engineering Education, 24(3), 545–557.
Committee on Science Engineering and Public Policy. (2006). Rising above the gathering
storm: Energizing and employing America for a brighter economic future. Washington,
DC: National Academies Press.
Committee on Science Engineering and Public Policy. (2010). Rising above the gathering storm,
revisited: Rapidly approaching Category 5.Washington, DC: National Academies Press.
Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and
results. American Journal of Physics, 69(9), 970–977.
Dinan, F. (2002). Chemistry by the case. Journal of College Science Teaching, 32(1), 36–41.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-
based learning: A meta-analysis. Learning and Instruction, 13, 533–568.
Duffy, H., Barry, C., Barrington, L., & Heredia, M. (2009). Service-learning in engineer-
ing science courses: Does it work? Paper presented at the ASEE Annual Conference &
Exposition, Austin, TX.
Dukhan, N., Schumak, M. R., & Daniels, J. J. (2009). Service learning as pedogogy for
promoting social awareness of mechanical engineering students. International Journal of
Mechanical Engineering Education, 37(1), 78–86.
Ebert-May, D., Derting, T. L., Hodder, J., Momsen, J. L., Long, T. M., & Jardeleza, S.
E. (2011). What we say is not what we do: Effective evaluation of faculty professional
development programs. BioScience, 61(7), 550–558.
Emshoff, J. G., Blakely, C., Gottschalk, R., Mayer, J., Davidson, W. S., & Erickson, S.
(1987). Innovation in education and criminal justice:Measuring fidelity of implementation
and program effectiveness. Educational Evaluation and Policy Analysis, 9(4), 300–311.
Etter, J.-F., & Perneger, T. V. (1997). Analysis of non-response bias in a mailed health
survey. Journal of Clinical Epidemiology, 50(10), 1123–1128. DOI: 10.1016/S0895-
Fasko, D. (2003). Case studies and method in teaching and learning. Paper presented at the
Annual Meeting of the Society of Educators and Scholars, Louisville, KY.
Felder, R. M., & Brent, R. (2009). Active learning: An introduction. ASQ Higher Educa-
tion Brief, 2(4), 122–127.
Ferber, R. (1948). The problem of bias in mail returns: A solution. Public Opinion Quar-
terly, 12(4), 669–676.
Froyd, J. E. (2005). The Engineering Education Coalitions Program. In National Acad-
emy of Engineering (Ed.), Educating the engineer of 2020: Adapting engineering educa-
tion to the new century. Washington, DC: National Academies Press.
Froyd, J. E., Borrego, M., Cutler, S., Henderson, C., & Prince, M. (in press). Estimates
of use of research-based instructional strategies in core electrical or computer engineer-
ing courses. IEEE Transactions on Education.
Gabel, C. (1999). Using case studies to teach science. Paper presented at the Annual Meet-
ing of the National Association for Research in Science Teaching, Boston, MA.
Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys.
Public Opinion Quarterly, 70(5), 646–675. DOI: 10.1093/poq/nfl033.
Hake, R. R. (1998). Interactive-engagement vs. traditional methods: A six-thousand-
student survey of mechanics test data for introductory physics courses. American Journal
of Physics, 66(1), 64–74.
Hall, G. E., & Loucks, S. F. (1978). Innovation configurations: Analyzing the adapta-
tions of innovations. Austin, TX: University of Texas at Austin Research Development
Center for Teacher Education.
Henderson, C., & Dancy, M. H. (2009). The impact of physics education research on
the teaching of introductory quantitative physics in the United States. Physical Review
Special Topics: Physics Education Research, 5(2), 020107.
Henderson, C., Dancy,M. H., & Niewiadomska-Bugaj,M. (2012). The use of research-based
instructional strategies in introductory physics: Where do faculty leave the innovation-
decision process? Physical Review Special Topics: Physics Education Research, 8(2), 020104.
Hsiung, C. M. (2012). The effectiveness of cooperative learning. Journal of Engineering
Education, 101(1), 119–137.
Jiusto, S., & DiBiasio, D. (2006). Experiential learning environments: Do they prepare
our students to be self-directed, life-long learners? Journal of Engineering Education,
95(3), 195–204.
Johnson, D. W., Johnson, R. T., & Smith, K. A. (1998a). Active learning: Cooperation in
the college classroom (2nd ed.). Edina, MN: Interaction.
Johnson, D. W., Johnson, R. T., & Smith, K. A. (1998b). Cooperative learning returns
to college: What evidence is there that it works? Change, 30(4), 26–35.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and develop-
ment. Englewood Cliffs, NJ: Prentice-Hall.
Koretsky, M., & Brooks, B. (2011). Comparison of student responses to easy and difficult
thermodynamics conceptual questions during peer instruction. International Journal of
Engineering Education, 27(4), 897–908.
Levin, B. (1997). The influence of context in case-based teaching: Personal dilemmas, moral
issues or real change in teachers’ thinking? Paper presented at the American Educational
Research Association, Chicago, IL.
Lochhead, J., & Whimbey, A. (1987). Teaching analytical reasoning through thinking
aloud pair problem solving. In J. E. Stice (Ed.), Developing critical thinking and
problem-solving abilities. New Directions for Teaching and Learning, no. 30 (pp. 73–
92). San Francisco, CA: Jossey-Bass.
Lundeberg, M. A., & Yadav, A. (2006a). Assessment of case study teaching: Where do
we go from here? Part 1. Journal of College Science Teaching, 35(5), 10–13.
Lundeberg, M. A., & Yadav, A. (2006b). Assessment of case study teaching: Where do
we go from here? Part 2. Journal of College Science Teaching, 35(6), 8–13.
Lundeberg, M. A., Levin, B., & Harrington, H. (1999). Who learns what from cases and
how? The research base for teaching and learning with cases. Mahwah, NJ: Erlbaum.
Macdonald, R. H., Manduca, C. A., Mogk, D. W., & Tewksbury, B. J. (2005). Teaching
methods in undergraduate geoscience courses: Results of the 2004 On the Cutting
Edge survey of U.S. faculty. Journal of Geoscience Education, 53(3), 237–252.
Macias, C., Propst, R., Rodican, C., & Boyd, J. (2001). Strategic planning for ICCD
clubhouse implementation: Development of the Clubhouse Research and Evaluation
Screening Survey (CRESS). Mental Health Services Research, 3(3), 155–167.
Mazur, E. (1997). Peer instruction: A user’s manual. Englewood Cliffs, NJ: Prentice-Hall.
Mazur, E. (2009). Peer instruction: An overview. Retrieved from http://www.turning-talk.
McNamee, L., Roberts, T., & Williams, S. (n.d.). Online collaborative learning in higher
education. Retrieved from
Millis, B., & Cottell, P., Jr. (1998). Cooperative learning for higher education faculty.
Washington, DC: American Council on Education.
Mills, S. C., & Ragan, T. J. (2000). A tool for analyzing implementation fidelity of an inte-
grated learning system. Educational Technology Research& Development, 48(4), 21–41.
Modesitt, K. L., Maxim, B., & Akingbehin, K. (1999). Just-in-time learning in software
engineering. Journal of Computers in Mathematics and Science Teaching, 18(3), 287–301.
Morgan, G., Leech, N., Gloeckner, G., & Barrett, K. (2011). IBM SPSS for introductory
statistics (4th ed.). New York, NY: Taylor & Francis.
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria:
Development, measurement, and validation. American Journal of Evaluation, 24(3),
315–341. DOI: 10.1177/109821400302400303.
National Research Council (NRC). (2012). Discipline-based educational research: Under-
standing and improving learning in undergraduate science and engineering. Washington,
DC: National Academies Press.
National Science Board. (2010). Preparing the next generation of STEMinnovators: Identifying and
developing our nation’s human capital. Arlington, VA: National Science Foundation.
Newson, T., & Delatte, N. (2011). Case methods in civil engineering teaching. Canadian
Journal of Civil Engineering, 38, 1016–1030.
Novak, G. M., Patterson, E. T., Gavrin, A. D., & Christian, W. (1999). Just-in-time
teaching: Blending active learning with Web technology. Upper Saddle River, N.J.:
O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implemen-
tation and its relationship to outcomes in K-12 curriculum intervention research.
Review of Educational Research, 78(1), 33–84. DOI: 10.3102/0034654307313793.
Oakes, W. C. (2009). Creating effective and efficient learning experiences while addressing the
needs of the poor: An overview of service-learning in engineering education. Paper presented at
the American Society for Engineering Education Annual Conference, Austin, TX.
Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurements, design, and analysis: An inte-
grated approach. Hillsdale, NJ: Erlbaum.
Pilzer, S. (2001). Peer instruction in mathematics. PRIMUS, 11(1), 185.
President’s Council of Advisors on Science and Technology. (2012). Engage to excel:
Producing one million additional college graduates with degrees in science, technology, engi-
neering, and mathematics. Washington, DC: Executive Office of the President.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engi-
neering Education, 93(3), 223–231.
Prince, M., & Felder, R. (2006). Inductive teaching and learning methods: Definitions,
comparisons, and research bases. Journal of Engineering Education, 95(2), 123–138.
Prince, M., & Felder, R. (2007). The many faces of inductive teaching and learning. Jour-
nal of College Science Teaching, 36(5), 14.
Prince, M., Borrego, M., Henderson, C., Cutler, S., & Froyd, J. (in press). Use of
research-based instructional strategies in core chemical engineering courses. Chemical
Engineering Education.Reeves, T., & Laffey, J. (1999). Design, assessment, and evaluation of a problem-based
learning environment in undergraduate engineering. Higher Education Research &
Development, 18(2), 219–232.
Rogers, E. M. (2003). Diffusion of innovations. New York, NY: Free Press.
Rozycki, W. (1999). Just-in-time teaching. Research & Creative Activity, 22 (1). Retrieved
Rubin, S. (1996). Evaluation and meta-analysis of selected research related to the laboratory
component of beginning college level science (Doctoral dissertation). Retrieved from Pro-
Quest Dissertations & Theses Full Text. (9623799)
Santi, P. (2007). Have they got it yet? Assessing student understanding of difficult concepts.
Paper presented at the ASEE Annual Conference and Exposition, Honolulu, HI.
Shooter, S., & McNeill, M. (2002). Interdisciplinary collaborative learning in mecha-
tronics at Bucknell University. Journal of Engineering Education, 91(3), 339–344.
Shymansky, J., Hedges, L., & Woodworth, G. (1990). A reassessment of the effects of
inquiry-based science curricula of the 60’s on student performance. Journal of Research
in Science Teaching, 27(2), 127–144.
Smith, D. (1996). A meta-analysis of student outcomes attributable to the teaching of science
as inquiry as compared to traditional methodology (Doctoral dissertation). Retrieved from
ProQuest Dissertations & Theses Full Text. (9632097)
Springer, L., Stanne, M. E., & Donovan, S. (1997). Effects of small-group learning on
undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Mad-
ison, WI: National Institute for Science Education.
Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning
on undergraduates in science, mathematics, engineering, and technology: A meta-anal-
ysis. Review of Educational Research, 69(1), 21–51.
Sudman, S. (1985). Mail surveys of reluctant professionals. Evaluation Review, 9(3), 349–
360. DOI: 10.1177/0193841X8500900306.
U.S. Department of Education. (2007). Report of the Academic Competitiveness Council.
Washington, DC: Author.
What Works Clearinghouse. (2011). Procedures and standards handbook. Washington, DC: United States Department of Education Institute of Education Sciences.