Why?
Scientists and science communicators who lead public engagement events often ask themselves: How do I know if they are getting it? Are they with me? Am I making a difference? Helping them answer such questions motivated the development of the On-the-Spot Feedback (OTSF) project.
How?
Through a research-to-practice National Science Foundation (NSF) project, over five years we developed a set of on-the-spot feedback (OTSF) strategies and professional development experiences to help scientists and other science communicators with their public engagement activities. This includes research-based guidance on effectively communicating science, monitoring audience interest and understanding, and using this information to make improvements to their science outreach. The project has developed resources for doing public engagement with OTSF strategies and tactics, and we had several professional development workshops and an active research & evaluation component investigating what has worked best.
Who?
The project has been led by the Astronomical Society of the Pacific (ASP), in collaboration with Oregon State University, the Portal to the Public Network via the Institute for Learning Innovation, and the National Radio Astronomy Observatory. We engaged in professional development with about 250 scientists during the project from multiple disciplines (including astronomy, physics, biological sciences, and environmental sciences), and they helped us research and refine our ideas.
When?
The project has had NSF funding for 5 years starting in 2018. Ongoing work includes communicating findings from the project, publication of research papers, and planning for follow-up work and funding.
OTSF Feedback-Response Cycle and OTSF Tactics
The project has developed, tested and refined a set of strategies and ideas depicted in the adjacent graphics here. These are the focus of the core Section 1 (“Using the Feedback-Response Cycle,” pp. 9-21) of the OTSF Guide. The seven OTSF tactics provide mechanisms to understand the motivation, interest, prior knowledge and understanding of one’s audience. They are not independent of each other, and in practice two or more tactics are often used together. Effective use of all the tactics is also supported by strong Listening and Observing, foundational skills of OTSF.
ON THE SPOT FEEDBACK ACTIVITIES
Professional Development
Over the course of the project, we conducted several professional development workshops online and in-person, for about 250 scientists from multiple disciplines. These helped us develop and refine our ideas and guidance for scientists doing public engagement, and we certainly “walked the walk” by incorporating feedback from participating scientists both “on-the-spot” and in subsequent workshops. We greatly appreciate the input provided along the way, especially from (a) the original “design testers” we did professional development with during the project’s earlier years, and (b) the scientists who wrote up their usage of OTSF tactics and contributed the 30+ examples provided in the Appendix of the OTSF Guide (pp. 65-137).
Research
The research focused on the scientists and their use of the OTSF strategy. Using the theory of planned behavior, the research sought to understand how the scientists changed based on the use of the OTSF strategy. The research team used a mixed-methods approach to collect data to better understand and validate the findings. Data collection included pre- and post-surveys using the theory of planned behavior, a self-report following the use of OTSF, observations, and interviews.
Evaluation
Evaluation helped the OTSF project team understand the process and experience scientists went through as they encountered, learned about, and decided to use (or not use) the OTSF approach in their public engagement. The evaluation zoomed in on the journey of learning and applying the technique, including revealing dynamics that could hinder or promote uptake of the model broadly.
