Changing STEM Education to be More Inclusive

The Seattle Public Schools partnered with me and my team at UW Seattle to revise instructional units to better include English learners and non-native English speaking stduents. My dissertation work analyzed how users at every level engaged with designed materials, leading to a boom in usage on the now-nationally important STEMteachingtools.org.

Problem
Solution Scope & Specifications

Design Process:

Teacher Surveys
Ethnography
First Findings
Second Design Attempt
Third Iteration
Key Learning

The Client's Problem: English learner students were not getting access to rich opportunities the current materials provided. This was deeply inequitable. We knew emerging multilingual students needed in-depth opportunities to practice speaking and listening in order to fully engage with the curriculum.

Scope of Solution: I led a 9-person research effort at the University of Washington and in Seattle Public Schools to synthesize student experience of these materials across 7 classrooms in order to deeply inform the field of improvements to be made in science education. My published STEM Teaching Tools have had 100,000+ views and continue to gain wide use by teachers nationally.

Tools

LucidChart
SurveyMonkey
SPSS
Interaction Analysis
Ethnography
Design-based Research
Excel
Word
Social Network Analysis

Team

Me (grad student & former teacher- turned UX researcher)
35 codesigner teachers
4 district leaders
6 collaborating university researchers
200 cooperating students

My Role

Take ethnographic fieldnotes and design surveys for teachers and students to take
Study in situ implementation of curricular materials

Teacher Surveys

In year 1-2, I led the design and analysis of a survey to understand teacher implementation of our codesigned materials. We used matrix Likert items to assess usefulness of each of the events. I customized the survey to include a "teaser" to remind teachers what they'd learned each day during the institute. Because of wifi constraints, I administered these on paper and hand-entered before analysis. I definitely prefer digital surveys!

Nowadays, I constrain entries with response validation as much as I can in order to get the data to come in clean and to reduce the likelihood of misinterpretation. I also test surveys ahead of time and have 1-2 users conduct cognitive interviews as they take the survey.

Ethnography

Teachers were invited to participate in classroom-level research, which included intensive audio and video recording, student work analysis, and some more interviews about teachers' experiences. I worked 1-1 with 7 teachers to learn how their students used the design materials.

We selected teachers who had diverse classrooms, prioritizing lower socioeconomic neighborhood averages, higher percentages of students classified as ELL, and neighborhoods that were geographically spread out across the district.

I made 110 clasroom visits in 9 months, recording 200+ hours of video and synthesizing them into fieldnotes. I built strong relationships with teachers and co-taught a number of lessons. And of course, I enjoyed being with students as they

First Findings

First Failure - Teachers weren't doing what we hoped.

One of the key findings from this project was the types of teacher talk moves that remained unsupported by our materials until 2016. With a couple of exceptions, teachers did not use the tools we designed to shift science learning to being student-centered, as teachers largely remained the holders of knowledge. We needed tools to help teachers ask better questions that were rooted in the science content.

A brainstorm session with science teaching expert Tana Peterman helped us connect the dots - we needed a table or flowchart of protocols to support teachers in building students' ideas as they were talking! The first version of the talk protocol digital tool was born.

Second Attempt

I shared this organizer in print format at a large teacher meeting in15 minutes in the summer of 2015. I got a ton of questions and sideways glances.

Despite my preparations and the printouts I shared, what I offered that day was too complicated and difficult to read. We needed to simplify. We needed some kind of common user interface that would make it easier to make decisions. We needed to use human-computer interaction principles.

We identified that the key use case was teachers who are making fast decisions on the fly in their classrooms. (Teachers make a LOT of decisions.)

Findings

We tracked Emerging Multilingual Learners through the 7 classrooms and used statistical analysis to figure out their participation, based on turns of talk and units of argumentation they brought up. I used SPSS to track their speech and Excel to build more qualitative, by-student analysis. We did not find a significant difference between classrooms with low, medium, or high use of these talk types, but it was initial and we did not anticipate any kind of correlation of that sort.

We did find an interesting relationship between students who were asked by a peer to "say more" about an idea. Multilingual learners added more ideas than their English-only counterparts, suggesting particular importance of strategies that deepen the quality of questions that peers can ask each other.

If classroom talk strategies were so helpful to multilingual learners, how could we improve our designs to help them include more of these moves?

Third Iteration

I worked with graphic designers and layout experts to design a digital tool, the Talk Formats Flowchart, that would guide teachers quickly through a selection process to choose an activity to better include multilingual learners.

The resulting tool had 2,500+ views after only 3 months of use. I continue to hear about the tool's wide usefulness at every conference I attend. It remains published on the editorially reviewed stemteachingtools.org housed in the prestigious University of Washington Institute for Math + Science Education.

Key Learnings

Access can be unpredictable. Even if you follow the rules for universal design, users can encounter barriers. These barriers are opportunities for us to learn to design better tools. I learned how to develop what people need, to stay close to the best data, and to use theoretical frameworks to guide design and analysis.

Page updated

Google Sites

Report abuse