Street Data: A Next-Generation Model for Equity, Pedagogy, and School Transformation
By Shane Safir and Jamila Dugan
In one of my favorite books about education, 2014’s Getting Schooled, the Vermont author and educator Garret Keizer describes slogging through a particularly distasteful assigned reading as part of a Personalized Learning Community (PLC) group at the school where he teaches. While Keizer objects to book’s slippery claims, its lack of evidence, its business-and-industry-inspired approach, and its dubious insistence that schools and teachers with limited budgets can – must – affect radical social change, he nevertheless manages to take a step back and to appreciate what’s good about the book:
“Taken at its best,” he writes, “and shucked of its corporate jargon, the authors’ argument sounds like a modest and perfectly reasonable appeal for schools to focus on what they’re actually able to do . . . as opposed to what they might wish to do” (61).
Change just a few words and you’d have the perfect description of Shane Safir and Jamila Dugan’s 2021 education book, Street Data: A Next-Generation Model for Equity, Pedagogy, and School Transformation. Taken at its best and shucked of its Critical jargon, Safir and Dugan’s argument sounds like a modest and perfectly reasonable appeal for schools to use qualitative data to improve culture and climate and student agency, as opposed to relying on reductive standardized testing data.
***
The first thing that’s noteworthy then, of course, is the blanket of Criticality that covers the book. It was an interesting read because even though this book was only written two years ago, because the movement for Critical Pedagogy is already cooling off, and because much of its then-cutting-edge rhetoric has been absorbed in / more carefully scrutinized by the mainstream, it’s the first book I’ve read that has caused me to think, “This book is very 2021.”
The Criticality in Street Data is both a philosophy of education as well as a means of inciting change. Street Data is – like Keizer’s book above – very much of the “school transformation” genre of ed books: it starts by ratcheting up the rhetoric, trying to get educators to buy into the urgent need to change the status quo. Back in 2011 (the era Keizer was writing about), the inciting cause was an apparently abysmal performance on standardized tests. By 2021, the inciting cause in Street Data is systemic oppression. The authors hit hard right out of the gate with the Critical rhetoric, ramping up the stakes that require you to act on the pressing educational change that they’re going to show you how to accomplish.
The Critical jargon is much more recognizable to me now (and less jarring) than it might have been in 2021, and it is heavy in the first few chapters. The term “white supremacy” is mentioned on page 1, and there follow many now-familiar Critical terms such as “systemic racism,” and “intersectionality,” along with other Critical terms like “spirit murdering,” “scientific colonialism,” and “decolonized ways of being.” There’s also the ubiquitous “characteristics of white supremacy culture” chart, courtesy of Tema Okun, and a whole section on ten types of “false equity.”
There is also the standard “urgency of transformation” rhetoric – the promise of a “next-generation model” for “school transformation,” the boogeyman of the NCLB-era, no-excuses neo-liberal goal of academic proficiency as measured by standardized tests, and the reminder that the authors’ chosen goal, equity, is not only urgent, but is a “moral imperative.”
Still, Criticality is not just a heap of rhetoric designed to incite action, but is also responsible for the book’s underlying goal. Although Street Data’s explicit goal is equity-as-learner-agency (which I’ll get to later), the actual, unstated goal of the book is equity as defined by Critical Pedagogy : the use of educational reform to liberate those who are oppressed due to particular identity-group characteristics (in the case of this book, mostly race). All the hallmarks of Critical Pedagogy are here in the supporting analysis:
- The view of education as an inherently discriminatory practice in the United states (we must transform the “white supremacist structure of schooling” [11]).
- The Critical-Marxist reproduction theory that the educational system “reproduces” or “perpetuates” unjust outcomes as a result of its biased foundations (“ . . . our systems, practices, and narratives are designed to perpetuate disparities in outcomes for marginalized students” [29) – and the imperative that educators must “acknowledge” this fact (29).
- The belief that reform is not incremental but transformational and radical.
- The belief that the reform requires the dominant group to take its cues by listening closely to the nondominant group (Safir writes that her aim in writing this book is no less than to “nourish a process of truth and reconciliation that frees us all from the grip of white supremacy” [11]) – facilitated by ethnographic / qualitative research and Freireian, quasi-dialogic educational methods.
“Truth and reconciliation” – ! That’s surely one of the most dramatic goals of education reform that I have ever read. But once again, the rhetoric is not just a wakeup call to change – it’s rooted in the very philosophical underpinnings of Critical Pedagogy, which posits the truth claim that oppression in a real and true sense occurs regularly in educational settings (“I am certain,” writes Safir, “that I participated in racism and white supremacy without being aware of it” [10-11]).
***
But what is interesting about Street Data is that after the first few chapters, Safir backs away from the strident rhetoric, and – somewhat – from the Critical goal. For example, when she really drills down to what she means, she does not call for a more traditional, Critical, Freire-ian educational outcome, such as the ability to understand – and the skills to change – the realities of oppression and discrimination in the world. That would be “equity” in the Critical sense, but that’s not where she goes.
Instead, when she does define “equity” (which takes until page 29), it is in fairly conventional, small-l-liberal terms: equity is “an approach to ensuring equally high outcomes for all [students] by removing the predicability of success or failure that currently correlates with any racial, social, economic, or cultural factor” (29). This sounds so jarring coming on the heels of “truth and reconciliation” – largely because this definition of equity sounds so typically American in its “anyone can succeed, no matter their origins” Jeffersonianism.
What’s even more surprising is the way that Safir defines “high outcomes.” She writes, “Our equity efforts truly begin when we redefine success as the cultivation of student agency and realign our measures of success to this goal” (100 – emphasis mine). So it is “agency” that equity efforts should aim toward, and Safir classifies four specific components of agency: a healthy valuation of one’s identity, the attainment of mastery, a sense of belonging, and a sense of efficacy.
So if one is connecting the dots (given the author’s earlier definition), equity means something akin to, ”an approach to ensuring equally high degrees of agency for all students.”
It’s a surprisingly ordinary goal, especially for a book that began with such dramatic, provocative rhetoric. It’s a little hard to tell, but I think Safir is pointing toward the notion that underserved students are rarely made to feel a sense of agency – and they should be. Certainly if that’s what she means by “equity,” then that’s hard to argue with. This is always a challenge I have with ed books – people are so often writing from such different perspectives, from such different local contexts, that what makes sense in one context might sound passe or unrealistic in another.
Still, taken in an objective sense, away from local context, the goal of “agency” points toward Safir’s broader educational progressivism, which is clearly both a response to what she sees (and which I too see) as the restrictive, reductive focus on basic “content” of the Test-and-Punish era, as well as a harkening back to a time – which Safir reminiscences about fondly – when she began her career in the 1990s afforded a great deal of freedom to teach and assess students in a more capacious way. Safir, in Deweyian fashion, says almost nothing in Street Data about content; her focus is on school-wide culture and climate and on individual dispositional outcomes. Indeed, student well-being is clearly an important goal for Safir; later in the book she makes this explicit, writing that her approach “represents a shift from evaluating outcomes to valuing well-being . . . a state in which educators and students experience healing, agency, joy, and connection . . . ” (76).
But as with Dewey, dispositional goals – or any sort of trans-academic goals – carry with them always a sense for me that they are something of a secondary or incidental outcome. For example, Safir highlights a former student who, after delivering a successful capstone presention in front of his mother, “felt an overwhelming sense of agency in having shared his knowledge publicly.”
“What test,” Safir wonders, “could possibly capture that?”
As a statement of response to the NCLB era, that’s surely a valid sentiment. But it is still the wrong question to ask. Surely no test can register a student’s emotional reaction to a situation (other than perhaps a psychological test). The better question is, “What test could possibly engender that feeling of agency?” And the answer is – clearly a challenging capstone project, with an relevant, engaging purpose, an authentic audience, with high standards and specific metrics of success in traditional academic disciplines, such as research, presentation, and public speaking. The student she describes felt a sense of accomplishment because he had completed a challenging assignment. That is an important feeling that we want all students to have, but to say that his elation or self-efficacy is the primary goal of the assignment seems to me to put the cart before the horse. Where, after all, did those high standards come from? Later in the book, Safir recommends that educators both use standards-based grading as well as a portrait of a graduate as vehicles for creating their *own* definitions of specific educational goals (and she even gives a few examples). Like Dewey, it’s as though Safir doesn’t want to take sides in these disputes about what students really need to know.
The meat of the book is devoted to a description of the titular method that the authors propose to collect data in order to improve – transform, really – the school culture: “street data” – which means on-the-ground qualitative data (or, as they put it in some places, ethnographic research) collected from students and families, particularly those “on the margins” in a given community. Safir defines street data as “a practitioner-driven layperson’s framework for conducting qualitative research in service of school transformation that drives toward equity and deep learning” (60).
There is much fanfare about the importance of this method and about the transformational potential of it – both in the form of Critical jargon that dramatically raises the stakes of the conversation, and also in good, old-fashioned urgency-of-transformation language. But when you brush past the 2021 rhetoric, you start to realize that what Safir is actually proposing to drive this transformational equity work is actually just basic qualitative research. When she listed her chosen prescriptions, I was very much underwhelmed: distributing surveys, holding focus groups, conducting exit interviews with students, funding “learning visits” to other schools, listening campaigns, ethnographies, and fish bowl sessions.
What I found hardest were that these actual prescriptions (at least until the last few chapters) were amazingly brief. In one instance, describing the methods that she advises for understanding research findings, Safir doesn’t even bother with a real description; instead she shares a link to a website where readers can presumably learn more (87). In the section on how to measure learner agency (which, again, Safir has suggested should be the new educational lodestar), all Safir has to recommend is that educators “administer a ten-question pre- and post-survey on a 1-4 Likert scale,”or “conduct agency interviews with a sample of students,” or “have students regularly complete a single-point rubric reflection on agency” (105-106). These are surely tried-and-true techniques of qualitative research, but they are hardly revolutionary suggestions, and hardly fleshed out with the sort of examples, details, and real-world case studies I would have expected.
Even more frustrating was the lack of recognition of the obvious challenges of qualitative research, which is time-consuming – often prohibitively so for most educators. But Safir never even acknowledges this challenge. The only reason she can think of for why educators fail to make “home visits” (presumably after the school day, which she recommends in order to gather “street data” from student families) is because teachers either don’t want to impose, or are simply afraid. Hmm, I wonder if there could be any other reasons why teachers don’t make house calls after school?
It was hard to tell sometimes who the intended audience of the book was: teachers, or administrators? Focus groups seem important to Safir, and so does improving school climate, and these seem like a stretch to imagine teachers focusing on. Either way, there is zero recognition that most educators simply don’t have time to do – to say nothing of training in – qualitative research. Time is money, after all, and qualitative research is often expensive in this way, but Safir glides right past this issue, assuring us this type of research can be done at “no cost.” The only thing Safir says as to why teachers rarely conduct qualitative research is that they are being “robbed of the opportunity” (60).
Nor does she pause to consider – or explain how to avoid – any of the other basic problems with conducting qualitative research: the problem of small sample sizes being unreliable, the difficulty of analyzing the data findings, or the susceptibility of a researcher’s pre-existing bias to influence the participants, the framing, or the conclusions.
This last issue – a researcher’s bias influencing the results of the research – seems particularly concerning in light of Safir’s insistence that educators embark on the process of ethnographic research with a firm ideological lens (Criticality) already in place. There is a tension here, in my view, between the belief in the importance of conducting research in the community and the belief in the importance of employing all of the things that Safir has spent the book telling us we should employ: a focus on Criticality, on “equity,” prizing student agency, and so one. She reminds educators that the truth they hear from marginalized stakeholders in interview sessions may be hard to hear, but what if that is because those parents believe the school is – let’s say – not being strict enough in punishing individual students? What if what one hears from the community is not a call for more Deweyian progressivism, more portfolios, more defenses-of-learning – but a call for more traditional, content- and book-based educational essentialism, more drill-and-kill, more rigid requirements? After all, a disdain for NCLB-style test-and-punishment is one thing, but good people in any community can disagree about educational goals and methods of attaining those goals – and what happens when what they want doesn’t look like what you thought it would be?
And then there’s another tension in the book: Safir is setting up street data and qualitative measurement as something of a holistic, humanistic foil for the cruelty of standardized testing and 20,000-foot-high “satellite data.” But in a sense, she is comparing apples to oranges. After all, most of her qualitative research really seems to be trying to take the temperature of the community, to measure culture and climate, rather than to measure educational outcomes (as NCLB was trying to do). Is this qualitative data a reliable or good way to measure student learning, on its own? That’s a hard question, and one that Safir doesn’t really address. Later in the book she seems to suggest standards-based grading as a way for teachers in classrooms to evaluate student learning, but the larger question still remains: How do we evaluate student learning more broadly?
In other words: how can the public, paying for our schools, know that students are learning? How can those who are running the schools and the school districts know that students are learning?
This is a really interesting question and one that I, as a classroom teacher, think about often. After all, if I design a rich, authentic assessment (which Safir, again, later in the book, seems to champion) and most students do well, then I will know students did well, but how is anyone outside the classroom to know that my students are really learning? And to what extent should they need to? These are challenging questions. Clearly NCLB and similar testing movements were inspired by a belief among the public that schools needed greater accountability – that they weren’t doing a good enough job of teaching students important information and skills, and that the public, which pays for the schools, needed a way to monitor schools’ effectiveness. In my view, most teachers (and I would include myself) are fairly defensive about this notion. “If we say they’re doing well, they’re doing well,” we want to tell anyone outside the classroom. But of course, the public does have a right to know, and perhaps, if we’re all going to be adults about this, we can admit that somewhere deep down this idea could be said to make intuitive sense.
But how can we do this? How can we ensure that students really are learning without resorting to some sort of norm-referenced standardized test? Do we really need to? These are hard questions, but ones that are not answered in the book.
And then there’s the question of quantitative research. Safir really critiques it hard in the book. It’s one thing to attack the legacy of NCLB, but it’s another to strawman quantitative research by portraying it as amounting to no more than the worst of NCLB. NCLB and the Big Tests were (are) bad not because they gave us quantitative data, but because they gave us bad quantitative data – reductive, limited, often-misleading – and because their high political stakes incentivized perverse and damaging changes to curricula, instruction, and sometimes to the material future of the school itself. And although they were intended to highlight the gaps between different groups of students in order to incentivize positive change, they too often unfairly labeled as failing many underprivileged students. But to conclude based on this experience that quantitative data itself is rotten to the core is a non sequitur.
And just as Safir never really attempts to address the challenges of qualitative research, she never really addresses how we got to such an overreliance on data in the first place, why that was so seductive. It’s not that she doesn’t try, but her attempt to do so is very misleading – really the most troubling passage in the book for me – when she attempts to trace our overreliance on big tests back to “the Western theory of knowledge known as empiricism” (14). The whole section seemed highly dubious to me; Safir quotes a book called Decolonizing Methodologies, which says that in “Western epistemology,” “Understanding is viewed as being akin to measuring. As the ways we try to understand the world are reduced to issues of measurement, the focus of understanding becomes more concerned with procedural problems . . . and of developing operational definitions of phenomena which are reliable and valid” (14).
It seems to me very misleading to say that the empirical tradition sees understanding as “akin to measuring.” I think it’s more accurate to say this tradition sees understanding as arising from observation, rather than from innate ideas or from received wisdom. Safir attempts to align her work not with this “white research” and “outsider research” – as she tells us it is called [14] – but with “indigenous and Afrocentric epistemologies” and “indigenous ways of knowing” and “ancestral knowledge” (17) which value the wisdom of tradition and of experience. Once again, I think Safir is really just trying to remind us of the importance of looking beyond cold data and into experience, rather than actually advising us to really jump ship to some kind of tribal knowledge epistemology. To her credit, later in the book when she is advising educators on how to analyze the qualitative data they’ve collected, Safir seems to adhere to fairly middle-of-the-road methods for analyzing data. But I had a hard time with this section because it was hard not to feel as though Safir was trying to score political points in the Critical tradition by strawmanning “western epistemology” while making a show of deferring to other “ways of knowing” (a showy, disingenuous move that I’ve never liked about the Critical movement), as well as the fact that she did such a shoddy job of showing us the true causes of our obsession with educational data (which are real, destructive, and which have been richly explored by other authors many times).
***
What’s interesting about Street Data is that the book progresses, the Critical rhetoric calms down, the strategies proposed get more sensible, and the explanations become fuller. Halfway through the book this starts happening. The fifth chapter is all about the teaching approach that Safir recommends to best achieve agency. While the chapter starts with some boilerplate Freire / Critical language about the need for a “pedagogy of voice,” suddenly we turn a corner, the Critical stuff is gone, and we’re talking about some fairly standard, sensible, left-of-center-but-not-too-far-left progressive-ed mainstays: allow revision, emphasize feedback over grades, reflection, portfolios, discussion-based strategies like Socratic seminars, authentic projects. Suddenly we’re spending almost ten pages learning in some depth about these strategies. For me, this is fairly routine stuff that I do now; for others, perhaps not – but at least it felt substantial and focused.
The next chapter is more of the same, with twelve pages spent discussing the importance of developing a profile of a graduate and designing a performance-based assessment system. Again, this was nothing new to me, but these are good, solid ideas that transcend the Critical approach, and Safir does even better in this chapter to include actual examples from real schools and practitioners.
Chapter seven is even more interesting, making the case for a practice Safir calls “public learning,” which seems to be a version of the classic “lesson study” protocol used to help teachers collaboratively analyze and improve their teaching. It’s here that the book seems most geared toward administrators, with lines such as, “As long as we ‘teach’ educators through a pedagogy of compliance, we will reproduce a pattern of passivity . . .” (149). Even still, the chapter feels very thin on details and specifics about how to find the time to make this work. The authors helpfully remind admin not to “make this one more thing,” but there aren’t many details on how to do this. Like conducting qualitative research, lesson study groups are great in theory, but very, very time-consuming in practice, and Safir doesn’t offer much help.
Interestingly, in chapter eight, toward the very end of the book, Safir returns us to “street data,” circling back again, giving us essentially the same ideas as before (interviews, dialogues), but with more depth and explanation. I didn’t understand why this section was so far away in the book from where she’d begun to discuss these techniques. Again, there was just as little recognition of the reality that qualitative research is time-consuming for busy educators. Her advice is: “Instead of speed-walking through school buildings and classrooms checking off tasks, lean into the luxury of slowing down and absorbing the little stories you encounter in the corridors and stairwells. Become an ethnographer rather than a building or classroom manager” (176). Sure, thanks – I’ll get right on that.
And as if the book were losing momentum, chapter eight returns us to the Critical rhetoric, with the obligatory Tema Okun chart of the characteristics of “white supremacy culture.” This section is an interesting blend of Brene Brown and Tema Okun. “Shame is a key tool in the playbook of white supremacy culture,” Safir reminds us (172).
***
Ultimately I like qualitative research a lot, and clearly the Big Testing movement was a failure, but I didn’t find Street Data’s methods particularly plausible. I’d love to be a school that’s inviting stakeholders to comment all the time, and to help set the direction – especially the people who aren’t normally part of the conversation – but that’s very, very time-consuming to run and to debrief and act on. Still, it’s a good goal. That said, I didn’t find Street Data particularly helpful in this sense. It didn’t really have more than passing references on how to do this sort of research, didn’t have long sections about how normal teachers or administrators can train themselves to conduct ethnographic research, how they can find the time in their schedules, and didn’t explain in detail case studies in the past when groups of educators have done this successfully. I’m sure it has been done, but that wasn’t really in the book.
Again, I thought this was a really compelling and interesting book, but I’m afraid that Street Data will stick in my memory mostly as a striking artifact of what the temperature was like in 2021.