As students were rushing to finish up assignments and exams in the week before winter break, their teachers were facing their own work crunch. Handed down from on high in the last days of the semester was a directive for teachers to submit--before the semester’s end--behavioral/mental health assessments for all of their students to a new system the district is using, Review 360, brought to us by the education mega-corporation Pearson.

Many teachers understandably balked at this for a multitude of reasons, including: the mountain of work it would mean in a compressed timeframe (some teachers at the middle school level can have well over a hundred students); the fact that parents had received no notification about the data being collected and redistributed about their children and had therefore not consented to the use of this new system; the apparent assignment of responsibility for data security by Pearson to individual teachers; possible access to student mental health data by a corporate education giant already making tremendous profits off of students; and the imprecision of the assessments and the potential future harm to individual students based on a teacher’s necessarily quick completion of that student’s profile.

The teachers’ union, United Educators of San Francisco (UESF) brought teachers’ concerns to the district in a formal way and the Board of Education (BOE) saw a presentation on and discussed Review 360 at their February 5, 2013 Committee of the Whole meeting. Unfortunately, despite the walkthrough by a product representative (introduced as having been both a school psychologist and a principal in Texas for over 30 years) that meeting did little to address the host of concerns swirling around the use of Review 360.

Originally developed by Psychological Software Solutions, Inc. (acquired by Pearson in July 2012), Review 360 is one of several tools that company sees as advancing its fundamental philosophy that “…we could use technology to help teachers better manage disruptive student behavior by making it easier for them to implement research-based strategies that we know work in the classroom.” Review 360 is intended to make it cheaper and easier to handle challenging students and it works by having teachers collect and enter detailed information about student behavior into a program that tracks and analyzes those data, that generates student profiles and reports that can be used at various levels of aggregation by administrators, and, according to the demonstration at the BOE meeting, can suggest tips for handling problem situations that are intended to be more constructive than sending students to the counselor’s office.

Seeking to do something other than shuffling students onto someone else is a good thing, but the reality that generates these types of referrals is significantly removed from this tool’s formulaic vision. The factors contributing to student behavior are incredibly varied in nature, with many dimensions at play, from learning differences to underlying cultural conflicts and miscommunication between teachers and students that is often invisible to the teacher, especially given that most teachers are white, while most of our students are not. As several of the BOE commissioners strongly noted, SFUSD has a track record of disproportionately identifying African-American students as needing either disciplinary action or special education services. In these circumstances, finding a more convenient mechanism for capturing perceptions of behavior doesn’t seem like a first priority intervention into the educational process. Better training in how to recognize and bridge cultural divides does.

Even if one could somehow set aside these larger contextual factors, Review 360 provides a very questionable mechanism for capturing assessments. SFUSD is focusing on the “Universal Screening” portion of this tool, which can be seen in the screen shot below.

Figure 1. Review 360 Universal Screening Input Window (from Pearson online user manual)

In this portion of the system, a teacher sees the set of students to evaluate and is asked to select from a set of extremely ill-defined values indicating how well each student measures up against a set of descriptive attributes. For instance, teachers must decide if a student “Never, Rarely, “Sometimes,” or “Frequently” is “Defiant or oppositional to adults,” “Fights or argues with peers,” or “Gets angry easily.” Common sense and reams of research tell us that one teacher’s “Sometimes” will be another teacher’s “Frequently.” Further, what one adult may see as oppositional behavior, another adult might see a precocious leadership or debate skills and any of these assessments are confounded by the socio-economic and cultural gaps between teachers and students.

The possibility for variance and error here is tremendous and the fact that these unstructured “assessments” may be used to make decisions about a given student or even worse, may turn into a persistent data point on student’s health record as that student grows into adulthood is even more concerning. Do we know if and how Pearson may be using these data and how these assessments may feed into the growing online profiles that are now commonplace in our networked world? If a child is identified as being a frequent liar today, will that be a mark against that student when he or she is all grown up and applying for jobs, health care or an apartment? A recent lawsuit against the federal Department of Education about violating student privacy and weakening the protections against sharing data with third parties indicates that this is a real and immediate issue.

The above concerns make Review 360 a worrisome tool to have in our schools and the SFUSD implementation of the system, even in this pilot mode, only compounded the problems. Student privacy and the consent of parents and guardians went by the wayside. No notices were sent to families in advance of the system implementation, and while the District’s legal counsel has asserted that according to the Family Educational Rights and Privacy Act (FERPA), there is no need for parental consent, it would have been far preferable to have taken that step. Without knowledge of the use of this assessment tool, parents and guardians have no ability to question any evaluations of their children. Although the range of expertise teachers need in practice is expanding to include all kinds of non-academic skills, teachers in general are not trained mental health providers, but this tool is asking them to make those kinds of evaluations. Additionally, teachers using this tool are not even receiving direct training. Instead, SFUSD is using a train-the-trainer approach and is even allowing school sites to determine at their own discretion which blocks of teachers should be asked to provide these assessments. Given that the teachers making these assessments are not mental health experts and given the problems previously described with disproportional identification of African-American students, this second-hand training is a false economy and a potential harmful misstep. In response to this at the BOE meeting, Superintendent Carranza compared these assessments to the vagueness of academic assessments and the fuzzy difference between an “A” and a “B” for any given teacher and across teachers. But the comparison is not apt; teachers undergoing extensive training in evaluating student work and they themselves are evaluated in that effort. Equally critical, students and parents get to see these academic assessments and can initiate e a conversation about them with the teacher and with administrators if need be. No such framework exists around the Review 360 assessments.

The impulse for bringing in Review 360 appears to be a good one--SFUSD is interested in identifying students who may need extra support as early as possible, but that is only one piece of this very complicated puzzle of educational inequity and this tool may introduce just as many problems as it is meant to solve.

Lisa Schiff is the parent of two children in the San Francisco Unified School District and is a member of Parents for Public Schools of San Francisco.