Proving the Correlation Between Poverty and Academic Success

One of the joys of summer is that I finally have the time to process the various reports that I gather during the school year but don't always have a chance to read; I'm starting off with Report Card on Washington State's Elementary Schools 2009 from the loyal opposition at the Evergreen Freedom Foundation.

What they've done is actually quite impressive--a rank order, from 1st to 1,127th place, of all the elementary schools in Washington State. I have some qualms about the methodology--these rankings are completely WASL-driven, which makes them rank in another sense of the word--but if we look at the report more as a discussion piece and less as an evaluative measure, it has some potential value.

(Might as well dispense with the snark right away--I'm a union guy, a teacher union guy, and as such fairly well disposed to not like anything, ever, that the EFF does. That said, I've had pleasant e-mail exchanges with a number of their staffers, and while their beliefs are diametrically opposed to mine, I won't question the sincerity of what they think.)

I think the over-reach, though, can be found right in the introduction to the report:

Previous studies have shown that variations in student results from school to school cannot be accounted for solely by the personal and family characteristics of a school’s students. Many other factors—including good teaching, counseling, and school administration—contribute to the effectiveness of schools. Indicators like low income describe past relationships between a socioeconomic characteristic and a measure of school effectiveness. It should not be inferred that these relationships will, or should, remain static. The more successfully schools enable all their students to succeed, the weaker will be the relationship between the family characteristics of students and their academic success. Thus, this socioeconomic indicator should not be used as an excuse for poor school performance.

Laudable, certainty. The great legacy of the Bush administration will be No Child Left Behind, and I say that with the utmost sincerity--the focus on disaggregated scores, and measuring all sub-groups, and making sure that all kids are coming along--that's what we should have been doing since the beginning. It's never a comfortable conversation, but it's the most important one we can have. That said, in the context of failing schools and failing kids, "excuse" is a loaded word. Poverty and race should never be the excuse for a school's low performance, but to ignore those metrics as even being factors would be a hell of a mistake to make.

And frankly, the report itself backs me up on this. Creating tables in Blogger is nigh-on impossible, but here you can see a spreadsheet that I made comparing the top 15 schools in the state, those that earned a perfect 10 on the rankings, to the bottom 20 schools in the state, those that earned less than a 2.0 on the scale. The interesting pieces:

• Of the bottom 20, 19 of them are majority-minority schools, where the majority of the students come from minority populations. Only Oakville Elementary breaks even, with 50% caucasian students.
  
• On the other hand, in the top 15 schools, white students are always the majority (all at 60%+). In 13 of the 15 the largest minority group is Asian students; in the other two (both from Spokane), the largest minority group is mixed-race students.
  
• Each of the bottom 20 schools has higher than 70% of the kids coming from low-income households. 4 of the schools are tribal schools and don't report the percentage of their kids who are low income. Of the 16 reporting schools, the average percentage of low-income students is about 83%.
  
• At the top 15 the average percentage of low-income students is 6%. One school (Libby Center of Spokane) has 23% low-income; if you take it out of the picture, the average drops to less than 5%. At two of the top 15 schools, less than 1% of the kids are low-income.

The Fraser Institute, which has been doing these analyses on Canadian schools for a number of years now, is quite explicit in calling the high schools winners and the bottom schools losers.

By their metrics, then, the way to be a winner is clear: be well-off and white. It's also very explicit who the report thinks the losers are. That's a shame.

Read more here, if any.

Randy Dorn on the WASL: KILL IT WITH FIRE!

The News Tribune sez it is so, and thus it will be so. I particularly like this comment from "jcathall", who may have been Randy's campaign manager because I don't know who was:

Can you give me one example of how Randy can add origniality and creativity to the process?

Personally I don't think Randy is competant to do the job. I only hope that he is incompetant enough not to hurt anything effectivly before my children graduate highschool.

People accuse Obama of having a Cult of Personality around him, but the Google shows us a lot about what people think Randy is going to do. From a search for "Randy Dorn will":

• ...be a forceful advocate for school funding
  
• ...replace the WASL with a simpler, fairer test
  
• ...replace the WASL with a better, fairer test
  
• ...replace the WASL with a fair and useful exam that is diagnostic
  
• ...correct this waste of taxpayer dollars by replacing WASL
  
• ...replace the WASL with a cheaper, fairer, less time-consuming, test
  
• ...Provide Leadership for School Funding

...et cetera, et cetera.

The take-home, then, is that Randy has a hell of an agenda facing him, but there's also a ton of pressure coming from the Legislature via the Basic Ed Finance Taskforce, as well as the various special interest groups like my own WEA.

Good luck and God speed, Superintendent Dorn. You're going to need it.

Read more here, if any.

Thoughts on the OSPI Race

Well, that happened.

Many were looking for a Gandolf to coming riding over the horizon and save the day, but that person never materialized. The six declared candidates are Terry Bergeson, Randy Dorn of the Public School Employees, and four perennial also-rans. At this point Bergeson has to be considered a lead-pipe cinch to win, buoyed as she is by positive WASL articles and the lack of any serious competition. The biggest name in the field after her is Randy Dorn, but when you haven’t even managed to get a decent campaign website up with two months to go until the primary, you’re not giving your campaign a fair chance.

What does Terry going back to Olympia mean? The WASL keeps the strongest supporter it has, though the legislature has taken some of the starch out. Interest groups like the WEA and Where’s the Math? take a licking, because they have been outspoken in their distaste for the incumbent. Strong standards remain the order of the day, though with frequent refinements.

Anyway you slice it, my summer just got a lot less busy. I wonder if the OSPI candidates will debate at all?

Read more here, if any.

WASL Test Administration Quiz Release Items

Actual questions from the WASL proctoring training test. And by actual, I mean mostly made up.

1) During the WASL test many students are confused by question #6. Mrs. Smith stops the class and shows them how to approach answering the question. Is this definitely OK, definitely not OK, or too close to call?

Answer: Definitely not OK. You may not expand on any WASL questions.

2) During the test many students are confused by question #6. Mrs. Smith stops the class and berates them for not recalling the three different lessons that she taught on probability; after a 10 minute screaming jag she retreats to her desk and sobs gently into her morning espresso. Is this definitely OK, definitely not OK, or too close to call?

Answer: Definitely OK. Neither screaming nor crying are banned by the test.

3) As students are taking the reading test a group called Mothers Militantly Against the WASL storms the school, breaks into the secure test storage facility, and holds tomorrow’s science WASL hostage for a $1,000,000 ransom. The principal refuses to pay the money. Is this definitely OK, definitely not OK, or too close to call?

Answer: Definitely not OK. The principal should pay any ransom money out of their building budget, or Title II Part C funds.

4) After losing her election for Superintendent of Public Instruction in 2008 Terry Bergeson returns to the classroom. In April of 2009 KOMO News 4 helicopters take video of her on her knees in the school parking lot, plaintively screaming to the heavens for relief from the ungodly testing monster that she helped spawn. Is this definitely OK, definitely not OK, or too close to call?

Answer: Definitely OK, and a delightful image to boot.

5) A teacher refuses to give the WASL and becomes a folk hero with ballads written about him. Meanwhile, his kids still end up taking the test, only their teacher isn’t in the room to be with them through the process. Is this definitely OK, definitely not OK, or too close to call?

Answer: Definitely not OK. C’mon, Carl.

Read more here, if any.

With Apologies to Doctor King

At WERA in March month Dean Fink, keynote presenter, made a comparison between the civil rights movement of Dr. Martin Luther King Jr. and the standards movement of today. I was curious, so I went to a secret underground library at OSPI to do some research, and I was shocked to find this previously undiscovered yet eerily prescient speech that Dr. King never had a chance to give. I present it to you here for posterity’s sake.

I am not unmindful that some of you have come here out of great trials and tribulations. Some of you have come fresh from giving seniors the math WASL for the 5th time. Some of you have come from districts where your quest for excellence left you battered by the storms of standardization and deflated by the ignominy that is canned curriculum. You have been the veterans of stifled creativity. Continue to work with the faith that TRS will provide for you in the end.

Go back to Spokane, go back to Stehekin, go back to Ridgefield, go back to Shoreline, go back to the Twin Cities and the Tri-Cities and the Emerald City, knowing that through legislative fiat this situation can and will be changed. Let us not wallow in the valley of despair—let us build a Sylvan Learning Center there for the districts that are in the third stage of failing AYP.

I say to you today, my teachers, so even though we face a long series of difficult days that blend into difficult weeks, and even though the year is long but the hours are short, I still have a GLE. It is a GLE deeply rooted in the EALRs, and it is a GLE that all Washington students share. Allow me to tell you about my GLE.

I have a GLE from math, that one day our standards will rise up and live out their true potential: to be equal to the standards of Massachusetts and California, which are much higher than our own.

I have a GLE for reading, that one day on the wide plains of Lincoln County the sons of Almira and of Coulee-Hartline will be able to sit down together in a classroom and read a passage and make inferences from that passage and write about them.

I have a GLE for science that one day even a screwed up hellhole like Rochester, a district withering under a plague of methamphetamine, strapped by a long series of levy failures, will be transformed into an oasis of hypotheses and controlled experiments and independent variables.

I have a GLE so that one day any one of our children will be able to attend a school where their chance for success will not be a function of the color of their skin but of the content of their character.

I have a GLE today.

Let GLEs ring from the snowcapped Cascades of Grant County!

Let GLEs ring from the amber waves of Whitman County!

But not only that; let GLEs ring from the fogbanks of Olympia, the bars of Ballard, and the overpriced hotel rooms of Long Beach!

From every county, let GLEs ring!

And when this happens, when we allow GLEs to ring, when we let them ring from every 2B school and consolidated school district, from every 4A powerhouse where the high school has more kids than several towns put together, we will be able to speed up that day when all of Washington’s children, black kids and white kids, low-SES or high-SES, free lunch or brown bag, will be able to join hands and sing in the words of the old spiritual, “Free at last! Free at last! Thank Bergeson Almighty, we are free at last!”

Read more here, if any.

When In Doubt, Pick C

In Maryland the debate over how best to test kids is touching on an interesting question--can multiple choice questions work? From the Washington Post:

Maryland plans to eliminate written-response questions from its high school exit exams to address long-standing complaints about how slowly test results are processed, state education officials said yesterday.

Beginning in May 2009, the Maryland school system will phase out "brief constructed responses" and "extended constructed responses" -- questions requiring a short or long written answer -- from its four tests covering algebra, English, biology and government, said Ronald A. Peiffer, the state's deputy superintendent for academic policy.

Eliminating those questions will allow the state to process test results up to four weeks faster than before, Peiffer said. The timing of the change means that the Class of 2009, the first group for which the test will count, will still be responsible for composing written answers.

There's the speed argument, but can multiple choice tests still be valid measures of what students know? How do you account for guessing and the old-school Genesis fan who goes with "Abacab" over and over again?

It's a question that's come up here in Washington just recently. In February there was a proposal to scrap the WASL in favor of a multiple-choice assessment; despite some bi-partisan support, it didn't pass. There's been a hearty debate about whether the WASL is that much better than the ITBS that many schools discarded in favor of the mandated state assessment.

To my mind, a written response is always going to show you more about what the kids know than filling in the appropriate bubble will. Even the best constructed multiple choice question will be inferior to an essay question if you want to measure insight and application.

That said, I'm also a big booster of the MAP assessment from the NWEA, because it's sooooo damned easy to administer, you get the results back the second the test is over, growth can be measured in a snap, the ability to rank order and apply percentile rankings is meaningful--the data that my school gets from the MAP is vastly superior to what we get from the WASL, and I'm saying that as the data guy and one of the building test coordinators.

We can't solve this through local control; to be able to compare districts and comply with the demands of NCLB, OSPI has to bigfoot. If you heard tomorrow that Terry Bergeson had seen the light and wanted to switch the WASL to a multiple choice format, what would your reaction be?


For a different take on the Maryland story, try District Administration magazine. Diane Ravitch brings her usual thoughtfulness to the limitations of multiple choice testing here; EdWeek does a great job with her blog, and it's a place well worth checking out often. A research-minded look at how to write good multiple choice questions can be found here.

Read more here, if any.

And Lo, the WASL Scores Were Released

I drove back from Olympia yesterday, which was extremely fortuitous timing because it gave me a chance to pick up quite a few of the daily newspaper between Seattle and Spokane. The headlines are telling:

Most Seniors Pass WASL; Concern Over Lower Levels from the Seattle Times
Math is WASL's Big Stumbling Block from The Olympian
Good WASL, If You Don't Count Math sez the Seattle PI
WASL Math Trips Up Students, offers the Tacoma News Tribune

Clearly the headline, then, is the math scores, but we could have seen that coming because of all that went on during the legislative session. Dr. Bergeson talked about it quite a bit during her press conference, and one suspects that all that is going on with math is also the raison d'etre for all the work that the State Board of Education has put in to revisiting the math requirements (pdf).

(as an aside, Terry, dear--make sure your presentation on the screen matches what you're trying to say to the reporters. That was kind of embarrassing that you would have so many glitches in what is arguably your most important speech every year)

Also of interest to many would be the list of failing schools and districts, with thanks to Jim over at Washington Teachers for getting on the story early. I'm glad to not be on either list, and I give all the best to those who are.

Here we go again!

Read more here, if any.

Kramer vs. WASL

WASL time also means WASL Editorial Time. They're often repetitive, but I really liked this take by Stephen Kramer:

Last year, for the first time in WASL history, teachers were prohibited from looking at any test questions or completed test booklets. Now teachers have no way of knowing what is on the test or how their students responded. As I monitored students taking the test last spring, I knew many of the things I'd seen in the past were happening again:

• Students inadvertently turning two pages in a test booklet, and missing all the questions on those pages.
  
  
• Good writers who were stumped, became discouraged and shut down because they couldn't quite figure out how to respond to the assigned writing prompt they had a full day to work on.
  
  
• Students whose hands were so tired by the afternoons on writing test days (there are two) that, when they were copying their final draft work into the test booklets, they purposely skipped entire paragraphs or pages of their first-draft writing.
  
  
• Students who became so disheartened, after repeated unsuccessful attempts to solve a particularly difficult math problem, that they gave up, lost confidence and randomly marked answers to the rest of the questions in the section.
  
  
• Students who struggled for so long on a couple of particularly confusing math or reading questions that, when they read a "no-brainer" question, they were convinced that they were being tricked -- and skipped over the obviously correct answer because they didn't think it could possibly be right.

This idea of not being able to look at the test your kids just took is stupid beyond words; what manager worth his salt doesn't check what's being produced on his watch? Kramer has written previously about his experiences giving the WASL before; I'd be curious to hear his take on what a useful testing system would look like.

Read more here, if any.

RETREAT!

After the math WASL passed away in November, it was only a matter of time. From the Olympian:

Some lawmakers are floating the idea of delaying the Washington Assessment of Student Learning reading and writing high-school graduation requirements, piggybacking off a proposal to delay the math requirement.

Current state law requires members of the high school Class of 2008 to pass all three sections of the WASL in order to graduate.

Only 58 percent of the state's juniors have passed the 10th grade WASL math exam. There are measures pending in the Legislature, supported by Gov. Chris Gregoire and state schools superintendent Terry Bergeson, that would postpone the math requirement until 2011.

Students did better in reading and writing than math, with 87 percent having passed the reading exam and 86 percent having passed the writing exam.

But Sen. Rosemary McAuliffe, D-Bothell, chairwoman of the Senate education committee, and her House counterpart, Rep. Dave Quall, D-Mount Vernon, said they're open to the possibility of delaying those requirements also.

"We can't just do math and not think about the kids who failed the other subjects," McAuliffe said, noting that there are still 16,472 of the state's juniors who haven't passed the reading portion and 17,444 who haven't passed the writing exam.

Many of those are minority, disabled and disadvantaged students, she said.

"What do we say to them?" she said. "We have to really answer to those kids who are struggling."

What do you tell them? In some cases you tell them to get to summer school, take advantage of the tutoring options that are available, that you're going to take the second math class instead of the second PE class. To others you say that they're going to take a different version of the WASL, because that's what their IEP calls for. To still others you offer the WAAS.

And to others, who don't try, don't show up, and expect a diploma just because, you say you're sorry but that's not how the system works any longer. You do it with empathy and you show them a way, but you do it because it's the right thing to do.

Rainy Prediction: The reading and writing sections will continue to be graduation requirements, but look for the Science WASL to go the way of math sometime soon. After four years of the math WASL 38.9% of the kids were meeting standard--it's only 35.0% on science so far, so the situation is worse.

Read more here, if any.