Monday, March 19, 2007

Black-and-white is not my favorite color

We got the written report from Bud's day-long assessment at the developmental clinic. Honestly, I have no right to be disappointed. They didn't promise anything more than they delivered and, in fact, they tried to warn me about the limitations of their assessment. But I heard "interdisciplinary," and I made assumptions: I assumed that each specialist would gather information in his or her own area of expertise, and I thought they'd drop it all together into a pot and look for what bubbled up, watching closely for the subtleties in the aromas, for the ways that the flavors mixed together, and then try to capture in words the unique delicacy that emerged.

My assumptions were only half right. Each specialist gathered information from his or her own area of expertise. Then they put each bit of information in a tidy little pile on a plate, spacing the piles out so that the flavors complemented each other but never really mingled together. And then they wrote a report that described each item on the plate, clearly and accurately, in black-and-white terms.

It's the black-and-white that I find most disheartening, I think. I'm not typically a fan of black-and-white. I prefer greys - and even then, I want to know more about them: are they more charcoal, or more slate? Are they a smokey grey, or more opaque? But there are few greys in this report. Instead, the report is a series of results from standardized assessment tools. It lists scores and averages and percentiles and standard deviations that, ultimately, have no real-life meaning or relevance to me. The end result is a stack of pages that reflect the testing process, but that do not seem to reflect much of my son.

In fact, I find myself getting defensive as I read the lists of percentiles. Part of it, I know, is simply the reaction of a mom who refuses to acknowledge that her son is anything less than "best" ("What do you mean my son won't be the starting pitcher for every little league game this season?") Beyond that, though, I take issue with the assumption that these tests are useful as a measure of Bud's ability, of Bud's performance, of Bud's intelligence.

Without question, these scores and percentiles are an accurate measure of Bud's ability to take these tests. But I would submit that these tests are not designed with Bud's unique ability in mind.

I was talking about it recently to a friend, who asked, "But didn't they use versions of the tests that weren't language-based? Didn't they assess nonverbal intelligence?"

Yes, yes. Of course they did. But they assessed nonverbal neurotypical intelligence. It seems to me that the only thing they confirm is that Bud is not neurotypical. They confirm that he's autistic.

And we already knew that.

My hunch is that if we really want to assess how Bud thinks - how well and how differently he thinks - then we have to cast aside our preconceived ideas about the ways we measure intelligence. We need to do more than try to measure the same things in different ways. We need to determine how to measure different things entirely.

We need to start working in a whole new paradigm.

Interestingly enough, it was a Gray who started me thinking about paradigms. I recently attended a presentation by Carol Gray, the originator of Social Stories (TM), in which she talked about the idea of paradigm shifts. She quoted Joel Barker from The Business of Discovering the Future: a "paradigm is a set of rules and regulations (written or unwritten) that does two things: 1) it establishes or defines boundaries; and 2) it tells you how to behave inside the boundaries in order to be successful."

Gray contends that if you're not operating within the correct paradigm, then it doesn't matter how hard or how efficiently you're working. Even the best made, most detailed map of New York City, she says, won't help you find your way around Chicago.

And that's just how I feel about much of the testing that was part of Bud's evaluation. It tested his ability to navigate a city he'd never visited. It neglected to ask him questions about his own hometown.

And, in retrospect, I can see that they told me that this might be coming. They cautioned that I might be disappointed. They told it to me in black-and-white.

But I was listening in grey.

19 comments:

Connie said...

You got it, mom -- once again, bravo. If only the system did begin to address all those shades of grey. They couldn't recognize a delicacy of any kind (let alone a unique one) or a subtle shade of grey (Gray :) -- and a COLOR -- well, that might take ten or twenty more years.

This is just what happened with my son in 1989 at the Shriver Center, and the neurologist was Margaret Bauman! Every question I asked the team (at the final meeting, with all nine "experts" sitting around the table), her answer was the same: they only had "anecdotal" information about that. Looks like not much has changed.

KAL said...

This is hard. I'm sorry they did not paint the vibrant, colorful Bud that you bring alive for us in your blog and instead outlined only the black and white in their report. Tests. I'm not liking them so much these days.

kristina said...

Which is why you have your own assessment of the assessment---and of Bud in multifarious shades of gray (grey). I've had to school myself to see these tests as tests, interesting exercises in pedagogical management, measurements, and musing.

Then, back to the boy!

Mom Embracing Autism said...

I stopped going to our original developmental pediatrician, and haven't been too eager to return to the neurologist we saw one time. In both cases I was turned off to them because of their immediate discussion of medication for my son after only a few minutes of observation. At least talk about some other approaches to teaching my son first! We are going to a new developmental pediatrician at a new hospital next month.

We just faced being steered toward a medical waiver for "mental retardation" in our state vs. the "developmental disability" waiver, all based on Samuel's IQ test. As you noted, the test measured his neurotypical intelligence, which I do not consider to be a true measure of his real intelligence.

"Paradigm" is one of my favorite words. I was first introduced to it in my banking days (when lots of changes were going on in the industry and therefore in our jobs). Little did I know that my whole life would experience a paradigm shift due to autism. I think this is a word all parents of children on the autism spectrum should come to know.

Lisa/Jedi said...

When our school district re-tested Brendan in fall 2005 (they re-test every 3 years) we were fortunate to have the testing psychologist sit down with us (myself & his 2 teachers) to explain the percentiles, etc. She concentrated mostly on the outliers, since they seemed to affect his performance the most, & we found this approach to be very helpful. It took a lot of the demoralisation out of the process & helped us to focus on where he needed the most help. Is there anyone who could sit down with you & do something similar? This might allow you to get the most out of the info...

Another Autism Mom said...

MOM-NOS, this is exactly how I felt last week, when I received the results of some standard tests done on my son. The only "high" score he got (which was a mere "moderately high") was on writing and reading skills - he's hyperlexic - and I thought, "What do you mean, moderately high? His writing skills are sky-high! He's only three-years-old and can write any letters, small or caps! And very quick! Nobody taught him that, he learnt it himself!"

But the thing is, while the numbers weren't very reassuring, the specialists were very excited about my son's progress and prognosis. Plus, things that he wasn't doing a month ago when the tests were done (like cutting shapes with scissors) he's already doing now, like a pro.

So I look at the numbers as criteria to define my son as autistic and different from the NT kids his age. There's no doubt that, at THIS POINT, he can't carry a conversation or use the toilet, he can't understand the dangers of crossing the street, and he can't sit still for circle time in school. So I can't expect him to score better on those ADOS/Vineland/Mullen/whatever tests right now. But he's learning new skills, he's developing. And he may surprise everyone in the next few years. People look at him and think he's smart, funny, affectionate and amazing. So I'm not gonna let those numbers bring me down.

Anonymous said...

Even for NT kids, IQ tests are only part of the picture. An average IQ score means you have average test-taking skills - nothing more. It certainly does not predict your success in life, your accomplishments, or your happiness. And that goes doubly for autistic kids, who just do not fit easily into testing norms.

My Lola absolutely would not comply with the therapists attempts to test her intelligence. Lack of compliance does not equal lack of intelligence, last time I checked. Like you said, all it tells me is what I already know - Lola is autistic. I do know that she has a great memory (for those things that interest her) and is very creative in her problem-solving (she can figure out how to get the oreos no matter how high I place them!), and I know those two things will take her far. I think that is enough for now.

Erin said...

This describes pretty well how I've felt after Tommy's assessments. It hurts to hear that my bright child rates with below normal intelligence. I know it's not true, but it's what the test says.
During his last assessment he was shown the picture of the rabbit with only one ear and when he was asked to tell the doctor what was missing, he said, "a carrot!"
When shown the picture of the wagon without a wheel, he told us that what was missing was the flowers!
Not correct, but certainly not wrong, and definitely intelligent and creative.
Lucky for us, the doctor was noticed this herself.

MOM-NOS said...

Erin: Not even wrong. Just a completely different paradigm.

And for more on the frustrations of standardized testing check out this post and this post, from This Mom, which chronicle the testing woes of the very clever, and not even wrong, Fluffy.

for what it's worth said...

Having done reports such a Bud's and having facilitated many of the assessments through the years, I will be the first to tell you that black and white is awful.
It's kind of the same perception I have of report cards. Report cards are snippets of information gathered at different times to justify the work done in the classroom. Are they accurate? Not in my experience. They are an assigned letter, period. If you want to know how your child is really doing, ask them. And in your case, asking may not give you all the answers, but observing does.

I don't know about Bud, but when we had the interdisciplinary team memebers in to do their various assessments or observations, the class dynamics changed immediately and so did the behaviour. There is a certain "safety" within the classroom, especially for kids with sensory issues. They get used to the sounds, routines and of course personnel. Bringing someone, anyone in can cause a difference in behaviour or comfort. Sadly, the reports never address that issue of grey. They report only what was observed and what tools were used, not the "why" of the matter.

My advice, just as with report cards, take the report for what it is. Allow it to serve it's purpose; likely to provide the services and supports necessary for Bud's success in learning. But, remember, YOU are the best judge of who Bud is and what his capabilities are. And if you say he is a beautiful charcoal grey, then that's what he is.

Incidentally, our district recently stopped testing for IQ for kids with ASD. The rationale was that there is no suitable test to accurately test kids who have severe communication issues.
Now they just need to find something helpful to do instead.

VAB said...

I posted a rather verbose comment here, and it looked like it had been published, but now it's gone. Any idea what happened?

MOM-NOS said...

VAB, I'm afraid it may have been eaten by Blogger. Can you recreate it?

Alyric said...

If your assessment was anything like the one done on my offspring quite a few years ago - don't believe it. You know it's really hard to get an apology out of someone fifteen years after having labelled a child as borderline mentally retarded when they are in fact of above average intelligence and this is a verbal child. This is how far out of whack the tests are for anyone on the spectrum. By the way, you might like to ask the testers for the confidence limits for these tests. If as is usual it's +/- 3 SD then the test results apply to 99.7% of the population - but note not the whole population. There's 0.3% for whom it is mathematically certain that the tests are invalid. A spectrum child is most likely to be among the 0.3%. Secondly, you might like to ask the testers, in detail, what use they are going to make of the results. They will say that they will use the results to tailor the IEP. Ask them to be very specific. I'll bet anything you like that they won't be able to get beyond a general level of motherhood statements, and I do like to see them squirm. Naturally, the testers believe in their tests, but really their main purpose is to reinforce their own sense of normality, which is about what you 'd expect from people going into psychology. If there isn't really a norm, then what is it they are doing for a living exactly?

kyra said...

you so got it, MOM-NOS. yes. a whole new paradigm. i've been saying the same thing as we go through the (sorry) asinine testing over here. ah well. they can all lay it out in black and white and we will still keep thinking, talking, and hearing in greys.

xx to bud. SO not even wrong and so very full of every single color there is.

VAB said...

Here's another try at the post that Blogger ate.

Our SLP said that she thought that the problem with most testing was that they plan in advance to test a particular set of skills. Then they test each skill until they get x number of wrong answers, whereupon they stop and go on to the next skill on the list. But there are subsets within skills which, according to her, can explored with what she calls interactive testing. Basically, offering increasingly difficult question in the areas in which they do well during the test. This means the battery of questions used changes according to how the test progresses, rather than being fixed in advance. You end up with more data, and it tends to look spiky, with various splinter competences standing out. When you see what a child is good at, you can use those competences as stepping stones in an educational plan.

As for me, if I were developing a test for neurologically diverse kids, I would first ask for a list of things that they do well, and then set about trying to discover what processing competences they use in order to perform those tasks. It would take a long time, I imagine, because you would have to keep testing different potential processing competences and there would probably have to be a lot of trial and error involved. But, in the end, you would hopefully have results that go some way towards accounting for the skill set of the child. Of course I would also be interested in the processing that they didn't do well, but listing weaknesses alone is of very limited practical use compared with the identification of a pattern of strengths and weaknesses.

As parents, we do this anyway. We know what approaches to learning are most likely to work, because we know where our children's strengths lie. But it would be nice to have a fresh perspective, because I am always finding strengths that I missed and I'm sure our guy has many others waiting to be discovered. More importantly, an independently established profile of strengths and weaknesses might actually be given consideration by the schools and other service providers.

In the meanwhile, I guess we have to take most evaluations as tick marks in our own process of elimination that slowly lets us narrow down the things that work best for our kids. One hopes for more from psychologists, but then one hope for more from weather forecasters and financial analysts and all the other people whose job it is to look at something far more complex than we are able to understand and tell us what's going on and what's going to happen next.

Mom to Mr. Handsome said...

I loved your analogy with the maps. Sometimes I search for ways to explain ideas and thoughts to others about Gabe and to myself. There are many times that the words escape me, leaving me a little frustrated. So close to understanding, but missing an important step :O)

Great post. I think I will cut and paste the map analogy for my own safe keeping when I just need the right words for Gabe. (If you don't mind)

Kristin

MOM-NOS said...

Kristin, I don't mind a bit - but, just to be clear, they are Carol Gray's words and not mine. (She is BRILLIANT, by the way, and a dynamic, engaging speaker. I've seen her twice and highly recommend her workshops!)

VAB - Thanks for re-posting!

Mom without a manual said...

Assessments/Grades are going to be worthless for our children. Bravo for everyone who realizes that! This way we can save ourselves and our children some heartache in the future.

I just worry about the other families with disabled children who don't realize that these "professional measures" are not all they are cracked up to be. I have a fellow autism family that was told (based on an IQ test) that their 5 year old was moderately MR. They used his IQ evaluation to place him in a self containment setting. They didn't feel that he could intellectually keep up with the mainstream setting.

I just get heartsick thinking that these other families don't realize that they (the parents) know their child best. Yes, the professionals have great insights and I don't want to discredit that. But so many parents think that the professionals are "all knowing". No one knows what the future holds and I won't have a standardized test tell me what my son will be like when he is a man.

Perhaps my cynicism is getting out of control again but this is just another example of how we have to fight to help our kids find their place in this world!

Joker The Lurcher said...

i so remember these reports bringing me down. most of the ones on my son were done when we were trying to get him into a special school so we needed them to dwell on the problems but seeing it all, as you say in black and white nearly always upset me.

one thing that it is vital to remember is what a lot of others have commented on and you also mention - the tests were not invented for kids like ours. it is like trying to get a cat to fetch a ball. i happen to know what my iq was when i was a kid as it was tested because of my wildness (!). my son's iq comes out significantly lower than mine yet i am pretty sure he is more intelligent than i was then and than i am now. it is the huge spikes and troughs due to the autism that make it a useless measurement.

we need to celebrate the brilliant things about our kids and not let them be measured against other kids. i'm not sure this wouldn't be a good approach for all parents, not just those of autistic kids!