I Broke My #1 School Counseling Rule (Here's What Happened)

The Tap Dance Illusion

Years ago, in a whole other life, I got hired to do something very specific. I was brought in to teach a man how to tap dance for a stage production. Now I need you to picture this guy, big guy, burly like a lumberjack, kind of bumbled around the room. He was not the kind of guy that you looked at and thought, oh yeah, he's a dancer. He had two left feet, bless his heart, with no dance background, no training, and we did not have much time. He was tap dancing while playing Judas in our production of Godspell. I'm not making that up. I was hired to teach Judas the Betrayer to tap dance. And this part still makes me laugh a little bit. One of his big numbers was a song called All for the Best. Tapping his heart out with all the charm on full blast, selling the audience on the idea that everything is gonna work out just fine. Keep that picture back in your head. Because y'all, we pulled it off. On opening night, he got up on the stage and he tapped like his life depended on it, and he looked good. He looked like somebody who had been dancing his whole life. The audience ate it up. And he actually went on to win an award for that role that year. But here's what nobody in that theater knew. If they had stopped that man on the street after the show and asked him to do something simple, like a shuffle ball change, one of the most basic steps there is in tap dancing, he would have not been able to do it because he never actually learned to tap dance. What we built was the illusion of a tap dancer. The costume, the choreography, weeks of me drilling the same eight counts into his body and his brain over and over until he could fake it. He performed tap dancing. He never acquired it. And this week I realized I just watched somebody do the exact same thing to school counseling for $12. Hey, school counselor, welcome back. This is part two of a conversation we started last week about what happens to counseling

The Paper Test Rule

when you take out the part that actually works and just keep the part that looks good. Last week, somebody else did the stripping away, but this week I did it myself on purpose. And I want to show you exactly what happened when I broke my number one school counseling rule. So if you're ready for some straight talk, my friend, some clarity on your work and maybe a little bit of rebellion, you are in the right place. I'm Steph Johnson, and this is the School for School Counselors podcast. So if you missed last week, here's the information that you need right now. I looked at a state-sponsored training that took a real counseling model, stripped it down till it pretty much didn't work anymore, and then handed it to school counselors as a real effective method. And out of that conversation, we built one simple tool that I called the paper test. The paper test is one question you ask of any resource before you use it. If you took all the paper away, every worksheet, every page, every slide, would there still be an intervention present? Would it still do anything? If yes, the worksheet was a support. But if nothing's left, all you've got is a prop. And at the very end of that episode, I told you I had done something I haven't done in years. I broke my own number one school counseling rule. Y'all, I do not buy printable counseling materials. I have not kept them in my school counseling office for a very long time. But to do this episode right, I wanted to run the paper test live on something real that school counselors are really buying. So I went on one of those big teacher resource marketplaces, I'm sure you know the one, and searched for anxiety counseling materials. I grabbed a random resource near the top of the results that wasn't a bundle, just a normal, popular, well-reviewed, inexpensive resource. And let me be honest here, my mindset going in was that I expected to find complete crab. I was ready to find junk. But what I actually found was more unsettling than finding junk, because the creator of this resource was not lazy. And they did something most other creators don't even bother to do. So before I tell you what I found, I need to lay two ground rules, and I mean this with everything in my heart. One, I'm not gonna name the resource. I'm not gonna name the product or the store or the person that made it because this is not meant to be a takedown of a human being. Because the truth is, this isn't about one resource, it's about the pattern, and it's everywhere. And the specific resource that I bought is just the example that happened to come up first. And two, this is a really important one. Everything I'm about to say is about the resource. It is not about the person. I have no doubt that the person that made this resource cares about kids. I think they worked really hard on this. I think they were trying to do it right. So none of my critique is about their heart or their effort or their intentions. It is purely about what is on the pages measured against what the research says treats anxiety. So we're talking artifact, not author. Hold me to that. Okay, so let me tell you what I found. I open it up, and first impression looks pretty good. It's long, it's beautifully

A “CBT-Based” Packet With Citations

produced. It's a multi-week individual counseling curriculum for anxiety, and it's aimed at the early elementary crowd. There are caregiver notes there in more than one language. There's a screening tool, there's progress monitoring, and right on the cover in this confident little badge, it says CBT based. And so then I flip to the back because that's how I read things. I read them the same way I read research papers week after week. You start with the summary up top, then you jump straight to the back to the results and references because that's how you find out if anything good happened. And then you go back through the middle to see how it all came together. So anyway, I flip to the back and I want you to hear how I felt in this moment because it was such a shock. Y'all, it had a reference list, a real one of peer-reviewed studies and two legitimate clinical textbooks. I have been shouting from the rooftops for this to happen on downloadable materials for years. And by ding dang, if I didn't stumble on one, my first crack out of the barrel. I thought that was really cool. Whoever made this didn't just slap CBT on the cover and call it a day. They went and found actual research and they cited it. So somebody clearly cared about this. And I want to keep hammering that in because I really do think that's true. And I'll be straight with you, this all kind of threw me for a loop because I came in ready to roast a resource. And instead, I'm looking at a resource that looks more rigorous than half of the PD I've set through in the last couple of years. So I actually had to take a moment and reckon with this. So we're gonna run this test together right now on the podcast episode. I'm gonna think through this out loud in real time so that you can hear two things happening at once. How the paper test actually works when you apply it to something, and how a resource like this is actually built underneath the pretty cover. So let's go. So the resource runs across several sessions. Here's kind of the arc of it, and I'm gonna take the paper away as

Running The Paper Test Live

we go so we can see what's actually left. Session one is an introduction. You build rapport, explain what worry is, play a little matching game. That's fine. It's a first session. Take the paper away and what's left. You building a relationship with a kid. Good. That part's good, and it doesn't need the worksheet. Moving on. Session two, identifying worry triggers. There are worksheets and scenario cards to name what makes the kid worried. Take the paper away. What's the intervention? Identifying. Naming. Okay, that's a real piece of CBT. All right, file that away. We're gonna come back to that in a minute. Session three is self-esteem and confidence. Session four is worry signals and calming the body with coloring sheets and deep breathing. Session five is problem solving and support. Session six, positive self-talk. Session seven, and remember this one is called modeling and practice. So let's take the paper away from the whole middle of that resource, sessions two through six, and what's the actual intervention? It's identifying worries, it's naming feelings, it's practicing positive self-talk. It is almost entirely teaching a kid to talk about their anxiety, to name it, rate it, color it, and describe it. And the next piece that sounds like it's finally going to put the skills to work, the one named modeling and practice, that's where the kids practice their coping skills against pretend scenarios, made up worries, and a little sensory game. They practice being brave about things that aren't real in the safest possible room with you right there. Then is session eight, termination in review, and you're done. So, did you hear it? Every single session is a real, recognizable component of CBT. The triangle, the feelings, the self-talk, the distortions. Every session looks exactly like the real thing. And at the end of eight weeks, you'd have a kid who could tell you all about their anxiety. They could name the triggers, list the coping skills, and here's the part that should really get your attention. They'd probably score better on the resources post-survey. Because one of the items on the post-survey is essentially, can you use a coping skill when you're worried? Well, of course their score is going to go up. You just spent eight weeks teaching coping skills. So the survey is partly measuring whether the kid absorbed the information, not whether the anxiety actually went anywhere. So the kid looks better on paper, but the worry is sitting right there where it always was, which is exactly the situation I had with my anger management group in the cupcakes that I talked about in the last episode. Exposure to a skill does not mean they have acquired the skill. A kid can perform anxiety resolution the same way a big birdly guy can perform a tap routine on opening night. Now, here's where I have to put my clinician hat all the way on because this is the part that's really important. And I'm going to walk through it slowly and intentionally because I want you to be able to follow every one of these steps.

The Active Ingredient In CBT

Okay, I want this to be airtight. CBT for childhood anxiety is real and it works. It has one of the strongest evidence bases we've got for kids. But CBT isn't one thing, it's a bundle of things, right? Psychoeducation, teaching the kid about anxiety, relaxation in breathing and calming the body, cognitive work in examining unhelpful thoughts, and exposure, gradual, careful exposure with support to help the student approach the thing they're afraid of instead of avoiding it, so that their own nervous system learns in real time that they can handle it. Those are the pieces. And the question that research has been working on for years is when CBT for anxiety works, which piece is actually doing the work? So let's walk through that in order. And I want to give you the best evidence first. Exposure keeps coming out on top. There's a randomized controlled trial by Billikan colleagues that took over a hundred anxious kids and put exposure-focused CBT head-to-head against a relaxation-based program. So they weren't testing the CBT against nothing. They were testing it against another very real treatment. And the CBT exposure group responded at about 57%. The relaxation group responded at a rate of 19%. So exposure didn't just win, it roughly tripled the response rate. Now, the flip side, and this is the part that should get your attention, because we talk about this a lot in schools. Relaxation and positive self-talk, the calming, the breathing, the think happy thoughts pieces are the components with the weakest evidence. Meta-analysis has found that adding relaxation to CBT for anxiety is associated with the same results, or in some cases, worse results than not adding it at all. Think about that for a minute. Adding relaxation is not just less helpful, sometimes it creates worse outcomes. And I'll be completely straight with you about this evidence because if I overstate it, I'll be doing the exact same thing I'm criticizing, right? I don't want to go there. The researchers themselves will tell you that we need more of what are called dismantling studies. Those are studies that would pull CBT apart piece by piece. And once we have more of those dismantling studies, we may be able to say all of this with complete certainty. That work is still undergoing and it's pretty scarce at this point. But here's what we do have: a controlled trial where exposure beat relaxation nearly three to one, a dismantling pilot that favored exposure on 10 of 12 measures, and a meta-analysis showing that relaxation adds little to nothing. Every arrow that we have is pointing in the same direction. Whatever the exact mechanism turns out to be, exposure consistently emerges as one of the most important ingredients in effective anxiety treatment. And the calming pieces consistently look like the part that matters the least. So let's hold this resource up to what the research actually says. I looked at the whole thing, every page, and asked, what is it built out of? So

Safety Behaviors And Hidden Harm

for the strongest evidence ingredient, exposure, I couldn't identify it anywhere in the resource. I looked through session after session for an anxious child, and I couldn't locate a single place where that child is gently, supportively walked toward a real thing that they're afraid of. And the weakest evidence ingredients, the calming, the breathing, the positive self-talk, that's most of the resource. So look at what we actually have here. This resource is built almost entirely out of the pieces that the research ranks lowest. And it leaves out the one piece the research ranks highest. It's not just missing the active ingredient, it is built around the components with the weakest support while leaving out the component with the strongest. The best supported thing is gone. The least supported things are front and center for eight weeks. And remember what this thing is. It is an individual counseling plan, one-on-one. In our world, pulling a child for individual counseling is the most intensive, most targeted thing we do. Y'all, that's the deep end of the pool. And those are the kids with real identified persistent need. And what's actually inside this resource looks much closer to prevention-oriented work than to intensive anxiety treatment. Psychoeducation, naming feelings, coping skills instruction, all great things. All things you'd happily do in a classroom guidance lesson for everybody. But we took that universal level content and we aimed it at the kids sitting in the most intensive seats we've got. The exact kids who needed the part that got left out. And it's gonna get a little bit worse. It's not just that the calming and the self-talk don't treat the anxiety. It's that when we teach this to an anxious student without exposure, it can make things worse. There's a name for this in the research. They're called safety behaviors. Little rituals that a student uses to make the scary feeling go away in the moment. And the anxiety literature consistently warns that safety behaviors can undermine the one thing that actually treats anxiety. Because every time the feeling goes down or goes away because of the kid's ritual, they learned the wrong lesson. They learned that the situation really was dangerous and they only got through it because of the breathing, the positive phrase, or the thing that they were holding in their hand. The relief is real, but the relief is also the trap. Let's make this a little bit more concrete. Think about the kid who constantly asks the teacher, Are you sure? Are you sure? Are you sure? Is it gonna be okay? Is there a fire drill today? Are you sure my mom is coming? And then the second they get an answer, they start asking the questions again. That's reassurance seeking. It's one of the most recognizable accommodation patterns in child anxiety. And accommodation is strongly linked with maintaining anxious avoidance over time. Every answer feels like help, but it teaches the child they couldn't have survived not knowing on their own. And a curriculum built on calm yourself, talk yourself out of it, get the relief with no exposure to ever test the fear, can feed that exact loop. And there's a second cost to this too, and it's the one that really bothers me the most. It's that this all lands on the kid. Because this approach teaches a child that anxiety is something you fix by thinking the right thoughts. Be positive, be rational, use your strategy. So when the anxiety doesn't go away, because they were never given the thing that works, the child doesn't come to the conclusion that the counseling method was incomplete, right? They're gonna conclude that it's them. They're gonna think everyone else can apparently think their way out of being scared, but I can't. So something must be wrong with me. I did the breathing, I said the positive thing, I did everything she taught me, but I'm still scared. So what is wrong with me? That is the bill that we are handing them to pay. We took a treatable problem, we put the costume of treatment on it, and when the costume didn't work, the kid walked away anxious and ashamed, now believing that their fear is a personal failing. Y'all, that is not a neutral outcome. That can do harm. Now, remember what stopped me at the start of all of this was the reference list. The real peer-reviewed citations that made me take this thing a little bit more seriously. I told you, I had to take a minute

When References Prove What’s Missing

and reckon with the fact that they were even there. So I went and I read them because of course I did, right? I read the actual studies this resource cites to prove it's evidence-based. So I'll tell you what was in them. One of the hallmark citations is a study called the Distillation and Matching Model, Chorpita and Day Laden. It's a pretty famous study. It took hundreds of randomized trials of child mental health treatment and asked one simple question. What's actually in the treatments that work? What are the pieces problem by problem? And for childhood anxiety, one of the most common pieces showing up over and over in the treatments that work is you guessed it, exposure. This resource cites it. And then it builds a whole arc of counseling without the thing that it mapped as central. Think about what that means. The evidence isn't pointing toward what's in the resource, the evidence is pointing toward what's missing from it. Another citation that was there is a modular CBT program for anxiety run in elementary schools by Gala and colleagues. Strong outcomes. Kids were still better a year later, so that's good. But if you look at what's in that program, exposure is one of its named core modules. Again, the creator cited a program built on exposure, but then they left the exposure out of theirs. And then one more, this one's almost tragically poetic because it mirrors the whole problem. Another study that they cite about the Feelings Club by Manassas is a prevention study. It was done on mild subclinical kids. And when you read this study, the CBT group didn't even outperform the kids who just did structure activities. Both groups improved at about the same rate. So it doesn't even show what it's being used to imply. But here's the thing about that one that's a prevention study. That's two. One. Prevention studies are mild, universal needs, the lightest piece of what we do. But it's being used in this context to support a resource that was sold for individual intensive counseling, the heaviest end of our work. The evidence is from one side of the spectrum, the product is sold for the other. And that's the exact same mismatch the curriculum itself makes. Light, universal level content aimed at the most intensive kids in the building. The mismatch in the reference list is the mismatch in the curriculum. It's the same error. So think about what that reference list actually is. It is not proof that the resource works. And you need to keep that in the back of your mind as you're evaluating resources. Read honestly, this reference list is a map of the very ingredient that the resource leaves out. The citations point straight at exposure like a set of arrows. But exposure is the one thing missing from the whole shebang. And almost nobody would ever know because who pulls the references on a cheap download and then actually goes out and reads them? Right? Maybe except for me. So here's kind of the clearest way I can say this. And I want to be careful. The research that this resource stands on is the research that points to the thing that's missing. I'm not telling you that the product was tested and failed. Nobody tested it, but I'm telling you that when you read its own evidence, its own evidence describes a treatment that this resource is not. Now, let's widen the lens a little bit because I can feel you might be bracing for me to just keep hammering on this one thing. But that's genuinely not where I want to go. I want to be up front in that I don't think I found a bad apple.

The Unvetted Marketplace Problem

I think I found an apple picked at random. And then I looked out into the orchard. And the orchard is really the story. Because here's the thing: you could run the paper test on almost anything in our school counseling world right now and watch it falter. Don't just take my word for it. Walk to your own shelf, look in your own digital drive, get in your filing cabinet, and run this paper test for yourself. The coping skills card deck, the growth mindset bundle, the self-regulation curriculum with the cute little animal on the cover, the mindfulness coloring pages sold for anxiety, the zones poster. Take the paper away from all of those and ask, where's the intervention? How many of them are a real thing that could survive without the printable? And how many of them just are the printable? I'm not saying it's all empty, but you have to run the test to know the difference. And most of us were never taught to do that. And here's where it stops being about any one creator anyway, and starts being about the machine they're selling it inside of. Let me tell you what the product page for the thing I bought actually conveyed. All right. We're not talking about the pages of the resource right now. We are talking about the sales listing itself. It told buyers in so many words that this was to be used as a tier three individual intervention. Tier three. Again, in our world, that is the deep end of the pool. That is the most intensive support we offer for the kids with the most persistent needs. Tier three is that small number of kids for whom the stakes are the highest. And the bar for counseling exposure should be highest. And the platform let a cheap packet with no exposure in it carry that label. There's no gate, there's no check, and nobody asked whether a scripted coping skills curriculum belonged anywhere near a tier three kid because there's no vetting on these platforms. And the listing made another promise too. The gist of it was it's fully scripted, so you'll feel confident running it. Think about that for a sec. The thing being sold isn't the kid's outcome, it's your confidence, the feeling of knowing what you're doing. That is the best description of a prop I have ever seen in school counseling marketing copy. It sells the counselor the feeling of competence. And it was written because the person who wrote it had to do exactly what the marketplace rewards. So let's bring in a little bit more research because I don't want this to rest on my opinion. There is a body of work I've referenced before, Harris, Archambald, and Shelton, that went looking at the quality of best-selling materials on these online platforms. And I'm going to be extremely honest about something. Their studies looked at history materials, not school counseling materials. The school counseling specific version of this study, as far as I can find, doesn't exist yet. So challenge to some of you out there. And so I'm not going to tell you that these guys tested our stuff. They didn't. But here's what they found in the materials they did study. 70% of the best-selling resources were moderate or low quality against an expert rubric. Because again, the platform doesn't vet anything. There is no quality gate at all. And when they compared expert quality ratings against the things that people actually used to shop, like star ratings, reviews, popularity, the relationship was weak. The pretty popular five-star stuff was not the good stuff. Aesthetics and marketing predicted sales. Quality did not. Now, is there any reason on earth to think school counseling materials are the magical exception to this rule? That the one corner of this giant, unvetted marketplace that may just happen to police itself for clinical soundness belongs to us. The thing that I pulled at random with its tier three label and its five stars and its missing active ingredient suggests the answer is no. But I will let you draw that conclusion for yourself. So this whole thing was never about one creator who cares about kids and made a thing. It's about a system with no gate, a rating that doesn't track quality, and a search bar that rewards the term tier three, whether or not there's anything to substantiate it. There is the apple, y'all, but then there is the whole orchard. A reference list is not always evidence. Sometimes it's a costume. And frankly, a costume is one of the easiest things in the world to put on. The question was never whether this thing looks evidence-based. The question is whether there's an actual intervention underneath it when you take the costume off. So last week I gave you the paper test. Take the paper away. Is there still an intervention? That's step one and it still works.

The Second Question To Ask

Run it on everything. But this resource taught me that we need one more move. Because really, this resource would have passed a quick overview, right? It's got citations, it looks rigorous. So here's the second question for anything that waves the term evidence-based at you. Don't stop at, does it have research? Ask, what did that research actually test? And is that thing really in there? You don't have to go pull every study, but you do have to know one thing. You have to know the active ingredient for the problem you're addressing. For anxiety, it's exposure. If you know that one fact, you can pick up a beautiful, well-sided, 200-page anxiety curriculum, ask where's the exposure, and watch the whole thing disintegrate in about 10 seconds. And if your next thought is, okay, but I don't know the active ingredient for all the things, for anxiety or grief or the kid who won't talk, then good, you're being honest, and that's a good place to be. Nobody handed us that knowledge in grad school in any way that was really usable to us, but it is knowable. And it's the difference between buying the costume and seeing through the facade. Knowing the active ingredient, problem by problem, kid by kid, y'all, that's the work. It's not a worksheet and it's not something you can learn in one podcast episode, but it is the thing that we do in the mastermind every single week. We take real cases, actual situations, and we figure out what would actually move the needle. Not what looks good in the binder, what works. The details and the links are in the show notes, and we would love for you to come learn with us. Now let's go back to my tap dancer, my burly two left feet Judas who got up under those lights and pulled off something

Real Help Beats A Great Performance

that genuinely looked like the real thing. I want you to understand that the applause was real. He earned the award for what he did on that stage, thousand percent. The illusion was good. I made sure of it. But he was not a tap dancer. He never became a tap dancer. And every single person in that audience clapped anyway because they couldn't tell the difference. Yeah, they weren't supposed to be able to, right? That was my job. But a scared student sitting across from you is not an audience. And what they need from you is not a performance of treatment that looks good for several weeks and leaves them exactly where they started, except now they believe it's their own fault. They need the real thing, the actual intervention. The shuffle ball change they can still do when the lights turn off and nobody's coaching them from backstage. So run the paper test. Take the paper away, ask where the active ingredient is. And when something can't pass that test, no matter how pretty it is, no matter how many studies are stapled to the back of it, have the nerve to call it what it is. It's not an intervention, it's theater. And a child in real distress deserves better than just a great performance. Hey, I'm Steph Johnson. I hope I didn't make your brain explode, but I really want you to consider this as you're starting to look at resources for the new school year. What is real and what is theater? What truly has an evidence base and what is just wearing a costume. Y'all, it is more important than you know. Take care.