Kathryn Schulz' Being Wrong - A Short Review

 "To f*ck up is to find adventure."

I wasn't sure if I wanted to keep reading Being Wrong: Adventures in the Margin of Error until Schulz made this point at the end of chapter two. I thought all of chapter one and most of chapter two were too abstract and philosophical for my taste. But then, in the last pages of chapter two, she set the hook and both my highlighter and I were in. That's when she wrote about the medieval knight errant and reminded readers that the Latin word errare, meaning "to roam", gave rise to the English words "error" and "errand". The English word "errant" can be used to express either idea. But enough about etymology. To be errant is to be wrong but also to be on an adventure.

In this spirit, Schulz gives us the 'Cuz It's True Constraint: we believe what we believe because our convictions (we believe) are based on facts. We're willing to accept the general notion that our beliefs may be biased, but if we pick a particular belief, we'll probably say, "but not that one". We don't believe things because it makes us feel better, smarter, or more in control. That's what other people do. Wrong is something that other people do and they do it for the wrong reasons. Schulz points out that even when we do admit we were wrong, it immediately becomes past tense because we no longer believe the same thing. We were wrong.

So why do we make errors? Because our brains are really good at inductive reasoning. We learn language by hearing others conjugate a verb a few times then we go out and conjugate it the exact same way in all situations until someone points out that we say "took" instead of "taked". We are all built to work on assumptions and we don't realize when that gets us into trouble. As Schulz phrased it, "every one of us confuses our models of the world with the world itself" (p. 107).

We silently substitute how we think the world works for how it actually works and then quickly convince ourselves of the certainty of our model. So if my model is how the world works and I see that your model is different, then your model and, therefore, your world is wrong. I can see all the ways that your world is wrong but I can't see how mine is. I see your certainty as laughable but my own as righteous. (It is worth remembering that this book was published in 2010 and written earlier than that so we now have a decade of social media to show just how right Schulz is about how we treat being wrong.)

To me, the beauty of the book is that Schulz doesn't turn it into a how-to on not being wrong, nor does she pontificate. She writes an ode to and explanation of "wrongology". The closest Schulz gets to telling readers what to do about being wrong is suggesting that it is about far more than getting our facts straight.

But facing up to the true scope and nature of our errors is also (and more self-evidently) psychologically demanding. Crucially, these two challenges are inseparable: if we can't do the emotional work of fully accepting our mistakes, we can't do the conceptual work of figuring out where, how, and why we made them. (p. 207)

There is a vulnerability in being wrong and it scares the crap out of us. So fixing our mistakes means fixing ourselves as well as our facts. But Schulz describes an "optimistic model of wrongness" in which error is a force that "imperceptibly helps...us human beings - to grow up" (p. 289).

So what does this have to do with coaching? Nothing. Everything. I take this book as an invitation to admit to and talk about being wrong. How does my wrongness affect my decisions? How does it affect how I treat those I see as wrong? How do I deal with my mistakes when they are made apparent? How do I treat those that show me my errors? As coaches, our relationships with our athletes (and so many others) are built on trust and a power imbalance. Our positions of power can raise the stakes on the errors we make. Do we rely on our power to make things "right"? Do we rely on some version of "because I'm the grown up and I know better" to paper over our misjudgements? When we rely on our power to manage our mistakes, we do so at the expense of the trust placed in us. Our models of the world encourage us to be certain we have done the right thing so we don't understand why faith in us erodes. That's not the place I want to find myself anymore. I'd rather be wrong. After all, that's where the adventure is.

When You Realize You're Not Even Close - Volleyball in Light of Skinner and Goldman's Optimal Strategy in Basketball

Citation:

Skinner, B., & Goldman, M. (2015). Optimal Strategy in Basketball. ArXiv:1512.05652 [Physics]. http://arxiv.org/abs/1512.05652

You know the feeling when you taste a new dish that is so good that you keep repeating to your friends how good it is after each bite?

 

You know the feeling when someone shows you the difference between what you've been doing and the way it should be done?

 

Have you ever felt both of those at the same time?

That weird mish-mash of thoughts and feelings is the result of reading Skinner and Goldman's "Optimal Strategy in Basketball". Add to that the dismay of knowing that they wrote it five years ago and I just discovered it now. The only thing that eases that dismay is the knowledge that I wasn't anywhere near prepared enough then to do anything about what I read. While I still may not be prepared enough now, I can at least see how to get there from where I currently am.

So what was so good about the paper that I couldn't stop telling my friends? The introduction was all it took. The authors clearly summarized what my friends and I have been fumbling around for months, if not years.

- The score difference fluctuates somewhat randomly throughout play.

- That net score difference is determined by the skill levels of the teams and the scoring strategies they use during play.

- There are three main principles that go into determining what the optimal use of scoring strategies should be.

Those ideas may not seem so earth-shattering but that is the amazing part. If those ideas are so plain to see, why hadn't I seen anyone else nail these down in some formal, empirical way until this paper? It's not until you try to explain something complex to others that you realize just how little you truly understand it.

That's where math starts to come in. The first of the three principles to consider is "allocative efficiency", or determining the frequencies at which certain players/plays should be used in order to maximize scoring. The authors demonstrate that the answers are not as simple as we think they are. It isn't as simple as using the average points scored by each player/play. We need to understand the marginal points scored. Marginal rates are derivatives. They explain how large the changes are between data points. The question shouldn't be "how many points do we expect to score running this play?" The question should be "how many more points would we expect to score if we ran that play one more time?"

The second principle is "dynamic efficiency", which the authors explain as an "optimal stopping problem". To maximize dynamic efficiency, teams must choose which shots to take and which ones to pass up within a given possession. Teams should shoot when they reach a point at which the expected value of the current shot is greater than the average expected value of continuing the possession. The expected value of a possession keeps going down as the shot clock keeps winding down so earlier shots are typically better than shots in identical circumstances later in the possession.

The third principle is "risk and reward". We can easily understand that there is a trade off between these two but the authors quantify this trade off in an insightful way. Increasing "risk" means that we decrease our chances of scoring our "average" number of points in order to increase the chances that we score a number of points that is much higher than our average. The downside of increased risk is not only the decreased chances of scoring our average but also an increase in the chances of scoring far less than our average number of points. So why would we risk, given how it negatively impacts our scoring? If we remember back to the introduction of the paper, we should care about the difference in scores more than how many points we score. If our opponents are likely, based roughly on average scoring, to outscore us then we have to take some chances to possibly increase our scoring. In statistics-speak, we are trying to fatten the tails of our scoring distribution. In COVID-speak, we are trying to flatten the curve.

So how do I think we can apply these ideas to volleyball?

I'm very early on in developing my thoughts on this but I view this post as a chance to sort of think out loud, which I find helpful.

When it comes to "allocative efficiency", I think that this encompasses both offensive play selection as well as setter choice. While there may be some play selection in transition offense, I think that, just by sheer volume of opportunities, we're mainly considering side out offense. Setter choice is always going to be constrained to some extent by reception/dig quality, which is only the beginning of the deep complexity involved in figuring this out. Let's start by considering a given play in a three-hitter rotation. We shouldn't just consider the attack efficiencies of the three front row attackers when running that play, we should compare how efficient they are as they are set more and more in that situation. We're looking to determine how to get the most expected scoring by leveraging each attacker as much as possible before their skill curve deteriorates too much. The better the individual attacker, the more the distribution will skew towards them. To some extent, we set different options to "keep our opponents honest" but mathematically, we are trying to maximize expected scoring. This is also an example of how the score fluctuations in a match are somewhat random. There are many factors that contribute to these situations and who the setter chooses to set can vary randomly within our maximal scoring scenario. We can achieve our allocative goals without strictly scripting what happens on every play. I think that allocative efficiency can also influence setter choice on less-than-perfect passes. To further complicate our thinking, this would mean not just knowing how to maximize scoring on a certain play with certain attackers but also how that efficiency may change as reception quality deteriorates. The abilities of the attackers determine this to some extent but there is always an element of allocation to consider.

I think that we need to understand how marginal scoring rates change in all of these situations before we can understand how to allocate most efficiently. I suspect that we currently only truly understand average scoring so when we start considering complicated situations, our thinking reverts to "set the attacker who is most efficient on average" which will not result in the best allocative strategy.

I think making direct "dynamic efficiency" comparisons between basketball and volleyball is a bit tougher mainly because our time constraints are much different than theirs. The rules of volleyball prevent lengthening possessions. With that said, I think that there are some interesting comparisons to be made. First, there is a constraint in terms of how close teams are to the end of a set. I have seen this referred to as the "red zone", or after 20 points. I have also seen research suggesting that 18-15 is a kind of point of no return, in that the trailing team is very unlikely to win the set under normal circumstances. We can study dynamic efficiency as a principle that governs decision-making as teams approach the end of sets. How should decision-making change as teams near 25 points? Does attack efficiency change in some important way as teams near 25 the way that expected points change as the shot clock or game clock nears zero? What impact is there on dynamic efficiency when a team is leading versus trailing? (Is that question really more about risk and reward?)

 

What about considering rotations as part of dynamic efficiency? Should teams be weighing how to shorten time spent in weaker rotations and lengthen time in stronger ones? Is this a different question from allocative efficiency? Is attack choice (line/angle, hit/tip, etc.) a dynamic choice, allocative choice, or both?

"Risk and reward" is probably the most straightforward because it is a meta-strategy so it isn't tied to how the game is played. Risk and reward is about choosing strategies and about appetite for risk. There are plenty of stories in the sports canon about underdogs adopting radical strategies to overcome heavy odds. I don't think that volleyball coaches consider themselves separate from such ideas. In discussions that I have had with friends about this subject, I have framed this in terms of women's collegiate programs. How does a college coach think about risk and reward? How willing is any given coach to gamble on less-certain strategies? It's not that the strategies don't work (we need to be wary of dualistic thinking and resulting), it's that the strategies are much more variable. Do coaches who are willing to risk have any data to support how much variance accompanies a given strategy? How frequently can a coach utilize such strategies and keep their job? Variance means getting a much wider variety of results (in terms of scoring, not necessarily outcomes) and that means living with much less certainty, which can be exhausting.

But what if you coach a college team that can usually make it to a postseason tournament but has very little chance of advancing far? Would you consider risking more, especially given that there may be little chance of overcoming the gap between you and your opponent? What if you coach a team that makes the tournament every year, advances a round or two and then needs some luck? How many years would you be conservative and hope that the stars align to make it to the finals?

And then there's the question of how do we risk? What strategies are available to our teams that we aren't currently employing to score? Aren't we trying to practice those things that we can't control well enough yet so that we can control them better? Is risking all about tailoring strategies to fit certain opponents beyond the small tweaks that we usually make? Is it about putting more eggs into a single basket than we feel comfortable doing? Do we have to risk in all facets of the game (serving, attacking, blocking, etc.) or is deploying a large change in one area enough? Can we risk with certain personnel decisions rather than team strategies?

 

And how would we measure the risks we would take? We need to employ strategies that we trust either fatten tails or shift curves in a favorable direction. Blindly risking is still risking but we can do better than that. We should gather some kind of data on the potential strategies, whether those come from other teams or our own team in situations where we are favored enough to experiment. It is a lack of rigor that can give risk a bad name.

There's a lot of research to be done, data to be gathered, and numbers to be crunched to get a handle on these ideas. But before we do the math, we still need to decide how to translate between what we know about volleyball and what Skinner and Goldman have taught us about basketball. Are there areas unique to volleyball that the authors didn't have to account for? Are there arguments they made that won't hold up in our game? We have some substantial (but manageable) theoretical work to do before the applied work. Skinner and Goldman did the theoretical work and began the applied work but then ended their paper with this massive caveat:

At a practical level, the biggest hindrance to quantitative basketball strategy is usually the difficulty of accurately estimating the efficiency of different offensive options. The usage curves f(p) are particularly difficult to estimate from easily-measurable statistics, and are necessary for a quantitative determination of optimal allocation. What’s more, the usage curves are really only robustly defined relative to a particular defense, and can vary strongly depending on the quality of the team’s opponent. A major advance in their determination may therefore provide the most important step toward enabling quantitative optimization of basketball strategy. (p. 14)

So it's going to be a long haul. We're not even close. But we can get there from here.

Academic Quick Hit - Acquiring Skill in Sport: A Constraints-Led Perspective - Davids, Araujo, Shuttleworth, and Button, 2003

Where I attempt to give a quick summary and opinion on an academic paper that connects to teaching, learning, and/or sport.

Why I think this paper matters:

- The authors, leaders in the field of ecological dynamics, give a brief summary of the constraints-led approach (CLA) framework of skill acquisition.

Citation:

Davids, K., Araujo, D., Shuttleworth, R., & Button, C. (2003). Acquiring skill in sport: A constraints led perspective. International Journal of Computer Science in Sport, 2(2), 31–39.

Type of Paper: Review/Opinion

The authors weave a short literature review together with their views on how to best structure learning environments for skill acquisition.

Highlights:

- Active participation and concentration on exploring the solution space by the learner is better for skill acquisition than satisfying task demands prescribed by a coach. (This can be loosely thought of as discovery learning.)

- There are three main task constraints that coaches can manipulate: equipment, practice structure, and augmented feedback (p. 33) (Augmented feedback usually refers to feedback from a coach rather than from performing the skill alone.)

- "practice structure should emphasize 'task simplification' rather than the more traditional technique of task decomposition" (p. 34) (Simplifying the whole skill rather than teaching "whole-part-whole")

- Feedback from coaches should use external focus of attention (from Wulf's OPTIMAL theory) and should be used more infrequently.

What I'm left wondering:

-How effective is external cuing as compared to internal cuing? Are there differences in the rates of improvement?

- How would I work with a novice athlete learning to serve? How would I use external cuing and different constraints to develop the skill? I know how I used to do it but incorporating a CLA means coaching very differently.

Academic Quick Hit - Metacognition in Motor Learning - Simon and Bjork, 2001

Where I attempt to give a quick summary and opinion on an academic paper that connects to teaching, learning, and/or sport.

Why I think this paper matters:

- It shows that random practice can be better than blocked practice for motor learning tasks. Most blocked/random research at the time was limited to cognitive tasks.

- It highlights how "illusions of competence" may influence current learning efforts.

- "One important speculation that arises from the present results is that learners who train under [random] conditions would be less likely to terminate practice before achieving the level of learning that is the goal of such practice and less prone to attempt a task for which they are unprepared" (p. 912).

Citation: 

Simon, D. A., & Bjork, R. A. (2001). Metacognition in motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27(4), 907–912. https://doi.org/10.1037/0278-7393.27.4.907

Type of Paper: Empirical Research

The authors studied college students, having them learn a task, predict how well they would complete the task in future trials, then tested them the next day.

Highlights:

- "Confidence was positively correlated with the amount of practice at the task, but confidence did not correlate with performance" (p. 908). Learners tend to overestimate how well they have learned a task when they learn in blocked practice.

- As practice trials went on, the difference in skill execution between blocked and random practice structure decreased. Any short-term advantages in learning from blocked practice were lost as more learning opportunities occurred.

- Learners in random practice were much more accurate in their estimates of their future performance.

What I'm left wondering:

- How well does this work predict learning/prediction/execution of more complex motor skills?

- At what point is a task considered "learned"? Is that when there is no significant decrease in performance between trials?

- If a task has been "learned", how often must it be practiced in order to maintain that status?

Academic Quick Hit - Van Raalte, et al.'s Relationship Between Observable Self-Talk and Competitive Junior Tennis Players' Match Performances

Where I attempt to give a quick summary and opinion on an academic paper that connects to teaching, learning, and/or sport.

Why I think this paper matters:

- It suggests a connection between our self-talk and our performances

- It reminds us that self-talk may be a lot more than what we can see

Citation:

Raalte, J. L. V., Brewer, B. W., Rivera, P. M., & Petitpas, A. J. (1994). The Relationship between Observable Self-Talk and Competitive Junior Tennis Players’ Match Performances. Journal of Sport and Exercise Psychology, 16(4), 400–415. https://doi.org/10.1123/jsep.16.4.400

Type of Paper: Empirical Research

The authors studied junior tennis players at a pair of tournaments, recorded their observable self-talk and had them complete a survey about their self-talk.

Highlights:

-Players who used more positive self-talk won more sets than those that used more negative self-talk (the authors make it clear that this is merely correlation and not causation)

- Players categorized self-instruction as positive, negative, and "other", which suggests that they view self-instruction differently depending on the circumstances

What I'm left wondering:

- How can we as coaches learn more about the unobservable self-talk?

- How can we help athletes move from negative to "other" or positive self-talk?

Academic Quick Hit - Gallimore and Tharp's What a Coach Can Teach a Teacher, 1975-2004

Where I attempt to give a quick summary and opinion on an academic paper that connects to teaching, learning, and/or sport.

Why I think this paper matters:

- The authors show the value of revisiting previous work and applying a different lens to learn new lessons.

- We see that great coaching is far more than what happens during practice and competition. We don't just show up and do our best work.

- We see great coaching isn't just about how good your practice plan is. Who we are during practice matters.

Citation:

Gallimore, R., & Tharp, R. (2004). What a Coach Can Teach a Teacher, 1975-2004: Reflections and Reanalysis of John Wooden’s Teaching Practices. The Sport Psychologist, 18(2), 119–137. https://doi.org/10.1123/tsp.18.2.119

(fun volleyball reference: this paper cites Marv Dunphy's unpublished doctoral thesis on Wooden.)

Type of Paper: Review (sorta)

The authors wrote a famous empirical research article in the 70s that quantified Wooden's coaching. In 2004 they revisited their work, critiqued it, and applied a more qualitative lens.

Highlights:

- Researchers, like coaches and most humans, often find themselves looking back on their previous work and wondering what they could have done better.

- "Had qualitative methods been used to obtain a richer account of the context of his practices, including his pedagogical philosophy, the 1974-1975 quantitative data would have been more fully mined and interpreted" (p. 119). Coaching, at its best, is an exercise in mixed methods research. The best research around it must account for both the qualitative and quantitative aspects of coaching.

- "It is now clear Coach Wooden’s economical teaching that we observed was the product of extensive, detailed, and daily planning based on continuous evaluation of individual and team development and performance" (p. 124). Wooden's seemingly effortless coaching was the product of massive amounts of care and attention in the gym and work and reflection outside of it. Just like great athletes, great coaches are working long before and long after practice and competition.

- Coach Wooden's work on his craft was so much more than finding the perfect drill. He dedicated himself to being more efficient but also to learning his athletes. He saw the importance of affirming each one of the athletes in his care and connecting with them in ways tailored to their individual personalities and needs.

What I'm left wondering:

- How can I learn more about pedagogy, particularly as it applies to sport coaching?  There's a great deal of work in this area (some of which I have read) and I think that this article, along with its predecessor, should encourage coaches to learn more about coaching from the research and not just from the coach on the court next to them.

Academic Quick Hit - OPTIMAL Theory by Wulf and Lewthwaite, 2016

Where I attempt to give a quick summary and opinion on an academic paper that connects to teaching, learning, and/or sport.

Why I think this paper matters:

There are two important ideas that I took away from this paper.

- If we want to maximize the motor learning and execution of the athletes in our care, then we should incorporate support for athlete autonomy into how we teach and coach.

- If we want to maximize the motor learning and execution of the athletes in our care, then we should use external foci of attention in our instruction and feedback.

Citation:
Wulf, G., & Lewthwaite, R. (2016). Optimizing performance through intrinsic motivation and attention for learning: The OPTIMAL theory of motor learning. Psychonomic Bulletin & Review, 23(5), 1382–1414. https://doi.org/10.3758/s13423-015-0999-9

Type of Paper:

Theoretical Review

Highlights:

- OPTIMAL is a backronym for Optimizing Performance Through Intrinsic Motivation and Attention for Learning.

- Intrinsic Motivation refers mainly to supporting autonomy in the participants (read more by Bandura or Deci and Ryan in particular) to positively influence learning. What do the authors think autonomy is? They quote Eitam, Kennedy, and Higgins (2013): "(the perception of) one’s actions having effects on the environment" (p. 1392)

- Attention for Learning refers mainly to using external focus of attention cues (as opposed to internal) to enhance motor performance and learning.

- Given that these two areas are in the title of the theory, the authors think that "motor learning cannot be understood without considering" intrinsic motivation and external focus of attention (p. 1384).

- There is a good amount of nuance and references to other work in the field, but the arguments for intrinsic motivation center on things that increase a participant's expectancies (their belief about their chances of successful achievement of a movement goal).

- "Importantly, rewards appear to exert their effects via expectation rather than receipt." (p, 1389) While the authors do point out that intrinsic rewards are better than extrinsic ones, they also point out that extrinsic rewards can increase expectancies if we believe that we can complete the movement goal. It's the thought of getting ice cream after a win that motivates if we think we have a chance of winning. This is a separate argument from rewards vs. punishments.

- "control over an assistive device can have a beneficial effect on learning, even if that device in and of itself is relatively ineffective" (p. 1393). Think of Dumbo's magic feather. The authors also comment that superstition and similar factors can also influence performance expectancies (p. 1387).

- The language we use in our instructions (to say nothing of our actual feedback) influences motor learning. (p. 1393)

- External focus of attention increases motor performance because it lessens thoughts of self. Lessening the engagement of "self" allows the body to use faster reflex loops in movement rather than slower conscious loops.

What I am left wondering:

- While the authors point out that autonomy-supporting behaviors and external focus of attention are beneficial to motor learning, all I have to reference are more research articles. This is the nature of the divide between researchers and practitioners, it isn't the job of researchers to provide ways to implement these ideas but I, as a practitioner, typically can't use the research to learn how to implement their ideas. I have to hope that the research closely resembles my setting or I have to hope that the principle they reveal transfers clearly to my setting.

- The article cites research in which expectancies are increased by giving participants misleading feedback and by lowering the difficulty level of trials. I don't deny that these methods can work, but I am left questioning if these are methods that coaches should use or use regularly. I feel uncomfortable with doing something that I perceive as being close to lying or with regularly lowering expectations versus supporting athletes differently when they perceive that they are failing.

- If you've read my post about Gallwey's Inner Game of Tennis, then you'll know that I like engaging the self but as an observer to actions rather than the initiator of those actions. Does the "self-as-observer" focus lessen thoughts of self in the sense that Wulf and Lewthwaite mean when discussing internal focus of attention? Is this mode of thought somewhere on the continuum between external and internal focus of attention? Has any research been done to test these ideas?

- My primary sport (volleyball) and most other sports have some object that athletes need to control or manipulate and that object lends itself to external focus of attention. What if you are a dancer or a gymnast and you are the object? How do we facilitate motor learning when external focus of attention is almost non-existent?