In This Issue
- Understanding D'
- T.O.V.A. Case of the Month
- Frequently Asked Questions
- Meet the T.O.V.A. Team!
Understanding D'
One of the most common questions that we receive in clinical support calls has to do with interpretation of d' (pronounced "d-prime"), one of the variables found in the T.O.V.A. report since version 7.0. In fact, we've noticed that it comes up in almost every conversation with T.O.V.A. users! Clearly, people are confused about the variable and are generally not sure how to interpret it.
While its calculation is (to put it gently) "nontrivial" (see page 6 of the T.O.V.A. Professional Manual for the details), d' is nevertheless relatively easy to understand conceptually. The d' statistic provides information about the subject's ability to discriminate the target stimulus from the non-target stimulus. It is calculated from the rate of "false alarms" (commission errors) and the rate of "correct hits" (correct responses). If a person is better at differentiating target from non-target, the difference between the two distributions is greater and the d' statistic is larger.
More technically, d' describes the distance (in standard deviation units) between two probability distributions, defined as response performance under "noise only" conditions, and the other under "noise plus signal" (see the web link at the end of this article for more information).
The d' statistic is one of a number of indices that was developed as part of the general body of statistics known as signal detection theory. The history of signal detection theory is rather interesting. This body of work came into its own with the invention of RADAR and SONAR, and the need to find ways of quantifying the performance of individuals in the then-new job category of RADAR or SONAR operator. Both "false-alarm" errors (e.g., mistaking a seagull floating on the water for a submarine periscope) and false-negative errors (failing to correctly identify an actual periscope) had real consequences, and the ability to discriminate "target" from "non-target" was at the core of good operator performance. Measurement and tracking of these abilities were necessary as part of training operators to make these discriminations, and d' is an essential statistic in modeling an operator's - or a test's - performance with a Receiver Operator Characteristic curve (something that we might get into in a future T.O.V.A. Times newsletter).
What can d' do for you when interpreting the T.O.V.A.? As you know, the T.O.V.A. reports two general categories of variables, those reflecting aspects of reaction time (Response Time and Variability) and accuracy of the subject's performance (Commission Errors and Omission Errors). D' is based on the occurrence of "false alarms" (Commission Errors) and "hits" (correct responses, calculated from total targets [Omission Errors - Anticipatory Responses]). In effect, d' collapses the information from both Commission and Omission Errors into a single score.
For most clinical interpretations, you will probably find that it is a whole lot easier to describe a patient's accuracy of performance in terms of Commission Errors (which reflect impulsivity) and Omission Errors (which reflect inattention). These terms are easily understood by everyone, much more so than reference to the patient's "discriminability".
The d' statistic does give you a sense of general performance across the four quarters of the task, and it is certainly a useful measure in research (where reducing the number of variables in an analysis yields an increase in power). You will also find that d' is helpful in comparing treatment effects across multiple T.O.V.A. administrations.
The d' statistic for Half 2 of the T.O.V.A. is also one of the scores that make up the ADHD Score (which we'll discuss in next month's newsletter).
We hope this helps you better understand when, and when not, to make use of the d' statistic. While it can be quite useful, much of the time you can safely ignore it.
If you are interested in learning more about signal detection theory, d' and Receiver Operator Characteristics, you can find an excellent interactive tutorial on the web at http://wise.cgu.edu/sdtmod/index.asp
T.O.V.A. Case of the Month: Clinical Teaching Case
A ten-year-old male was referred with typical presenting problems of inattention, impulsivity and difficulty in both school and social activities, including his favorite activities, sports. Previous assessments documented an above average IQ and no evidence for a learning disorder. Behavior ratings by the teacher and parents were indicative of ADHD, and the history was negative for co-morbidity. (Note: Co-morbidity is the rule, not the exception, with ADHD cases.)
The baseline (no medication) T.O.V.A. data (standard scores) were:
| Variable | QTR 1 | QTR 2 | QTR 3 | QTR 4 |
|---|---|---|---|---|
| Response Time Variability | 100 | 55* | 101 | 88 |
| Response Time | 120 | 97 | 113 | 99 |
| Commission Errors | 87 | 71* | 85 | 99 |
| Omission Errors | 99 | 101 | 101 | 96 |
- significantly deviant from the norm
ADHD Score: -1.45
Beginning with the most important variable, Response Time Variability, we see that it significantly worsens from quarter 1 to 2 and becomes significantly deviant. His performance significantly improves (and normalizes) in quarter 3 when the test condition changes to frequent targets, and many individuals become more stimulated and "wake up" while some become overstimulated, and their performance rapidly deteriorates. This subject does well in quarter 3, but his performance significantly decreases in quarter 4 by 13 standard score points (almost one standard deviation) although it remains within normal limits. It looks like he can function well (as regards variability) for 5-6 minutes and then he becomes significantly inconsistent in both test conditions, making information processing problematic.
Indeed, when we look at Response Time, we see the same pattern - he significantly slows down in quarters 2 (23 points) and 4 (14 points) although RT remains within normal limits. Now we can safely speculate that he has more problems with boring tasks (Half 1) than more stimulating tasks (Half 2) since both RTV and RT decline more in the first half than in the second. He has problems, though, in both conditions.
His Commission Errors (impulsivity) worsen significantly and become deviant in quarter 2, and then they significantly improve in quarter 3 and again in quarter 4. Thus, it appears that he becomes impulsive in a boring task, but actually becomes less impulsive with more stimulation and does not become overstimulated and impulsive, at least as measured by the T.O.V.A. (While his Anticipatory Responses - a measure of guessing - increased in quarter 4, they remained within normal range.)
His Omission Errors (inattention) remain in normal range throughout the test.
The ADHD Score is more positive than -1.80 so it is "inconclusive". It doesn't mean that he doesn't have ADHD. When it's more negative than -1.80, we can say that the profile is similar to that of a group of individuals with ADHD (diagnosed independent of the T.O.V.A.), and that would lead us to more strongly consider ADHD as the diagnosis. That doesn't apply in this case.
What do we have here? Clearly the T.O.V.A. is significantly deviant from the norm and compatible with an attention disorder. Interestingly, we also have some evidence that he does worse in boring tasks and seems to respond (improved impulsivity) to toward task stimulation. These findings are helpful in selecting effective techniques to help him attend better to task and guide treatment interventions. In this case, the School and the Home Intervention Reports (the outlines of which are part of the T.O.V.A. software) can be written to highlight shortening especially boring tasks, etc., and modulating stimulating activities with the expectation that this will be helpful.
In this particular case, subsequent T.O.V.A. testing revealed normalization of the test two hours after a challenge dose of 18 mg of Concerta. After a clinical trial four weeks of medication, behavior ratings and repeat T.O.V.A. testing were within normal limits. Plans are to repeat behavior ratings and T.O.V.A. tests every six months to make any necessary adjustments.
This protocol was selected to illustrate that we can learn more about a person than just whether they have an attention disorder from the T.O.V.A.
Frequently Asked Questions
"What would be the effect on the T.O.V.A. for a 15-year-old male who has smoked substantial marijuana most days for the past two years, but did not smoke any the day of the test?"
When taking into account the effects of substances on T.O.V.A. performance, there are a number of factors to consider, including the dosage, the build-up and duration of effects, and concomitant use of other psychoactive substances (such as caffeine and nicotine) and medications.
In this particular case, since the subject has been on the drug for so long, his THC level is consistently high 24 hours a day. The THC effects will persist for several weeks after the last ingestion although the concentration of THC will gradually decrease.
Like all psychoactive substances, the effect of THC on attention varies with dosage. In low doses, Response Time Variability and error rates decrease, and Response Times become faster. Anecdotal reports indicate that low doses of THC may normalize the symptoms of ADHD. However, in higher doses (like the subject is taking), inattention and processing times worsen and become more inconsistent.
It is very difficult to interpret the test results in these circumstances. Our advice is to talk with the subject and to determine whether he is willing and able to discontinue the marijuana for at least three weeks before repeating the baseline T.O.V.A.
We have similar problems with caffeine and nicotine use since both, in low concentrations, can improve what otherwise might be a deviant T.O.V.A. performance. Thus, a normal baseline performance in someone using these substances is not drug-free. Conversely, if they are on high doses of these substances, their performance may be worsened. If someone is using considerable amounts of one or both substances, and you ask them to go "cold turkey" the morning of the test, acute withdrawal symptoms can result in a worse performance. The best way to handle this situation is to ask that they gradually reduce their use over a two to three week period and then test them early in the morning before they have their first cup or smoke (knowing that there may be some lingering but minimal effects).
In general, prescribed medications for depression, anxiety, seizures, etc., are long term, on-going treatments and should not be interrupted for testing. Instead, we advise that these medications be considered part of the baseline and listed in the other (than challenge) medication blanks on Form 5 so that the clinician will remember to allow for them when interpreting the test results. As an example, anti-depressants do tend to improve test results although they are not as beneficial as psychostimulants.
On the other hand, if, during the course of ADHD pharmacotherapy, new medications are prescribed for other conditions, you may need to obtain a new baseline once those medications are stabilized to be sure that they are not adversely affecting attention and impulsivity.
T.O.V.A. Director of Education and Research, Dr. Steve Hughes
Meet Steve Hughes, PhD, LP, ABPdN, Director of Education and Research for The TOVA Company. Steve's training as a clinical psychologist and specialist training in pediatric neuropsychology led him to a career-long interest in accurate diagnosis and appropriate treatment of children and adults with attention disorders. Steve has been with The TOVA Company for just over two years. He and his family live in St. Paul, Minnesota.
My work with The TOVA Company involves roughly equal parts teaching, clinical and research support, and T.O.V.A. product development. I enjoy the variety and challenges offered by my work, and I especially enjoy discussing cases with T.O.V.A. providers in clinical support calls and meeting T.O.V.A. users when I teach workshops or at meetings.
I was born in Minnesota and, with the exception of a year in Vancouver, British Columbia (where I completed my psychology internship), I've lived in or around the Twin Cities all my life. I received my BA, MA, and PhD from the University of Minnesota. After completing a two-year postdoctoral fellowship in pediatric neuropsychology at the University of Minnesota Medical School, I joined the faculty in the Department of Pediatrics as an assistant professor.
For most of my career, I performed and supervised neuropsychological evaluations of children, adolescents, and young adults suffering from a wide range of developmental, genetic, medical, and behavioral disorders. I found clinical work to be rewarding and intellectually challenging, and there was always something new working in a major medical center! The opportunity to teach and work with graduate students, interns, and post-docs added additional rewards - the teacher always ends up learning the most!
My life took an unexpected turn about two years ago when I received a call from my friend, Andrew Greenberg (Director of Development for The TOVA Company). Would I consider doing some contract work for The TOVA Company? Attend conferences? Teach workshops? Help develop a future version of the T.O.V.A.? Of course!
The rest is history. Within the year, I had taken leave from the university and joined The TOVA Company full-time. While I still do some teaching and research in my old academic digs, the majority of my time is now devoted to ensuring that The TOVA Company continues to lead the field in the assessment of attention and that T.O.V.A. users are getting all they can out of the world's most popular objective measure of attention.
As the son of two educators, teaching workshops about attention and the T.O.V.A. comes second nature. I am very proud of the positive feedback that I receive from T.O.V.A. workshop attendees, and we are always working to add new and interesting material. As a researcher, I enjoy the applied nature of product development with the T.O.V.A., and I'm looking forward to feedback from our customers when Version 8.0 is released some time in the months to come.