Catherine DeSoto and Robert Hitlan have published an FAQ site Frequently Asked Questions about DeSoto and Hitlan (2007) to answer questions and refute some inaccurate, misleading and even erroneous critiques of their important article published in the November 2007 article of the Journal of Child Neurology Blood levels of mercury are related to diagnosis of autism: A reanalysis of an important data set. Most of the FAQ deals with questions raised on Autism Street in A Tale of Two Tails by bloggers Interverbal and Do'C. The two bloggers published a critical review of he DeSoto and Hitlan paper on Autism Street. The Desoto-Hitlan FAQ site exposes many information gaps in the Interval-Do'C critique.
DeSoto and Hitlan responded very politely to the critique and the FAQ site is important reading for anyone with a serious interest in autism, although it will undoubtedly be difficult for many of the 87 Autism Street cheerleaders who published their gloating sarcastic commentaries to read with an open mind. Interverbal posted an interesting comment:
Comment by Interverbal — 18 November, 2007 @ 4:06 pm At this time I don’t think we have any plans to write a letter to the editor. Speaking only for myself, I am anxious to see how Ip et al respond to DeSoto & Hitlan.
As one who does not share Interval's and Do'C's knowledge of statistical methodologies I might have been confused in saying that I too look forward to any reply by Ip et al. As I read DeSoto and Hitlan's FAQ it appears to me that they are saying that Ip et al already acknowledged their errors by the time the DeSoto article was published in November:
Q. Why did you take the time to write about a mistake that had already been corrected by the authors?
A. We didn’t. The mistake had not been found until we found it. We are the correctors. Again, some blog sites have unfortunately served to confuse this issue.
Q. What was really so wrong with the Ip 2004 article?
A. Based on their retraction which appeared in the same issue issue as our article, the mean for the autistic group was wrong, the standard deviations were wrong for both groups, the stated statistical significance in 2004 was way off. The means as they reported them in 2004 result in a significant t test by any standard…meaning that the autistic group had significantly more mercury in their blood than the control group. This is indisputable (or should be). It would not matter if a one tailed or a two tailed test was used. All interested parties should use their original data from the 2004 article and calculate the t value and p value (or put the numbers into an online t test calculator-- see "how can I check the original numbers myself?"). Their original stated level of statistical probability was off by almost 10 fold.
The data set they provided in 2007 misses conventional significance by a hair using their original statistical technique. Some blog sites such as Age of Autism have also pointed out that Ip et al overstated their findings in 2004. This means that the conclusions they made reached way beyond their findings. This is less serious compared to flubbing your stats, but I will note it for completeness.
If I am right in my reading of the DeSoto and Hitlan FAQ site commentary then I assume that Interval and Do'C will publish a retraction of their analysis on Autism Street. I will hold my breath until then. No, no ... on second thought ... I better not. I don't know if Interval and Do'C have the integrity to admit error. My lungs could explode, or I could implode, waiting for the retraction.
Too bad Interval didn't write that letter to the editor.
autism
Hi Harold,
ReplyDeleteActually, I am pleased that Drs DeSoto and Hitlan took the time to write the FAQ page. Do’C and I did make some errors (as have others) and I am nothing, but grateful that they cared enough to correct them.
You write “As I read DeSoto and Hitlan's FAQ it appears to me that they are saying that Ip et al already acknowledged their errors by the time the DeSoto article was published in November”
They did. What I was hoping was that Ip and company would have some comments about several aspects of DeSoto’s reanalysis. This is not clear in my comment. I believe that I did make this clear later in the comments section.
Also, I am not very fond of writing letters to the editor. I don’t find that they are usually worth the time. Just my opinion.
There are several semantic and factual problems in our critique and in the comments. There are also a lot of gray areas, that seem very peripheral on second glance. I would like to clean up our article in these regards. Do’C will be unavailable for a while so this will take a few weeks.
However, there are several issues that were raised in our original critique or that emerged after Dr. DeSoto’s first visit that seem to be genuine problems or that certainly are genuine problems in the DeSoto & Hitlan article. I also have some problems with what DeSoto & Hitlan wrote in their FAQ. I would like address this at some point in the near future.
Some examples of problems would be: DeSoto & Hitlan reference an article as stating that an 11 month old boy was presumptively diagnosed with autism before a formal diagnosis of mercury poisoning was made. This is inaccurate.
Also, DeSoto claimed in the comments that there is a scientific consensus that a portion of the increase in autism is caused by diagnostic substitution and the rest is caused by a real increase. To our knowledge there is no consensus paper stating such. Also no reference was given.
There are other problems also. None of these were addressed in the new FAQs page. They probably should be.
So… while I would like to clean up the critique, we will certainly be keeping the critique. We also will be waiting patiently to see if the important issues that we brought up, but did not make it to FAQ page, are addressed.
Best wishes for you and your family.
nterverbal
ReplyDeleteThanks for your comment and for wishing my family well. I extend best wishes to you and yours. And I look forward to reading your revisions.
I am not in any "camp" when it comes to mercury/vaccine and autism issues. I received considerable hostility on local autism newsgroups a few years ago when I posted news of a medical journal article asking questions about Wakefield's theory. I also wrote early on that my son seemed autistic from birth although we didn't think of him as autistic, we just noted odd behavior and lack of speech development.
It also seems obvious that the expansion of the DSM criteria and greater awareness (including via the internet) have contributed SUBSTANTIALLY to the increase.
But to me even those factors do not NECESSARILY explain the incredible increases ENTIRELY. Since my son's diagnosis 10 years ago - which was AFTER the DSM revisions - the prevalence rate has changed from 1 in 500 to 1 in 250, 166 and now 150. In the UK the autism prevalence is estimated at 1 in 100 and 1 in 98 in NJ.
As long as the actual causes or contributors to autism are not known with any reasonable certainty it is unreasonable to say that these incredible increases are ALL due to social/diagnostic changes.
You may disagree with whether there is a consensus of thought about a combined genetic/environmental causation theory of autism but there is in fact a growing number of respected researchers who are of the view that environment plays a triggering role in development of autism.
DeSoto and Hitlan performed a valuable service in pointing out that the initial Ip data had been improperly ananlyzed. And the central point of their article - that it is important to analyze such data properly should be beyond dispute. Particularly when Fombonne and others reference the Ip study as one authority for the absolutist view that mercury plays no role in autism causation.
I have never accepted that there is enough evidence to conclude that mercury is behind the autism increases. But I have never accepted, and still do not after the California report, that we can conclusively rule it out either. You know the arguments about the trace elements in some vaccines but vaccines are not the only source for mercury contamination in our environment. Anyone of a certain age with dental fillings knows that. Anyone who has read warnings about consumption of certain fish products knows that.
As I read DeSoto and Hitlan, and I am not a statistician, I never read their article as attributing autism increases to vaccines or even definitively to autism increases just that the original conclusions ruling out a mercury autism connection arose from faulty analysis.
I also understood from the article that the Ip data does SUGGEST that there might be correlations if not a causal connection between mercury and autism. That would be important to know if borne out by further study. I hope your future revisions addresses central, important, issues. Internet autism debates tend to digress into irrelevant side paths and central issues simply become bludgeons used without any analysis let alone the careful analysis provided by DeSoto and Hitlan. We need less digression and more focus on what is important, understanding the causes and nature of autism, finding treatments and cures.
I appreciated your serious and considerate post. I had expressed doubts that you would respond seriously. My doubts were ill founded.
I moderate the comments to this forum for a number of reasons, liability, profanity, hostility,lack of courtesy, endless redundant and repetitive argument etc. I thank you for your courteous and reasonable comment.
Thanks Harold,
ReplyDeleteYou raised some important issues in your response. I would like to add some add some comments that I think will help support why we argued, what we argued. All lot of this relates to autism epidemiology which is an area of formal study for me.
You write “Since my son's diagnosis 10 years ago - which was AFTER the DSM revisions - the prevalence rate has changed from 1 in 500 to 1 in 250, 166 and now 150. In the UK the autism prevalence is estimated at 1 in 100 and 1 in 98 in NJ.”
-The DSM revisions came in 1994 as you note. However, the first study to actually use the new criteria was Bertrand et al. (2001).
-The 1 per 166 and 1 per 150 are equivalent. Either can be considered correct. You can use recent studies to justify both. Sometimes a study will land between these and one group will say 1 per 166 and another will say one per 150. It doesn’t matter in the statistical sense. The confidence intervals are always in bounds of either number.
-The UK numbers come from Baird et al. (2006). They were using the ICD-10 criteria. Compare this to the 1 per 166 found by Chakrabarti & Fombonne 2001 & 2005 who were using a modified DSM-IV method.
-The 1 in 98 n NJ comes from databases, not from ground level epidemiology like the above studies. Specially, the researchers used the special education IDEA data combined with a medical dataset. Many states only had one dataset. People get very excited when they talk about why New Jersey has the highest number out of any State. People forget or don’t know that many States only included the IDEA data.
-Assignment to an IDEA service category does not constitute a medical diagnosis. This is very hard for many people to realize. All the IDEA data records is assignment to a service category by an IEP team which includes the parents, administrator, special ed teacher, regular ed teacher, and sometimes other professionals like a school psychologist. It is this group that decides what service label the child will receive and a child only gets one service label.
-Administrative Datasets like the medical data used to find 1 per 98 in NJ typically do not control for the 6 statistical threats. They are open to tremendous error and variability.
You write “You may disagree with whether there is a consensus of thought about a combined genetic/environmental causation theory of autism but there is in fact a growing number of respected researchers who are of the view that environment plays a triggering role in development of autism.”
Well for that matter, I too think there could be combined genetic and environmental trigger. That is not an issue in contention in our critique. The issue was whether there was a scientific consensus over the proportion of real increase vs. increase due to diagnostic substitution. There is no consensus in this regard, at this time. That is not an opinion, it is a fact.
You write “DeSoto and Hitlan performed a valuable service in pointing out that the initial Ip data had been improperly ananlyzed. And the central point of their article - that it is important to analyze such data properly should be beyond dispute.”
It is beyond dispute, it has never been in dispute.
You write “But I have never accepted, and still do not after the California report, that we can conclusively rule it out either.”
Neither do I. Neither does the CDDS. What the CDDS themselves, myself, and others have been saying again, and again, and again, and again, is that CDDS data can not be used for purposes of establishing prevalence and incidence. There are tons of real problems with that dataset that CDDS themselves acknowledge and are crystal clear about. The CDDS is not a “gold standard” in autism epidemiology. That term was first applied by a politician. It has never been echoed by any reputable scientist.
What the report does do; is put a nail through the coffin of the false belief that the CDDS data shows a decrease. Ironically this is what Do’C, myself, and others have been saying for over two years.
As to mercury from other sources. I made some predictions way back in 2005 as to what would happen if the autism levels didn’t drop in the CDDS. Apparently I was correct. See for yourself:
http://interverbal.blogspot.com/2005/12/modest-prediction-or-two.html
My problem is that mercury, being a natural element found in the Earth and oceans, is never going to be out of our lives. There is always going to be a fallback even if we take away one unnatural source. There is always going to be an “okay we took it out of lights, but there are still more kids with autism, so maybe it is the…….”It becomes a shell game after a while.
I really don’t know if thimerosal is a major or minor cause of autism. However, I am certain that in an area of so much uncertainty, of so much complexity, of so many theories that make sense on the basic theoretical level, that we need rigorous science, facts, and crystal clear thinking. This is the backstory to our critique. This was not the critique of scientists who found and error, but the critique of several aspects of their reanalysis and their justifications for how we should view the new data.
Thanks again.
Harold wrote: "DeSoto and Hitlan responded very politely to the critique and the FAQ site is important reading for anyone with a serious interest in autism, although it will undoubtedly be difficult for many of the 87 Autism Street cheerleaders who published their gloating sarcastic commentaries to read with an open mind."
ReplyDeleteHi Harold. I did read the FAQ, and I have to say, I'm appreciative that Dr.s DeSoto and Hitlan took the time to put it together. While I disagree with your assessment:
"The Desoto-Hitlan FAQ site exposes many information gaps in the Interval-Do'C critique."
I can see how anyone without a relevant background or education in physiology or medicine may think this to be the case. That being said, we did make a couple of minor mistakes and assumptions. We'll address those very soon - I promise. With the release of their FAQ, We'll also have more to say. (You're not surprised, I'm sure.)
Originally posted under a different comment:
ReplyDeleteHi Harold,
Please feel free to move this comment as it concerns DeSoto and Hitlan (2007).
Do'C and I have published our errata in "Part 1" on Autism Street.
Part 3 concerns our remaining criticisms. You may or may not find it interesting.
http://www.autismstreet.org/weblog/?p=165#more-165