Thursday, May 31, 2007

Discussing the Lidcombe RCT

Ora writes:
I've just read your September 2005 letter to the BMJ "The statistical fluctuation in the natural recovery rate between control and treatment group dilutes their results. The statistical fluctuation in the natural recovery rate between control and treatment group dilutes their results" You wrote: I find a 10% probability that a "minimum worthwhile difference between the two arms set at 1.0% syllables stuttered" occurs due to statistical fluctuation and not due to treatment effect. The standard aims for a less than 5% chance that the observed results occur by chance (usually expressed as p<.05), whereas you're observing that this trial achieves only 10% (p<.10). Is that what you're saying? If so, 90% is obviously not as powerful a result as 95%, but it's not worthless either.
I show that two control groups (i.e. I assume no treatment effect) have a worthwhile difference in 10% of all cases, because the natural recovery rate causes the groups to start at a different baseline (they have a different percentage of natural recovers by chance) (This effect gets smaller with increasing sample size). Then I argue: If with no treatment effect, I have a 10% chance of seeing an effect, then something is wrong because I should expect that in 0% of the cases there is an effect because both groups receive no treatment. So their p-value (which they claim less than p<0.01) MUST BE wrong and higher, because the t-test assumes that they are the same. So their statistics is wrong, and they must re-do their stats. What exactly the value is is not easy to compute.
Possibly instead of choosing an effect size (1.0% difference in syllables stuttered) that allowed them to claim p<.10, they should have claimed a smaller effect size (for example, 0.5% difference in syllables stuttered) that would have supported the stronger claim of p<.05.
What you are doing is shifting the criteria to get a result! See my arguments above, we need a strong signal to be sure that no other systematic effect can destroy it.
Though I don't fully understand your argument in your Sept 2005 letter, I would agree if you're merely saying that their results may not be as strong as they're suggesting. But on the other hand it seems to me that your claim that "the random control trial did not show efficacy despite their claims" is also too strong; 90% may not be 95%, but a 90% number is still strongly suggestive.
I am saying that the p-value is much higher, but I do not know how much higher.

However, there is at least one more crucial flaw in the study that I haven't talked about in the letter but I did in my talk which supports my claim that they have not convincingly shown efficacy. They only observe the kids for 9 months. Let's assume that the only thing Lidcombe does is to accelerate the recovery of those kids that would have recovered anyway, then I see a treatment effect over the short-term. Here is how it works. Lets assume at the start 100% of the kids stutter, and after 2 years only 20% of the kids stutter, those that do not recover naturally. I also assume that 40% recover naturally per year, so I have 40% first year, 40% second year, and 20% don't recover. If I run the experiment for one year. Then in the control group only 40% recover, but in the treatment group 80% will recover (remember I assume that Lidcombe makes the kids who recover naturally recover faster). And so I have a 40% higher recovery in the treatment group!!! But if I wait for one more year, the control group will have the other 40% kids recover naturally, and then there is NO difference any more between the two groups!!! So they need a longer observation period to control for this effect! Another aspect might be to look for relapses of kids...

Saturday, May 26, 2007

Mark Onslow and Lidcombe at ISA


At ISA, I attended Mark Onslow's workshop on the Lidcombe program. He didnt seem to be very pleased when I told him during the workshop that his random control trial is flawed. The day before, I had given a talk (which Mark did not attend, I think) on why the random control trial did not show efficacy despite their claims. Sue Block was not too pleased, and said that random control trials are widely used in all kinds of areas. Of course, this is the typical response without addressing my arguments. Why can no-one say: Tom, I have listened to your arguments. I don't agree with your argument on slide 10, because of X and Y. That's why you are wrong. No, everyone says but RCTs are widely used, or the RCT's statistics have been computed by a statistician. Which kind of scientific debate is this??? Anyway, I met Sue Block later at the B&withnoB, and we had a more constructive debate.

Several people approached me regarding my talk or even congratulated me for standing up against Lidcombe. I just want to emphasise that I am not really against Lidcombe, just against their claim that they have shown efficacy with an RCT. I am not an ideologue who feels a strong urge to defend his/her view of the world. Here are my thoughts:

1) I have been to the workshop, and overall the program seems to make sense and I dont think it will harm the children at the very least. In fact, it does encourage good parent-child interaction, which should be beneficial anyway. So it might well prevent greater behavioural, social and psychological effects from developing.

2) What I do criticise is that Mark Onslow claims that research has shown efficacy. However, as I said before, they have proven anything. In fact, during the workshop he himself showed some long-term data that suggests that only 84% of the children recover, which is close to natural recovery.

3) My guess is that Lidcombe might well help kids to recover faster that would have recovered anyway, might well ease tension between kid and parents (the therapist acting as supper-nanny), and for the non-recovers reduce some secondary effects, BUT I doubt very much that it makes the kids fluent that would not have recovered anyway. To do this, the program is not interventionist enough and one needs to do more active speech work.

4) If I had a stuttering child and the choice of doing nothing or doing Lidcombe, I would certainly do Lidcombe with them.

5) I get the feeling Mark does not really know how to handle my statistical arguments, but his statistician should have: Mark Jones. He has never properly addressed my issues I have raised by replying to my email or my rapid response in the British Medical Journal.

6) Lidcombe might well have an effect but on a behavioural (less severe secondary effects like grimaces or strong blocks) and psychological (avoidance and shame) level, which is much more difficult (if not impossible) to capture in a trial.

Monday, May 21, 2007

ISA: Too expensive at average quality

Despite the beautiful site, I found the congress quite expensive, especially as I had to pay it all by myself. And in general I found Cavtat and Dubrovnik quite expensive for the quality offered. I spent several holidays in Italy, and I have to say that the food in Italy is much better and similar prices. The people are far more friendly and you are less treated like a walking euro/dollar symbol.

For example, the gala dinner food was not very good. First of all, they only served champagne at the reception on the top terrace, not even water or orange juice. When I asked for orange juice, the waiter said: No. Everywhere else, they would have organised something, but not here. Big Minus. The buffet food was mass production. They even managed to have factory cakes as dessert: you know the one you can buy in the supermarket next door in boxes of 100 or 200! And the fish risotto was far too salted.

Next comes my hotel room. I booked through the conference organiser's recommended travel agent, because I assumed that they would make sure that the deals are fair. So I paid 70 Euros for a very very simple room in a flat complex 5 minutes from Hotel Croatia, and with breakfast at Hotel Croatia. As I stayed one day longer, I was curious how much the room in the neighbouring bed and breakfast would cost. I paid 20 Euros!!! So in fact they made at least 40 Euros per day. This was a complete rip-off, and I think it was irresponsible of the conference organisers not to check whether the deals offered by the agents were fair!!!

Then the day trip to Montenegro cost 90 Euro. I didn't go, because I was still upset by the way they "stole" 5 times 40 Euros from me. 90 Euro per person for a day trip is ridiculous. The bus costs at most 15 Euros per person, then lunch 20 Euros, and the guide 5 Euros, so in total 40 Euros at most! Again every single person was ripped off by 50 Euros!! Again, I blame the conference organisers for not making sure the deal very fair.

I did tell one of the organisers and someone from ISA that I found the price far too high for the services offered. And I think it is especially unfair for the many visitors from poorer countries. OK, I shut up now... :-)

A beautiful site for the ISA congress

ISA is the umbrella organisation of the national stuttering associations, and has a congress every four years. The last congress was is Perth (Australia). This year's congress was in Croatia in Cavtat near Dubrovnik. The conference itself was held in Hotel Croatia which is beautifully set on a hill next to the coast. The first picture shows the hotel's terrace where conference participants sat during breaks. The great thing was that they had a free wireless network! The next picture shows the nice hotel beach. I spent two days there. There was no sand, but it was OK. I also went swimming there.
The most useful room in the hotel was the disco/bar Posseidon in the cellar. The design is all Austin Power style, and a big dance floor in the middle. It was open every day, and often people from the conference would get together there. I even managed to convince Velda Osborne, Chair of the BSA and member of the Older Generation, to join in after the gala dinner. The next day she blamed me that I had been a bad corrupting influence on her, and that I "made her" stay at the bar until 3 o'clock in the morning. When she went up to the reception to get a taxi back to her hotel, the porter said "But Madam, it is 3 o'clock in morning, there are no taxi." And then I think he drove her back...Picture are from the net. If someone has better pictures, pls let me know!

Sunday, May 20, 2007

Summary of ISA

Sorry I havent posted for a week. I needed to recover from the intense ISA conference in Dubrovnik! :-)

I really should have posted real-time from the conference, but unfortunately I didnt. As soon as I get too many different inputs, my focus gets hijacked.

Here are the topics that I will talk about:

0) A beautiful site for ISA

1) The high costs of the conference

2) Tom and Lidcombe.

3) AAF Devices and their usefulness.

4) Jaik at ISA.

5) People from all around the world.

Tuesday, May 15, 2007

Back from ISA / Live feeds

I am back from the ISA Congress from Dubrovnik (Croatia). I had a great time, and I will write more soon.

There is/was a live feed for the ISA Congress: here. I am not sure whether there are recordings of the key note speeches. If you know more, pls let me know.

Sorry, I wanted to post this information much earlier. By the way, the live feeds were done by Manfred. (I forgot his family name). Many thanks to him. This allowed people from around the world to be part of the ISA congress.

Monday, May 07, 2007

ISA(II)

Tim Saltuklaroglu spoke about the impact of stuttering on the physiological state of the listener. He showed that the listener has a lower heart rate and more blinking. Clearly, the listener reacts. There were many questions.

At the end, Illia and Natalia Rasskazova showed their software NEODIALOG that trains effective communication by measuring intonation.

More soon, my battery is nearly empty.

ISA live (I)

Right now, I am in a talk at the ISA conference. Already after the first slide, I fundamentally disagree with the lecturer! So let me talk about the research session that I chaired this morning.

Patrick Kelly talked about his experience as a therapist for a rural area in Ireland. He has worked there for more than 10 years and sent out a questionnaire to all his past children that he treated. He reports are very high recovery from stuttering; I think 44 out of 46.

Then Jelena Tadic showed evidence that many children who stutter also have articulatory problems. However, her sample was quite small, less than 20. So I am a bit skeptical.

Then Soster claimed that the voice of people who stutter shows difference to normal speaker's voice. I am a bit skeptical, and suggested that she should do the reverse test, i.e. is it possible to identify stutterers by only listening to their voices?

Sunday, May 06, 2007

Off to the ISA conference

I am off to the ISA (International Stuttering Association) conference in Dubrovnik (Croatia). I might be blogging from there, depending on the availability of Internet Access.

Thursday, May 03, 2007

Too much blogging..

Too much blogging about stuttering is bad for you. Here is what happened to The Stuttering Student. :-) Funnily enough, when I met him in San Diego, he came across as rather calm and a bit shy!!!

Tuesday, May 01, 2007

Competition from junior stuttering brains

Oh my god, the competition is getting younger: The Kids Speak Blog.

Monkeys, Hands, and speech


Have a look at this BBC science article: here:
They analysed the way two types of apes that are closely related to humans use hand and limb gestures to communicate. The scientists found that apes used gestures more flexibly than the way they used facial and vocal expressions. They say the findings support the theory that human language developed through the use of hand gestures.

The most appealing argument for me is that for most people the right hand is the dominant hand, which is controlled by the left-side brain. And speech and language is located only in the left-side brain. This is more than a coincidence.

I am talking about this connection, because people who stutter have subtle dual tasks defficiencies with hand or finger movements: see previous post. Maybe this is all related?