Can introspection help?

Carl Joakim Gagnon has posted the following question:

I'd like a post on how you see the general relation between neuroscience and that oldest guide to how the mind works, introspection. Surely the only way to even design and understand experiments is coming at them from both angles?

...I've often noticed (through introspection) that when my mind races and I have a hard time focusing, I stutter; when I force myself to say one-word-after-another, cancelling all blocks with great deliberation, I seemingly stutter more badly for the first seconds, and then, if I allow my mind to slow down and go into synch with my speech, I'm completely fluent. This is all at a very slow pace. If I continue with that, resisting my extremely strong urges to lose focus on a single thought-process, and see each word in my mind before I say it, it becomes easier within a few minutes, and I can speed up just a bit. If I continue this with various strangers throughout the day (also using breathing techniques), I'm almost completely fluent by the end of the day. The urge of the mind to skip back into its default, unfocused, stuttering "track" remains, but will presumably lesson or go away with time.

So that's the introspection side of things. Any way to relate that first-hand report to neuroscience (to approach the problem from the other angle)?

Here are my views on the use of introspection or first-person reports.

1) There is no conflict whatsoever between a first-person experience (introspection) and a third-person science.

2) A first-person report is never wrong as long as the report only contains experiences. For example, no-one can tell you that your statement "I like Tom's blog", "I am not scared when I stutter", or "I like pink" is wrong.

3) Can you use such first-person reports in helping to advance third-person science? I always listen to first-person reports as they may offer food for thought and inspiration to look for new avenues. However, I am playing with the devil because even if you are a professionally trained thinker / scientist you will almost inevitably commit logical fallacies when trying to generalise the first-person report or your experience to a population.

4) To summarise, introspection of my own experience or a first-person reports can be a useful inspiration to construct theories, but they need to be tested with the scientific method.

5) Here are some of the pitfalls that I constantly encounter:

a. You cannot know whether an effect is present in all stutterers or only specific to the person who gives the first-person report. You can only find that out by doing a statistical analysis.

b. Reports might use the same words, but they might be different meaning. The words are very loosely defined. Your fear of a threat is not my fear of embarrassment. Your block is my slight hesitation.

c. Memories gets re-written every time they are re-called, and many memories are a mixture of a seed of true memory, and post-hoc interpretation.

d. Virtually always the reports are not actually reports of immediate experiences but a coherent story that your mind has woven from experiences and interpretation of what the gaps could be. The readers of first-person reports finds the untangling very difficult and tricky.

e. The greatest pitfall I see with introspection is the logical fallacy "Correlation is not necessarily causation". For example, "I started stuttering when my brother was born, and hated not being the focus of attention anymore" are factually correct, but for many this sounds like great evidence for a conflict between siblings being the cause of my stuttering". But I could also have written "I started stuttering [at age 3 at the age everyone starts stuttering]", "my brother was born [when I was age three which is not very surprising as parents have children at a 2-3 year interval], and "I hated not being the focus of attention anymore [like millions of other kids]". Now suddenly correlation between the three statements sounds like a coincidence and not like a casual link between them.

So to summarise, I use introspection or first-person reports for inspiration, but you are playing with the fire. And almost everyone that uses them gets burned. But luckily for them they don't notice the burns. :-)

Note: Some scientists refuse the use of introspection and first-person experiences as "impure science", but they are wrong. Playing with the fire can be highly insightful, if you are careful.