Tuesday, October 30, 2012

The Inferiority of Expert Judgment

What follows applies to a vast range of topics, but is of particular interest (to me) in how it relates to the practice of medicine.

I quote from the book by Daniel Kahneman, Thinking, Fast and Slow.

In the slim volume that he later called "my disturbing little book," [Paul] Meehl reviewed the results of 20 studies that had analyzed whether clinical predictions based on the subjective impressions of trained professionals were more accurate than statistical predictions made by combining a few scores or ratings according to a rule. In a typical study, trained counselors predicted the grades of freshmen at the end of the school year. The counselors interviewed each student for forty-five minutes. They also had access to high school grades, several aptitude tests, and a four-page personal statement. The statistical algorithm used only a fraction of this information; high school grades and one aptitude test. Nevertheless, the formula was more accurate than 11 of the 14 counselors. Meehl reported generally similar results across a variety of other forecast outcomes, including violations of parole, success in pilot training, and criminal recidivism.

Not surprisingly, Meehl's book provoked shock and disbelief among clinical psychologists, and the controversy it started has engendered a stream of research that is still flowing today, more than fifty years after its publication. The number of studies reporting comparisons of clinical and statistical predictions has increased to roughly two hundred, but the score in the contest between algorithms and humans has not changed. About 60% of the studies have shown significantly better accuracy for the algorithms. The other comparisons scored a draw in accuracy, but a tie is tantamount to a win for the statistical rules, which are normally much less expensive to use than expert judgment. No exception has been convincingly documented.
The range of predicted outcomes has expanded to cover medical variables such as the longevity of cancer patients, the length of hospital stays, the diagnosis of cardiac disease, and the susceptibility of babies to sudden infant death syndrome.... In every case, the accuracy of experts was matched or exceeded by a simple algorithm.
Why are experts inferior to algorithms? One reason, which Meehl suspected, is that experts try to be clever, think outside the box, and consider complex combinations of features in making their predictions. Complexity may work in the odd case, but more often than not it reduces validity. Simple combinations of features are better. Several studies have shown that human decision makers are inferior to a prediction formula even when they are given the score suggested by the formula! They feel that they can overrule the formula because they have additional information about the case, but they are wrong more often than not.  
Another reason for the inferiority of expert judgment is that humans are incorrigibly inconsistent in making summary judgments of complex information. When asked to evaluate the same information twice, they frequently give different answers. The extent of the inconsistency is often a matter of real concern. Experienced radiologists who evaluate chest X-rays as "normal" or "abnormal" contradict themselves 20% of the time when they see the same picture on separate occasions. ... A review of 41 separate studies of the reliability of judgments made by auditors, pathologists, psychologists, organizational managers, and other professionals suggests that this level of inconsistency is typical, even when a case is reevaluated within a few minutes. Unreliable judgments cannot be valid predictors of anything.

*  *  *  *  *  *  *

Truth is...there's a reason why it's called practicing medicine.

Tuesday, October 23, 2012

Happy Birthday, Lady Liberty

October 28, 1886, the Statue of Liberty was unveiled in New York's harbor in a ceremony attended by President Grover Cleveland and witnessed by a steamship crowded with European immigrants arriving to the shores of America with weary eyes and huge dreams.

Most people have heard the phrase, "Give me your tired, your poor, your huddled masses yearning to breathe free," but how many of us have ever read the complete poem engraved into Liberty's base? It was written in 1883 by Emma Lazarus, and is titled "The New Colossus."

Not like the brazen giant of Greek fame,

With conquering limbs astride from land to land;
Here at our sea-washed, sunset gates shall stand
A mighty woman with a torch, whose flame
Is the imprisoned lightning, and her name
Mother of Exiles. From her beacon-hand
Glows world-wide welcome; her mild eyes command
The air-bridged harbor that twin cities frame.
"Keep, ancient lands, your storied pomp!" cries she
With silent lips. "Give me your tired, your poor,
Your huddled masses yearning to breathe free,
The wretched refuse of your teeming shore.
Send these, the homeless, tempest-tossed to me.
I lift my lamp beside the golden door!"

Truth is...while many a dream has been shattered amid our fields of amber grain, and one would be tempted, in the deafening roar of political mud-slinging, to think we've gone beyond the point of no return, there still lives here in the land of the free and the home of the brave a hope that will not die and a commitment to peaceful coexistence that will not waver.

Tuesday, October 16, 2012

Murder or Medical Procedure?

In my humor blog, Almost the Truth, there is a recurring feature called "Almost the News," where I quote actual headlines and then skewer their intent with my twisted perspective on word meaning and syntax. (You can open the latest example in a new window by clicking here.) Because of this, I tend to scan the newspaper headlines frequently...and because of that, I recently found the following Associated Press piece (PG Warning...this is slightly graphic):

Woman Convicted In Fetus Abduction

A Milwaukee woman who confessed to trying to steal a baby by attacking a pregnant woman and slicing out her full-term fetus was convicted Thursday of killing them both.

Jurors deliberated for less than two hours before finding [the woman] guilty of two counts of first-degree intentional homicide in the October 2011 deaths of the mother and fetus. [She] faces a mandatory life sentence...though the judge could allow for the possibility of parole.

A key piece of evidence during the trial was a videotaped police interview in which [the defendant] described her attack.... She told investigators she was desperate to have a son, that she had faked a pregnancy, and that she devised a plan to steal an unborn baby as her supposed due date approached.

*  *  *  *  *  *  *

Apparently, one's intent is of utmost importance when it comes to matters of ending a human life. (For more on the humanity of the pre-born, see my previous post, On Abortion.) If a person is selfishly wanting a child but inadvertently kills it, that is homicide. But if a person is being paid by the mother to kill the child on purpose, that is a legally-defensible constitutional right.

Truth is...intentionally taking a human life, no matter what his or her size, level of development, environment, or degree of dependency, is taking a human life.

Tuesday, October 9, 2012

Do You Trust Me?

Aladdin is trying to help the young incognito princess to escape their pursuers. They're running hither and yon and come upon what seems to be a rooftop dead end. Extending an invitation to jump off the roof for crying out loud, the street rat says to the damsel in distress, "Do you trust me?"

There is no time to explain or debate or come up with an alternate plan. She needs to decide...Do I entrust my well-being to this guy, or do I head off and take my own chances?

Methinks it almost always comes to a point like that for people considering Christ.

A person can study books and weigh opinions and philosophize about altruism and social justice, but sooner or later it's going to have to come down to a decision...Do I entrust my well-being to this guy, or do I head off and take my own chances?

Truth is...it's not about what Jesus taught; it's about who he claimed to be and what you're going to do about that. It's not about believing ideas; it's about trusting a person.

Tuesday, October 2, 2012

Press On

Billy Sprague has been vagabonding around Christian music and youth ministry for a few decades (you'll notice the decidedly-dated duds in the attached video). This song is a fun, upbeat encouragement to keep on keepin' on, no matter what happens. The amazing thing is that it was born, I believe, out of the pain of Billy's fiance's death. 

I was down in the valley of the shadow of death 
Where the passion for life drained like blood from my chest 
And it took more than my will just to take a step 
When the compass of hope was gone 

In a silence so black that I wished for the blues 
every desperate prayer seemed like heaven refused 
and some days I found faith meant just tying my shoes 
and it was all I could do to press on. 

Press on, mi amigo; press on, mon ami; 
walk on in the face of the mystery. 
Though the night hides the light through the darkness til dawn, 
tie your shoes, my dear friend, and press on. 

On the ocean so lonesome I was not left alone-- 
had some heavyweight friends when my heart was a stone, 
And they carried the heartache and made it their own 
When the currents of sorrow were strong. 

One said, "I pray your memories will not drag you down, 
not be anchors but treasures of the love that you've found," 
and his kind words turned hurt into comfort somehow 
and a wind in my sail to press on. 

Press on, mi amigo; press on, mon ami; 
walk on in the face of the mystery. 
Though the night hides the light through the darkness til dawn, 
lean on me, my dear friend, and press on. 

Though the days seem like years--there may be giants in our fears-- 
Ah, but they who sow in tears--they shall reap in joy; 
They shall run and leap for joy. 

Press on, mi amigo; press on, mon ami; 
walk on in the face of the mystery. 
Though the night hides the light and the journey is long, 
tie your shoes, my dear friend, and press on.

PRESS ON  -  Billy Sprague & Jim Weber
c.1993 Paragon Music Corp / Skin Horse, Inc.

*  *  *  *  *  *  *

Truth is...I've never met anyone suffering from being over-encouraged. So bookmark this and use it to your advantage the next time you get the short end of the stick.