Category: Methodology

Needed Revolutionary Ideas Often Come From Outsiders

(p. 103) . . . where knowledge is no longer growing and the field has been worked out, a revolutionary new approach is required and this is more likely to come from the outsider. The skepticism with which the experts nearly always greet these revolutionary ideas confirms that the available knowledge has been a handicap.”

Source:
W. I. B. Beveridge as quoted in Meyers, Morton A. Happy Accidents: Serendipity in Modern Medical Breakthroughs. New York: Arcade Publishing, 2007.
(Note: ellipsis added.)

Medical Innovator “Maintained a Healthy Skepticism Toward Accepted Wisdom”

(p. 103) Barry Marshall, a lanky twenty-nine-year-old resident in internal medicine at Warren’s hospital, was assigned to was assigned to gastroenterology for six months as part of his training and was looking for a research project. The eldest son of a welder and a nurse, Marshall grew up in a remote area of Western Australia where self-sufficiency and common sense were essential characteristics. His personal qualities of intelligence, tenacity, open-mindedness, and self-confidence would serve him and Warren well in bringing about a conceptual revolution. Relatively new to gastroenterology, he did not hold a set of well-entrenched beliefs. Marshall could maintain a healthy skepticism toward accepted wisdom. Indeed, the concept that bacteria caused stomach inflammation, and even ulcers, was less alien to him than to most gastroenterologists.

Source:
Meyers, Morton A. Happy Accidents: Serendipity in Modern Medical Breakthroughs. New York: Arcade Publishing, 2007.

“It Is Often Essential to Spot the Exceptions to the Rule”

Baruch Blumberg was awarded the Nobel Prize in 1976:

(p. 98) . . ., Blumberg learned an invaluable lesson: “In research, it is often essential to spot the exceptions to the rule–those cases that do not fit what you perceive as the emerging picture…. Frequently the most interesting findings grow out of the ‘chance’ or unanticipated results.”

Source:
Meyers, Morton A. Happy Accidents: Serendipity in Modern Medical Breakthroughs. New York: Arcade Publishing, 2007.
(Note: ellipsis added.)

Theory Said Giant Bird Could Not Fly, But It Flew Anyway

(p. A3) Scientists have identified the largest flying bird ever found–an ungainly glider with a wingspan of 21 feet or more that likely soared above ancient seas 25 million years ago.
Until now, though, it was a bird that few experts believed could get off the ground. By the conventional formulas of flight, the extinct sea bird–twice the size of an albatross, the largest flying bird today–was just too heavy to fly on its long, fragile wings.
But a new computer analysis reported Monday [July 7, 2014] in the Proceedings of the National Academy of Sciences shows that the bird apparently could ride efficiently on rising air currents, staying aloft for a week or more at a stretch.
. . .
“You have to conclude that this animal was capable of flapping its wings and taking off, even though it is much heavier than the theoretical maximum weight of a flapping flying bird,” said Luis Chiappe, an expert on flight evolution at the Los Angeles County Natural History Museum, who wasn’t involved in the project. “Our modern perspective on the diversity of flight is rather narrow,” he said. “These were very unique birds.”
. . .
“This was a pretty impressive creature,” said avian paleontologist Daniel T. Ksepka at the Bruce Museum in Greenwich, Conn., who conducted the analysis of the bird’s biomechanics. “Science had made a rule about flight, and life found a way around it.”

For the full story, see:
ROBERT LEE HOTZ. “U.S. NEWS; Giant Bird Was Able to Fly, Scientists Find; Computer Analysis Shows Ancient Glider Could Get Off the Ground, Defying Conventional Theories of Flight.” The Wall Street Journal (Tues., July 8, 2014): A3.
(Note: ellipses, and bracketed date, added.)
(Note: the online version of the article has the date July 7, 2014.)

Serendipitous Discoveries Are Made by “Accidents and Sagacity”

(p. 6) “Accident” is not really the best word to describe such fortuitous discoveries. Accident implies mindlessness. Christopher Columbus’s discovery of the American continent was pure accident–he was looking for something else (the Orient) and stumbled upon this, and never knew, not even on his dying day, that he had discovered a new continent. A better name for the phenomenon we will be looking at in the pages to follow is “serendipity,” a word that came into the English language in 1754 by way of the writer Horace Walpole. The key point of the phenomenon of serendipity is illustrated in Walpole’s telling of an ancient Persian fairy tale, The Three Princes of Serendip (set in the land of Serendip, now known as Sri Lanka): “As their highnesses traveled, they were always making discoveries, by accidents and sagacity, of things they were not in quest of.”
Accidents and sagacity. Sagacity–defined as penetrating intelligence, keen perception, and sound judgment–is essential to serendipity. The men and women who seized on lucky accidents that happened to them were anything but mindless. In fact, their minds typically had special qualities that enabled them to break out of established paradigms, imagine new possibilities, and see that they had found a solution, often to some problem other than the one they were working on. Accidental discoveries would be nothing without keen, creative minds knowing what to do with them.

Source:
Meyers, Morton A. Happy Accidents: Serendipity in Modern Medical Breakthroughs. New York: Arcade Publishing, 2007.
(Note: italics in original.)

Modelers Can Often Obtain the Desired Result

(p. A13) After earning a master’s degree in environmental engineering in 1982, I spent most of the next 10 years building large-scale environmental computer models. My first job was as a consultant to the Environmental Protection Agency. I was hired to build a model to assess the impact of its Construction Grants Program, a nationwide effort in the 1970s and 1980s to upgrade sewer-treatment plants.
The computer model was huge–it analyzed every river, sewer treatment plant and drinking-water intake (the places in rivers where municipalities draw their water) in the country. I’ll spare you the details, but the model showed huge gains from the program as water quality improved dramatically. By the late 1980s, however, any gains from upgrading sewer treatments would be offset by the additional pollution load coming from people who moved from on-site septic tanks to public sewers, which dump the waste into rivers. Basically the model said we had hit the point of diminishing returns.
When I presented the results to the EPA official in charge, he said that I should go back and “sharpen my pencil.” I did. I reviewed assumptions, tweaked coefficients and recalibrated data. But when I reran everything the numbers didn’t change much. At our next meeting he told me to run the numbers again.
After three iterations I finally blurted out, “What number are you looking for?” He didn’t miss a beat: He told me that he needed to show $2 billion of benefits to get the program renewed. I finally turned enough knobs to get the answer he wanted, and everyone was happy.
. . .
There are no exact values for the coefficients in models such as these. There are only ranges of potential values. By moving a bunch of these parameters to one side or the other you can usually get very different results, often (surprise) in line with your initial beliefs.

For the full commentary, see:
ROBERT J. CAPRARA. “OPINION; Confessions of a Computer Modeler; Any model, including those predicting climate doom, can be tweaked to yield a desired result. I should know.” The Wall Street Journal (Weds., July 9, 2014): A13.
(Note: ellipsis added.)
(Note: the online version of the commentary has the date July 8, 2014.)

“Different Structural Models Can Fit Aggregate Macroeconomic Data About Equally Well”

(p. 1149) There is an apparent lack of encompassing-forecasting and economic models that can explain the facts uniformly well across business cycles. This is perhaps an inevitable outcome given the changing nature of business cycles. The fact that business cycles are not all alike naturally means that variables that predict activity have a performance that is episodic. Notably, we find that term spreads were good predictors of economic activity in the 1970s and 1980s, but that credit spreads have fared better more recently. This is of course a challenge for forecasters, as we do (p. 1150) not know the origins of future business cycle fluctuations. Much needs to be learned to determine which and how financial variables are to be monitored in real time especially in an evolving economy when historical data do not provide adequate guidance.
Explanations for the Great Recessions usually involve some form of nonlinearity. The sudden nature of the downturn following the collapse of Lehman is consistent with nonlinearity being part of the transmission mechanism. At the same time, we lack robust evidence of nonlinearity from aggregate low-frequency macroeconomic data. Essentially, there is an identification issue as different structural models can fit aggregate macroeconomic data about equally well.

For the full article, see:
Ng, Serena, and Jonathan H. Wright. “Facts and Challenges from the Great Recession for Forecasting and Macroeconomic Modeling.” Journal of Economic Literature 51, no. 4 (Dec. 2013): 1120-54.

Climate Models Allow “the Modeler to Obtain Almost Any Desired Result”

Integrated assessment models (IAMs) are the commonly-used models that attempt to integrate climate science models with economic effect models. In the passage quoted below, “SCC” stands for “social cost of carbon.”

(p. 870) I have argued that IAMs are of little or no value for evaluating alternative climate change policies and estimating the SCC. On the contrary, an IAM-based analysis suggests a level of knowledge and precision that is nonexistent, and allows the modeler to obtain almost any desired result because key inputs can be chosen arbitrarily.

As I have explained, the physical mechanisms that determine climate sensitivity involve crucial feedback loops, and the parameter values that determine the strength of those feedback loops are largely unknown. When it comes to the impact of climate change, we know even less. IAM damage functions are completely made up, with no theoretical or empirical foundation. They simply reflect common beliefs (which might be wrong) regarding the impact of 2º C or 3º C of warming, and can tell us nothing about what might happen if the temperature increases by 5º C or more. And yet those damage functions are taken seriously when IAMs are used to analyze climate policy. Finally, IAMs tell us nothing about the likelihood and nature of catastrophic outcomes, but it is just such outcomes that matter most for climate change policy. Probably the best we can do at this point is come up with plausible estimates for probabilities and possible impacts of catastrophic outcomes. Doing otherwise is to delude ourselves.

For the full article, see:
Pindyck, Robert S. “Climate Change Policy: What Do the Models Tell Us?” Journal of Economic Literature 51, no. 3 (Sept. 2013): 860-72.

Environmentalists Seek to Silence Those Who Dare to Disagree

(p. A13) Surely, some kind of ending is upon us. Last week climate protesters demanded the silencing of Charles Krauthammer for a Washington Post column that notices uncertainties in the global warming hypothesis. In coming weeks a libel trial gets under way brought by Penn State’s Michael Mann, author of the famed hockey stick, against National Review, the Competitive Enterprise Institute, writer Rand Simberg and roving commentator Mark Steyn for making wisecracks about his climate work. The New York Times runs a cartoon of a climate “denier” being stabbed with an icicle.
These are indications of a political movement turned to defending its self-image as its cause goes down the drain.

For the full commentary, see:
HOLMAN W. JENKINS, JR. “BUSINESS WORLD; Personal Score-Settling Is the New Climate Agenda; The cause of global carbon regulation may be lost, but enemies still can be punished.” The Wall Street Journal (Sat., March 1, 2014): A13.
(Note: the online version of the commentary has the date Feb. 28, 2014, and has the title “BUSINESS WORLD; Jenkins: Personal Score-Settling Is the New Climate Agenda; The cause of global carbon regulation may be lost, but enemies still can be punished.”)

The Krauthammer column that the environmentalists do not want you to read:
Krauthammer, Charles. “The Myth of ‘Settled Science’.” The Washington Post (Fri., Feb. 21, 2014): A19.

Many Important Medical Articles Cannot Be Replicated

The standard scientific method is more fallible, and less logically rigorous, than is generally admitted. One implication is to strengthen the case for allowing patients considerable freedom in choosing their own treatments.

(p. D1) It has been jarring to learn in recent years that a reproducible result may actually be the rarest of birds. Replication, the ability of another lab to reproduce a finding, is the gold standard of science, reassurance that you have discovered something true. But that is getting harder all the time. With the most accessible truths already discovered, what remains are often subtle effects, some so delicate that they can be conjured up only under ideal circumstances, using highly specialized techniques.
Fears that this is resulting in some questionable findings began to emerge in 2005, when Dr. John P. A. Ioannidis, a kind of meta-scientist who researches research, wrote a paper pointedly titled “Why Most Published Research Findings Are False.”
. . .
. . . he published another blockbuster, examining more than a decade’s worth of highly regarded papers — the effect of a daily aspirin on cardiac disease, for example, or the risks of hormone replacement therapy for older women. He found that a large proportion of the conclusions were undermined or contradicted by later studies.
His work was just the beginning. Concern about the problem has reached the point that the journal Nature has assembled an archive, filled with reports and analyses, called Challenges in Irreproducible Research.
Among them is a paper in which C. Glenn Begley, who is chief scientific officer at TetraLogic Pharmaceuticals, described an experience he had while at Amgen, another drug company. He and his colleagues could not replicate 47 of 53 landmark papers about cancer. Some of the results could not be reproduced even with the help of the original scientists working in their own labs.

For the full commentary, see:
GEORGE JOHNSON. “Raw Data; New Truths That Only One Can See.” The New York Times (Tues., JAN. 21, 2014): D1 & D6.
(Note: ellipses added.)
(Note: the online version of the commentary has the date JAN. 20, 2014.)

The first Ioannidis article mentioned above is:
Ioannidis, John P. A. “Why Most Published Research Findings Are False.” PLoS Medicine 2, no. 8 (August 2005): 696-701.

The second Ioannidis article mentioned above is:
Ioannidis, John P. A. “Contradicted and Initially Stronger Effects in Highly Cited Clinical Research.” JAMA 294, no. 2 (July 13, 2005): 218-28.

The Begley article mentioned above is:
Begley, C. Glenn, and Lee M. Ellis. “Drug Development: Raise Standards for Preclinical Cancer Research.” Nature 483, no. 7391 (March 29, 2012): 531-33.