Thursday 23 October 2008

The Fallibility of Statistical Models

Who are the culprits that caused the meltdown of the global financial system which in all probability will lead to a severe global recession? A lot has been said about the sheer greediness of mortgage originators, investment banks and their cosy relationships with rating agencies. Roman Frydman, Michael Goldberg – authors of Imperfect Knowledge Economics – and Edmund Phelps – 2006 winner of the Nobel Prize in Economics – argue in an article published in the Financial Times (October 19) that another major cause is often overlooked, namely "failure to acknowledge that market participants and regulators alike have only imperfect knowledge about the forces and mechanisms driving asset values and the broader economy."
"We rely on markets because we know of no other way to take account of the myriad bundles of knowledge and intuition that individuals use in determining economic values, such as those for financial assets. Asset prices often undergo long swings away from historical benchmark levels, followed by “corrections”, because this is how markets “discover” a sensible range of values."

"Such price reversals are a source of risk that is not recognised by standard risk-management methods. Even more importantly, these methods ignore the very nature of a capitalist economy: it generates new ways of doing things. Hence, economic relationships and patterns that applied in the past are eventually replaced by new ones."

"Many of our regulations are designed to achieve transparency of information. Public companies must make available their financial statements on the theory that investors can then decide how much of an asset to hold. What the crisis has demonstrated is that more than information is required for prudent investment decisions. Financial markets need regulation to bring to light the imperfect knowledge of those who are in the business of providing assessments of financial assets."

"Finance and economics professors devoted their talents to developing abstruse, yet simplistic, models that left out the imperfection of knowledge. Universities have produced two generations of financial engineers who sold the idea that academic models could safely neglect market discovery of risk and prices."

"The ratings agencies used statistical models that projected historical default patterns to continue. These patterns showed very low loss rates, thanks to ever rising prices since 1997. With such low loss rates, triple A ratings appeared to be justified. Brave new models tempted the industry to abandon proven prudential procedures, which combined their own judgment and more formal criteria. To be sure, agencies apply stresses to their current procedures. But these stresses are hidden in ratings reports and, as recent events have painfully demonstrated, are woefully inadequate."
"Had the agencies been required to make explicit how their ratings would have changed under the alternative assumption that house prices would fall back to benchmark levels, the markets would have feared greater loss rates, lowering the demand for mortgage-backed securities. This would have reduced the volume of mortgages originated and thus ultimately the amount of bad paper that banks ended up holding. Requiring the agencies to rate securities under one or more pessimistic scenarios as well as the optimistic one would make it harder for the agencies to deliver rosy ratings in return for business from the investment banks."

The authors propose that agencies should be required to report at least two ratings: one assuming that historical patterns will continue and at least one other assuming reversals in the trends of major variables. No single individual or institution can render a definitive judgment on the riskiness of securities. Only markets have aggregate knowledge that is not given to anyone in its totality.

No comments: