Thursday, June 21, 2007

the black swan

it was scottish titan david hume who gave form to the problem of induction for the duration of this civilization. his masterwork, 'enquiry concerning human understanding', gave the line of reasoning which has concerned all philosophy since and indirectly gave rise to the embrace of irrationality which has been since called romanticism.

Matters of fact, which are the second objects of human reason, are not ascertained in the same manner (as relations of ideas); nor is our evidence of their truth, however great, of a like nature with the foregoing. The contrary of every matter of fact is still possible; because it can never imply a contradiction, and is conceived by the mind with the same facility and distinctness, as if ever so conformable to reality. That the sun will not rise to-morrow is no less intelligible a proposition, and implies no more contradiction, than the affirmation, that it will rise. We should in vain, therefore, attempt to demonstrate its falsehood. Were it demonstratively false, it would imply a contradiction, and could never be distinctly conceived by the mind.


hume (and later bertrand russell) thereafter illustrates the weakness of human reason, hopelessly simple and linear in the face of an extremely subtle, complex and nonlinear world in which the seemingly obvious -- that confirmed by a long chain of observations -- does not in the slightest give certainty as to the very next occurence.

as science is becoming aware, people are biologically developed to expect what has been recently experienced and incrementally developed, and are generally incapable of considering correctly the odds and effects of rare and momentous events; we are instead given by nature to reduce the world to a platonic form that our limited brains can more easily comprehend. these biological/emotional biases taint our ability to process information objectively and account for the strange but shattering events that (though we are designed to be oblivious to them) dominate our existences. to make simple sense of complexity, we fabricate and narrate simple causes and effects that impose a perhaps-logically-consistent but backward-looking and incorrect linear progression on events, in order to make them seem as though they occurred with the ordered regularity of a perfect watchwork.

in truth, however, looking forward no event is certain -- all are merely probable. with our ability to imagine the possibilities of the abdication of the probable inadequately constructed, we are left (in the absence of reflection) to persistently underestimate uncommon and undesirable likelihoods and their consequences. karl popper first termed these rare and momentous events black swans -- after the late-17th-century discovery of the australian black swan, prior to which a fascinated european society had never conceived of anything but white swans -- a concept being popularized to some extent by nassim taleb.

when hume's world-shaking treatise (itself a black swan) was first published it was savaged by london critics, who had never been exposed to its like and utterly missed its essential point -- it is, after all, deeply counterintuitive. as if to annote its ubiquity, the biological reaction that compels animals to reduction and conformity didn't exempt hume in spite of his cognitive awareness of his reviewers' errors -- he took sick for weeks.

what has any of this to do with the cubs?, one may reasonably ask. in truth, it seems a very great deal.

many inveterate cub fans, for instance, are dedicated to the idea that the club is a perpetual loser -- that it cannot win a world series, and is not winning for some mechanical reasons of questionable contrivance that will always prevent it from winning. these include but are not limited to the incompetence of ownership (regardless of the owner), the park they play in, the time of day they play, the players they put under contract (regardless of the players), how they treat and/or teach the players (regardless of how they are treated or taught), how they are managed on the field (regardless of the manager), how they are managed off the field, and the fans they play in front of.

please note -- any and all of these conditions do certainly have real effects upon the ability of the club to win a world series. but the tendency among human observers is to radically simplify the complexity of the network of conditions and presume reductive linearity -- ie, "the cubs play too many day games, and so cannot win a world series" -- to such an extent as to invalidate the value of the observation. in truth, one can have no idea how the cubs would behave if they, holding all else constant, played every game in the evening. what little evidence there is (gathered since 1988) indicates that very little has changed with an increasing proportion of night games. that has not prevented a very detailed and loosely-founded narrative from springing up around the concept, which colorfully includes drunken players (observed and unobserved) out until all hours of the morning.

few (and certainly not including this writer) seem well prepared to accept that 98 years of failure may in fact be essentially random, or at least contain a very significant component of chance. fewer still are prepared to accept that many of the detailed plans they are convinced must be executed in order to improve the chances of the club winning a world series are of highly questionable effect. people are physically constructed so as to minimize the role of chance in determining outcomes -- and so they widely do, preferring instead to create an illusion of rational actors performing a sequence of deeds that can and will mechanically surmount all obstacles (ie, "if they just do a, b and c, they'll certainly do well" -- how often it is said!).

of course, one can allow that the acts of people alter the probabilities of certain outcomes over others -- it is not the fact of man's effect on processes and results that is to be questioned, only the proper evaluation looking forward of the value and direction of the possible effect on complex systems, particularly in contraposition to the intended effect. when this page advocated the signing of jason schmidt this offseason, for example, it noted:

schmidt is something of a difficult prospect to assess. he has long been one of the most abused starters in baseball -- ranking top ten in pap every year since 2002 -- and it is difficult not to believe that the wear and tear is compounding the normal ageing of his talent. schmidt's deteriorated velocity has episodically been the talk of san francisco, and of his last four halves of baseball three have been accompanied by whip in excess of 1.42. even if he remains a good pitcher -- one who compiled 45 pitching runs above replacement even in a disappointing 2005 campaign -- this is probably no longer the dominant figure that drove the 2003 giants.


yet in spite of that gloom, it concluded:

there can be little question that even a declining schmidt would be, even if falling short of high expectations, an improvement on what the cubs have heading to the mound today. and there's always the chance that he could experience a continuation of his former glory.


a great many others put quite a lot less circumspection into their view that this. schmidt has since met with something like disaster this season, and had the cubs signed him they may very well be even worse off than today.

was it probable that schmidt would have made the cubs a better team? it certainly appeared so at the time -- very few if any watchers of the cubs were opposed to signing him; those who dismissed the probability did so on specious and narrated grounds other than potential performance, such as, "he won't leave the west coast", or, "he won't sign for the amount of money the cubs would give him". schmidt had compiled five consecutive years of at least 29 starts ending in 2006, during which time he was one of the greatest pitchers in baseball. in spite of evidence of pitcher abuse that, in the revisionist narrative of retrospect, seems to loom very large, schmidt was one of the most sought-after free agents in the game.

and he was so in part because human beings have great difficulty properly appreciating the nonlinearity of injury probability -- we cannot account for the black swan, that which has not been observed, even when erudition and cognition tell us that the probability of a rare but disastrous event is higher than normal thanks to measurable conditions. it is in fact much more likely that we construct a flimsy narrative to dismiss the warning flags of cognition in order to "go with our gut" and "act on common sense" -- which generally consists of projecting with too great a degree of certainty an inherently unstable and tenuous status quo.

the most obvious and pressing situation to which this applies to the cubs today is carlos zambrano -- who, not dissimilarly to the 2006 version of jason schmidt, is one of the very best pitchers in the game over the previous five years but who is also, cognition warns us, at elevated risk as gauged by measures of pitcher abuse.

this page has at times unwittingly mischaracterized the consequences of pitcher abuse as something cumulative, piling up linearly in accordance with pitcher abuse points and so forth. it has noted possible onset injuries and periods of reduced performance as potential warnings. as valid as such observations may or may not have been, this is, it should be seen, a reduction of the black swan -- the more difficult truth is that a pitcher like schmidt or zambrano may remained uninjured to the very moment of disaster in spite of years of concerning readings. there may be no damage in his arm whatsoever, in spite of mechanical changes that, however real and actual, amount to a confirmation of previously held opinion selected with bias.

a black swan need offer no such solace as accumulated damage. to borrow an analogy from taleb, much as with a turkey being fattened for thanksgiving day, everything may (from the perspective of the turkey) appear utterly benevolent and even better than could be expected -- all this extra food! -- until the holiday arrives. in this case, it is worse -- we don't know on which day the holiday falls. the appointment will be random. indeed, it may never come.

that doesn't, of course, invalidate the results of cognition. zambrano is at elevated risk, and will remain so for at least as long as he continues to top charts of pitcher stress. we can infer that the black swan of career discontinuity is probably nearer him than to many others, perhaps most any other. and if it strikes, all the narrative fallacies that have arisen around zambrano to excuse ourselves from the results of cognition -- that he is a superman, that he is more strongly constructed than other pitchers, that he is lucky, that he is a special case, that pitcher abuse as a concept isn't real -- will be exposed for what they are. and if it does not strike, zambrano will become an anecdotal example incorrectly used to justify continued ignorance and defiance of difficult probabilities.

this page can have no idea of when or even if zambrano's day of reckoning will arrive within his career. all it can know is that, speaking probabilistically, he is in greater danger than he may at first appear to be to human beings working under presumptions of reductive linear narratives -- and that that greater probability should inform the club as the trading deadline approaches and decisions have to be made about zambrano's future with the team.

No comments: