Painting By Numbers is a book I’ve wanted to write for a long time. BUT, as I begin the roll-out of its promotional campaign, I first want to acknowledge the books which should serve, with Painting By Numbers, as a syllabus of sorts. These are books which inspired me to write my own, expand on the topics I raise, and address the issues of numerical uncertainty in specific industries and sectors. All of them should be on your radar if you are passionate about this topic. My hope is that, after readers get acquainted with the concepts at the elementary, anecdotal level I present them, they will move on to the deeper and broader treatments available from these experts. Links to their Amazon pages are provided for convenience.

The Signal and the Noise, Nate Silver, Penguin Group, New York, New York, 2012.

This book should be considered a modern bible on the limitations of forecasting and prediction, but also on how prediction can be improved. I’ve recommended it to many friends and several have taken me up on it. If I ever teach a class on this subject, I will warm up the students with Painting by Numbers and then use The Signal and the Noise as the main text. The breadth of Silver’s topics and discussion points are, well, breathtaking. He tackles numerical analysis in baseball, election polling, climate change, gambling, weather forecasting (different from climate change), epidemics, financial markets, chess and much more. If my work is known for one thing, I hope it will be that it achieved more with respect to brevity and simplification. The Signal and the Noise is an investment of time and brain cells but well worth the sacrifice of both.

Mindware: Tools for Smart Thinking, Richard E. Nisbett, Farrar, Strauss and Giroux, New York, New York, 2015.

I reference Mindware in the text, because of the author’s unabashed warnings regarding the limitations of multiple regression analysis (MRA), perhaps the most prevalent numerical analysis conducted in research (especially the social sciences). Nisbett also observes that “our approach to hypothesis testing is flawed in that we’re inclined to search only for evidence that would tend to confirm a theory while failing to search for evidence that would tend to disconfirm it.” Nisbett’s book is very readable. While his focus is on reasoning in general, experiments, and the philosophy of knowledge, his central question is very similar to mine: How well do we know what we know?

The Laws of Medicine: Field Notes from an Uncertain Science, Siddhartha Mukherjee, TED Books/Simon & Schuster, New York, 2015.

This slim volume, by the Pulitzer Prize winning author of The Emperor of All Maladies, reveals why ‘the laws of medicine are really laws of uncertainty, imprecision, and incompleteness.” They are, in fact, the ‘laws of imperfection.’ Probably the greatest piece of wisdom I got from this book is that even a perfect experiment is not necessarily generalizable. In other words, even if all of your statistics prove that your experiment ran perfectly, that doesn’t mean your results can be extrapolated to larger or different populations or even repeated for an identical sample.

In my view, the medical profession is particularly rife with arrogance and inability to face the limits of certainty. Mukherjee courteously holds the collective profession up in front of a mirror, pointing out the flaws in what he concedes is a relatively young area of science.

Willful Ignorance: The Mismeasure of Uncertainty, Herbert Weisberg, John Wiley & Sons Inc, Hoboken, NJ, 2014.

Weisberg tackles the subject of uncertainty from the perspective of the general process of scientific discovery and uses engaging stories about scientists and “thinkers” throughout history to illustrate his points. Like Nisbett, he also thinks statistical analysis has approached “a crisis” (paraphrasing the back flap copy). One of his central tenets is that “this technology for interpreting evidence and generating conclusions has come to replace expert judgment to a large extent. 

“Scientists no longer trust their own intuition and judgment enough to risk modest failure in the quest for great success.” And this corollary: “Instead of serving as a adjunct to scientific reasoning, statistical methods today area widely perceived as a corrective to the many cognitive biases that often lead us astray.” It isn’t the role of science to provide answers; it’s to refine the questions. It’s a readable text but falls squarely between an academic textbook and one attempting to popularize science concepts.

Automate This, Christopher Steiner, Portfolio/Penguin, New York, New York, 2012.

The book’s subtitle, “How Algorithms Came to Rule Our World,” suggests that Steiner’s focus is how human activities are being automated through bots governed by algorithms. “Algorithms,” he writes, 

“operate much like decision trees, wherein the resolution to a complex problem, requiring consideration of a large set of variables, can be broken down to a long string of binary choices.” 

Binary choices are ones computers can make. But this statement also shows that an algorithm is just another form of numerical analysis. Of all the books I recommend, Steiner’s scares me the most. Consider this: 

“Of the nearly one billion users in Facebook’s system, the company stores up to a thousand pages of data, including the type of computer you use, your political views, your love relationships, your religion, last location, credit cards…” (Remember, it was published in 2012). Think about that with respect to the privacy and national security debate.

At one time, the federal government forced AT&T to cooperate for national security in ways no one wants to remember. Now, imagine when the social media sites  of our modern world have your information wrong, when they have drawn the wrong conclusions from your digital footprints! Steiner also describes a company which has developed a bot that “sucks in box scores from sporting events, identifies the most relevant aspects, and writes a story built around those aspects of the game. Is this the end of sports journalism as we know it?

Models.Behaving.Badly.: Why Confusing Illusion with Reality Can Lead to Disaster, on Wall Street and in Life, Emanuel Derman, Free Press/Simon & Schuster, New York, New York, 2011.

Derman is a physicist turned Wall Street “quant” and was one of a plethora of authors weighing in on the financial crisis and great recession of 2007/2008. Derman brings into the discussion the idea of models and metaphors: 

“Models stand on someone else’s feet. They are metaphors that compare the object of their attention to something else that it resembles. Resemblance is always partial, and so models necessarily simplify things and reduce the dimensions of the world.” 

But this later quote is priceless in its utility for understanding: “Once you understand that a model isn’t the thing but rather an exaggeration of one aspect of the thing, you will be less surprised at its limitations.” 

This is similar to what Nisbett is trying to convey about MRA, which limits the researcher to one aspect of the thing, and thus loses the context of all the other influences on that one thing (e.g., a measured, independent variable). Although Derman focuses (mostly) on financial models, he explains very well the limitations of models for economics, global climate, and other broad situations compared to those used in physics.

An Engine, Not a Camera, Donald Mackenzie, The MIT Press, Cambridge, Mass., 2006.

If more people read and understood Mackenzie’s account of his deep research into valuation models for financial derivatives and the inner workings of financial markets, the world of investment would probably be very different. Mackenzie shines a bright light on the purpose of most models—to create a version of reality and then capitalize on that reality. In this case, Mackenzie argues persuasively that the Black-Scholes model for options pricing, which did indeed by most accounts change the field of finance, was developed to drive a market (engine) rather than reflect a market (camera). His analysis lends evidence to a broader contention, that the “invisible hand” of the market is anything but, that markets are deliberately constructed for the entities which will participate in that market. 

To my way of thinking, An Engine, Not a Camera is about uncertainty at its highest level, as it casts doubt on the entire notion of a “free market,” “Markets are not forces of nature, they are human creations,” he writes. To which I would add (as I suggest in the chapter on business models), models today are primarily used to create new markets and new realities, not expand our understanding of the human condition.

Useless Arithmetic, Orrin Pilkey and Linda Pilkey-Jarvis, Columbia University Press, New York, 2007. 

This is an example of a book that focuses on a specific field of applications identified in the subtitle, “Why Environmental Scientists Can’t Predict the Future.” This quote sums up what you are going to learn from the Pilkeys: “The reliance on mathematical models has done tangible damage to our society in many ways. Bureaucrats who don’t understand the limitations of modeled predictions often use them.” Even if you consider yourself an environmentalist, Useless Arithmetic is very useful for understanding how math models are used and abused.

Merchants of Doubt, Naomi Oreskes and Erik M Conway, Bloomsbury Press, New York, 2010.

As I note, uncertainty is something used to create doubt. In particular, the authors take aim at scientists and researchers pressed into service (and well paid) to blow up what is left of scientific uncertainty on highly charged political and cultural issues to impede progress on the issues of the day. They go as far to accuse such experts as turning doubt into a “product.” The issues they tackle include smoking and cancer, the ozone hole, global warming, acid rain, and other ecological issues. Unlike many of the other books listed, the authors in particular assess the public and political debates around these issues, not the scientific method. Health effects of smoking were turned into a great debate, funded by “big tobacco,” after the scientific evidence was rapidly drawing the conclusion, assert the authors. Among the important tenets of wisdom imparted is that balance in reporting is not giving equal weight to both sides, but to give accurate weight to both sides. Some “sides” represent deliberate disinformation spread by well-organized and well-funded vested interests, or ideological denial of the facts.

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, Cathy O’Neil, Crown, New York, 2016.

You’ve probably inferred from the title that this book aims to be provocative and incendiary first. It certainly accomplishes that. O’Neil tackles data and modeling through the prism of social justice and power structures. But her metaphor is precious because it reveals how models evolve into WMD. One of her best examples is the US News & World Report college ranking system. Over several decades, it became the standard for college rank and therefore the object of intense manipulation so that colleges could improve on the rank. She observes (correctly in my mind) that all of the emphasis on the rating and its following among parents doesn’t do a damn thing for the quality of education. When a school’s objective becomes figuring out how to “game the ranking,” it’s no different than my attempt to game my rankings of colleges to favor the school I had already selected, as I illustrated in the opening chapter. O”Neil applies her analysis to getting insurance, landing a job, obtaining credit, on-line advertising and other aspects of unfairness in modern life. 

An Introduction to Mathematical Modeling, Edward A Bender, Dover Publications, Mineola, NY,  1978. 

Here, the term “introduction” refers to very mathematics-intensive theory and applications, optimization routines, and probabilities. The first chapter, “What is Modeling?” does a good job of laying the groundwork for those who wish to skip the math. 

Measurements and Their Uncertainties, Ifan G. Hughes and Thomas P.A. Hase, Oxford University Press, Oxford, England, 2010.

This book, focused on error in physical sciences, also gets complicated in a hurry, but again, the first chapter is well structured and offers good foundational material. It starts with the overriding point that “there will always be error associated with that value due to experimental uncertainties.” It goes on to classify uncertainties as random errors, systematic errors, and mistakes. While most discussions of uncertainty and error (mine included) focus on extrapolation, or extending a curve fit to data past the original measured data (or making inferences into the future using data from the past), this book reminds us that interpolation can be just as insidious. Interpolation refers to assuming the shape of the curve or line or graph between the measured data points. While this is a textbook, it is graphically rich rather than mathematically intensive (authors assume that computers will be doing most of the math).

Interpreting Data, Peter M Nardi, Pearson Education Inc, Boston, 2006.

This book keeps to the straight and narrow of how data analysis is applied in experiments. It notes in the introduction that “it is written in non-technical everyday language…” With passages like “Pearson r correlations are for interval or ratio levels of measurement…Many researchers, however, use these correlations for dichotomies and for ordinal measures, especially if there are equal-appearing intervals,” I’m not convinced of the everyday language. Nevertheless, I found it useful as a refresher on how experiments are designed, data taken, results analyzed, and conclusions drawn.

A Demon of Our Own Design, Richard Bookstaber, John Wiley & Sons Inc, Hoboken, NJ  2007; and Lecturing Birds on Flying, Pablo Triana, John Wiley & Sons, Hoboken, NJ, 2009.

Both of these books are focused on financial engineering and were blessed in being well-timed with the collapse of financial markets and the world economy. They cover similar territory and both insinuate that financial markets are imperiled by the way modeling is applied. The subtitle for Demon is “Markets, Hedge Funds, and the Perils of Financial Innovation,” and the subtitle for Lecturing Birds is “Can Mathematical Theories Destroy the Financial Markets?” However, everything I read tells me that things have only gotten worse, so unless you are seeking recent historical perspective, I’d supplement these two books with some more recent titles.

20% Chance of Rain, Richard B Jones, Amity Works, Connecticut, 1999 

This book wants to be “Your Personal Guide to Risk,” as its subtitle urges. Written by an industry colleague in my consulting work, who spent decades in the machinery insurance business, it’s not really about modeling or uncertainties, but about risk and how we measure risk through probabilistic assessment. Jones stresses the uncertainty boundaries around any risk assessment and that “perception creates risk reality.” He also offers this bit of timeless wisdom: “Statistics do not, and cannot, prove anything. The field of statistics is incapable of this. Statistics can provide information to help us make decisions, but the decisions are still ours to make.” Today, statistics and numerical analysis in general are being used so decisions can be made for us (automation, digital algorithms, market construction, even on-line dating and hookup). We’d better all have a thorough understanding of their limitations

What does one avid reader write about the latest novel from one of, if not the, leading American author of contemporary literary fiction?

I feel like I’ve grown up with this guy through his novels. The Names was an enabler of my addiction to literary fiction when I read it in the early 1980s. Underworld is one of my all time favorite “big books.” I’ve read everything the guy’s published since Underworld, including his short stories in the The New Yorker, and gone back to read his earlier works from the 1970s, including Great Jones Street, End Zone, and White Noise.

Zero K is about a son coming to grips with his wealthy father’s impending cryogenic preservation in a personal pod at one of those remote sites where they’d have to kill you if they told you its location. About all we know about it is it is “land traveled by nomads for thousands of years.” His father wants to “own the end of the world.”

No one reads DeLillo for plot or character. You read DeLillo because he is a master at extrapolating the contemporary zeitgeist to its future state. He anticipates our future without sensationalizing it. He makes you look at it, not immerse you in it.

And you read DeLillo, or at least I do, because his writing is so exact, so precise, without ever resorting to lofty words for the sake of making the reader look them up. When he uses such a word in Zero K, he has his narrator step back and examine it, define it for you.

Zero K is a meditation on life extension, on the notion of not wanting to die, on having the means (money) to choose not to die. It probes the idea of money and science as the new religion. After all, you better have faith in the scientists who are going to restore you if you’re going to be preserved for a few decades or centuries.

Most of all, it challenges the impact of those the preserved leave behind. Isn’t that what death is all about? The dead are dead. The living have to make sense of it all. In this case, they aren’t dead, though, but a state in-between.

DeLillo uses this idea of being preserved, for later reincarnation on this planet, to explore elements of modern life, death, and everything in-between. Because being preserved for your rebirth in an enhanced physical state for whatever future society holds in store is the ultimate betwixt and between life and death, I suppose.

My observation on DeLillo is that his novels have a detachment quality which has grown over time, not linearly perhaps, but a trendline (Great Jones Street is an example of an early work with this detachment quality). This detachment hasn’t worked so well in all of his later novels. I thought Cosmopolis was kind of a dud but Point Omega was fantastic.

Detachment is extremely effective in Zero K. Maybe it works because we (the characters, us readers) are not attached to either life or death as this story unfolds. We’re observers with the narrator. It’s not the surround sound kind of effect most novels have.

The narrator is uber self conscious, almost annoyingly so at times. He ponders everything. I mean, everything. And DeLillo pauses time as part of the reading process for the narrator to ponder. Most of the time, the narrator seems detached from himself even.

The detachment of the narrator from his father, his girlfriend, his mother in law, and the death of his real mother, his girlfriend’s abandoned son, adopted from an old Soviet state, all this detachment also allows DeLillo to get away with passages which in other novels might be considered explanation getting the way of the story. We’re observing Jeffrey Lockhart, the son, ponder his father’s impending non-death.

There’s much pondering going on, but I can assure you it’s not a ponderous novel.

This is the kind of detachment which permeates this novel, from the last few pages when the narrator son is back in New York: “On public elevators, I direct a blind gaze precisely nowhere, knowing that I’m in a sealed box alone with others and that none of us is willing to offer a face open to inspection.” A nice metaphor for the preserved in pods half a planet away, but also cold hard truth for elevator riders everywhere.

Characters, narrator, elevator riders, readers – all are on the outside looking into something and away from others.

This is how pod residents are prepared: “You are completely outside the narrative of what we refer to as history. There are no horizons here. We are pledged to an inwardness, a deep probing focus on who and where we are.”

Think about this the next time you watch someone with face in iphone smack into something or someone. DeLillo is extrapolating from here to Zero K. I think. After all, the narrator Jeffrey tells us in the middle of the book, “what else was there for [my father] to acquire” but “the billionaire’s myth of immortality.” The “pod would be his final shrine of entitlement.”

Detachment fully serves what I perceive to be DeLillo’s purpose here, so the reader senses this in between space along with the characters, especially the narrator who has the obligation but no desire to witness his father’s preservation and privilege.

What is it like not to be attached to life or death? Zero K nails that one.

Assia Djebar, a highly acclaimed Algerian author, died earlier this year. I just finished her novel So Vast the Prison (1995). It does not read like a novel (certainly not a conventional or even non-conventional one). Judging from the brief bio info I’ve read, it is as much, or more, memoir and autobiography laced with historical fiction. The language is beautiful. It is perfectly titled. The way the universal plight of women is addressed takes no prisoners. Like some paintings I love, it is amorphous and fluid, and even confusing at times. But I stayed with it and realized my own limited perspective was getting in the way.

The novel covers a hundred years, or at least several generations, across France and Algeria. At times I was completely confused. Was this fiction? Was it a memoir? Some of the passages seemed to come straight from a journal. Or was it simply an epic poem? The narration starts with a long sequence about a lover, really an infatuation, never to return to this seemingly critical individual. It hops from generation to generation. Suddenly, it is all about making a film about a woman’s life, or the life for women, in Algeria. There are brothers and mothers (including in laws) and cousins and and aunts and uncles and sisters and friends appearing on stage, usually briefly. Men, as you may guess, don’t fare well. Husbands are brutal. Leaders are oppressive and dictatorial and battle-wagers.

But I had to go back to the title to “get it.” So Vast the Prison. That is the theme. That is what ties all of this together…tightly, unambiguously, unabashedly. Excepting perhaps a lucky few, the women of the world live in various forms of prison. Their bodies and child-bearing, prison. Their men keep them in prison. These same men maintain the cultural norms which perpetuate these life sentences. Other women, clinging to the traditional ways, fortify these prisons. Across generations, continents, and time, so vast is the prison.

Once again, this theme I’ve written about before (http://jasonmakansi.com/the-global-american-footprint-in-fiction/) emerges. Novels which are “big” outside America (and often highly acclaimed) address sobering issues, issues of life and death, survival, war, oppression, the human condition; meanwhile, the big American novels seem frivolous by comparison, dealing with the minutiae of life in the suburbs, the vagaries of popular culture, privilege, entitlement, or, as I often see it referred to these days, “first world problems.” More to the point, novels like So Vast the Prison deal with the impact of the first world’s footprint on the rest of the people living on this planet, in this case the colonization of Algeria by the French and the ensuring war for independence, as well as internal conflicts. But that’s just a backdrop for the Godzilla footprints of men all over women all across the ages.

So Vast the Prison is beautiful, lyrical writing. It may not have fit my notion of what a novel should be, but through it, I “felt” what a great deal of feminist literature has been trying to tell me for decades.