Severn Darden, one of the founding members of Chicago’s Second City comedy troupe, had a routine in which he played a Professor of Metaphysics.

“Now, why, you will ask me, have I chosen to speak on the Universe rather than some other topic?’” he would begin in a thick accent. “Well, it's very simple. There isn't anything else!”

In the Information Economy, the same could be said for the Value of Information (VoI).

It started over half a century ago with a seminal 1966 article entitled Information Value Theory by Professor Ronald A. Howard of Stanford. When I sat in on Howard’s class as an Adjunct Faculty member in the mid-1990s, I was amazed that with all my years of technical education I had never been exposed to this fundamental idea. And I continue to be surprised at how few people are aware of this concept today. I believe that the Internet of Things (IoT) is about to change all that.

My epiphany came during a recent presentation by W. Allen Marr, Founder and CEO of Geocomp, a Boston area geotechnical engineering firm that determines how the earth will respond when you build a bridge or skyscraper on it or drill a tunnel through it. Marr started by pulling out his smart phone, which displayed a live map of Chesapeake Bay, with colored dots representing the recent movements of sensors embedded in a tunnel currently under construction. Then he went on to discuss the use of sensors in an earthen dam, discussed below, which for me sealed the deal on the connection between the Value of Information and the Internet of Things.

First, here is my own informal definition of VoI. In any situation in which you can imagine saying I wish I had **found out** such and such before I had to **decide** between this or that, ask what you would have been willing to pay to go from **decide** then **find out **to **find out **then** decide**.

For example, you can **decide** to buy a stock or not today, and then **find out** tomorrow if it goes up or down. How could you **find out** what a stock was worth in the future and then **decide** whether to buy it? Easy. Stock options let you do just that, so option pricing is a special case of VoI.

Here’s another example. Suppose you like to drive your Ferrari fast over a stretch of road where you know there is a 20% chance of a radar trap with an associated $500 speeding ticket. You must **decide** how fast to drive, then **find out** if you will get a ticket. Your expected loss is 20% x $500 = $100. What is the value of information provided by a radar detector with a 90% accuracy? You get to **find out** if the detector goes off, then **decide **to slow down. Now there is only a 2% (10% x 20%) chance of getting a ticket, so your expected loss is 2% x $500 = $10. The VoI is the difference or $90.

Note that if you are driving a clapped out 60’s vintage VW Bus on the same road, you have nothing to **decide** about speed. You need to keep the pedal to the metal just to keep up with traffic. Without a decision that could be changed by the information, VoI is zero.

But let’s get back to Marr’s dam story.

Suppose the acceptable rate of failure for an earthen dam is once in 10,000 years. And the dam in question looks pretty good until someone points out that it is upstream of a nuclear facility. Uh oh. Now the rules say you need to reinforce it to a rate of one failure in 1 million years. So get out your checkbook, because to patch it up to that strength will cost $800 million.

But here is an IoT idea. Consider a sensor network embedded in the dam that has a 99% chance of detecting a failure before it happens. And suppose that the $800 million patch job could be done quickly and would still have a 99% chance of saving the dam after the sensor network goes off. We have gone from **decide** to spend $800 million, then **find out** if we really had to, to **find out **if the dam will fail then **decide **to spend the $800 million.

So, what is the value of the information provided by the sensor network? Of course, one must really look at the net present value over an extended period, the reliability of the sensor network, etc, etc, but let’s start with the first year. We have an operational sensor network which reduces the likelihood of dam failure to the goal of about 1 in 1,000,000, but since we did not reinforce it, there is still about 1 chance in 10,000 that the sensors will detect that the dam is unhealthy, in which case we will need to spend the $800,000,000. So, our expected cost is roughly $80,000 for a savings (VoI) in the first year of $799,920,000. So does the sensor network cost less than that? Are you kidding? It’s $500,000. And according to Marr, doing the economics for 30 years, including monitoring and maintenance of the network, adds another $2 million. Marr calls this application of real-time monitoring to detect and respond to emerging risks “Active Risk Management.” The actual details are more complex and the statics assume an “average” dam, but you get the idea. Data from sensors can provide great value.

Marr’s presentation made me wonder about the total value of the information coming from each of the other 20 billion things on the internet. And this led to the theme of this year’s Annual Conference: **Data, Decisions, and the Value of Information**.** **

I am happy to announce that Allen himself will be a highlighted speaker, along with other pioneers in information economics and the internet of things.

For example, Doug Hubbard, author of the popular “How to Measure Anything” series, has made a career out of VoI. He has discovered that ignorance of this subject leads to an ironic outcome, which he calls Measurement Inversion. When he ranks the effort that firms put into measuring things next to the information value of those measurements, he finds that they go in “exactly” the wrong direction. That is, the most effort is spent collecting the least valuable information.

Another long-time supporter of ProbabilityManagement.org who will also be presenting is Steve Roemerman, CEO of Lone Star Analysis, a Dallas-based firm working in logistics, aerospace, and oil & gas. They have been a pioneer in IoT, with lots of practical experience. According to Steve, “In more than one of our IoT engagements, we found the customers already had all (as in 100%) of the information they needed.” The real problem was to integrate the information for making better predictions and decisions. Steve also warns that “brute force sensor deployment for its own sake is one reason we see IoT deployments fail.” This only reinforces the need to understand the concept of VOI both with the information you have already, and the information you are planning to acquire.

© Copyright 2019 Sam L Savage

Limbic Analytics from ProbabilityManagement.org RSS]]>No, no, no! We don’t mean a killer app using the Excel Data Table, like SIPmath simulation. We mean an app that will kill your data table, for example, your SIPmath simulation.

It all started last week when the Dice Calculator (Excel file), which is supposed to instantaneously roll a pair of dice 10,000 times, suddenly started taking ten seconds on one of our machines. Flash back to the late 1980’s when Bill Sharpe, Nobel Laureate in Economics, discovered that the data table in Lotus 1-2-3 could perform simple Monte Carlo simulations. We tried this in Excel in the early 1990s, and although it showed great promise, it often caused the spreadsheet program to crash unceremoniously. When we discovered in 2012 that the Excel Data Table could instantaneously perform tens of thousands of calculations of the Rand() formula, we were ecstatic. Furthermore, using the Index formula, Excel could read SIPs as well. With interactive Monte Carlo simulation available on every desktop, we were able to get corporate sponsorship for ProbabilityManagement.org, and we incorporated as a 501(c)(3) nonprofit in 2013.

But where were we? Oh yes. There are only two things that really keep us at up at night.

The first is that Jensen’s Inequality (the strong form of the Flaw of Averages) will be declared to be Fake Math. We have been working together for decades, offering a money back guarantee to our consulting clients on the validity of this well-established mathematical result. If the internet deems it false, our careers are over.

The second nightmare is that the Excel Data Table, which has done for simulation what penicillin did for bacterial disease, ceases to work. This would spell the end of SIPmath for Excel.

So, when something that was supposed to be instantaneous took ten seconds, we freaked out. We re-installed Excel twice on the offending machine, but nothing worked. Then we realized that the installation process was so seamless that it left all the Excel add-ins in place. By process of elimination we found that one of our own experimental add-ins was slowing down any instance of the Data Table by orders of magnitude.

Here’s the scoop. Some formulas in Excel are known as Volatile, because they recalculate with each keystroke. Most formulas do not have this feature. For example, if cell A1 contains =B1+C1, then A1 will not re-calculate unless either B1 or C1 change. RAND(), on the other hand is Volatile. Since it doesn’t depend on anything it needs to change with every keystroke.

**Warning: **Do not use a SIPmath model in Excel while another workbook is open that contains RAND() or it will run very slowly.

We have known that for a long time. But what does that have to do with an add-in? Well, our add-in had Excel worksheets built into it for use as templates. They didn’t use RAND(), but they did use other Volatile functions, such as OFFSET. Worse they used OFFSETs that drove hundreds of other cells. It was like having hundreds of Volatile cells in Excel all the time, whenever the add-in was loaded.

**Updated Warning: **Do not use Volatile functions in the vicinity of SIPmath models. That is, close all worksheets with Volatile functions before using a SIPmath model. You can use RAND() in a SIPmath model, but not the model next door. And there are some other exceptions that seem to work. But please be careful or your models will grind to a halt just as you are making that great analytical pitch.

To better understand this phenomenon, we created a killer app in Excel that destroys the performance of the Data Table in any worksheet. At first, we planned to publish it as a download with this blog. But on second thought that would be like publishing plans for a weapon of mass destruction, so we are keeping it in hermetically sealed container in the lab.

To learn more about Volatile functions, see http://www.decisionmodels.com/calcsecretsi.htm.

An F4 Phantom with the Air Force Academy Chapel in the background

I just returned from the 87th Symposium of the Military Operations Research Society at the Air Force Academy in Colorado Springs. ProbabilityManagement.org had a proud showing. Shaun Doheney, PM Chair of Resources and Readiness Applications, Connor McLemore, PM Chair of National Security Applications, and I gave a total of four presentations. Despite being on the last day of the conference, Shaun and Connor delivered two sessions on **Readiness Modeling: Changing the Question from “Ready or Not?” to “Ready for What?”**, which drew standing room attendance. See Shaun and Connor’s recent blog and access their slides and models on PM’s Readiness page.

Connor McLemore

The field of Operations Research (OR) grew out of the application of mathematical analysis to the tremendous resource allocation problems of World War II. After the war, OR took on additional names, such as Management Science, Analytics, and others, but it all boils down to analyzing your options and figuring out mathematically how to do the most with the least. The primary professional societies are INFORMS (the Institute For Operations Research and the Management Sciences) and MORS (the Military Operations Research Society).

My father, L. J. Savage, was in the thick of war time OR at Columbia’s Statistical Research Group. In the early 1940’s he worked with future Nobel Laureates Milton Friedman and George Stigler. They tackled such problems as determining whether a fighter should carry six 50- or eight 30-caliber machine guns, and the best strategy for hunting enemy submarines. My own PhD research was on the Travelling Salesman Problem, a classic OR problem.

But back to the symposium. The meeting made me realize just how heavily Military Operations Research has been influenced by the incomparable OR Department of the Naval Postgraduate School (NPS) in Monterey, California. The school provides active duty military and other government employees as well a few international students with rigorous graduate education, mostly master’s degrees and some PhDs. Areas include Engineering, International Studies, Computer Science, Business, and OR. I first visited the NPS OR department in the early 1990’s when my dear friend and former department chair, the late Rick Rosenthal, invited me down from the Stanford OR Department to give a talk. I found it unlike the typical academic programs in OR, which are often quite theoretical and PhD-dominated. First, NPS students start out with military discipline so they all pay attention. Second, they are learning through the solution of real military problems, for which doing the most with the least may have life or death repercussions. Here is a place where there is every reason to stay and work because the results matter. And with its spectacular setting on the shore of Monterey Bay, there is no reason to go anywhere else. It was love at first visit.

NPS OR has played an outsized role at ProbabilityManagement.org. Shaun and Connor are both grads, and Connor also taught there, introducing SIPmath. Phil Fahringer, a Lockheed Martin Fellow and the nonprofit’s primary contact at that organization, has an OR degree from NPS as well. In Colorado Springs I reconnected with many others from NPS whom I have known over the years and realized what a powerful intellectual network they represent. I also had the pleasure of an extended conversation with Doug Samuelson, a prominent OR Analyst whom I had only known peripherally. I proposed that the OR department at NPS was the Harvard Business School of Operations Research. Doug disagreed and said I was being charitable to Harvard.

Shaun Doheney, Chair of Resources and Readiness Applications

Connor McLemore, Chair of National Security Applications

*For want of a nail the shoe was lost. For want of a shoe the horse was lost.For want of a horse the rider was lost.For want of a rider the message was lost.For want of a message the battle was lost.For want of a battle the kingdom was lost.And all for the want of a horseshoe nail.*

The proverb “For Want of a Nail” describes how seemingly inconsequential details can lead to a disaster in military readiness, and is a valuable lesson for us all. For those of us who make decisions or support decision-making involving risks or uncertainty, we need to have an answer to the question, “are we ready?” Of course, that question should almost always be followed by the question, “ready for what?” Are we ready to respond to the next natural disaster? Are we ready to mitigate market volatility? Is our energy infrastructure ready to handle the increased demand this summer? Is our city ready for the expected increased growth over the next five years?

We (Connor McLemore and Shaun Doheney) have had military Operations Research experience, and have been working with Dr. Sam Savage here at ProbabilityManagement.org on an improved representation of military readiness. This provides a framework that we believe is useful, logically consistent, and most importantly is simple enough for adoption by military decision makers and those support such decision-making. As a poster child of poor military planning see the PowerPoint and Excel model describing the failed mission to rescue the American hostages in Iran in 1980.

One of the key components to this readiness representation framework is the ability to roll up readiness in a logical, mathematically sound, and intuitive way. To paraphrase Dr. Savage in his recent blog titled, Why Was RiskRollup.com Available?, if squadron A has a 60% chance of accomplishing the mission and squadron B has a 70% chance, then if we send them both is there a 130% chance of success?

Recent improvements in our ability to account for uncertainty allow us to rethink approaches to representing military readiness. To demonstrate our approach, we’ve created a few prototype models that you may download here.

We hope that you’ll join us during the upcoming Military Operations Research Society (MORS) Symposium when we give presentations and a tutorial on this work. While improved readiness accounting across the military and business or enterprises will likely be an evolutionary process with inputs from numerous stakeholders, the key in almost all situations is to “start small and reinforce success,” as Shaun likes to say. And as Connor likes to say, “Go Navy; beat Army!” But that’s a blog for another time!

Datasaurus Arithmetic

Data set HAP and PY

The three great milestones of manned flight were the Wright Brothers in 1903, the lunar landing in 1969, and the lithium ion laptop battery of the 1990s. This last breakthrough allowed me (while buckled into an airline seat to control my ADD) to develop a data set to dent the steam-era concept of correlation. I was on a flight from the East Coast to San Francisco, and over Denver I reached my goal: two variables, called HAP and PY, which had zero correlation, but nonetheless displayed a clear interdependency, as shown.

As I mentioned in my earlier blog on Virtual SIPs, I am not the only one poking fun at statistical concepts with ridiculous scatter plots. Alberto Cairo, a professor of Visual Journalism at the University of Miami, has a downloadable data set called Datasaurus, which has several X,Y pairs of data points, with identical summary statistics and correlation, but wildly different scatter plots. Alberto created his masterpieces with an interactive tool called DrawMyData from data scientist Robert Grant.

Never one to leave the bizarre well enough alone, I could not resist creating a model called Datasaurus Arithmetic, in which you may perform SIPmath calculations on the various patterns in Alberto’s dataset. Above we see the marginal distribution of X and Y (which I call Dino and saur), along with calculations involving the sum, product and quotient of X and Y while preserving the Jurassic joint distribution of X and Y.

If you teach statistics or data science, I urge you to download the file and compare the scatter plots and summary statistics of Alberto’s other included data sets.

Limbic Analytics from ProbabilityManagement.org RSSHubbard Decision Research and KPMG have launched a short Risk Management survey, which I urge you to take and to forward to others before March 10. It only takes 6 – 7 minutes to fill out and will help us better understand this important but poorly defined field.

Doug will be presenting on The Failure of Risk Management at our Annual Conference in San Jose in March, and I am eager to get his first impression of the responses. And don’t forget that Tom Keelin, inventor of the Metalog distributions, will also be there. The next generation SIPmath Standard, which leverages Doug’s HDR Distributed Random Number Framework and Tom’s Metalogs, will facilitate a more quantitative approach to Enterprise Risk Management.

Take the Hubbard/KPMG survey© Sam Savage 2019

Limbic Analytics from ProbabilityManagement.org RSS]]>If the risk of a power outage in City A next year is 60% and the risk of an outage in City B is 70%, then the risk of an outage across both cities is 130%, right? Obviously not, but what is it? Before the discipline of probability management, you couldn’t just add up risks. But today, you can represent the uncertainty of an outage in each city as a SIP as shown, where a 1 indicates an outage in that city. Simply summing the SIPs row by row provides the number of failures across both cities, then using the “Chance of Whatever” button in the SIPmath Tools you will find that that the risk of at least one failure across both cities is 88%. This pastes the following formula into the spreadsheet.

=COUNTIF( Sum, ">=1") / PM_Trials, where PM_Trials is the number of trials.

I am currently working with Shaun Doheney and Connor McLemore to apply these idea to Military Readiness, and Shaun will be presenting the MAP Model at our upcoming Annual Conference.

How do I know? I recently bought RiskRollup.com, ConsolidatedRiskManagement.com, and ConsolidatedRiskStatement.com for $11.99 each. I probably won’t be able to retire on these investments, but I’ll bet I get a decent return.

The holy grail of consolidated risk management is to optimize a portfolio of mitigations to provide the best risk reduction per buck. You might think that if people aren’t even rolling up risk today, we must be years away from optimizing. But that is not true. The concept of SIPs and SLURPs was in use in the field of stochastic optimization (optimizing under uncertainty) long before probability management was a gleam in my eye. This is the technique we applied at Royal Dutch Shell in the application that put probability management on the map. The scenarios of uncertainty generated by stochastic optimization are effectively SLURPs, and I argue that they are too valuable in other contexts not to be shared in a corporate database.

We are honored that a pioneer in stochastic optimization, Professor Stan Uryasev of the University of Florida, will also be presenting at our Annual Conference. I know I have a lot to learn from him. I hope you will join us in March.

More on rolling up risk and a discussion of the Consolidated Risk Statement are contained in a December 2016 article in OR/MS Today.

Limbic Analytics from ProbabilityManagement.org RSSDecades ago, I discovered that few managers were benefiting from probabilistic analysis. Despite widely available simulation software such as @RISK and Crystal Ball, most people lacked the statistical training required to generate the appropriate distributions of inputs.

“But wait a minute,” I thought to myself. “The general public still uses light bulbs even though they don’t know how to generate the appropriate electrical current.” After some research I discovered that there is a power distribution network that carries current from those who know how to generate it to those who just want to use it.

So why not create a Distribution Distribution network, to carry probability distributions from the people who know how to generate them (statisticians, econometricians, engineers, etc.) to anyone facing uncertainty?

Great idea, but it took me a while to figure out the best way to distribute distributions. Eventually I arrived at the SIPs and SLURPs of probability management, which represent distributions as vectors of realizations and metadata which support addition, multiplication, and any other algebraic calculation, while capturing any possible statistical relationship between variables. This concept even works with the data set invented by Alberto Cairo, made up of SIPs I call Dino and saur [i].

A Scatter Plot of Alberto Cairo’s Dino and saur

Once Excel fixed the Data Table, it became possible to process SIPs in the native spreadsheet, which greatly accelerated adoption [ii]. SIPs and SLURPs have been a simple, robust solution, although they do require a good deal of storage.

Before I thought of SIPs, I had thought of and abandoned an idea involving snippets of code which would generate a random number generator when they arrived on a client computer. I called this approach the Generator Generator (well, that was for short—the full name was the Distribution Distribution Generator Generator). The advantage of such a system is that the storage requirements would be tiny compared to SIPs, and you could run as many trials as you liked. It might not be possible to capture the interrelationships of Dino and saur, but at least some forms of correlations could be preserved.

Recent breakthroughs from two comrades-in-arms in the War on Averages have made the Generator Generator a reality and allowed it to be incorporated into the SIPMath Standard. One key ingredient is Tom Keelin’s amazingly general Metalog System for analytically modeling virtually any continuous probability distribution with one formula.

Another is Doug Hubbard’s latest Random Number Management Framework, which in effect can dole out independent uniform random numbers like IP addresses while maintaining the auditability required by probability management. This guarantees that when global variables such as GDP are simulated in different divisions of an organization, they will use same random number seed. On the other hand, when simulating local variables, such as the uncertain cost per foot of several different paving projects, different seeds will be guaranteed. This allows individual simulations to be later aggregated to roll up enterprise risk. Doug’s latest generator has been tested thoroughly using the rigorous dieharder tests [iii].

At ProbabilityManagement.org, we have wrapped these two advances into the Open SIPmath Standard for creating libraries of virtual SIPs, which will take up a tiny fraction of the storage of current SIP libraries. We hope to release the tools to create such libraries at our Annual Meeting in San Jose on March 26 and 27. Tom, Doug, and I will be presenting there, along with an all-star cast of other speakers. I hope we see you there.

Limbic Analytics from ProbabilityManagement.org RSS

[i] http://www.thefunctionalart.com/2016/08/download-datasaurus-never-trust-summary.html

[ii] Savage, S.L. Distribution Processing and the Arithmetic of Uncertainty, Analytics Magazine, November/December 2012.

[iii] https://webhome.phy.duke.edu/~rgb/General/dieharder.php

March 26 - 27, 2019

San Jose, CA

SIPmath is a broad-spectrum cure for the Flaw of Averages, which impacts all plans involving uncertainty. With this in mind, our 2019 Annual Conference casts a wide net over a variety of probability management applications. I urge you to look through the abstracts.

We have many great speakers lined up, including:

Deborah Gordon – Director, City/County Association of Governments, San Mateo County

Max Henrion – CEO of Lumina Decision Systems and 2018 Ramsey Decision Analysis Medal Recipient

Doug Hubbard – author of

*How to Measure Anything*and*The Failure of Risk Management*Tom Keelin – Inventor of the Metalog Distribution & Chief Research Scientist at ProbabilityManagement.org

Michael Lepech – Associate Professor of Civil and Environmental Engineering, Stanford University

Harry Markowitz – Nobel Laureate in Economics (via live webcast)

Greg Parnell – Military Operations Researcher & Professor at the University of Arkansas

Stan Uryasev – Risk Management Expert & Professor at the University of Florida

Topics covered include:

Analytics Wiki Development

Applying in SIPmath in Human Relations

Military Readiness

Municipal Risk Management

Applied Economics

Probabilistic Energy Forecast

Bridge Safety

Water Management

On September 17, I delivered a one-hour webinar previewing my Winter Quarter course in Project Risk Analysis in Stanford University’s Department of Civil and Environmental Engineering. This course will apply the discipline of probability management to such problems as risk return tradeoffs in R&D portfolios and rolling up operational risk across assets such as gas pipelines. Although the entire 57-minute webinar is available, I recommend the following excerpts.

Limbic Analytics:

Connecting the Seat of the Intellect to the Seat of the Pants

•The Concept

•The Cartoon

•The Audio Logo

•Interactive Simulations in Excel

•SIPs (Stochastic Information Packets)

Conveying Uncertainty as Data

The 60-Cycle AC Current Standard for Conveying Uncertainty as Data

•Getting Distributions from Forecasts

•Using Libraries with the SIPmath Tools

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>A common fork in the road to hell is arrived at when, in the face of uncertainty, the boss demands: “Give me a number.” You may be tempted to respond with, “Would you settle for an average?” But even the *correct*** average** of the uncertain duration of a task, demand for a new product, or labor hour requirements for a job, leads to a host of systematic errors that guarantee that your plans will be

Technically you should say to the boss, “Here’s the probability distribution of the number you want.” But I don’t recommend that if you want to keep your job. Instead, the latest version of the SIPmath™ Modeler Tools, both the free version and guilt-free $500 Enterprise version, now include the new “Chance of Whatever” button.

Just put your cursor in the cell where you want the chance of whatever to appear, then specify the uncertain cell that needs to be greater or less than your boss’s specified goal. Then click OK. Now as you change your goal, the chance cell will immediately update. So, next time the boss demands a number, you can respond with, “What do you want it to be? I can tell you the chance of meeting your goal.”

Brian Putt, Chair of Energy Practice at ProbabilityManagement.org, has a new video on how to use this feature of our tools. Check it out.

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>

Tom Keelin

We are happy to announce that Tom Keelin, inventor of the Metalog system, will join ProbabilityManagement.org as Chief Research Scientist. Tom is Founder and Managing Partner at Keelin Reeds Partners, former Worldwide Managing Director of Strategic Decisions Group, and co-founder of Decision Education Foundation. He holds a PhD in Engineering-Economic Systems from Stanford University.

On their own, **Metalogs** represent an unprecedented, unified approach to creating analytical formulas to represent probability distributions derived from data. Coupled to the HDR Random Number Management Framework from **Doug Hubbard**, they are leading to a new generation of SIPmath in which SIP libraries, which currently may contain millions of data elements, will be reduced to a few lines of code. These in turn will create virtual SIPs on an as-needed basis, without losing the fundamental properties of additivity and auditability that are the hallmarks of the discipline of probability management.

Watch for an upcoming blog post on the combined use of the SIPmath, HDR, and Metalog standards.

Related Reading: Tom Keelin’s Metalog Distributions

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>Since much of my income is from consulting, I have devoted resources to reaching out to appropriate clients. I can’t count the number of engagements I’ve gotten this way because there aren’t any. All my engagements have dropped in from out of the blue.

“But how about your 2009 book?” you say. “That was marketing on a grand scale. Some would have even called it selling out. You must have had customers breaking down your door after that.”

Nope. There was a horrific worldwide recession and I lost my key clients instead of getting new ones.

“But things are going great now, right?” Absolutely, and I am deeply thankful. But this was due to dumb luck, such as the improved Data Table function in Microsoft Excel, which enabled SIPmath, and stumbling upon adult supervision in the nick of time.

None of my successes have been planned and none of my plans have been successful. So, I don’t ** plan** (much to the consternation of my adult supervisors). Instead, I

So, when I heard that three Italian physicists (Pluchino, Biondo, & Rapisarda) had written a paper called “Talent vs Luck: the role of randomness in success and failure,” I was all ears. Among other things, they address the question of why, if talent is distributed along a bell curve, that wealth is extremely skewed with the top few percent of the population owning the lion’s share. They created a simulation that shows how chance drives the disparity in the distributions of talent and wealth. Inspired by the physicists, Dave Empey [1] and I built our own SIPmath model in Excel (available on our Models page) to explore similar principles. Our model shows that chance plays a role, but that disparity in income can arise without it. NOTE that unlike the physicists’ model, ours is not calibrated to reality, and is merely designed to give directional results.

Free models, like free advice, are worth what you pay for them. The admonition of George Box, that “all models are wrong, but some are useful,” applies in spades to economics, where Chaos Theory is always lurking a few decimal places away. I think the Italians would agree with me that such models do not provide “right answers” as much as “right questions.”

With the above caveats in mind, our model has the following elements.

1. We start with 50 agents, whose talents are measured in IQ score, normally distributed with mean of 100 and standard deviation of 15. These are assigned at the beginning and do not change during the simulation.

We also endow the agents with an initial wealth distribution, which may be uniform, or skewed either toward the high or low intelligence agents.

2. . We then simulate two forms of IQ-based income (wealth accumulation) over twenty years; either ** adding** wealth proportional to IQ or

3. We also allow for additional ** Chance Events** that can impose independent positive or negative impacts for each agent.

4. A heatmap displays the relative wealth by year each agent for a single trial. It is fun to crank up the uncertainty, press the <calculate> key, and watch the unsuspecting agents succeed or fail beyond their wildest simulated dreams.

5. Given the above calculations, we run 100 simulated trials of final wealth for each of the 50 agents, effectively generating a simulated population of 5,000 agents over which we calculate the final wealth distribution.

A key result of Pluchino, Biondo, & Rapisarda is that the final wealth in their simulation (which was more complex than ours) was very skewed even though talent was normally distributed. Our model indicates that you can’t sneeze without creating a skewed distribution of final wealth. For example, suppose there is no uncertainty, and all agents start with equal wealth, that increases each by a percentage proportional to their IQ. This is analogous to agents with investments that grow at different rates. Then you get the distribution of final wealth shown below.

Here we have the top 1% of the population holding 10% of the wealth. Adding additional uncertainties makes the skew worse, but talk is cheap, I suggest that you download the model here and play with it yourself.

[1] Director of Software Development at ProbabilityManagement.org and programmer of the SIPmath™ Modeler Tools.

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>by Sam Savage

The other night I was reading *Behave*, Robert Sapolsky’s magnificent book on human behavior, when something grabbed my attention. On page 35, Sapolsky describes two psychological experiments. In the first experiment, the subject is presented with a deck of cards, is told that half are red and half are black, and asked how much they would wager that the top card is red. Because there is an even chance of the top card being red or black, the risk-neutral bet (that is, the maximum payment you would make for a wager that pays you $1 if you win) would be 50 cents.

In the second experiment, the subject is told that the deck consists of red and black cards, and has at least one red and at least one black card. When the subject is asked to consider the same wager as before, again the risk-neutral bid is 50 cents because neither red or black is more likely to appear than the other.

So, what’s the difference between these two experiments? In the second one, the subject’s amygdala (the emotional center of the limbic system, which triggers the fight or flight response) lights up like a Christmas tree when viewed with functional MRI! The explanation for this strong reaction is that the ambiguity of the second deck induces anxiety[1]. The subject knows there is one red and one black card, but what are the rest of the cards? Experiments like these bring scientific rigor to the emerging field of Behavioral Economics.

The relevance to probability management is that in our discipline, uncertainty is communicated in SIPs (Stochastic Information Packets), which are randomly shuffled potential outcomes similar to the first deck of cards. With SIPs, the uncertainty is unambiguous. You can take your time and look at each number in advance, which is comforting, but you know only one will be selected when the uncertainty is resolved. This is unlike traditional simulation, which generates random experiments on the fly, thereby driving accountants bonkers. Instead, SIPs contain metadata, including provenance, so you know that they are not just something the cat dragged in. Then, when the accountants come knocking, you can say, “We are basing our decision on fifty million deterministic numbers. How about auditing these for us?” That’ll get them off your back for a few days.

This discussion also highlights the difference between the two famous feuding schools of probability, the *frequentists* and the *Bayesians*. The frequentists define the probability of an event as the proportion of times the event occurs in a large number of identical experiments. For example, if you actually wagered on *Red* over a thousand standard shuffled decks, you would win about 500 times, and the relative probability of red to black would be defined as 50%. A true frequentist might have trouble with deck two because they would not know how to define the repeatable experiment. The Bayesians, for whom my father was a major evangelist, think of probability as being subjective, and determined by the risk-neutral wager you would make on the outcome. Bayesians have no problem putting a relative probability of 50% on the outcomes of experiment two, which suggest that perhaps members of the two schools could be identified by what their amygdalae do in MRI machines.

In his book *The Black Swan*, Nicholas Taleb coined the term Ludic Fallacy to warn against "the misuse of games to model real-life situations." His warning should be heeded. However, I define the Ludic Fallacy-Fallacy to be the belief that you can manage real-life uncertainties *without* first understanding the simple arithmetic of dice, cards, and spinners. One thousand auditable potential outcomes may not contain any black swans, but it is way better than the industry standard of using a single average number to represent an uncertain future.

And speaking of games, an associate’s son is a star Little League baseball player. During the regular season his team dominated the other local teams, composed of kids he had known and played against for years. They easily made it into the playoffs with teams from other cities. At that point, facing the ambiguity of the unknown opposing players, his mother told me that the poor kid’s amygdala went up in flames of pre-game anxiety. They made it all the way through the playoffs, finally losing in a tight game in the fourth and final round. I asked if his anxiety had persisted during the championship play, and was told that by the second game “he had learned to play with deck two.”

So, think of probability management as “Uncertainty Light,” designed to calm those with Post Traumatic Statistics Disorder during the regular season. But don’t be lulled into complacency. You’ll never make the playoffs if you can’t deal with the ambiguity of the second deck.

To learn probability management applications, sign up for our Fort Worth workshop or an upcoming webinar.

[1] Doug Hubbard, author of the popular *How to Measure Anything* series, uses a variant of the card experiment called the Urn of Mystery, which shows the importance of drawing even a single sample before you wager in case 2 above. You may download Doug’s Urn simulation here.

]]>

by Sam Savage

Sandbagging is the practice of padding one’s budget to avoid running out of money in the face of an uncertain forecast. Suppose, for example, that ten managers each have independent uncertain annual expenditures that average $10M. Let’s assume they all cover their butts by forecasting the 90th percentile, which turns out to be $11M (the Sandbags). Now they each have only a 10% chance of blowing their budget.

Next, the CFO rolls these forecasts up to get $110M (the Sum of the Sandbags). And suppose the enterprise can also tolerate a 10% chance of exceeding the overall budget. The problem is that due to the diversification effect, there is only about one chance in 1,000 that the CFO will blow through all $110M. Why? Suppose one manager, Paul, ends up exceeding his budget at the end of the year, while another, Peter, has extra cash. Then the CFO can borrow from Peter to pay Paul, and all is well. So mathematically, given the options to balance across the portfolio at the end of the year, the 90th percentile at the line item level turns into something like the 99.99th percentile at the enterprise level.

To achieve the desired 90% confidence, the CFO might need only $105M, which we refer to as the Sandbag of the Sum. So, in this case, $5M is just lying around gathering dust instead of being available as investment capital. If you don’t think that’s a big deal, go out and try raising $5M sometime. And this problem only compounds as you roll up layers upon layers of fat through a multi-tiered organization. In the above, and most examples, the Sum of the Sandbags is greater than the Sandbag of the Sum (the number you should budget at the portfolio level given your organization’s risk tolerance). But the inequality can sometimes go the other way with asymmetric distributions, and you can’t do this stuff in your head.

When I wrote the first edition of *The Flaw of Averages*, there was no practical way to solve this problem on a universal scale. Today, however, thanks to the Open SIPmath™ Standard, anyone with a spreadsheet can easily perform the necessary calculations with uncertain budgets. What remains is the re-alignment of the numerous stakeholders involved. Impossible, you say? Someone who has done this the hard way without SIPmath is Matthew Raphaelson, who first introduced me to the Sandbag Problem years ago. Matthew is a former senior banking executive with 25 years of experience, which includes being CFO of a large business unit. He is also chair of Banking Applications at ProbabilityManagement.org.

He stresses that some managers may use probability as an excuse for lack of accountability. “At the end of the day,” says Matthew, “managers – not machines – need to own their forecasts and be accountable for their results.” He warns that “a company that relies solely on centralized models will be met with smirks and shrugs when it attempts to distinguish between forecast errors and performance misses.”

Matthew, who has been on the front lines of numerous budget wars, describes five stages of managerial development for tackling the sandbag issue.

**Education**

Make managers aware of the problem, and how today there is a practical solution.**Communication**

Understanding percentiles, and communicating uncertain estimates as auditable data.**Models and Data**

Convert existing data infrastructures to handle SIP libraries instead of numbers. This is no big deal and can be done with current software.**Incentives and Cultural Change**

The “nobody gets in trouble for beating a forecast” mentality is the root cause of the sandbagging problem. Gamification can both provide new incentives and train managers to become better forecasters in the face of uncertainty.**Analysis and Action**

Once uncertainty becomes auditable, it may be systematically reduced in a continual improvement process.

Matthew and I have written on this subject for the Banking Administration Institute (BAI).

And there are two separate documented SIPmath models available below that perform thousands of simulation trials per keystroke to connect the seat of your intellect to the seat of your pants.

SandbagCalc

Demonstrates basic sandbag math

Model from BAI article

Banking example with revenues and expenses

I will end with a war story from Matthew, which foretells the nature of the battle ahead.

“In the 1990s, I asked managers to give me a ‘50th percentile’ forecast to avoid the sandbag problem. Apparently, this guidance wasn't as clear as it needed to be. One manager's monthly expense results kept coming in lower than forecast, to the point where it was clear there had to be some bias. I re-affirmed with the manager that he provided 50th percentile forecasts. ‘Oh, absolutely,’ he said. Probing a bit, I asked if this meant there was a 50% chance that actual expenses would come in lower than forecast each month. ‘Yes, that's what it means,’ he said. And so, is there also a 50% chance that actual expenses would come in higher than forecast each month? ‘Oh no, there is almost no chance of exceeding our forecast....’”

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>by Sam Savage

After receiving his PhD in Decision Analysis from Stanford, Tom spent 40 years in analytical consulting, including an 18-year stint at the prestigious Strategic Decisions Group, where he was Worldwide Managing Director. Tom was struck by general management’s inability to compute uncertainties, and has developed a flexible family of continuous data-driven probability distributions based on pragmatic consulting experience. The Metalog distributions, as he calls them, combine mathematical elegance with computational efficiency [i], [ii], [iii].

To put Metalogs in perspective, I remind the reader that the theory of probability and statistics is powerful and elegant. But so is the steam locomotive, and they were developed around the same time. By the 1970’s, computational approaches to statistics such as bootstrapping arose. These were based on the brute force of computer simulation instead of 19th century calculus. Although Metalogs are also based on simple mathematical principles, they are intended to be fit to data sets, not adjusted by parameters such a mean and standard deviation. And they output the ideal food for simulations: inverse cumulative functions. These functions are the most common way to generate random variates in simulations. The Excel function NORMINV(rand(),Mean,Sigma), for example, will produce Normal random variables with the specified mean and standard deviation with every press of the Calculate Key.

The informative Metalog Distributions website contains extensive documentation and implementations in numerous environments, including Excel and R. We have already implemented some of the Metalogs in the SIPmath™ Modeler Tools as described below. You can also download any of the Excel templates from the Metalog website and use them with the tools. Just be sure to replace the “random” cells in the templates with either RAND() or HDR generators from the SIPmath tools.

It is still early innings for Metalogs. For example, last year Tom and I discovered how to generalize the concept to solve a vexing problem in simulation. Suppose you are modeling an uncertain number of risk events, such a transformer failure. Each failure will cause a fire with a skewed, lognormally, distributed adverse consequence. On a given simulation trial you may get 3, 5, 8 or some other number of failures, and need to add up 3, 5, 8 or some other number of lognormal distributions. But you don’t know in advance how many you will have so you don’t know how many you need to generate. Until our approach with the Generalized Metalog, there was apparently no closed form solution for expressing sums of lognormals. We (mostly Tom) wrote this up for publication, and with his help we built sums of lognormal and triangular distributions into the Enterprise SIPmath™ Modeler Tools. Tom is now Chair of Data-Driven Distributions at ProbabilityManagement.org, and we will keep you apprised of future Metalog developments, several of which are in progress.

All the latest versions of the tools support the SPT (symmetric percentile triplet) Metalog, which can produce a wide range of distribution shapes as shown below [iv].

Furthermore, Tom has written a nice tutorial on their use in the SIPmath Tools.

The sums of identical triangular and lognormals are implemented in the Enterprise version of the tools, as described below.

Suppose your organization is subject to a risk characterized by an average of 5 adverse events per year, each with a consequence that is lognormally distributed with a 50th percentile of $1Million, and a 90th percentile of $3Million.

The steps below show how to model this situation in the Enterprise SIPmath Tools

1. Poisson number of events

After initializing the file, we model the number of events per year as a Poisson variable.

2. Creating a sum of IID lognormals based on the Poisson number of events

We then create a lognormal in cell E5, checking the box on Sum multiple IIDs box (IID stands for independent, identical distributions). The number of lognormals to sum will be the number of events generated in cell C5, which varies with each simulation trial.

3. Specifying Risk as Output

We now specify E5 as an output of the simulation named “Risk” (cell E4) and denote cells F4 through G7 for a sparkline histogram.

4. Querying Statistics

Once the output is specified, you may specify any statistics, such as percentiles as shown below.

Now if you change any of the inputs (C3, E3, F3) the model will instantly update. And like all models created with the SIPmath modeler tools, the file is pure Excel, and uses no macros or add-ins, so you may share it with 1 billion of your closest friends.

[i] Keelin, T.W. and Powley, B.W., 2011. Quantile-parameterized distributions. Decision Analysis, 8(3), pp.206-219. https://pubsonline.informs.org/doi/abs/10.1287/deca.1110.0213

[ii] Keelin, 2016. The Metalog Distributions. Decision Analysis, 13(4), pp.243-277. https://pubsonline.informs.org/doi/10.1287/deca.2016.0338

[iii] MetalogDistributions.com

[iv] From http://metalogdistributions.com/images/TheMetalogDistributions.pdf

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>Life is full of helpful sounding procedures for improving your memory, losing weight, landing that perfect job, etc. I have found most of these to be worthless clichés with one notable exception: the five-step process of the renowned computer scientist Donald Knuth. In fact, it is the only thing I am religious about.

When I was studying computational complexity in graduate school in the early 1970s, I was exposed to Knuth’s multi-volume set on computer science, much of which went over my head. But early in one of the volumes he lays out the five steps of writing a computer program, which I have found invaluable in many settings. I state these in the context of analytical modeling, which I do more of these days than programming.

Decide what you want to do.

What is the purpose of the analysis? Who is the audience?Decide how to do it.

Is a spreadsheet adequate for the analysis or will I need a more powerful tool? Will I model time discretely or continuously?Do it.

Put fingers to keyboard and press appropriately.Debug it.

Of course it didn’t work as planned. Who do you think you are, Einstein?Trash steps 1 through 4 now that you know what you really wanted in the first place.

The power of recursion!

I’ll bet your organization spends a lot of time in steps 1 and 2 and calls it planning. I say, get to step 3 with a primitive prototype as quickly as possible. You will then be at step 4 before you know it, which qualifies you for the true enlightenment of step 5.

I consider myself a black belt in the Step Five Process. When I start a new modeling project, I am completely confident that I don’t know what I want, so I only spend 3 seconds on step 1. I give myself much longer on step 2, 30 seconds. If it takes longer than that, I quit. Step 3 is where the time comes in. I put on headphones, switch to my Eagles Channel on Pandora (as much as I love classical music, it does not work here), and typically work for 15 to 30 minutes before finding the fatal flaw, which I must debug. I don’t spend a lot of time debugging at step 4, maybe 5 minutes, because I know that step 5 is inevitable, and I can’t wait to start again on what I now think I wanted in the first place.

When do I terminate the Step Five Process cycle? When my model is dead! A living model is always evolving in this manner.

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>According to George Bernard Shaw, “The single biggest problem in communication is the illusion that it has taken place.” The poster child for this conundrum is the word *risk*.

You worry about the risk of XYZ stock going down, but I’ve shorted it, so I worry about XYZ going up. I offer you $200,000 in cash or a coin toss worth 0 or $1 million and you take the cash. Bill Gates risks the coin flip.

Bottom line: risk is in the eye of the beholder.

Yet most risk management techniques serve up “risk” as a single number, or worse, a color on a heat map, which is blind to risk attitude. No wonder Doug Hubbard, author of *The Failure of Risk Management: Why It’s Broken and How to Fix It*, argues that most current methodologies “are no better than astrology.” Both Doug and I agree that a promising approach is the computer simulation of uncertainty, but most risk simulations are siloed, and cannot be networked together into integrated systems. This will take a degree of standardization which is just emerging, and which will need to take place on multiple levels.

As an analogy, consider the standardization of financial statements.

**Uniform Formatting defines how things look **(let's go with the green stripe).

Formatting helps organize information visually, but does little else. The risk management version is a heat map. Don’t get me started!

**Uniform Calculations define what things mean.**

Risk calculations are typically done three ways:

Not at all

Doug Hubbard and co-author Richard Seiersen deride calculations with heat maps as “Orange times fish plus purple times birds equals Pee Wee Herman.”With averages

The good news: averages are easy to calculate with. The bad news: they lead to the Flaw of Averages. I will address this in a future post, but you’d better use an incognito browser when you read it.With simulations

Simulations preserve the uncertainty of risky situations, allowing results to be viewed according to the beholder’s individual risk attitude.

Here an important calculation is:

Risk = Likelihood of Failure x Consequence of Failure

If you treat Likelihood and Consequence as single numbers in this expression, stay tuned for my upcoming diatribe. If you use it in a simulation which conveys uncertainty, then game on!

**Uniform Representations define how things are communicated. **

Hah! You thought we were done. But if you can’t communicate the results in an actionable way you are stuck in your own silo.

Hindu-Arabic numerals are so entrenched that we don’t even realize we have a choice. At ProbabilityManagement.org we are working with others on an analogy for conveying uncertainty between simulations.

Our open SIPmath Standard represents uncertainties as arrays of simulated trials called Stochastic Information Packets (SIPs). Doug Hubbard is developing a family of portable random number generators that have already been adopted by the SIPmath Tools available on our website, and may eventually enable massive networked simulations that communicate across the economy. Tom Keelin's new representation for probability distributions, called Metalogs, also appears in our tools. I will devote a future blog to Tom’s mathematically elegant and practical invention.

The theme of our Annual Conference in San Jose on March 27th and 28th is Standardized Risk and Doug, Tom, and I will be presenting. I hope you can join us.

Do you have ideas of your own that you want to share with the world? Send us an email.

Sam L. Savage

Executive Director

by Sam Savage

In his 2011 book, *Thinking Fast and Slow*, Daniel Kahneman divides the human thought process into a fast, intuitive component, System 1, and a slower analytical side, System 2. This is a useful dichotomy, and I refer to these systems as "the seat of the pants" and the "seat of the intellect," as sensitively portrayed by Jeff Danziger in my 2009 book, *The Flaw of Averages.*

Kahneman claims that System 1 is bad at understanding statistics because it can only focus on one thing at a time, and System 2, which can handle many things at once, is slow and lazy and may not be consulted in the heat of decision making.

But when you put System 1 and System 2 on what Steve Jobs called a Bicycle for the Mind (a computer), you can connect the seat of the intellect to the seat of the pants and fundamentally change your thought process.

This is one of the advantages of SIPmath for performing probabilistic analysis. First the SIP itself, as an array of thousands of potential outcomes, is "one" thing that contains "many" things. Second, because SIPmath simulations in Excel yields results in real time by evaluating thousands of possibilities per keystroke, it can tap into our limbic system, with its tens of millions of years of evolution.

Limbic Analytics from ProbabilityManagement.org RSS© Copyright 2018 Sam Savage]]>