Below we find another video, this time from IBM, that paints a vision of the data enriched world in our not too distant future.  Most of the things we use and interact with as we go through our day will all be connected and talk to each other on the same network.  The items will work together to make our lives more simple and efficient.  For instance, our alarm clock may check our calendar for the day’s first meeting time and location.  It will then work backwards using public transport networks (traffic patterns and ferry schedules) to calculate when we should be woken up.

The video is very nicely done, but it does raise some questions about the effects that this kind of artificial intelligence or automated decision making will play in this new world.  Are we really building the smarter planet that IBM intends?

The video talks about the DIKW triangle.  At the bottom level is Data–expanding and hard to make sense of.  As you go up the triangle Information gets extracted from the data.  The information is then used to create Knowledge.  Finally, the acquisition of enough knowledge leads to Wisdom.  My concern is that as we make the systems around us smarter and allow them to obtain the ability to make decisions for our life or perform certain actions, are we doing it at the expense of our own abilities to navigate through life?

I guess this is by no means a new conflict.  After all, I haven’t remembered a phone number of a friend or contact since my 7th grade girlfriend.

The Economist recently published an excellent article painting a picture of the the emerging data explosion and of the challenges and opportunities that this explosion will present.  Apparently some are even calling it an “industrial revolution of data”!  The article explains the role that I aim to play in the revolution with the following paragraph:

Chief information officers (CIOs) have become somewhat more prominent in the executive suite, and a new kind of professional has emerged, the data scientist, who combines the skills of software programmer, statistician and storyteller/artist to extract the nuggets of gold hidden under mountains of data. Hal Varian, Google’s chief economist, predicts that the job of statistician will become the “sexiest” around. Data, he explains, are widely available; what is scarce is the ability to extract wisdom from them.

Data, Data Everywhere, The Economist

You can’t fault Mr. Varian for adding some wishful thinking to his prediction.  I can say with 95% confidence that statisticians will not suddenly become sexy anytime soon.  Other than that, his assertions are true.

In the last post, I exposed the mounting challenge related to our abilities to draw valuable conclusions from the swell of data that will be available to us in the near future.  In order to benefit from this influx of information, companies will need to adopt a scientific approach to understand and create actions from this information.

Fortunately, the sort of scientific and numbers driven approach has been evolving for the past century.  It started with Fredrick Taylor studying the movements of workers in a factory and has evolved into process improvement philosophies like Total Quality Management, Six Sigma and Lean thinking.  Over that time period they have moved from assembly lines, to service business like delivery, restaurants, and medical practices, and now are starting to take shape in white collar environments including banks and insurance companies.

Applying these concepts to web companies and consumer product behavior are the next step in this evolution.  Because the disciplines are heavily focused to operations inside the company, many people assume they are not good for evaluating what is happening outside the walls of the organization.  In fact, this assertion couldn’t be further from the truth.

In lean thinking, everything starts with the customer.  Our business need is to provide the exact item required at the highest quality at the lowest cost in the shortest lead time.

Getting the Right Things Done, Pascal Dennis

Process improvement takes place by using the aspects of you business that you have control over to better serve the needs of the customer.  Acquiring voice of the customer data at the outset is the most important aspect of the philosophy.  The preceding quote applies to businesses with traditional operating model including production and cost of goods sold.  But as we apply this idea to the web where variable costs are marginal, it would differ slightly.

With lean thinking applied to the web, it is still all about the customers.  The website must provide the exact service required at the highest quality while the delivering the highest level of intuition and ease of use to the user.
This guy.

In my next post I’ll take a closer look at the tools that make up the process improvement methodologies and how they should be applied to the web.

What follows is a brilliantly animated video that demonstrates the technological opportunities and challenges that lie in the very near future.  As computing power continues up the curve of Moore’s Law and chips get smaller, faster, and cheaper, technology will be embedded into more and more of the products we buy.

As the video suggests, the information captured on these items will not simply be isolated to the devices themselves, but rather they will be communicable to the environment around them using wireless technology.

Think about the prevailing technology from 10 years ago. Connecting to the internet required you to plug your desktop computer into the wall. Watching video on your computer more often than not involved a DVD drive. Your cell phone was only used to make calls and had the capacity to store about 200 phone numbers. Given the advancement embodied by products, services, and technologies like the iPhone, Hulu, GPS, and WiMax, it is hard put a limit over the plausible advancements we can expect to see in the next decade. Appliances, vehicles, tools, furniture, even your clothes will all be a part of the network. The social web will evolve from people and the websites they visit to people and the items they use each day.

Just as websites do today, all of these new sources will create mountains upon mountains of data. Companies will be able to find out precisely how their products are used. They can use this information to make their products and services more effective. When morally permissible, they may also sell this information to others that can benefit. The winning companies of this new age will be the ones that can efficiently detect meaningful trends and ask targeted questions in order to reach insightful conclusions that lead to action.

As technological advancements allow managers to become less pragmatic and more scientific with their decision making, the companies that embrace this new kind of analysis and attract personnel that can effectively extract valuable action from mountains of data will be the most successful.

In the wake of the pedal recall and the controversy that has surrounded it, Toyota is being called out for its failure to adhere to the management philosophies that it helped pioneer over the past half century.  When this kind of failure occurs it is justifiable for both insiders and outsiders to question or reevaluate the system and environment that led to its creation.  Reevaluation is a wise process that leads to learning and getting continuously better.  It is, however, unwise to have a knee jerk reaction and come to the conclusion that the entire management philosophy is now debunked as some are doing.

My take is that nothing is wrong with the philosophy; rather, the management of Toyota has strayed over the past years. An example of this can be found in the following excerpt.  Six years ago, when accidents started occurring Toyota thought the root cause was due to either the floor mats or a more serious malfunction of the gas pedal itself.

By month’s end, Mr. Yon updated his NHTSA case file with a memo. Itsaid NHTSA had decided to limit the probe to incidents involving briefbursts of acceleration, and would exclude so-called “long duration”incidents in which cars allegedly continued racing down the road aftera driver hit the brakes.

The reason: Investigators decided it would be more effective toisolate any possible defect by zeroing in on shorter incidents, aTransportation Department official said. The shorter incidents lookedmore like “pure cases of engine surging due to a possible defect,” theofficial said. Longer incidents were excluded because they showed moresigns of driver error such as mistaking the accelerator for the brake.

Of the 37 incidents, 27 were categorized as long-duration and not investigated. On July 22, 2004, the probe was closed because NHTSA hadfound no pattern of safety problems.

Secretive Culture Led Toyota Astray, WSJ Feb 8th

The father of the Toyota Production System and Lean Manufacturing, W. Edwards Deming, would ask one question to retort the thinking behind the decisions in this passage. By what method? Deming believed it was foolish for managers to draw any conclusion or formulate any kind of plan without the data to back it up.  Conjectures and forward thinking are an integral function of leadership and building a successful organization.  But rather than following conjecture blindly, management must instead use experiments to test out assumptions and draw conclusions from the resulting data.  In this case Toyota should have done more analysis before reaching their optimistic conclusion.  This whole fiasco could have been dealt with six years ago and lives would have been saved.

Often times one of the most overlooked aspects of carrying out a project is quantifying the business value an organization can expect to recoup by investing time and resources into that project. Of course consensus can still be built without taking a look at the numbers. Projects still move forward when everyone in the room agrees that taking action is a “no-brainer”.

Unfortunately for the organization’s shareholders, the decision makers will put this common phrase into practice by absent-mindedly skipping the upfront work of quantifying the end benefit to the organization. Instead, in many cases they will simply spend the money that has been set aside for their department and the emphasis for carrying out projects shifts from delivering value to staying under budget and meeting deadlines.

In these sorts of organizations, projects that actually destroy business value can be deemed as a success because they were completed cheaper and sooner than originally anticipated. Picture a toll road that is constructed out of gravel. It might be cheaper and quicker to build than a highway, but if no one pays to drive on it the project can obviously not be considered a success. The reward for the time and effort spent to build the project will never be recouped.

By instead putting in the effort to calculate the business value upfront, the organization sets the correct tone for all future activities related to that project. It not only answers but also provides justification for the most important question of all: Why take action?

• Pre Project
  From the very beginning, building a financial justification allows the organization to compare the project against other alternatives. It also develops the goals of the project and paints the end picture of what a successful project looks like from day one.

• During the Project
  During the implementation of the project the business value should continuously be revisited as uncertainty reduces. Known facts will replace and revise the estimates and assumptions from the initial plan. If evidence presents itself that the project will destroy value or that other alternatives are more attractive, knowing the business value gives the organization the power to abandon the project before digging a deeper hole.

• Post Project
  After the project is completed, it gives the organization an idea of what to measure. The organization will know if the project is successful and what steps can be taken to further increase business value.

There are at least three ways to improve business value. Among them are:

1. Higher Quality Output
   Improving the quality of the output of an organization or department will make the output more valuable or increase the demand for that output. Calculating business value anticipates a getting higher price for or an increased demand for goods or services.
2. Cost Reduction
   Cost reduction involves cutting wasteful activities that do not contribute to delivering value to the customer. Reducing the costs of producing goods or delivering services will allow the organization to increase productivity. Productivity increases open the door to higher profit margins by the allowing the customer to produce the same amount with fewer resources or by producing more with the same number of resources. Productivity gains can also increase market share by allowing the company to offer products at a lower price.
3. Risk Reduction
   A more complex and less understood form of increasing business value involves reducing the amount of risk the organization must bear to stay in business. Activities in this group include providing insurance against catastrophic events, fulfilling regulatory requirements, or shifting money around on a company’s balance sheet. This is the most complicated of the three because it does not directly involve supply or demand. These kinds of projects don’t necessarily add quality to output or reduce the cost of production. In order to justify them you must calculate the expectancy and cost of a catastrophic event that could put the company out of business. They are especially hard to justify because until the circumstances of catastrophic event occur, their value is not fully realized.

There is a degree of interrelatedness between all of these value propositions. For instance, if a company were to improve the quality of their product it would increase the demand for that product. It would also reduce waste and costs due to rework or warranty replacements. It may also reduce liability related to massive product recalls or fighting a class action suit against defective products. Each one of these angles should be considered when quantifying the benefits of a potential project.

In some organizations there is simply no emphasis on calculating the reward for actions taken. Energy is focused solely on “doing more with less” and staying within budgets.

While doing more with less is a means to create business value, it is not effective unless the end effects of the actions taken are also considered. In many cases cost cutting measures lead to reduced quality, which then affects profits, and business value is destroyed.

In other “no brainer” cases, employees simply perceive the value gains as incalculable. Customer facing departments like product design will argue that it’s impossible to predict the effects that certain features or product offerings will have on customer demand and at what price to offer the item or service.

Departments several steps removed from the customer that instead serve internal business needs will argue that the ambiguity is more intense. In many cases it may be hard to calculate the benefits of increased employee satisfaction or improved communication.  This is no excuse to skip the step of justifying decisions. In these situations it is beneficial to build a model that substitutes assumptions for the missing data. The construction of these models preserves the capability to make informed decisions. Rather than debating an end dollar figure given for the delivered business value, the emphasis is shifted to the validity of the model and the assumptions which it is based.

Buy in from other decision makers comes from obtaining their agreement on these assumptions. Rather than pulling these assumptions out of thin air, they should be developed using comparable scenarios from the past data or experience. As the project goes on and uncertainty shifts to known information, the model becomes more accurate and the picture of the business value that is produced or destroyed becomes concrete.

Most organizations have their own preferred methods of calculating the business value of projects and weighing them against other alternatives. The prerequisite to each of these methods is that the model gives scheduled value generation on a periodic basis (either months or years). For example, a project may present the following schedule of value delivery.

Period   Cash Flow
Year 0     $ -750,000
Year 1     $  150,000
Year 2     $  500,000
Year 3     $  400,000
Year 4     $  200,000
Year 5     $  200,000

The following are several of the most common methods to describe summarize the value that is created in the schedule above:

Monthly Payback
This is the simplest of all methods. It is simply a statement of how long it will take to recoup the investment made into the project. Given the data above, the payback would be calculated as the following:

Payback = -750,000 + 150,000 + 500,000 + (400,000 x 0.25)
Year 0        Year 1      Year 2             Year 3
or
2 years 3 months payback

The value of this method is its simplicity.  One aspect of the risk of the project can be judged by how long it will take to see the benefits. The shortfall of this method is that it does not consider what happens after the breakeven point has been met.

Net Present Value
Calculating the net present value or NPV of a project provides a more complete picture of the business value delivered because it includes the project’s entire lifespan in addition to also factoring risks and alternatives.

The reason for this lies in the concept of the time value of money. Compounding interest allows wealth to grow over time. Every person on the planet has the opportunity to invest money into an account that will collect interest. With this in mind, it follows that collecting $100 today is more valuable than collecting exactly $100 five years now. By accepting the money today and taking it straight to the bank, interest will accumulate allowing you to pull more than $100 when it is withdrawn five years from now.

Exactly how much will the $100 be worth after five years? The answer depends on the interest rate offered by the account that the money was deposited. If a company has a bank that will offer 10% interest rate compounded annually with no risk of default that means in the long run the company should only invest in projects that will result gains better than the 10% per year offered by the bank. The 10% from this example represents the discount rate for that company. The discount rate is used in NPV calculations to compare the value of money gained in future periods against its value in today’s terms by relating it to the other investment opportunities available to the company. If the effects of a project result in a positive NPV, the project is better than alternatives. Projects that calculate to a negative NPV should not be pursued because they are not better than investing in alternatives.

The following formula is the first step in calculating NPV. It should be performed for each period.

Present Value for Each Period = Future Value / (1 + Discount Rate)^Period

Assuming the company has a 10% discount rate and the given data above, the Net Present Value calculates to the following:

Period   Future Value        Calculation            Present Value
Year 0     $ -750,000       -750,000/(1+.10)^0         $ -750,000
Year 1     $  150,000         150,000/(1.10)^1          $  136,364
Year 2     $  500,000         500,000/(1.10)^2          $  413,223
Year 3     $  400,000         400,000/(1.10)^3          $  300,526
Year 4     $  200,000         200,000/(1.10)^4          $  136,603
Year 5     $  200,000         200,000/(1.10)^5          $  124,184
Net Present Value:       $  360,900

First, the value generated each year is calculated in today’s terms. Notice that money now is more valuable than the same amount in future years. Working backwards, the chart also shows that given our 10% investment alternative, $124,184 will be worth exactly $200,000 five years from now.

Once each period’s values a have been calculated in today’s terms they are now valid to add together to create the Net Present Value. The value generated by this project is worth $360,900 in today’s dollars. As other projects’ NPV are calculated, the decision makers have an apples-to-apples comparison to justify their actions

Calculating the internal rate of return or IRR for a project is very similar in concept to the NPV calculation. The distinction is that the value derived by the project is communicated as percentage of annual return on the investment. Similar to the 10% return offered by the bank in the previous example, our project has its own annual return value known as the IRR.

To calculate the IRR, take the value delivered each period and then calculate the discount rate that gives the project an NPV = 0. The values from the above project give us the following equation:

0 = -750,000 + 150,000/(1+r)1 + 500,000/(1+r)2 + 400,000/(1+r)3 + 200,000/(1+r)4 + 200,000/(1+r)5

Plug these values into a solver or use the IRR function in Excel to solve for r. In this example the IRR is 28%. The higher rate of return (as opposed to the 10% offered by the bank) shows that the company should invest in this project. If the IRR is lower than 10% or lower than other project alternatives it may not be advisable to pursue the project.

This article was picked up by the Application Development Knowledge Center at eWeek.com.  Check it out here.

A recent Gartner article revealed CEO’s greatest concerns for 2009.  The # 1 concern is layoffs and restructuring.  #2 on the list is to “pare the corporate efforts down to those that are central to the company’s short-term survival while not killing off its future.”

How well will your IT department fare when the time comes to make adjustments to the budget?  The answer can be predicted depending on your response to the following questions:

• Does your IT department deliver solutions that are critical to the strategic goals of the company?
• How long does it take to deliver value to the business users?  Can you react quickly enough to meet the tactical needs of the business?

For those of you that get a little nervous while answering either of these questions, this article will introduce you to the delivery methodology utilized by business process management—a technology that has been shown to reduce costs by 20% in the first year and is the #1 priority for CEO’s in 2009. BPM does this by:

• Focusing on processes that are strategically important to the company.
• Putting solutions into the hands of the business side as quickly as possible.

The following graphic displays the difference between how BPM is delivered, and that of a traditional IT development project.

The Traditional Approach
To kickoff traditional IT projects, business analysts will take time from the business users to capture all the details needed to construct the final application. Then, after several months of interviews, the analysts toss a massive requirements document over the wall to the developers.  The developers transcribe the requirements and begin building the application.  The business is not brought back into the process until the application is close to the final release.

When the application is finally ready for release many months or even years later, the organization can only cross its fingers in hope that the process that was described in the requirements document is still in practice today.  At any point before or after the release has been delivered, the business users’ change requests are usually met with frustration from IT because they are departures from the functionality that was agreed upon in the requirements document.

The BPM Approach
BPM delivery is focused on forming a partnership with the business and meeting their needs as quickly and efficiently as possible.  Rather than building requirements to kickoff the process, the first step is to discover and boost the understanding that all business users, owners, and downstream participants have of the process.
With this increased knowledge and understanding, stakeholders can develop a list of well-informed improvements to the process(es) that are related to strategic business goals that have been established at the executive level.

Analysts then work with developers to implement processes that address the highest ones on the list.  The developers’ playback releases to the business users every 4 to 6 weeks in order to be certain that the process is meeting their expectations.

And as soon as a release meets the minimum requirements, the business can start utilizing the application while more functionality is built into the process.

What It Means
Rather than pointing back to contracts or signoffs on requirements documents, the business users and IT department are working as a team to build the most valuable application possible.  Changes are expected and welcomed.  Down the line, after the process has been in production for a while, research can be done using the data that has been collected, as well as simulation and optimization tools, to develop further enhancements to the process.

With a partnership that is formed using this approach, the IT department is able to deliver value to the business immediately and constantly.  The process discovery and improvement exercises at the start of the effort give the business users ownership and empowerment in the implementation process.  Each new release delivers more value to the business than the one before it.

On the other hand, without such partnership, the business users view the requirements building process as valuable time out of their day, change requests are met with frustration from IT, and everyone has their fingers crossed that the final application will deliver value.

A Recent Example
The benefits of this new kind of delivery approach were recently on display at a large financial services company that implemented several processes in a short timeframe.  To start out, the IT department had a list of process requests they had received from the business. The analysts conducted interviews with management to establish the largest challenges that were facing the company in the coming year.  Among them was a looming set of compliance standards that they were far from fulfilling efficiently.   Meeting these standards was the highest for priority for the IT department.  As a result, the processed related to these requirements were at the top of the list.

After that, subsequent sessions were conducted with the business users to increase the understanding of the current process.  During the sessions, analysts used a collaborative mapping tool to create a model of the current process.  Participants with differing perspectives on the same process were able to paint their own pictures of the process and then come to an agreement about what was actually happening day to day.  From just those sessions, the business users came up with ideas about what should be done to address the current problems.  They immediately recognized that the IT department was working for them.  The business users looked forward to seeing the new process implemented very soon.

The analysts took the process maps and the lists of improvements to the developers to decide the size and scope of each one.  Together the analyst and developers came up with a delivery roadmap that laid out which features would be implemented in each release.  Within five weeks of kicking off the discovery of the process, the business users were able to see click through a working prototype that met their functional needs.  As the releases continued, business users gave feedback on how screens should look and also requested changes to the process.   Because the partnership had been formed and the goals of IT and business users were aligned, the developers had no problem adjusting their approaches.

Utilizing the BPM delivery approach, the IT department ended up delivering four processes within a six month timeframe—successfully meeting the business’s pressing need of fulfilling regulatory and compliance standards.

When CEO’s look to pare their efforts and resources, which approach would they be more willing to preserve in tough times?