AKF Partners

Abbott, Keeven & Fisher Partners Partners in Technology

Growth Blog

Scalability and Technology Consulting Advice for SaaS and Technology Companies

Avoiding the Policy Black Hole

April 18, 2018  |  Posted By: Pete Ferguson

During due diligence and in-depth engagements, we often hear feedback from client employees that policies either do not exist - or are not followed.

All too often we see policies that are poorly written, difficult for employees to understand or find, and lack clear alignment with the desired outcomes.  Policies are only one part of a successful program - without sound practices, policies alone will not ensure successful outcomes.

Do You Have a Policy …?

Early in my career I was volunteered to be responsible for PCI compliance shortly after eBay purchased PayPal.  I’d heard folklore of auditors at other companies coming in and turning things over with the resulting aftermath leading to people being publicly humiliated or losing their job.  I suddenly felt on the firing line and asking “why me?”

I booked a quick flight to Phoenix to be in town before the auditor arrived and I prepared by walking through our data center and reviewing our written policies.  When I met with the auditor, he looked to be in his early 20s and handed me a business card from a large accounting firm.  I asked him about his background; he was fresh out of college and we were one of his first due diligence assignments.  He pulled out his laptop and opened an Excel spreadsheet and began reading off the list:

  • Do you have cameras?  “Yes,” I replied and pointed to the ceiling in the lobby littered with little black domes.
  • Do you record the cameras?  “Yes,” and I took him into the control room and showed him that we had 90 days of recording.
  • Do you have a security policy?  “Yes,” and I showed him a Word Document starting with “1.1.1 Purpose of This Policy ....”

Several additional questions, and 10 minutes later, we were done.  He and I had both flown some distance so I gave him a tour of the data center and filled him full of facts about square footage and miles of cable and pipes until his eyes glossed over and his feet were tired from walking and off he went.

I was relieved, but let down!  I felt we had a really good program and wanted to see how we measured up under scrutiny.  Subsequent years brought more sophisticated reviews - and reviewers - but the one question I was always waiting to be asked - but never was:

“Is your policy easily accessible, how do employees know about it, and how do you measure their comprehension and compliance?”

My first compliance exercise didn’t seem all that scary after all, it was only a due diligence “check the box” exercise and didn’t dive deeper into how effective our program was and where it needed to be reinforced.

While having a policy for compliance requirements is important, on its own, policy does not guarantee positive outcomes.  Policy must be aligned with day-to-day operations and make sense to employees and customers.


The Traditional Boredom of Policy

Typically policy is written from the auditor’s point of view to ensure compliance to government and industry requirements for public health, anti-corruption, and customer data security standards. 


Image Credit: Imgur.com

Unfortunately, this leads to a very poor user experience wading through the 1.1.1 … 1.1.2 … . Certainly a far deviation from how a good novel or any online news story reads.

I’ve heard companies - both large and small - give great assurances that they have policies and they have shown me the 12pt Times New Roman documents that start with “1.1.1 Purpose of This Policy …” as evidence.

I had to argue the point at a former position that the first way to lose interest with any audience is to start with 1.1.1 … and with Times New Roman font in a Microsoft Word document that was not easy to find.  It was a difficult argument and I was instructed to stick with the approved, and traditional, industry-accepted method.

Fast forward a decade later and our HR Legal team was reviewing policy and invited me to a meeting with the internal communications team.  Before we started talking documents, the Director of Communications asked me if I’d seen the latest safety video for Virgin Atlantic Airlines.  I thought it a strange question, but after she told me how surprised and inspired by it she was, I took a look.

VA thankfully took a required dull and mundane US Federal Aviation Administration ritual and instead saw it as a differentiator of their brand from the pack of other airlines.  Whoever thought a safety demonstration could also be a 4-minute video on why an airline is different and fun?!?  Up until that point, no one!  Certainly not on any flight I had previously flown.

Thankfully, since then, Delta and others have followed their example and made something I and millions of airline crews and passengers had previously dreaded - safety policy and procedure - into a more fun, engaging, and entertaining experience.

While policy needs to comply with regulations and other requirements, for policies to move from the page to practice they need to be presented in a way employees clearly understand what is expected - so in writing policy, put the desired outcome first!  The regulatory document for auditors can be incorporated at the end of each policy or consider a separate document that calls out only the required sections of your employe handbook or where ever your company policies are presented and stored.

Clarifying the Purpose of Your Policy

In her article “Why Policies Don’t Work,” HR Lawyer Heather Bussing boils down the core issue: “There are two main reasons to have employment policies: to educate and to manage risk.  The trouble is that policies don’t do either.”

She further expounds on the problem in her experience:

“ … policies get handed out at a time when no one pays attention to them (first week of employment if not the first day), they are written by people who don’t know how the company really works (usually outside legal counsel), and they have very little to do with what happens.  So much for education.”

As for managing risk, Bussing points out that policies are often at odds with each other, or so broad that they can’t be effectively enforced.

“Unless it is required to be on a poster, or unless you can apply it in every instance without variance, you don’t want policies.  Your at-will policy covers it.  And if you don’t follow your policies to the letter, you will look like a liar in a courtroom.”

Don’t let your online policy repository feel like a suppository - focus on what you want to accomplish! 

Small and fast-growing companies typically have little need for formalized policies because people trust each other and can work things out.  But as they grow it has been my experience that often the trust and holding people accountable - which sets fast growing companies apart as a cool place to work - get replaced with bureaucratic rituals cemented in place as more and more executives migrate from larger, bureaucratic behemoths.  If the way policy is presented is the litmus test for the true company culture, a lot of companies are in trouble!

Policy must be closely aligned to the shared outcomes of the company and interwoven into company culture.  Otherwise they are a bureaucratic distraction and will only be adopted or sustained with a lot of uphill effort.  In short, if people do not understand how a policy helps them do their job more easily, they are going to fight it. 


Adapting Policy To Your Audience

In the early days of eBay, the culture was very much about collectables, and walking through the workspace many employees displayed their collections of trading cards, Legos, and comic books.  When it came time to publish our security policies, we hired Foxnoggin - a professional marketing strategy company - and took the time to get to understand our culture and then organized a comprehensive campaign to include contests, print and online material, and other collateral. 

They helped formulate an awareness campaign to educate employees and measure the effectiveness of policy through surveys and monitoring employee behavior.

To break away from the usual email method of communication, we got and held employee attention with a series of comic books which included superheroes and supervillains in a variety of scenarios highlighting our policies.

An unintended consequence from our collector employees was that they didn’t want to open their comic books and instead kept them sealed in plastic.  To combat this, we provided extra copies (not sealed in plastic) in break rooms and other common areas and future editions were provided without the bags.  The messages were reinforced with large movie-style posters displayed throughout the work area. 

This approach was wildly popular among employees located at the customer support and developer sites and surveys showed that security was becoming a top of mind topic for employees.  Unfortunately, this approach was not as popular with Europeans - who felt we were talking down to them - and by the executives coming from more stodgy and formal companies like Bain & Company or GE and particularly unpopular with execs from the financial industry after the purchase of PayPal.

Intertwining policy into the culture of your organization makes compliance natural and part of daily operations.

Make Sure Your Message Matches Your Audience

President and CEO of Lead From Within Lolly Daskal writes on Inc.com:

“... sometimes the dumbest rules can drive away the best employees … too many workplaces create rule-driven cultures that may keep management feeling like things are under control, but they squelch creativity and reinforce the ordinary.”

Be creative and look at the company culture and how to interweave policies.  Policies need to be part of the story you tell your employees to reinforce why they should want to work for you.

Nathan Christensen writes in his Fast Company article: How to Create An Employee Handbook People Will Actually Want to Read, “let’s face it, most handbooks aren’t exactly page-turners. They’re documents designed to play defense or, worse yet, a catalog of past workplace problems.”

Christensen recommends “presenting” policies in a readable and attractive manner.  It must be an opportunity to excite people in meeting a greater group purpose and cause.

Your policies need to match your company culture, be in language they use and and understand, and the ask for compliance needs to be easily enough for a new employee to be able to explain to anyone.

Writing Content Your Audience Will Actually Read and Understand

According to the Center for Plain Language - which has the goal to help organizations “write so clearly that their intended audience understands what they are saying the first time they read or hear it” - there are five steps to plain language:

  1. Identify and describe the target audience: “The audience definition works when you know who you are and are not designing for, what they want to do, and what they know and need to learn.”
  2. Structure the content to guide the reader through it: “The structure works when readers can quickly and confidently find the information they are looking for.”
  3. Write the content in plain language: “Use a conversational, rather than legal our bureaucratic tone … pick strong verbs in the active voice and use words the audience knows.”
  4. Use information design to help readers see and understand: Font choice, line spacing, and use of graphics help break up long sections of text and increase the readability score.
  5. Work with target user groups to test design and content: Ask readers to describe the content and have them show you where they would find relevant content.

As an illustration, here is a before and after comparison of the AARP Financial policy on giving and receiving gifts:

In reading the “before” example, my eyes immediately glazed over and my mind began to wander until the mention of “courtesies of a de minimus ... “ Did the guy who wrote that go home that night to his family and instruct his kids, “you will need to consume a courtesise of a de minimus amount of broccoli if you want videogame time after dinner”?  I sure hope not!

On the “after” example, notice the change in line spacing, switching of font and use of bullet points.  Overall the presentation is a lot more conversational and less formal.  It also has a call to action in the title starting with two verbs “give and accept …”

I’d add as the 6th step to remember K.I.S.S. - Keep It Simple Stupid!  You get a few seconds to grab your audience’s attention and only a few more minutes to keep it. 

As a content editor, I was feeling proud of myself when I distilled 146 pages of confusing policies, procedures and “how to” down to 14 pages over the course of several weeks.  But when I mentioned this to my wife, she said “you are going to make them read 14 pages?!?” 

So I looked at it a few days later with fresh eyes and realized I could condense it down again to two pages by making it more of a table of contents with a brief description of each bullet point and then include links after each section if employees wanted to learn more, and I was able to retain a font size of 14 and plenty of white space.

In reading the two pages, people would understand what was expected of them and could easily learn more - but only if they were interested.

Write policy in language a new employee will quickly understand and be thoughtful in how much you present to employees on their first day, week, and month.


Document Readability is How You Show Your People Love - And Soon To Be the Law In the EU

Speaking more in terms of content marketing, VisibleThread author “Fergle” quotes Neil Patel, columnist for Forbes, Inc, as stating “content that people love and content that people can read is almost the same thing.”  Yet, as Fergle points out, “a lot of content being created is not the stuff people love. Or read.”

“Content that people love and content that people can read is almost the same thing.”

Writing content with the aim of it being easy to read as something people love may seem a bit altruistic.  But for information regarding data privacy, it is also soon to be the law - at least in the EU and for any international policy which would reach an EU resident.  On May 28th of 2018 the General Data Protection Regulation (GDPR) goes into effect.  From the GDPR :

“The principle of transparency requires that any information and communication relating to the processing of those personal data be easily accessible and easy to understand, and that clear and plain language be used.”

There are a number of ways to measure readability ease and grade level of your content, and a good communications expert will be able to help you identify the proper tools.

Scores are a good benchmark, but don’t forget the most important resource for feedback - your potential audience! 

Buy them lunch, have them come and review your plan and provide their feedback.  Bring them back in later when you have content to review and provide an environment where they can be brutally honest - again a communications expert outside of your department will help provide a bit of a buffer and allow your audience to be open, honest, and direct.

But don’t just write policy to comply with due diligence or for policy’s sake - be sure it is part of the company culture, easy to search, and placed where and when your employees or customers will need it.  When there are shared outcomes between compliance and how employees operate, policy is integrated and effective. 

Timing is Important

Think of ways to break down your policy content not just by audience, but by timing and when the information will actually be relevant.

In retail, the term “point of sale” refers to the checkout process - when taxes, final cost and payment are all settled.  The placement of “last minute items” at the POS is very carefully, and competitively assigned only to items with a high ROI measured by the amount of inches each item takes up on the limited shelf space.  This careful placement has also been adopted to the online marketplace when you add an item to your shopping cart and a prompt arises to add additional items others have also purchased with your item.

This same methodology in thinking should be applied to where - and when - you introduce your policies to your audience.

We made the mistake for years of pushing our travel safety program and policies for everyone during new hire orientation when only about half of the population traveled and most of them wouldn’t be traveling for several weeks or months.  It made a lot more sense to move the travel policies to the travel booking page.

If you only give out corporate credit cards to Directors and above, there is no sense pushing policies on spend limits to the global population.  It makes a lot more sense to push the policy when someone is applying for the card and as a reminder each time their credit card expires and they are being issued a new one.

Your audience will appreciate only being told what they need to know when they need the information and will be more likely to not only retain the information, but to comply!


    For similar content on our Growth Blog, click here


Know How You Will Measure Successful Outcomes

Perhaps the most important question to ask when designing policy is “how we will know we are successful?” 

Having good policy written in a clear and concise manner and stored in an easy to find location is still a very passive approach.  Good policy should evolve as your company evolves and should be flexible and realistic to business, customer, and employee needs.  It must be modeled by company leadership and hold true to the daily actions of your company.

Tests at the end of annual compliance training are only a “check the box” measure of compliance.  Think back to how much you actually learned - or, better yet, retained - the last time you were subjected to hours of compliance training!

If metrics cannot support that your policies are known and followed, then you need to re-evaluate the purpose of your policies and if they are contributing to the benefit of your employees and customers or just ticking compliance boxes.

While compliance is important, compliance alone does not make for better business practices or a competitive edge.  Effective, measurable compliance protects your employees and provides value to your customers.

Getting Started

Subject-matter experts are often too close to the policies to be objective.  A little tough love is needed and it is best to bring in experts in marketing and communications who will not be biased to the content, but biased to the reader who is the intended audience. 

A good communications plan will cover the following:

  • Be clear on the desired behavior the policy is to encourage and enforce - and that behavior is streamlined with the overall company purpose
  • Identify the target audiences and each of their self-interests
  • Outline which channels each audience is receptive to (email/print/video, etc.)
  • Identify the inside jargon and language styles needed
  • Decide when and where each audience will want to find relevant information
  • Plan how often policies will be reviewed - and include as many stakeholders as possible in the review process
  • Decide how implementation of policies and compliance to the policies will be measured

Only AFTER the communications plan is agreed upon - with plenty of input from representatives of the target audiences - should the content review begin.  Otherwise the temptation from subject-matter experts will be to tell people everything they know.


Pulling it All Together

Poorly written policies that are difficult for employees to search or find do little to meet the mission of policy: to provide a consistent approach to how your company does business and satisfies regulatory compliance.  Policies on their own do not make for good operations or guarantee overall success.  Remember the true test of policies is not whether they exist, but if they are tightly aligned and incorporated into daily operations, how they contribute to the success of your employees and customers, and if their effectiveness can be measured in a tangible way.

—-

Experiencing growing pains?  AKF is here to help!  We are an industry expert in technology scalability and due diligence.  Put our 200+ years of combined experience to work for you today!

Get this article and others like it by signing up for our newsletter.

Subscribe to the AKF Newsletter

Permalink

How to write concisely

April 11, 2018  |  Posted By: Geoffrey Weber

The Three Sentence Rule

Variations of the Three Sentence Rule have been around for a long time.  The differences are multiplicative but the base rule is a useful and often necessary tool to teach concision to the wordy.

Say what you need to say in THREE sentences, or less.

Anyone who has been through flight school learns how to be concise on the radio.  Who are you?  Where are you?  What do you want?  That happens to be three sentences.  “Palo Alto Tower, Cessna 15957X; 5 miles southwest of SLAC with information Echo; request landing.”  The need to be precise, accurate and speedy is a requirement at a tower as busy as Palo Alto tower.  Controllers have no patience because there are 12 other aircraft waiting to communicate.

As technologists, we are generally rewarded for producing details, the more the better.  Engineers have to be obsessed with substance; their work is about precision and there are no shortcuts when it comes to building complex tools.  It wouldn’t make any sense to try and distill how Cassandra compares to a relational database, in three sentences, in a room filled with colleagues, at a meet-up.

But what if our CEO asks us about Cassandra?  How can we possibly explain to someone who is just a wee bit tech-illiterate the differences between two very different data stores? Moreover, why on earth would we try and distill that down to three sentences??  Let’s start at the beginning… before there were databases there were Hollerith Cards…

Lack of brevity is a death sentence to any technologist who finds themselves interacting with non-technologists on a regular basis.  We see this as a common anti-pattern for CTOs; some never learn the difference between a novel, a paragraph, or sentence and why each has utility.

The controller in the tower could care less about why we’re in an airplane today, that we’re stopping at the restaurant for the traditional $100 hamburger and that we need to be home for dinner tonight:

  • Who are you?
  • Where are you?
  • What do you want?

Rewind:

  • Cassandra is a new database technology.
  •  
  • It’s very different than what we use today.
  •  
  • It will lower costs in the next 12 months.
  •  

That is the CEO-version of Cassandra in three sentences.  “What is it called, why should I remember it, what does it do for me?”

At AKF Partners, we believe that technology executives need to start practicing a version of the three sentence rule as soon as they transition into their first leadership role.  Specialists in Operations roles have an advantage because of the daily chaos and need for ongoing communications: “Customer sign-in unavailable for 15 minutes; 100% of our customers are impacted for 15 minutes; we are restarting the service for 100% operations in 10 minutes—update in 20 minutes.”

  • What happened?
  • What is the impact?
  • When is it going to be fixed?

There’s a practical reason for such precision: most CEOs are consuming information on tiny screens, sometimes over really bad internet (Detroit Airport) at 2 in the morning, and news also just arrived about a sales crisis in Europe, there’s a supply-chain issue in India, and the Wall Street Journal is doing a feature about the product that’s going live next week.  If we mentioned Hollerith, or even thought about it for a second, we’re Exploring Alternative Employment.  If they have a moment to breath, they can ask for more detail.  Or maybe next week.

Sometimes we’re required to communicate when there’s no answer.  Try this:

  • Impact update,
  • Standard procedures failed, assembled SWAT,
  • Updates every 15 minutes until resolution.

An equally important rule for executives is the “No Surprise Rule” (stay tuned) and zero sentences are as fatal as 4 sentences at the wrong time.  Keeping a CEO waiting for 2 hours until root cause is determined is stupid.

The final place to consider the Three Sentence Rule is the boardroom itself.  Most boards members are not going to read through the 200 page board deck, and our ten minutes to discuss the Cassandra project is unlikely to resonate with most of the attendees.  Up and coming executives understand the need for absolute precision.  Steve Jobs could do it with a single slide:

Three sentences, if you count the background gradient as a sentence. 

For the Board:

  • We’re introducing new technology next fiscal year.
  • It’s called Cassandra.
  • A year from now I will demonstrate how it increased EBITDBA by $2M.

In summary:

  • Anything can be explained in 3 sentences
  • Even concepts so fantastic they seem magical
  • If you don’t believe me, Books in 3 Sentences

At AKF Partners, we can help with mentoring, coaching and leadership training. 

Subscribe to the AKF Newsletter

Permalink

Microservices for Breadth, Libraries for Depth

April 10, 2018  |  Posted By: Marty Abbott

The decomposition of monoliths into services, or alternatively the development of new products in a services-oriented fashion (oftentimes called microservices), is one of the greatest architectural movements of the last decade.  The benefits of a services (alternatively microservices or micro-services) approach are clear:

  • Independent deployment, decreasing time to market and decreasing time to value realization– especially when continuous delivery is employed.
  • Team velocity and ownership (informed by Conway’s Law).
  • Increased fault isolation – but only when properly deployed (see below).
  • Individual scalability – and the decreasing cost of operations that entails when properly architected.
  • Freedom of implementation and technology choices – choosing the best solution for each service rather than subjecting services to the lowest common denominator implementation.

Unfortunately, without proper architectural oversight and planning, improperly architected services can also result in:

  • Lower overall availability, especially when those services are deployed in one of a handful of microservice anti-patterns like the mesh, services in depth (aka the Christmas Tree Light String) and the Fuse.
  • Higher (longer) response times to end customers.
  • Complicated fault isolation and troubleshooting that increases average recovery time for failures.
  • Service bloat:  Too many services to comprehend (see our service sizing post)

The following are patterns companies should avoid (anti-patterns) when developing services or microservices architectures:


The Mesh

Mesh architectures, where individual services both “fan out” and “share” subsequent services result in the lowest possible availability. 


Deep Series

Services that are strung together in long (deep) call trees suffer from low availability and slow page response times as calculated from the product of each service offering availability. 


The Fuse

The Fuse is a much smaller anti-pattern than “The Mesh”.  In “The Fuse”, 2 distinct services (A and B) rely on service C.  Should service C become slow or unavailable, both service A and B suffer.


Architecture Principle:  Services – Broad, But Never Deep

These services anti-patterns protect against a lack of fault isolation, where slowness and failures propagate along a synchronous path.  One service fails, and the others relying upon that service also suffer. 

They also serve to guard against longer latency in call streams.  While network calls tend to be minimal relative to total customer response times, many solutions (e.g. payment solutions) need to respond as quickly as possible and service calls slow that down.

Finally, these patterns help protect against difficult to diagnose failures.  The Xmas Tree pattern name is chosen because of the difficulty in finding the “failed bulb” in old tree lights wired in series.  Similarly, imagine attempting to find the fault in “The Mesh”.  The time necessary to find faults negatively effects service restoration time and therefore availability.

As such, we suggest a principal that services should never be deep but instead should be deployed in breadth along product offering boundaries defined by nouns (resources like “customer” or “sales”) or verbs (services like “search” or “add to cart”).  We often call this approach “slices instead of layers”.
How then do we accomplish the separation of software for team ownership, and time to market where a single service would otherwise be too large or unwieldy?

Old School – Libraries!

When you need service-like segmentation in a deep call tree but can’t suffer the availability impact and latency associated with multiple calls, look to libraries.  Libraries will both eliminate the network associated latency of a service call.  In the case of both The Fuse and The Mesh libraries eliminate the shared availability constraints.  Unfortunately, we still have the multiplicative effect of failure of the Xmas Tree, but overall it is a faster pattern.

“But My Teams Can’t Release Separately!”

Sure they can – they just have to change how they think about releasing.  If you need immediate effect from what you release and don’t want to release the calling services with libraries compiled or linked, consider performing releases with shared objects or dynamically loadable libraries.  While these require restarts of the calling service, simple automation will help you keep from having an outage for the purpose of deploying software.


AKF Partners helps companies architecture highly available, highly scalable microservice architecture products.  We apply our aggregate experience, proprietary models, patterns, and anti-patterns to help ensure your products can meet your company’s scale and availability goals.  Contact us today - we can help!

Subscribe to the AKF Newsletter

Permalink

Security Considerations for Technical Due Diligence

April 10, 2018  |  Posted By: Greg Fennewald

It seems as though a week cannot go by without news reports of yet another data breach at a large, recognizable company.  One wonders what has been compromised but not yet detected or announced. 

Security issues are perceived far differently than other technology issues.  Consider an example of “Dilly Dilly Fidget Spinners has hard coded IP addresses in their code base” – most people would infer little if anything from that fact, while a minority would shake their heads and feel nauseous.  On the other hand, “Dilly Dilly Fidget Spinners suffered a data breach affecting thousands of customers” is likely to have a negative perception from everyone who hears about it.  The public sensitivity to all things security warrants a thorough investigation of security practices and incidents prior to any investment.

What should a potential investor look for in regard to information security during a due diligence effort?  The answer to that question will vary widely based on the market segment of the potential investment, but there are some common considerations for information security

Common Security Considerations

1. Fit the Risk

Security posture should fit the risk a company faces.  A company providing financial services or healthcare has a far higher risk to manage than a company involved in consumer product pricing and availability.  The security policies, regulatory compliance and certifications, and operational practices should fit the risk.  Going beyond the appropriate degree of security adds cost and may not make business sense but is far superior to inadequate security.

A security program that fits the risk profile for the company can be a business enabler.  Security programs consume time and cost money – establishing the right fit and balance can conserve resources.  Alternatively, a poor fit can add drag to a company and damage the business.  Consider industries that have a strong reputation for security and face significant regulatory requirements, industries such as financial services and insurance.  An experienced security professional with a banking background moves to a telematics company and is determined to bring bank level security to his new role.  The telematics company deals with route optimization and fleet maintenance management.  It does not process credit card payments or store PII.  Bank level security would be a horrible fit that adds cost without benefit and ultimately damages the culture.

2. Security Minded Culture

Security awareness and accountability should be part of the culture.  Well written policies do not accomplish much if they are not internalized and emphasized by leaders.  Technology leaders must treat security in the same manner as they treat availability, quality of service, and engineering productivity - by establishing transparent goals and objective metrics by which those goals are measured.

An excellent resource for security awareness training is the OWASP Top 10 Application Security Risks list.  The top 10 list is revised periodically as security threat vectors morph.  The top three risks from the 2017 list are injections, broken authentication, and sensitive data exposure.  More information can be found here.

3. Validation via Recurring Testing

Recurring testing is a hallmark of successful security programs.  Areas to test include employee security policy training, 3d party network penetration tests, static code vulnerability testing, and drills to rehearse information security policies such as a security incident response plan.  Testing validates the policies and practices are effective and part of the company’s culture.

QA automation is needed for agile product development that seeks rapid iteration and market discovery.  75% code coverage or greater is recommended.  Incorporating automated security testing into the overall testing program is a smart move.

4. Cover the Basics

Basic security hygiene items that should be considered table stakes today include role-based access with audit trails, closing server ports by default and opening them by exception, segregating networks, logging production access, and encrypting data at rest.  None of these actions are particularly difficult or expensive.  Implementing them demonstrates security awareness and commitment.  Controlling who can access data in a taciturn server farm, logging that access, and encrypting the data is a pretty good start to effective security.

How AKF Can Help

AKF Partners has performed hundreds of due diligence efforts over the last 10 years and is comprised of technology professionals that have walked the walk at widely recognized companies such as eBay, PayPal, and General Electric.  Our security expertise comes from living the reality of technology, not an auditing course.

Subscribe to the AKF Newsletter

Permalink

Technical Due Diligence: Did We Get It Right

April 4, 2018  |  Posted By: Geoffrey Weber

Technical Due Diligence: Did We get it Right?

AKF Partners have performed 100s of technical due diligence engagements on behalf of strategic investors, private equity, venture capitalists and others.  As such, we have amassed a significant repository of data, of patterns and anti-patterns, and the personal characteristics of successful and unsuccessful executive teams. Moreover, our teams have been on the “other side” of the table countless times as executives for companies being acquired or looking to acquire. 

It’s not rare, however, when a potential client asks: how accurate is your technical due diligence?  How can I trust that 8 hours with a company is enough to evaluate their technology stack?  We love this question!

Due diligence, whether financial or technical is about assessing the risk the investor or acquirer is taking on by investing money into a company.  It’s less about trying to identify and measure every gap and more about understanding where significant gaps exist relative to the investment thesis, calculating the risk to which those gaps expose the investor, and estimating the cost of closing the most critical gaps.

Due diligence is remarkably similar to playing poker: not any one player at the table has complete information about the cards remaining in the deck and the cards held by each player.  Great poker players combine an ability to calculate probability nearly instantly with respect to the possible combination of cards as well as an ability to read the other players; looking for “tells” that inform the player as to whether their opponent is bluffing or playing with a pair of Aces heading into the “flop.”  Great poker players seamlessly combine science and art.

At AKF, we’ve developed a formal checklist of questions around which we build our due diligence practice.  In the big picture, this includes evaluating People, Technology, and Process. Our experience suggests that companies cannot build the right technology without the right people in charge, with the right individual contributors displaying the right behaviors to maximize success.  Further, building reliable performance and predictable scalability (process, in other words) is of little value if they’re built on top of a weak technology stack.

People >> Technology >> Process

Read more about technical due diligence and AKF Partners here.

So how do we know we’re right?

First and foremost we need to understand the backgrounds, responsibilities and behaviors of the team present in the room with us.  A dominate CEO that answers even the most technical questions, shutting out the rest of the team, is a red flag. We might ask specific questions to others and ask the CEO to let us listen to what others at the table have to say.  If we’re still road-blocked, then our assessment will be very specific about the behavior of the CEO and we might ask for additional time without the CEO present. Another red flag: a CTO answering a technical question while the VP of Engineering’s face turns purple; an engineering manager choking a piece of paper to death while listening to the CTO review architectural principles or a senior leader refusing eye contact. . . the list of cues is nearly endless.  We’ve seen all of them.

Seeing red flags early in an engagement is wonderful, and it allows us time to adjust the agenda on the fly.  If we’re hearing something that sounds implausible, we will focus on the area in question and fire off repeated questions.  Nothing helps like asking the same question three times in three different ways at three different times during an engagement.  We can then judge the quality of the answers by the variety of answers.

Everything is Perfect

The biggest red flag of all: when things are too good.  No down time; the architecture scales across all three axes of the AKF Scalability cube without human interaction;  obscure or bleeding-edge technologies implemented without significant issues, and, most of all, always on time/under budget.  Each of these red flags highlights an area that begs for further digging. This is a good time to start asking for real data. Let’s see the logs in real-time; demonstrate that 2-factor works as described and let’s watch a build.  This is a good time to get familiar with Benford’s Law which states “in many naturally occurring collections of numbers, the leading significant digit is likely to be small.”  Math is useful here just as it was for Bernie Madoff and just as it is for the poker player seeing a 5th Ace being dealt.

Assessments with significantly dysfunctional teams are a little easier and our assessments reflect this by candidly reporting that we could not validate certain facts or statements due to these dysfunctions.  Either we come back and dig more deeply or we meet with a different group of people with the investor.

The Truth about Broken Things

While we occasionally see these anti-patterns, we’re much more likely to see positive patterns:

  1. A leader that is capable of expressing a vision and behavioral standards (a.k.a. culture) and then seeing the echoes of those standards from more junior members of the team.  Huge green flag – maybe even a checkered flag for the finish line. 
  2. Finding flaws in the architecture and watching the technical team members finally see the flaw themselves and begin to energetically discuss ways to resolve the flaws.
  3. A willingness to listen to different ideas and a recognition that the technology team may not have all of the answers. 

Example: Dozens of times we’ve heard teams say their monolithic database is fine, peak load is 10%, no worries here.  But there is no answer for how to handle a 10x event that will take the database down.  There are plenty of examples of surprise 10x events: think of serving ads online when a famous singer or actor suddenly dies. 

A willingness to admit that such a possibility exists coupled with a desire to build load generating tools and capacity planning processes is another green flag.  Properly building these tools and then designing a credible scalability plan will absolutely reduce future operation risks. Those are critical cues we’re looking for during a due diligence session.

The best green flag of all:  “I don’t know.” This is not an admission of failure, this is an admission of the fact that our systems are so complex, it’s impossible for a single person to understand all of the dependencies.  “I don’t know” followed by “let me go get someone who does know” is one of the biggest green flags we see.  “I don’t know” is a green flag – “I always know” (hubris) is a red flag.  We seek teams that seek truth – not stories.

An important dimension to explore is the maturity of the company versus other companies in their industry as well as other companies of their size and age compared against the universe of companies.  A company of 10 people that has invested significant time focusing on scale has likely not spent enough time building out a product with the necessary features to capture the market. A private company that has existed for 15 years with > $100M in sales but has no capability to perform capacity planning has underinvested in architecture.  We’ve seen companies in all stages of maturity and size and our data helps structure our diligence recommendations: we can compare virtually any company against our universe of experience.

Technical due diligence is not about calculating a risk number to 6 digits of precision.  It’s not just science, and it’s not just art: it’s both.  The art is in ensuring alignment of people, process and technology - particularly the people component.  The science is in applying a data set garnered over 11 years to help identify where the largest issues do (or are likely) to exist. The end result is a report that outlines risks and plausible solutions written by a team of people who’ve had a combined 200 years experience doing the real work.

 

Subscribe to the AKF Newsletter

Permalink

Societal Consequences of the Uber Autonomous Vehicle Death

March 22, 2018  |  Posted By: Marty Abbott

It is sad and unfortunate, but the inevitable has finally happened; we’ve suffered our first death from an autonomous vehicle.

The Uber Autonomous Vehicle fatality in Tempe is odd, as there are several contributing factors:

  1. The pedestrian was crossing the street at night outside of a cross walk.  Jaywalking, as it is commonly called, is against city ordinances in Tempe, AZ.
  2. The pedestrian evidently didn’t see the car’s lights, or hear the car approaching.
  3. The safety assistant in the vehicle, who was meant to help avoid such crashes by taking control of the car, was not paying attention at the time of the crash and apparently had a prior felony conviction (raising the question of how she was hired in the first place).
  4. The vehicle’s collision avoidance system failed somehow to either detect the individual or take the appropriate action upon detection.

These factors raise several immediate questions regarding who (or what) is to blame for the incident:

  1. Who’s at fault?  The jaywalking pedestrian?  The safety assistant for negligence?  Uber for potential vehicle failures?
  2. If either the assistant or Uber bear the blame, does this rise to the level of a crime?  Vehicular manslaughter for instance?

Technology Advancements and the Benefits They Bring Almost Always Have a Price

First, to be clear, technical advances very often come at peril to human life. 

Advances in both flight and space travel have both resulted in several deaths over the last century – the manner of death being impossible before the advancement. 

While per capita deaths associated with automobiles are lower today than they were for horse related transportation in 1900 , the fact remains that the introduction of the automobile increased fatalities for a number of years through at least 1930

Power transmission to homes may be linked to leukemia in children.  While the jury is out regarding whether smart phones cause brain cancer, the “selfie” phenomenon they’ve enabled has been implicated in some deaths .

Even seemingly harmless entertainment devices such as televisions have caused fatalities.

We Have a Lot to Gain by Moving Forward

While sad, and in this particular case completely avoidable (the pedestrian could have crossed at a cross walk, could have avoided the vehicle, and the safety assistant could have been paying attention), this should not halt the advancement of research in this area.  Yes, we should pause briefly to understand and correct the cause.

But we also need to realize that the benefits to society are immense and cry for rapid progress and adoption:

  1. A likely overall reduction in vehicular accidents and fatalities as the technology progresses and gains adoption.  Driver attention problems (texting, cell phones) go away.
  2. Lower insurance rates as overall vehicle related claims decline.
  3. An elimination of alcohol related driving crimes and accidents.
  4. A reduction in the overall cost of living for many Americans who need flexible transportation in metro areas, but struggle to afford a vehicle.
  5. A reduction in the cost of living for American families who may only need one car if it could return home for other duties, but must buy two because each car remains with its owner wherever he or she goes.
  6. A reduction in vehicle related pollution and its associated climate effects as vehicle ownership declines and affordable autonomous ride sharing increases.
  7. Fewer traffic delays as autonomous vehicles select better routes, lowering commute times.
  8. Less road congestion as fewer vehicles compete for the limited infrastructure.
  9. Lower infrastructure costs longer term, as less road maintenance is required and one day the need for street lights go away.  Taxes similarly drop.
  10. Lower local taxes as the need for traffic related law enforcement declines over time.

Unfortunately Implementation and Adoption Will Likely Slow Down

While the benefits of autonomous vehicles are clear, Autonomous Vehicle (AV) deaths will provide fodder for special interest groups to slow down AV legalization:

  1. Several unions, including those related to livery services, will strive to keep their member base employed and either sue to stop implementation or fund political action committees to influence legislation biased against AVs.
  2. Because the secondary car market is likely to see a significant decline in demand (who needs a used car if one can get a ride service easily at lower overall cost?), car dealerships will fund PACs to similarly sway politicians.
  3. Automobile manufacturers who today see a near term opportunity with Autonomous Vehicles, may determine that vehicle sales overall in the new car industry could decline and join existing PACs
  4. Other unions and businesses reliant upon vehicle ownership for employment of some (or all) of their member base or for their very existence (car washes, gas stations, “Big Oil”, police departments, maintenance facilities, etc) may also join PACs.

While there are many societal benefits in adopting the AV, there are certain interest groups which have a lot to lose with their wide-scale adoption.  These interest groups will almost certainly mobilize and look to stall progress and in so doing keep society from reaping the benefits.

Summary

Death associated with technical advancement is nothing new and while we should strive to limit it, we must expect it.  While we stand to gain a great deal from autonomous vehicles, we must be wary of entrenched interests that may attempt to use events like this one to block their adoption.

 

Subscribe to the AKF Newsletter

Permalink

Balancing Team Autonomy and Anarchy

March 19, 2018  |  Posted By: Daniel Hanley

Agile teams often seem to be confused with the meaning and value of autonomy within a team.  Is it the autonomy to decide review what path to take to accomplish a goal?  Is it the autonomy to choose what tools and processes they will employ to accomplish that goal?  Is it both?  To answer this, we need to first unlock the riddle of where autonomy drives value creation and where and how it destroys value within an enterprise.


Balancing Team Autonomy and Anarchy
As an aid to this discussion, consider the following analogy:  We have the autonomy to determine what roads and paths we will take to a destination when driving a vehicle, but we are governed by speed limits, right of way (which side to drive on, stop signs, stop signals and other road signs), emission standards, and both vehicle and personal licensing.  Put another way, we are completely empowered to determine WHAT path we take, and WHY we take it.  We are much more limited in HOW we get to a place (on road, off road, speed, only licensed vehicles, etc) and WHO can do it (only sober, licensed drivers).  How does this apply to a fully autonomous technology team?  The value-creating autonomy within a team deals with WHAT paths a team takes to accomplish a goal and the reasoning as to why that approach is valuable.  A failure to provide structure and rules for HOW something is accomplished through architectural principles, coding standards, development standards, etc. can start to escalate the costs of development and cause repeating failures maintenance.  Not sharing best practices through standards means that teams are bound to repeat mistakes and cause customer interruptions.  While there is a narrow line between autonomy and anarchy, the difference in their effect on an organization is significant.

Consider the matrix below which differentiates the notion of decision making (What path gets taken to a result and Why that path is taken) from governance (the rules around How and Who should do something).  Autocracy (complete top down decisions and governance) occupies the upper left and the spectrum moves toward Anarchy in the bottom right (bottom-up decisions with little to no alignment to organizational goals and vision, and a complete lack of governance and best practice adherence).  Autonomy (as in an autonomous agile teams) exists in the upper right quadrant of the AKF’s Team Autonomy 2x2 with a high level of innovation because teams share best practices through governance but are free to experiment with paths to achieve desired outcomes.  Freedom in deciding what path to take to a destination or desired outcome is valuable, however it should be balanced with the governance to validate that past lessons learned (be they security related, operationally related, or simply development related) are properly applied. 

Similarly, we can plot the decision making authority for “WHO and HOW” something gets done as compared to the “WHAT (paths) and WHY (a path is taken)”.  In so doing, autonomous agile teams exist in the realm where leadership enforces standards, but the team is the deciding authority for what path to take.  Anarchy (bad) exists when the team makes all decisions as learnings codified within shared standards are not applied.  Warring tribes emerge when leadership defines a path but leaves the decision of who and how to the teams (similar to the “bad leadership” example in the prior matrix).  Autocracy exists when the leadership is responsible for all decisions.  The reason we identify this as “medium” innovation is that we assume the leadership is at least experientially diverse.  But innovation is low relative to autonomous teams because we aren’t tapping the innovation capabilities of the teams themselves - the intellectual capital in which we’ve so heavily invested.


How to Find the Balance of Freedom and Governance
Teams should be built around the suggestions from AKF’s white paper on Organizing Product Teams for Innovation: small, cross functional teams built around a service, who are empowered to be autonomous and work independently on their own.  Autonomy should be defined within the rules of the organization, inform the organization’s architectural principles, and drive adherence through leadership.  This is by no means a notion that the organization should avoid cross functional empowered teams!  As we state in our white paper, Organizing Product Teams for Innovation, “We still have executives developing strategy, functional teams (product management, etc.) defining subservient strategies and roadmaps.  But the primary identities of these folks are embedded within the teams that implement these solutions.”

It is easy to confuse the notion of empowerment and autonomy.  Empowerment is an action of delegation coupled with the assurance of resources and tools to complete the desired outcomes to the delegated party.  It is only through empowerment that autonomy can be achieved within an organizational hierarchy.  Further, it is only through empowerment that autonomous agile teams can be established.  But both empowerment and autonomy need to have rules governing their action or issuance.  Specifically, an agile team is empowered to be autonomous with the following constraints:  following development protocols and standards, adhering to architectural principals, adhering to established best practices regarding test coverage, etc. and ensuring that you achieve the “non-functional requirements” codified within the Agile Definition of Done (more on this later).


Like this article?  Share it with friends and subscribe to the newsletter here.

Have your autonomous technology teams been free to make decisions that do not align with the vision of the company?  Are you fearful of switching to Agile because of the rampant anarchy they will exhibit?  AKF Partners has over 200 combined years of experience helping companies ensure that their organizations and architectures are aligned to the outcomes they desire.  We’d love to help you achieve the success you desire.

 

Subscribe to the AKF Newsletter

Permalink

SaaS Migration Challenges

March 12, 2018  |  Posted By: Dave Swenson

More and more companies are waking up from the 20th century, realizing that their on-premise, packaged, waterfall paradigms no longer play in today’s SaaS, agile world. SaaS (Software as a Service) has taken over, and for good reason. Companies (and investors) long for the higher valuation and increased margins that SaaS’ economies of scale provide. Many of these same companies realize that in order to fully benefit from a SaaS model, they need to release far more frequently, enhancing their products through frequent iterative cycles rather than massive upgrades occurring only 4 times a year. So, they not only perform a ‘lift and shift’ into the cloud, they also move to an Agile PDLC. Customers, tired of incurring on-premise IT costs and risks, are also pushing their software vendors towards SaaS.

But, what many of the companies migrating to SaaS don’t realize is that migrating to SaaS is not just a technology exercise.  Successful SaaS migrations require a ‘reboot’ of the entire company. Certainly, the technology organization will be most affected, but almost every department in a company will need to change. Sales teams need to pitch the product differently, selling a leased service vs. a purchased product, and must learn to address customers’ typical concerns around security. The role of professional services teams in SaaS drastically changes, and in most cases, shrinks. Customer support personnel should have far greater insight into reported problems. Product management in a SaaS world requires small, nimble enhancements vs. massive, ‘big-bang’ upgrades. Your marketing organization will potentially need to target a different type of customer for your initial SaaS releases - leveraging the Technology Adoption Lifecycle to identify early adopters of your product in order to inform a small initial release (Minimum Viable Product).

It is important to recognize the risks that will shift from your customers to you. In an on-premise (“on-prem”) product, your customer carries the burden of capacity planning, security, availability, disaster recovery. SaaS companies sell a service (we like to say an outcome), not just a bundle of software.  That service represents a shift of the risks once held by a customer to the company provisioning the service.  In most cases, understanding and properly addressing these risks are new undertakings for the company in question and not something for which they have the proper mindset or skills to be successful.

This company-wide reboot can certainly be a daunting challenge, but if approached carefully and honestly, addressing key questions up front, communicating, educating, and transparently addressing likely organizational and personnel changes along the way, it is an accomplishment that transforms, even reignites, a company.

This is the first in a series of articles that captures AKF’s observations and first-hand experiences in guiding companies through this process.


Don’t treat this as a simple rewrite of your existing product - answer these questions first…

Any company about to launch into a SaaS migration should first take a long, hard look at their current product, determining what out of the legacy product is not worth carrying forward. Is all of that existing functionality really being used, and still relevant? Prior to any move towards SaaS, the following questions and issues need to be addressed:

Customization or Configuration?
SaaS efficiencies come from many angles, but certainly one of those is having a single codebase for all customers. If your product today is highly customized, where code has been written and is in use for specific customers, you’ve got a tough question to address. Most product variances can likely be handled through configuration, a data-driven mechanism that enables/disables or otherwise shapes functionality for each customer. No customer-specific code from the legacy product should be carried forward unless it is expected to be used by multiple clients. Note that this shift has implications on how a sales force promotes the product (they can no longer promise to build whatever a potential customer wants, but must sell the current, existing functionality) as well as professional services (no customizations means less work for them).

Single/Multi/All-tenancy?
Many customers, even those who accept the improved security posture a cloud-hosted product provides over their own on-premise infrastructure, absolutely freak when they hear that their data will coexist with other customers’ data in a single multi-tenant instance, no matter what access management mechanisms exist. Multi-tenancy is another key to achieving economies of scale that bring greater SaaS efficiencies. Don’t let go of it easily, but if you must, price extra for it.

Who owns the data?
Many products focus only on the transactional set of functionality, leaving the analytics side to their customers. In an on-premise scenario, where the data resides in the customers’ facilities, ownership of the data is clear. Customers are free to slice & dice the data as they please. When that data is hosted, particularly in a multi-tenant scenario where multiple customers’ data lives in the same database, direct customer access presents significant challenges. Beyond the obvious related security issues is the need to keep your customers abreast of the more frequent updates that occur with SaaS product iterations. The decision is whether you replicate customer data into read-only instances, provide bulk export into their own hosted databases, or build analytics into your product?

All of these have costs - ensure you’re passing those on to your customers who need this functionality.

May I Upgrade Now?
Today, do your customers require permission for you to upgrade their installation? You’ll need to change that behavior to realize another SaaS efficiency - supporting of as few versions as possible. Ideally, you’ll typically only support a single version (other than during deployment). If your customers need to ‘bless’ a release before migrating on to it, you’re doing it wrong. Your releases should be small, incremental enhancements, potentially even reaching continuous deployment. Therefore, the changes should be far easier to accept and learn than the prior big-bang, huge upgrades of the past. If absolutely necessary, create a sandbox for customers to access new releases, but be prepared to deal with the potentially unwanted, non-representative feedback from the select few who try it out in that sandbox.

Wait? Who Are We Targeting?
All of the questions above lead to this fundamental issue: Are tomorrow’s SaaS customers the same as today’s? The answer? Not necessarily. First, in order to migrate existing customers on to your bright, shiny new SaaS platform, you’ll need to have functional parity with the legacy product. Reaching that parity will take significant effort and lead to a big-bang approach. Instead, pick a subset or an MVP of existing functionality, and find new customers who will be satisfied with that. Then, after proving out the SaaS architecture and related processes, gradually migrate more and more functionality, and once functional parity is close, move existing customers on to your SaaS platform.

To find those new customers interested in placing their bets on your initial SaaS MVP, you’ll need to shift your current focus on the right side of the Technology Adoption Lifecycle (TALC) to the left - from your current ‘Late Majority’ or ‘Laggards’ to ‘Early Adopters’ or ‘Early Majority’. Ideally, those customers on the left side of the TALC will be slightly more forgiving of the ‘learnings’ you’ll face along the way, as well as prove to be far more valuable partners with you as you further enhance your MVP.

The key is to think out of the existing box your customers are in, to reset your TALC targeting and to consider a new breed of customer, one that doesn’t need all that you’ve built, is willing to be an early adopter, and will be a cooperative partner throughout the process.


Our next article on SaaS migration will touch on organizational approaches, particularly during the build-out of the SaaS product, and the paradigm shifts your product and engineering teams need to embrace in order to be successful.

AKF has led many companies on their journey to SaaS, often getting called in as that journey has been derailed. We’ve seen the many potholes and pitfalls and have learned how to avoid them. Let us help you move your product into the 21st century.

Subscribe to the AKF Newsletter

Permalink

Managing Risk with Technical Due Diligence

February 20, 2018  |  Posted By: Greg Fennewald

You should not buy a home without an inspection by a licensed home inspector and you should not buy a used car without having a mechanic check it out for you.  Diligence - it just makes good sense.  Similarly, it is prudent to include technical diligence as part of the evaluation for a potential technology company investment.


Diligence Informs Risk Management

Private equity and venture capital firms typically evaluate many areas preceding a potential investment.  The business case, legal structure, competitive analysis, product strategy, financial audits and contractual landscape are all examples of diligence deemed necessary prior to an investment.  A company with a great product but three years left on an extremely expensive office lease will probably have a lower value.  Breaking the lease or living with it until the term expires means higher costs and thus lower EBITDA.  A hot start up with an inexperienced CFO that has run on cash-based accounting from day 1 and is rapidly approaching $6 million in annual revenue needs to move to accrual-based accounting.  That takes time and effort and possibly a talent search - this affects the value of the investment. 

But what about the technical underpinnings of the product itself?  A company with a solitary production database and a marketing analyst with access to directly query that database is likely headed for performance and availability incidents.  Single points of failure create a high probability of non-availability.  Solutions that don’t allow for seamless and elastic scalability may run into either capacity or cost of operations problems. 

Preventing these incidents and altering the conditions that enabled them to exist takes time and effort.  All of these assessment areas boil down to risk management.  Further, understanding the cost of fixing these solutions helps a company understand their true cost of investment.  Your investment includes not just the “PIC” or capital that you put into the company - it also includes all the costs to ensure continuing operations of the product that enables that company.  A comprehensive diligence including technical diligence will prepare the investor to make an informed business decision - know the risks and adjust the value proposition accordingly.

Technology Risk Areas

Technology risks can be grouped into four broad areas - Architecture, Process, Organization, and Security.  Each area has several subordinate themes.

Architecture - subordinate themes are availability, scalability, cost control.


• Commodity hardware - Corollas, not Carreras
• Horizontal scalability - scale out, not up
• Design for monitoring - see issues before your customers do
• N+1 design - everything fails eventually
• Design for rollback - minimize the impairment
• Asynchronous design - stateless systems

Process - subordinate themes are engineering, operations, and problem management

• Product management - a product owner should be able add, delay, or deprecate features from an upcoming release
• Metrics - development teams should use effort estimation and velocity measurement metrics to monitor progress and performance
• Development practices - developers should conduct code reviews and be held accountable for unit testing
• Incident management - incidents should be logged with sufficient details for further follow up
• Post mortem - a structured process should be in place to review significant problems, assign action items, and track resolution
• PDLC - the Product Development Lifecycle should align with the company’s desires to be customer driven (not desirable in most cases) or market driven (resulting in the highest returns and fastest saturation of any market)


Organization - subordinate themes are PDLC (Product Development Lifecycle) structure, product alignment and team composition

• Product or Service Alignment - cross functional teams should be aligned by product or service and understand how their efforts complement business goals
• Agile or Waterfall - if “discovering” the market or choosing the best possible product for a market then Agile is appropriate - if developing to well defined contracts then waterfall may be necessary.
• Team composition - the engineer to QA tester ratio should ideally exceed 3.5:1.  Significant deviations may be a sign or trouble or a harbinger of problems to come
• Goals - measurable goals aligned with business priorities should be visible to all with clear accountability

Security - subordinate themes are framework, prevention, detection and response

• Framework - use NIST, ISO, PCI or other regulatory standards to establish the framework for a security program.  The standards do overlap, think it through and avoid duplication of effort.
• Policies in place - a sound security program will have multiple security related policies such as employee acceptable use, access controls, data classification, and an incident response plan.
• Security risk matrix - security risks should be graded by their impact, probability of occurrence, and controlling measures
• Business metrics - analysis of business metrics (revenue per minute, change of address, checkout value anomalies, file saves per minute, etc) can develop thresholds for alerting to a potential security incident.  Over time, the analysis can inform prevention techniques.
• Response plan - a plan must be in place and must have regular rehearsals.

Technology Cost Impact on Investment Value

Technology costs can have a significant impact on the overall investment value.  Strengths and weaknesses uncovered during a technical diligence effort help the investor make the best overall business decision.

Technology costs are normally captured in 2 areas of the income statement, cost of revenue (production environment and personnel) and operating expenses (software development).  Technology costs can also affect depreciation (server capital purchases) and amortization (pre-paid licensing and support).  These cost areas should be reviewed for unusual patterns or abnormally high or low spend rates.  It is also important to understand the term of equipment purchase, software licensing, and support contracts - spend may be committed for several years.

Cost Cautions - tales from the past

• Support for production equipment purchased from a 3d party because the equipment is old and no longer supported by the OEM.  Use equipment as long as possible, but don’t risk a production outage.
• Constant software vendor license audits - they will find revenue, but the technology team that leaves their company vulnerable on a recurring basis is likely to have other significant issues.
• Lack of an RFP or benchmarking process to periodically assess the cost effectiveness of hardware, software, hosting, and support vendors.  Making a change in one of these areas is not simple, but the technology team should know how much they should pay before a change is better for the company.

Technical Debt

A technical diligence effort should also identify the level of technical debt and quantify the amount of engineering resources dedicated to servicing the technical debt.

Technical debt is a conscious choice to take a shortcut in the technology arena - the delta between the desired or intended way and quicker way.  The shortcut is usually taken for time to market reasons and is a sound business decision within reason.  Technical debt is analogous in many ways to financial debt - a complete lack of it probably means missed business opportunities while an excess means disaster around the corner. 

Just like financial debt, technical debt must be serviced, and it is serviced by the efforts of the engineering team - the same team developing the software.  AKF recommends 12% to 25% of engineering effort be spent servicing technical debt.  Whether that resource allocation keeps the debt static, reduces it, or allows it to grow depends upon the amount of technical debt.  It is easy to see how a company delinquent in servicing their technical debt will have to increase the resource allocation to deal with it, reducing resources for product innovation and market responsiveness.

Put It All Together

The investor has made use of several specialists in an overall diligence effort and is digesting the information to zero in on the choice to invest and at what price.  The business side looks good - revenue growth, product strategy, and marketing are solid.  The legal side has some risks relating to returning a leased office space to its original condition, but the lease has 5 years to run.  Now for technology;

• Tech refresh is overdue, so additional investment is needed or a move to the cloud accelerated - either choice puts pressure on thin margins.
• An expensive RDBMS is in use, but the technology team avoids stored procedures and keeps their SQL as vanilla as possible - moving to open source is doable.
• Technical debt service is constantly derailed by feature requests from sales and marketing.  Additional resources, hired or contracted, will be needed and will raise the technology run rate.  More margin pressure.
• Conclusion - the investment needed to address tech refresh and technical debt changes the investment value.  The investor lowers the offer price.

Interested in learning more about technical due diligence? Here are some due diligence do’s and don’ts.

How AKF can help

AKF has conducted hundreds of technical due diligence studies over the last 10 years.  One would want an attorney for a legal diligence effort and one would want a technologist for a technical due diligence.  AKF does technology right.  Read more about our technical due diligence offerings here

 

 

Subscribe to the AKF Newsletter

Permalink

The AKF Partners Security Insights Cube

February 13, 2018  |  Posted By: Marty Abbott

Necessary But Insufficient Security Reviews

From a security perspective, tech product companies far too often focus solely on various ISO and/or NIST audits to help inform their view of how they manage risk within their company and their products.  The problem with the standards that exist today is that none of them tread deeply enough into the waters of detection and prevention of malicious activities within products.  Instead, they focus more on the processes of response, identification, notification, employee access, etc.

While these activities (and audits) are necessary, they are insufficient to ensure that we properly manage risk (and prevent malicious activities) in our products.  As we’ve written previously, erecting barriers and hiding behind big walls may make you feel better and help you sleep at night – but it’s not going to keep the bad guys from scaling your walls and taking your stuff.

The Online World is Getting Scarier
Consider the following secular trends for online products:
• A continuing mix-shift of commerce from retail to online.  Within the US today, excluding certain goods, this number stands at a meager 9% of total commerce in 2017 up from 1% in 2002.  If one excludes extremely high dollar items (vehicles, etc) the percentage of sales is significantly higher.  Growing at a slightly higher than linear rate since 2002, this number should easily double within the next 7 years.  From the perspective of a malicious hacker, this is a growth in opportunity.

• Developing and established nations outside of N. America and Western Europe continue to invest heavily in STEM-based education.

• Overall employment in many of these countries is comparatively low outside of what Western Nations provide through off-shore contracting opportunities.  Combined with recent nationalistic trends and a desire to “keep jobs at home” or not “offshore jobs” there is a strong possibility that demand for offshore agencies will decrease over time.

• Some nations within the set of nations spending heavily on STEM education, have created cyber-institutes promoting cyber and security related warfare capabilities.

• A smaller set of the nations described above have heavily promoted state sponsored cyber warfare initiatives, setting these teams (e.g. the PRNK’s Unit 180) against corporate infrastructure within the United States. 

• The barrier to entry for malicious actors to be effective in attacking corporate assets has declined.  Hacker communities commonly share exploits and malware, and certain nation-states (e.g. Russia and N. Korea) have contributed to hacking toolsets, thereby decreasing the barrier to entry for a malicious actor and resultingly increasing the supply of said malicious actors.

• Extradition from other countries for crimes committed, especially those with which the US is not allied, is difficult to impossible.  View this as a low perceived cost of committing a crime.  If you cannot be prosecuted, there is no to low perceived cost of committing the crime.

• Crypto-currency (e.g. Bitcoin) provide a near untraceable means of selling stolen data, or holding systems for ransom.

The resulting forces of these meta or secular trends are clear: 

1) The value of being a malicious actor has increased as the supply (in terms of sales/value) continues to increase.  View this economically as an increasing opportunity for crime.

2) The barrier to entry to become a malicious actor is decreasing.

3) The cost in terms of prosecution, if performed outside the US is low to zero.

These points combine to make one clear outcome:  Cybercrime and cyberterrorism (fraud, malicious use, etc) will rise as a percentage of revenue transacted online.

To help combat this rising malicious activity, we need new models and approaches to help us think about how to Identify and Prevent bad actors from doing horrible things.


Enter the AKF Security Insights Cube.


If It Isn’t Real Time It Is Worthless

The AKF Partners Security Insights Cube is predicated on the notion that all the data it addresses is accessible in near-real-time.  This alone is a considerable barrier for many companies.  Identifying fraudulent activity after credit cards are processed, for instance, is simply too late.  We want to know that bad people are entering our neighborhood and at our door – not that they stole something from our house yesterday.

The lower left corner of the cube is the starting point for any solution – the point at which you are flying blind and have no real time data.  Again – getting data from 15 minutes ago or 24 hours ago is as useless in driving a product as it is in driving a car or flying a plane; you simply have no idea what is going on.


X Axis

The X axis of the cube evaluates the breadth of data available to an organization in real time.  The far left is “zero real time data”.  Progressing to the right of the axes are increasingly valuable risk related data points from real time key performance indicators like logins, add-to-carts, check-outs, auth activity (and failures), searches, etc.  Moving further right, we may keep all session data such that we can interrogate and perform behavioral analysis and pattern matching.  The far right of the axis is the point at which we keep absolutely everything, increasing the optionality of how we may interrogate the data for risk management and malicious activity prevention purposes.


Y Axis

The Y axis of the cube evaluates the activities performed upon the X axis data by an organization.  Clearly here the X axis sets an upper bound on what’s possible on the Y axis.  For instance, it would be hard to understand “Who, What or How” something happened if we didn’t first store session data to be analyzed.  From a GDPR perspective, PII can be anonymized if necessary in session information.  As with most analytics oriented system, maturity progresses from doing nothing, to “reporting” capabilities that illuminate “what is happening” (typically employing performance indicators), to answering “Who, Why and How” to finally predicting what will happen and preventing malicious activities in real time.


Z Axis

The Z axis of the cube deals simply with the depth, or duration, that data is kept.  We rarely suggest that data be kept forever, but there is great value in ensuring that past patterns can be analyzed to create behavior models for scoring risk and blocking activities.  A handful of years is typically appropriate for most commerce solutions, slightly more data for fintech solutions.


AKF Partners performs security reviews of technology products.  Our approach evaluates security among several dimensions and includes components of NIST and ISO standards, but is tailored to the needs of online product companies. 

Subscribe to the AKF Newsletter

Permalink

 1 2 3 >  Last ›