A remarkable deception detection study was just published and – given this moment in human history where Artificial Intelligence (AI) and Machine Learning (ML) algorithms are being implemented within organizations – its findings deserve deep understanding by due diligence pros and their supervisors. The research is entitled, “Lie detection algorithms disrupt the social dynamics of accusation behavior”[i] and it provides a road map for organizations looking to leverage insights from deception detection algorithms.

Creation of a Deception Detection Algo

Researchers first created a deception detection algorithm using machine learning. They elicited 1,536 true and false statements from 786 people to train their algorithm using a standard 80:20 technique. If the authors could later fool a deceptive detection judge then they were awarded £2.00.

Overall the accuracy of the AI developed algorithm was incredibly comparable to other similar efforts described in the deception science literature. Namely, accuracy was 66.86% in detecting deception. Also similar to other algorithms of this type, the accuracy at detecting truthful statements was just 52.94%. In other words, its Type I and Type II error rate are very different.

By contrast, the Deception And Truth Analysis algorithm has been double-blind, scientifically test with a 88.4% accuracy, a Type I error rate of 11.3%, and a Type II error rate of 14.3%.

Measuring People’s Accuracy, Payoffs, and Accusation Rates

After the deception detection algorithm was created a separate group of 2,040 individuals were recruited to judge the statements created by the first group. Deception detection judges were told that 50% of the statements they assessed were deceptive and 50% were truthful. The 2,040 people were divided into four different test groups:

No Choice

• 1. Baseline – These people had no choice to access the deception detection algorithm’s assessments. The baseline group also did not know of the existence of the algorithm until after they had conducted their assessments.
• 2. Forced – These people had no choice and were automatically given the deception detection algorithm’s assessments.

The next two groups were informed about the existence of a deception detection algorithm and given the option to request the result of the deception detection algorithm for a cost of £0.05. However, some of the people making the request were randomly blocked from seeing the result and were told that the assessment was unavailable. These people were refunded their £0.05.

Choice

• 3. Blocked – These people had the choice to access the algorithm but randomly were told that they could not see the result of the algorithm. This was done so that the researchers could statistically determine if people had faith in the AI algorithm (as represented by their requesting its insights) were more accurate in their personal deception assessments.
• 4. Choice – These people chose to and could see the result of the algorithm.

Participants in all four groups were incentivized to make accurate assessments and for each correct judgment received £0.50.

Among the paper’s key findings are:

1. That people are very poor at deception detection normally (54.45%[ii]) and even worse when they are trying to detect it in the written word (46.47%). This finding jibes with similar research conducted several years ago that found people are just 50.0% able to detect deception in the written word, or transcripts of the spoken word.[iii] It also jibes with the results of hundreds of scientific studies that have found people are 54% accurate in deception detection judgments.
2. People who are given, rather than choosing to see the results of a deception detection algorithm have much higher and statistically significant outperformance; 56.47% (p = 0.003) vs. 50.78% (not significantly different from chance guessing).
3. The payoff from being forced to receive the results of the algorithm is 21.52% higher than not having access to the algorithm at all. When people have the option to choose to see the algorithm’s assessment their payoff is just 2.53% higher than not having it.
4. Furthermore, the people who made the most money are those who had the highest trust in the algorithm’s results, making 36.09% more money than those who were neutral in deciding whether to trust the algorithm or not.
5. People do not like to accuse others of deceptive behavior. Without access to a deception detection algorithm the rate of accusation is just 19.22%. This result is shocking given that participants in the study were told that 50% of the statements they would be assessing were deceptive. Given access to the algorithm’s results – either by choice or no-choice – increases the accusation rate to 31.08%. For those that had the highest belief in the power of the algorithm they made the most accusations at 40.54%. Note: this is still well below the 50% of statements that participants should have expected to be were deceptive.
6. When people experience high levels of guilt in accusing someone else of being deceptive they are much less likely to make an accusation.
7. Women are generally better at deception detection in the written word than men, 52.49% vs. 49.32%, though this result is not statistically significant.
8. By education, women who have psychology degrees are the best at deception detection, including those that accepted the results of the algorithm, at 60.0%. While those with an engineering degree are the worst at just 39.29%. For men, those scoring best are those with a social science degree other than psychology at 55.91%. While the worst are those with a political science degree at just 41.18%.
9. Those people who are least familiar with AI and ML actually are better at deception detection than those with a greater familiarity, 61.04% vs. 49.76%. This speaks to the general skepticism of AI among people and the harm of rejecting its insights.
10. People are more willing to request algorithmic predictions when they believe (1) it outperforms an average human (+28.03%), (2) it outperforms themselves (+19.45%), and (3) the probability of false accusations is low (-15.69%).
11. Last, people are more willing to purchase algorithmic predictions when they believe (1) it outperforms an average human (+£0.0645 or +12.90%), (2) it outperforms themselves (+£0.0471 or +9.42%), and (3) the probability of false accusations is low (+£0.0325 or +6.50%).

If you are a due diligence professional charged with assessing the trustworthiness of statements made by people then you should trust the assistance able to be provided by AI/ML algorithms. Not only is your accuracy improved, but also the amount of money that you can make. Additionally, people are both skeptical of AI/ML solutions, as well as reluctant to accuse others of deception, in general. If you are a supervisor of due diligence professionals – such as in investment management, insurance underwriting, the law, human resources, private investigations – you should mandate that the results of deception detection algorithms be provided to your staff to improve their capabilities and to save you money.

---

[i] Von Schenk, Alicia; Victor Klockmann; Jean-François Bonnefon; Iyad Rahwan; & Nils Köbis. “Lie detection algorithms disrupt the social dynamics of accusation behavior.” iScience(2024)

[ii] Bond, Jr., Charles F. and Bella DePaulo. “Accuracy of Deception Judgments.” Personality and Social Psychology Review. Volume 10, Issue 3 (2006): pp. 214-234

[iii] Kleinberg, Bennett & Bruno Verschuere. “How humans impair automated deception detection performance.” Acta Psychologica. 12 January 2021

Back in January Cloud Quant[i] published a comprehensive whitepaper, Outperforming the Market with Measures of Deceptive and Truthful Language in Regulatory Filings, that independently validated that Deception And Truth Analysis Scores may be used to create many highly competitive investment strategies. These strategies outperform benchmarks and do so with greater consistency and lower volatility than benchmarks. Here is a summation of the ground covered so far: 

• Independent Validation: DATA Handily Beats the S&P 500
• Independent Validation: DATA Handily Beats the Russell 2000

In this week’s article we cover more a more esoteric strategy. Namely, using a combination of factors available on the DATA platform. The full details of the trading strategy are detailed in their whitepaper.[ii]

For those new to Deception And Truth Analysis, we have built an algorithm based on the findings of deception science which has over the last 100+ years and in 8,000 pieces of research identified behavioral differences between deceivers and truth tellers. We then use Natural Language Processing to look for more than 30 behavioral differences. Our algorithm is grounded in science, not machine learning and it has never seen a stock price. Instead, we measure human behavior and it appears to be the case that financial markets are slow to price managements’ behaviors, but that to do so is extremely beneficial to generating excess returns.

The graph below shows the value of a $1.00 initial investment in CloudQuant’s dollar neutral trading strategy constructed using outputs on the DATA platform versus the 90-day Treasury Bill, as represented by the ETF equivalent ‘BIL.’

As you can see, the dollar neutral strategy using DATA Scores bests the 90-day Treasury Bill strategy over the period 2Q 2011 through 1Q 2023 $1.2213 versus $1.0752. This is an excess return at the end of the period of 14.61 percentage-points.

Further detail of the DATA dollar neutral strategy is shown below where the performance each year is shown, along with the annual excess return of the DATA dollar neutral strategy versus the 90-day Treasury Bill.

  

As you can see, above, the DATA dollar neutral strategy outperforms in all but one full-year (i.e. 2019) and one partial year (2023). 

Below is a comparison of the DATA dollar neutral strategy versus the 90-day Treasury Bill, summaried below:

  

First, the DATA dollar neutral strategy’s average monthly, quarterly, and annual performance exceeds that of the 90-day Treasury Bill. These amounts are +0.09% per month, +0.27% per quarter, and +1.01% per year. Not shown in the chart above is the average daily return of the DATA dollar neutral strategy versus the 90-day Treasury Bill. Outperformance is present there, too: 0.007% versus 0.002%.

Second, the DATA dollar neutral strategy’s maximum return in a month of 1.14% versus the 90-day Treasury Bill’s maximum return in a month of 0.40%. This is an outperformance for the DATA dollar neutral strategy of +0.74%. Maximum returns for the DATA dollar neutral strategy are also larger than that of 90-day Treasury Bills for quarterly periods, +1.97% vs. +1.03%, and for yearly periods, +3.48% vs. +2.03%.

Third, in terms of Maximum Drawdown, the DATA dollar neutral strategy underperforms the 90-day Treasury Bill for the monthly and quarterly periods: -0.62% vs. -0.04%, and -0.60% vs. -0.04%. The maximum drawdown of the DATA dollar neutral strategy for yearly periods is +0.55% vs. -0.13%.

Next, the returns of the DATA dollar neutral strategy are more consistent than those for the 90-day Treasury Bill, which we can evaluate by looking at the proportion of months, quarters, and years of outperformance. Here the DATA dollar neutral strategy versus the 90-day Treasury Bill shows outperformance in: 89 of 145 months; 32 of 48 quarters; and 11 of 13 years.

If you were to invest in the DATA dollar neutral strategy versus the 90-day Treasury Bill then you would have outperformed on most days, as well as in most months, quarters, and years. Last, your returns would have been more consistent, meaning that your entry point and exit point are more likely to have generated outperformance.

---

[i] Another organization also independently validated the DATA platform with interesting large cap results and features in our article “Independent Validation: Twice Validated Large Cap Outperformance.” Available upon request.

[ii] For those interested in CloudQuant’s methodology we refer you to the whitepaper, pages 23-29.

In this week’s article we further feature the findings of CloudQuant[i]who independently validated that DATA Scores, in general, are predictive of future small cap stock price movements in their whitepaper, Outperforming the Market with Measures of Deceptive and Truthful Language in Regulatory Filings. Earlier this year we featured similar results for large cap stocks in “Independent Validation: DATA Handily Beats the S&P 500.” For those new to Deception And Truth Analysis, we have built an algorithm based on the findings of deception science which has over the last 100+ years and in 8,000 pieces of research identified behavioral differences between deceivers and truth tellers. We then use Natural Language Processing to look for more than 30 behavioral differences between deceivers and truth tellers.

Our algorithm is grounded in science, not machine learning and it has never seen a stock price. Instead we measure human behavior and it appears to be the case that financial markets are slow to price managements’ behaviors, but that to do so is extremely beneficial to generating excess returns.

In short, CloudQuant identified multiple outperformance strategies based on DATA Scores. In this article we summarize their findings in implementing a small cap strategy. We leave the nitty gritty details to their whitepaper.[ii] Briefly described they found that focusing on small cap stocks whose management teams showed a reduction/improvement in how much high risk, likely deceptive language, was in their annual 10(k) reports, and quarterly 10(q) reports leads to noteworthy outperformance of the DATA small cap strategy versus the Russell 2000.

The graph below shows the total return results of CloudQuant’s use of DATA Scores to construct a small cap trading strategy to compete against the Russell 2000, whose ETF equivalent is ‘IWM.’

By Jason A. Voss, CFA

As you can see, the small cap strategy using DATA Scores bests the Russell 2000 over the period 170.22% total return versus 129.42%. The excess return at the end of the period was 40.80%. The maximum total return outperformance occurred on 30 March 2023 with an excess return of 42.83%. Again, as a reminder, the DATA algorithm measures human behavior and has never seen a stock price. The maximum underperformance of the DATA strategy actually took place on the very first day of the strategy, 1 April 2011, at -0.80%.

Further detail of the DATA small cap strategy is shown below where the performance each year is shown, along with the annual excess return of the DATA small cap strategy versus the Russell 2000.

  

As you can see, above, the DATA small cap strategy appears to be more defensive with outperformance taking place in each of the down years for the timeframe shown. In up years the performance is more mixed.

Additional key details from the head-to-head comparison of DATA versus the Russell 2000 are summarized below:

  

First, the DATA small cap strategy’s average monthly, quarterly, and annual performance exceeds that of the Russell 2000. These amounts are +0.11% per month, +0.23% per quarter, and +0.56% per year. Not shown in the chart above is the average daily return of the DATA small cap strategy versus the Russell 2000. Outperformance is present there, too: 0.038% versus 0.034%.

Second, the Russell 2000’s maximum return in a month was 21.07% vs. the DATA small cap strategy’s maximum return in a month of 20.73%. This is an underperformance of 0.34%. However, for the quarterly and annual periods, the DATA strategy’s maximum returns exceed those of the Russell 2000.

Third, in terms of Maximum Drawdown, the DATA small cap strategy performs better than the Russell 2000 for the monthly, quarterly, and annual periods. Specifically, the DATA small cap strategy versus the Russell 2000 in its: worst month outperforms by 0.16%; its worst quarter outperforms by 1.47%; and its worst year outperforms by 4.50%. In other words, the DATA small cap strategy preserves capital better than the Russell 2000.

If volatility is your thing, then you will be happy to know that this outperformance is also present when looking at the standard deviation of returns of the DATA small cap strategy versus the Russell 2000 are [not shown in the chart above]:  daily returns 1.04% versus 1.12%; monthly returns 3.70% versus 4.05%; quarterly returns 7.51% versus 8.07%; and annual returns 11.24% versus 13.32%.

Next, the returns of the DATA small cap strategy are more consistent than those for the Russell 2000, which we can evaluate by looking at the proportion of months, quarters, and years of outperformance. Here the DATA small cap strategy versus the Russell 2000 shows outperformance in: 80 of 145 months; 30 of 48 quarters; and 9 of 13 years.

If consistency of positive returns is more important to you then the DATA small cap strategy versus the Russell 2000 shows it up: 52.89% versus 52.37% of days [not shown]; 62.07% versus 62.07% of months; 72.92% versus 68.75% of quarters; and 69.23% versus 69.23% of years.

If you were to invest in the DATA small cap strategy versus the Russell 2000 then you would have outperformed on most days, as well as in most months, quarters, and years. Additionally, from a capital preservation perspective your maximum drawdown would have been better and you would have outperformed the Russell 2000 in each of the four years in which small caps had negative returns. Last, your returns would have been more consistent, meaning that your entry point and exit point are more likely to have generated outperformance.

[i] SolActive has also independently validated the DATA platform with interesting results that will feature in a forthcoming article.

[ii] For those interested in CloudQuant’s methodology we refer you to the whitepaper, pages 23-25.

Over the last several weeks we featured validation results from both CloudQuantand Solactive that demonstrate the power of Deception And Truth Analysis (DATA) Scores to improve investment returns for large cap investors. In this week’s article we focus on another key validation insight. Namely, that investment markets price DATA signals slowly. This is interesting for several reasons:

1. DATA is measuring something that markets care about but are slow to price. This is demonstrated by the average return advantage monthly of 0.51%, quarterly of 1.46%, and annual of 6.39% (2008 thru 1Q 2023 for all).
2. Gigantic investors have the opportunity to implement trades based on DATA signals.

As a reminder, the DATA algorithm has never seen a stock price. Nor do we train a model using machine learning. Instead, we have built a deception and truth detection algorithm based on the one hundred years of findings of deception science and its more than 8,000 papers. Our Natural Language Processing algorithm looks for more than 30 behavioral differences between deceivers and truthtellers. That DATA Scores are predictive of future securities prices means that markets do price what we are measuring even though our assessments are blind to stock price movements.

---

We would love to speak with you about how we can help your organization improve its results by better assessing the trustworthiness of the people whose words you rely on. Click here for a demo meeting. 

---

As you can see in the chart below from CloudQuant’s whitepaper that independently validated the importance of DATA Scores in improving returns both longer Lookback Periods and longer Holding Periods result in much greater Sharpe Ratios.

 

For the shortest time period shown above with a Lookback Period of 30 trading days and a Holding Period of 20 trading days (50 total trading days) a Sharpe Ratio of around 0.90 is achieved for the period 2008 thru the first quarter of 2023.

Stock markets are not open every day of the year which means that there are only 0.77 trading days per every calendar day, on average. Thus, 50 trading days is around 65 calendar days, or two months. CloudQuant’s work focused on DATA Scores rendered on companies’ annual 10K and quarterly 10Q regulatory filings.

So, on the short end of the time horizon, investors are pricing the informational content of DATA Scores only after two months after the publication of regulatory filings.

Meanwhile, using a Lookback Period of 70 trading days with a Holding Period of 80 trading days results in Sharpe Ratios in excess of 1.05 for the period 2008 thru the first quarter of 2023. Again, 150 trading days is equivalent to around 194 calendar days, or almost six months.

In other words, financial markets do value the informational content of DATA Scores, but they are exceedingly slow to price the information contained therein. Why might that be?

Before providing a hypothesis about why financial markets price the informational content of DATA Scores slowly, another key piece of research needs to be shared with you. Long before Deception And Truth Analysis received independent validation from CloudQuant we conducted many of our own validation tests.

In one such test we evaluated whether DATA Scores rendered on company annual reports would successfully catch the 10 largest financial scandals of all time. Depending on how you measure the success of this validation work, we either caught 9 of the 10 largest financial scandals of all time, or 10 of 10. Regardless, one of the interesting insights from this work is that DATA Scores for these scandal companies were measured as deceptive in the aggregate for an average of 6.2 years (!) in advance of the scandal becoming public knowledge. Again, our conclusion is that markets are slow to price what our signals are measuring.

Our theory for why markets care about the information content of DATA Scores, but price is slowly, is that most investors spend all of their time evaluating the 1.7% of the information content of annual and quarterly reports that is quantitative. They do their financial statement analysis, their ratio analysis, and their valuation work on that very narrow slice of information.

Yet, the quantitative figures are simply the outcomes driven by managements’ choices. In turn, those choices are driven by their behaviors. If you have the capability of measuring behaviors then you have a means of not only earlier predicting outcomes, but a more accurate estimation of outcomes, too. Better yet, wouldn’t it be great if there were a technology that could quantify behavior based on an underlying science of behavior? That is exactly what Deception And Truth Analysis strives to do, and based on CloudQuant’s work, it is what we do.

One of the most frequent requests we receive at Deception And Truth Analysis is to independently validate our claims that DATA Scores are predictive of future stock price movements.

After a nearly six month review we are pleased to report that we have received that independent validation from CloudQuant[i] with the publication of their whitepaper Outperforming the Market with Measures of Deceptive and Truthful Language in Regulatory Filings. Specifically, they found that DATA Scores provide a significant return advantage for different capitalization investment strategies, as well as in various sector strategies. In this article we will discuss CloudQuant’s independent validation that shows DATA handily beats the S&P 500.

Now the only question, with respect, is: Are you interested in generating alpha, or not?

CloudQuant constructed a portfolio that featured a large portion of the most truthful companies as assessed by their DATA Scores. What they found is: 

[* For those interested in CloudQuant’s methodology we have summarized it below.]

First, CloudQuant constructed a trading strategy that took advantage of some of the findings from their study. Namely, they used the period of 2008 through 2019 to define their parameters, and with a testing period of 2020 through March 2023. Also, they used the default settings described below, but with two modifications: a) the portfolio was long-only and b) the Percentile Rank Cutoff was set to the top or bottom 35%, depending on the sector.

This resulted in an Annualized Net Outperformance Return of 6.26% above the equal-weighted S&P 500. The idiosyncratic return registered a cumulative 86.29% outperformance for the preceding 15 years. For this trading strategy the turnover was relatively low at just 2.2%. These results assumed trading costs of 5 basis points per trade.

Here are additional details from this test as shown in the whitepaper’s Figure 3.5.1:

Second, for the highest performing sectors there is outperformance for at least 80 days after a position is initiated.

Third, for most of the sectors, the market does not begin to price the DATA Score assessment until 30 days after the publication of 10-Ks and 10-Qs (i.e. the day that DATA assesses these documents and publishes its DATA Scores). DATA speculates that this is because most investors respond to and price the quantitative financial results such as EPS rather than deceptiveness or truthfulness. These outcomes are driven by the management of a company’s linguistic choices; which, in turn, are driven by their behaviors. DATA very specifically is designed to measure deceptive and truthful behaviors. Thus, a lagged response makes sense.

Finally, because CloudQuant found that holding period is an important parameter they varied both the Lookback Period and the Holding Period and found that there is a consistent increase in Sharpe Ratio as the Lookback Period increases from 30 to 70 days. This means that DATA Scores are a useful metric throughout an entire quarter with peak performance at 20 and 80 Holding Period Days with 70 Lookback Period Days. To quote CloudQuant, they say:

---

“[This is] a very interesting result. Typically, datasets used for investing will exhibit a very rapid (minutes to several days) decay in their returns resulting in very limited ability for large investment managers to utilize them in a significant way.”

---

1. Data assessed. CloudQuant assessed DATA’s U.S. SEC 10-Q and 10-K filings dataset, DATAbase, as well as the Managements’ Discussion & Analysis sections of these documents.
2. Data quantity. In total there were 6.7 million data points from 213,522 earnings filings from 6,191 companies over 14 years.
3. Biases assessed. CloudQuant tested for both LookAhead Bias and Survivorship Bias and found both to be low.
4. Data examined. CloudQuant evaluated three primary outputs from DATA, our DATA Score, the average DATA Score for deceptive fragments, and the average DATA Score for Truthful fragments. Very importantly, DATA’s algorithm has never seen a stock price. Instead, DATA’s assessments are an NLP assessment of known-to-science behavioral differences between deceivers and truth tellers.
5. Transaction fees. For each entry or exit trade they assumed a 5-basis point transaction fee.
6. Portfolio Composition. Their index composition was not static and was reassessed and redefined at the commencement of each calendar year to ensure that the universe of stock symbols remained representative of their respective market capitalization categories. This means that the strategies adapt to Mergers and Acquisition activity, ascendant companies, and company size dynamics.
7. Rebalancing. Two methods of rebalancing were used: a) constant holding period with event-based entry; and b) daily rebalancing with a fixed lookback window and a maximum holding period.
8. Position Entry. Multiple lookback periods were considered, as well looking at the percentile ranking of companies based on their DATA Scores. Once DATA Scores were converted into percentiles, CloudQuant also established threshold values for long and short positions.
9. Position Exit. Two methods of exiting positions were examined; a) daily re-balancing with a fixed lookback window; and event based entry with a constant holding period.
10. Portfolio Weighting. Two methods of weighting portfolios was considered; a) equal weight, and b) an event-driven approach, where the sizing is adjusted based on the signal count for each entry date.
11. Entry Trade Execution Lag. To account for the fact it can take a long time to fully purchase a given position, CloudQuant looked at execution times ranging from 0 to up to 50 trading days.
12. Holding Period. Holding periods were varied from between 20 to 80 days. This allowed CloudQuant to evaluate the decay rate of the signals, potential investment capacity and performance characteristics for medium to long-term investments.
13. Portfolio Structure. Two types of portfolio structure were considered; a) long-only, and b) dollar-neutral.

For its back tests conducted on the S&P 500, CloudQuant’s default parameters were as follows:

1. Portfolio Structure: dollar neutral.
2. Lookback Period: 60 days.
3. Cross-Sectional Percentile Ranking of raw DATA Score.
4. Entry Trade Execution Lag: 0 days.
5. Holding Period: 40 days.
6. Percentile Rank Cutoff Threshold: 0.50/0.50
7. Portfolio Weights: equal-weighted and the date range is 2008 through 2022. 

For a full list of parameters, please refer to the whitepaper, Section 2, pages 2-9.

This is the first of six articles we are authoring to summarize CloudQuant’s and Solactive’s independent validation of DATA Scores’ ability to predict future stock price movements. Here is an overview of this and forthcoming articles:

1. DATA Handily Beats the S&P 500 – A summary of CloudQuant’s large cap findings.
2. DATA’s Stock Picking Batting Average – Solactive found that DATA Scores make the right judgment call on stock’s going up a very large amount of the time.
3. DATA Handily Beats the Russell 2000 – A summary of CloudQuant’s small cap findings.
4. DATA Measures Something Unique – A summary of CloudQuant’s latency findings that demonstrate that it takes markets many weeks to price the behaviors that DATA is measuring.
5. DATA’s Industry Performance – A summary of CloudQuant’s industry findings. The report card indicates that DATA is excellent in picking some industries, less so in others.

    

[i] Solactive has also independently validated the DATA platform with interesting results that will feature in a forthcoming article.

In this Key Scientific Paper Redux of “Can AI Read the Minds of Corporate Executives?,”[i] we summarize the findings of researchers who were interested in whether or not using Natural Language Processing may be used to assess disclosures from companies in their regulatory filings that would allow them to predict future earnings surprises in subsequent quarters. The answer to that question is: yes. This finding comports with our own back-test analyses that finds that there is still signal contained in within quarterly and annual reports over a year later.

The researchers considered three different high-level approaches to extract meaning from the words contained in company 10-Ks and 10-Qs from 1993 thru 2021 and had them compete against one another to predict future earnings surprises. Specifically, those methods are:

1. Method 1: Sentiment.

   a. A keyword sentiment lexicon developed by the researchers Loughran and McDonald in 2011 that had human-based researchers encoding words in financial reports as to their meaning.

   b. A tweaking of Google’s pre-trained Large Language Model BERT (Bidirectional Encoder Representations from Transformers) known as FinBERT, created by Huang, et al. in 2022.

   c. Simply looking at the length of the MD&A and Risk Factors sections.

2. Method 2: Bag-of-Words.

   a. A manual word classification scheme along with a regression model similar to that employed by Jegadeesh and Wu in 2013. Here the meanings of words are classified and then given weights by a regression model.

   b. The same type of approach as above, but as suggested by a difference scheme proposed by Manela and Moreira in 2017.

3. Method 3: Hierarchical Transformer Approach LLM. The method proposed by the authors of the paper who use a Large Language Model in a novel way. Specifically, they focus on:

   a. The MD&A and Risk Factors sections of reports.

   b. Given that current quarterly earnings announcements are noisy, they train their machine learning algorithms on next quarter’s earnings announcement surprises.

For each of the Methods’ performance summarized below the authors’ criteria was to rank stocks based on their earnings surprise forecasts into quintiles and then evaluate the out of sample performance of the High-minus-Low strategy that buys the highest quintile category and sells the lowest quintile category. Furthermore, for these quintile portfolios they looked at both the equal weighted (EW) and value weighted (VW) portfolio returns. Last, they controlled for additional factors that might skew their results, such as cross-sectional and time-series regressions with the time horizon and various firm characteristics. 

For Method 2 the researchers used various statistical and machine learning methods to develop weightings for the words identified as predictive. Which statistical method was used is shown in the parentheses, below.

Below we do not describe the full voluminous output of the researchers. But the authors also looked at CAPM returns, as well as Fama-French 5-factor and 6-factor returns. For those interested in these measures of performance, please consult the paper which is publicly available.

1. Method 1 performance is as follows:

   a. Keyword sentiment lexicon: EW = -2.1%; VW = +4.5%.

   b. FinBERT: EW = +15.0%; VW = +31.3%.

   c. Length of MD&A disclosures: EW = -18.9%; VW = -26.4% (in other words, shorter disclosures perform better).

   d. Length of Risk Factors disclosures: EW = +18.6%; VW = -8.9% (in other words, shorter disclosures perform better).

2. Method 2 performance is as follows:

   a. Bag-of-Words classification, version 1 (Ordinary Least Squares): EW = +22.4%; VW = +3.9%.

   b. Bag-of-Words classification, version 2 (Loughran & McDonald OLS): EW = +18.7%; VW = +21.9%.

   c. Bag-of-Words classification, version 3 (Elastic Nets): EW = +33.3%; VW = +18.1%.

   d. Bag-of-Words classification, version 4 (Lasso): EW = +22.0%; VW = +10.3%.

   e. Bag-of-Words classification, version 5 (Support Vector Regression): EW = +40.0%; VW = +41.8%.

   f. Bag-of-Words classification, version 6 (XGBoost): EW = +25.6%; VW = +39.7%.

   g. Bag-of-Words classification, version 7 (Random Forest): EW = -4.4%; VW = -19.1%.

   h. Bag-of-Words classification, version 8 (Feed Forward Neural Networks): EW = +18.7%; VW = +21.7%.

3. Method 3 performance is as follows:

   a. Frozen BERT: EW = +40.7%; VW = +43.0%.

   b. FtBERT: EW = +43.9%; VW = +56.1%.

Among the findings of the researchers are that traditional Natural Language Processing techniques are not able to identify future positive or negative changes in firms’ valuations. Next, off-the-shelf Large Language Models (LLMs), even those trained on financial targets, while good at predicting future earnings surprises, are not better than simpler estimators such as the lengths of companies’ MD&A and Risk Factors sections of their quarterly and annual reports. Third, fine-tuning and training LLMs is a solution. The researchers’ offering, FtBERT provides superior results in identifying future positive or negative changes in earnings. Last, the researchers find that there is robust unpriced information contained in both quarterly and annual reports.

Quotes of Note

• “The abundance of signals and complexity in disclosed information leads to investors inattention to subtle but important signals even in the most foundational to the corporate reporting process items such as quarterly and annual, 10-Q (10-K), reports.”
• “Cohen et al. (2020) also show that the subsequent [earnings] announcement does indeed reflect information which the market neglected to react to in the previous quarter’s announcement.”
• “Finally, we bring attention to the valuable information content of 10-Q and 10-K reports. We also find that market participants react very slowly to this information, largely due to high disagreement about its interpretation.”    

---

[i]Chapados, Nicolas, Zhenzhen Fan, Russ Goyenko, Issam Hadj Laradji, Fred Liu, and Chengyu Zhang. “Can AI Read the Minds of Corporate Executives?” SSRN. 27 June 2023

This week we provide another redux for a key scientific paper, “Executives vs Chatbots: Unmasking Insights Through Human-AI Differences in Earnings Conference Q&A.”[i] In particular, the researchers used Large Language Models (LLM) in a very clever way. Namely, they asked LLM to answer questions about a company’s performance and then contrasted the content of answers directly from management to the answers of the LLMs.

Their key assumption is that the LLM’s answers would be made up of already known/likely already priced information about the company. By contrast, the portions of the answers from management not captured by the LLM would be considered to be new and unique information that deserved greater scrutiny by investors. This approach allowed them to create a new measure which they refer to as the Human AI Difference, or HAID.

At Deception And Truth Analysis we also believe that some of the researchers’ key findings support our own beliefs about the content shared by management on earnings calls. We have found, for example, that the average DATA Score of earnings calls is 29.7% versus an average DATA Score of 8.1% for quarterly and annual reports. In other words, earnings calls are 3.67x more truthful. Why would this be?

We have hypothesized that management carefully manages the narratives of earnings calls and that they conduct a form of theater to ensure that the very best light is shown on company performance. Good information is readily shared, whereas bad information is either omitted or has spin applied so as to influence investor perceptions.

Fascinatingly, the researchers in this study find that their HAID score is, in fact, positively correlated with higher future earnings, as well as a higher propensity to issue future earnings guidance. In other words, the degree of new information shared by management is directly proportional to good future outcomes of the company. In short, management does manipulate the investment community when disclosing information on earnings calls.

The steps to creating the Human AI Difference (HAID) are straightforward:

1. They use three LLMs – ChatGPT, Google Bard, and a freeware LLM – to generate responses to investor questions based on existing information related to macroeconomics, industry trends, and firm conditions. Since LLMs try to provide optimized responses to user prompts, they hypothesize that these responses are a close approximation to the expected or already known and priced information about a company.
2. By examining the semantic differences between management teams’ answers and those provided by the LLM, they believe that the quantifiable differences in semantics serve as a proxy for the informativeness of the Q&A section of earnings calls. This semantic similarity percentage was then subtracted from 1 in order to generate the HAID. For example, if the semantic similarity between the answers provided by the LLMs and management was high, say 85%, then the HAID would be 1 – 0.85 = 0.15. This would indicate a lower new information content provided in the Q&A than if the similarity had been, say 45% where the HAID would be 1 – 0.45 = 0.55.
3. One of the key problems identified in using LLMs is that they can invent information or answers that are complete fabrications. These fabrications are known as hallucinations. In order to minimize this problem, the researchers provided the LLMs with the same information that investors listening on the call would have at their disposal prior to participating in the call. For example, the details provided by companies in their earnings press releases.
4. In total the researchers examined 190,538 earnings calls, as well as stock price information before and after the calls (from CRSP), as well as earnings estimates (from I/B/E/S). Along with this information the researchers used other financial data about the company and other measures. All of this led to a final sample of 104,932 earnings calls between 2004 and 2020 and corresponded to 5,570 unique companies.
5. The researchers hypothesized that a higher HAID would capture more informational content and lead to more-informed trading activity, reduce information asymmetries, and lead to greater market liquidity. The market action was summarized using absolute cumulative abnormal stock returns and abnormal trading volumes during and after the earnings calls, as well as the number of analyst forecast revisions and the accuracy of their estimates, too.
6. To ensure that other key performance metrics were not the source of the pricing and volume actions they controlled for other business metrics, such as firm size, market-to-book ratios, leverage, R&D, ROA, stock price volatility, analyst coverage, special items, and institutional investor ownership.
7. Another hypothesis considered is that a higher HAID is more likely to convey good news rather than bad news as measured by higher future earnings growth.
8. Last, the researchers also hypothesized that a higher HAID ratio is also likely more associated with earnings guidance on the part of management. This rests on an assumption that when management expects good news they are more likely to want to share that guidance on earnings calls.

1. HAID is strongly positively associated with the absolute cumulative abnormal return and abnormal trading volume in the two-days trading window before and after the conference call date. Specifically, a move in HAID from the 10th to 90th percentiles is associated with a 0.143 higher abnormal trading volume and a 2.6% higher absolute cumulative abnormal return.
2. They found that a higher HAID results in smaller analyst forecast errors of 11.1%. They also found less analyst forecast dispersion of 4.7%. Dispersion here is measured as the level of disagreement among analysts’ ratings.
3. Regarding liquidity, the researchers found that HAID statistically and significantly negatively correlated with the bid-ask spread, as well as the Amihud Ratio. Specifically, an increase in HAID from the 10th to 90thpercentile resulted in a 3.8% lower bid-ask spread and a 5.4% lower Amihud Ratio.
4. That earnings calls contain more good news than bad news is supported by the fact that a higher HAID ratio is associated with higher future earnings growth of 1.3%.
5. An increase in HAID also results in a 15.2% increase in the probability that management issues quantitative earnings guidance.

This study’s researchers find that the use of LLMs, as well as the use of traditional semantic similarity measures in Natural Language Processing provide new and unique information for investors. Additionally, they find that the amount of new information disclosed by management is predictive of multiple future outcomes including: higher abnormal returns, higher abnormal trading volumes, smaller forecast errors by analysts, less variability in all analysts’ forecasts, smaller bid-ask spreads, and higher liquidity. Additionally, and most interesting to DATA, the researchers found that a relatively higher ratio of new information is associated with future good outcomes at companies. In other words, this is evidence that when management discloses new information it is typically good news, not bad news.

Regarding the HAID ratio being hypothesized to contain more information about good news rather than bad: “This conjecture is driven by the incentives of managers to elaborate on positive developments and disclose private information, while being motivated to conceal negative news.”

The DATA Score for this article is 93.91%, or 99.99th %-ile truthful, or very low risk.  

---

[i]Bai, John (Jianqiu), Nicole Boyson, Yi Cao, Miao Liu, and Chi Wan. “Executives vs Chatbots: Unmasking Insights Through Human-AI Differences in Earnings Conference Q&A.” SSRN Posted 22 Jun 2023 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4480056 

Our apologies, but it has been awhile since we summarized a key scientific paper. This is mostly because there has been a dearth of new substantive research. Fortunately, two compelling pieces of research recently were published. This redux features, “The tangled webs we weave: Examining the effects of CEO deception on analyst recommendations,”[i]provides fascinating insight into investment professionals’ responses to deceptive CEOs before deceit is publicly known. In other words, are investment professionals capable of detecting deception from CEOs? In short, the researchers found that analysts are prone to assigning superior recommendations to deceptive CEOs.

Incidentally, our own published scientific research, “Investment Professionals’ Ability to Detect Deception: Accuracy, Bias and Metacognitive Realism” found that investment professionals:

1. Are worse than the general population at detecting deception; 51.8% versus 54% accuracy.
2. Have a 25.5% overconfidence in their deception detection abilities.
3. Have a 21.2% truth bias meaning that by default they trust they are being told the truth much more than they actually are told the truth.

Similar to the work of Deception And Truth Analysis (DATA), the researchers created a an algorithm to evaluate the level of deceptiveness in company communications; in this instance, earnings call transcripts. However, unlike DATA the researchers built their model using machine learning, though they did use the findings of deception science to amplify the results of their model.

Interestingly, they report an overall accuracy of their model of 84.18% which compares favorably to DATA’s reported accuracy of 88.4%. Similar to DATA, the researchers validated their model using multiple contexts where accuracy ranged between 73% and 82.8%.

Here are the hypotheses tested by the researchers:

1. A CEO’s use of deception will be positively related to analysts’ subsequent recommendations.
2. The positive relationship between a CEO’s use of deception and analysts’ recommendations will be negatively moderated by their history of deception. Prior use of deception will weaken the positive effects of subsequent deception.
3. The positive relationship between a CEO’s use of deception and analysts’ recommendations will be positively moderated by the reputation of the analyst. That is, All-Star analysts will give better recommendations to deceptive CEOs than Non-All-Star analysts.
4. The negative moderation of a CEO’s history of deception on the relationship between deception and analysts’ recommendations is weakened by analyst reputation. That is, All-Star analysts will give better recommendations to deceptive CEOs who have a greater history of being deceptive, compared to Non-All-Star analysts.

Relative to the above hypotheses, the researchers found the following:

1. Hypothesis One was supported by the results with there being a positive relationship between deception and analyst recommendations, p = 0.027.

   a. In particular, the probability of a Hold, Sell, or Strong Sell recommendation is 27.7% lower for deceptive CEOs. Truthful CEOs have a 59.0% chance of receiving one of these ratings, whereas deceptive CEOs have only a 31.3% chance.

   b. The probability of a Buy or Strong Buy for deceptive CEOs is 68.7% vs. just 41.0% for truthful CEOs.

   c. Last, when considering upgrades, deceptive CEOs have a 47.7% better chance of an upgrade when compared with truthful CEOs, p = 0.030.

2. Hypothesis Two was supported by the results and indicate that there are diminishing returns to continued deceptiveness, p = 0.011. CEOs were defined as having a history of deception if their measured deception was consistently 1.5 standard deviations above the mean level of deceptiveness/ truthfulness.

   a. CEOs with a history of normal deceptiveness benefit much more from deceptiveness when deceiving for the first time with a 12.4% increase in their chance of receiving a Buy or Strong Buy recommendation (68.5% chance versus CEOs with a history of deceptiveness’ chance of 56.1%).

   b. The chance of a newly deceptive CEO receiving a Hold recommendation was 31.5% versus 43.9% for CEOs with a more consistent history of deceptiveness).

   c. In terms of upgrades, newly deceptive CEOs have a 21.6% greater chance of an upgrade than consistently deceptive CEOs, p =0.024.

3. Hypothesis Three found only weak support with there being a similar influence of CEO deception on both All-Star and Non-All-Star analysts where the CEO is newly deceptive. Specifically, All-Star analysts gave Buy or Strong Buy recommendations 68.9% of the time for newly deceptive CEOs, whereas Non-All-Star analysts gave similar recommendations 68.4% of the time, p = 0.363.

4. Hypothesis Four was supported by the data with CEOs with a history of deceptiveness receiving higher ratings from All-Star analysts than Non-All-Star analysts. Here p = 0.009.

   a. All-Star analysts give Hold or worse ratings just 41.2% of the time versus 44.2% for Non-All-Star analysts.

   b. All-Star analysts give Buy or Strong Buy ratings 58.8% of the time versus 55.8% for Non-All-Star analysts.

   c. Changes in recommendation show that All-Star analysts upgrade consistently deceptive CEOs 5.3% more of the time, p = 0.007.

1. The percentage of documents that were evaluated as deceptive by the scientists was 31.0%. Note, this is similar to the percentage of deceptiveness found in DATA’s own work of 37.9%, even though the methodologies used are very different. One noteworthy difference is that DATA’s percentage of deceptiveness covers a period that is six years longer than the research currently being summarized. Additionally, research by the University of Chicago (2023) found a similar proportion of deceptiveness (i.e. 32%), though they labeled the companies in their dataset as those having malfeasance and likely fraud.
2. The current period assessment of deceptiveness was correlated to a history of deceptiveness of 54.2%. In other words, there is a persistence in the deceptiveness of firms. This also comports with findings in DATA’s own work. That is, deceptive firms tend to continue to be deceptive, and truthful firms continue to be truthful, though there are noteworthy state changes.

Each of the above findings support similar findings in DATA’s own scientific research. Namely, that investment professionals are vulnerable to deceptiveness on the part of CEOs and that familiarity with CEOs leads to overconfidence as well as a decrease in accuracy at detecting deceptiveness. Thus, it is important for investment professionals to have a systematic and scientific second opinion that is a part of their due-diligence.

• “These findings underscore the importance of awareness of potential deception in CEO communications and the need for continuous scrutiny, learning, and adaptability among analysts.”
• “Our study suggests that analysts can be manipulated more easily and cheaply than previously thought.”
• “[A]nalysts often spend minimal time verifying the accuracy of a firm’s earnings due to the challenging nature of uncovering deception, and thus they tend to accept the data they receive as truthful.”
• “There are also significant repercussions for wrongly accusing a CEO of deception, including loss of access to privileged information…This makes analysts less likely to risk their reputations and relationships without substantial evidence of deception.”
• “McCornack and Parks…argued that as individuals become closer, they tend to grow in confidence that they can detect each other’s deception – further increasing their truth bias and lowering their accuracy.”
• “[T]he linguistic patterns of deception have been consistently identified across multiple disciplines such as psychology, linguistics, communications, law, criminal justice, computer science, accounting, and finance, underscoring its reliability for measurement.”

  

The DATA Score for this document is 55.98%, or 97.00th %-ile truthful, or very low risk.

---

[i]Hyde, Steven J., Eric Bachura, Jonathan Bundy, Richard T. Gretz, and Wm. Gerard Sanders. “The tangled webs we weave: Examining the effects of CEO deception on analyst recommendations.” Strategic Management Journal. 2023; 1-47

Deception And Truth Analysis (DATA) is proud to announce that at long last we have earnings call transcripts, as well as other company transcripts on our DATAbase Platform thanks to our partnership with S&P Global. What follows is a Case Study analysis of ServiceNow (ticker: NOW) where we demonstrate the power of combining the information found in the company’s earnings call transcripts and its regulatory filings together. 

In short, the crucial information – that the company’s gross margins were under significant pressure – and knowledge of which would have saved investors around 20% is contained in its regulatory filings, not its earnings call transcripts. But that said, the ultimate and subtle admission of gross margin compression leading to deceptive behavior on the part of management is finally revealed in an earnings call. In other words, investors needed a tool like DATA and our ability to rapidly surface actionable investment insights and then summarize them in easy-to-digest tools.

One of the initial assumptions many of our Clients make about DATA is that we only are useful in uncovering the worst kind of malfeasance: fraud. But one of the reasons we love this Case Study is that ServiceNow’s story is not one of DATA flagging a fraudulent company, far from it. Instead, we are highlighting our power to provide everyday, mundane sorts of insights that reside in abundance on our platform and that our Clients rely on us to unlock on a daily basis and that can add crucial basis points to your returns.

In ServiceNow’s situation the company found its growth flagging and brought in a new CEO in late 2019. But his efforts to help the company compete seemed to flag as indicated by its eroding gross profit margins which is a fact revealed by DATA’s assessment of NOW’s regulatory filings and starting early in 2021, months before this issue was acknowledged on its earnings calls.

As 2021 progressed the company’s most high risk fragment in its regulatory filings continued to be about its gross margins. Eventually this led to an accounting change by the company regarding its Useful Lives and that was finally revealed in October 2021 in an earnings call. Shortly after this earnings call the company hit its all-time high in stock price. Subsequently, the stock price has never recovered and has been dead money since.

The above scenario is not fraudulent, but it is deceptive, and is all too typical at companies. They face top-line and bottom-line growth pressures. They try their best to accomplish this. When investor expectations become difficult to deliver they then pull what levers they can. In ServiceNow’s case it appears that they cut prices on their products to grow the top-line. This, of course, hurt their gross margins. When this did not work, in order to maintain expectations they appear to have pulled the “accounting policy discretion” card.

But this above sequence mirrors what we find at DATA all the time. Namely, looking at the financial results is a lagging indicator. Numbers are driven by management’s choices. In turn, these choices are driven by their behaviors. If you have a way of measuring managements behaviors, such as evaluating the amount of their deceptive and truthful language you get a sneak preview of what is ultimately baked into their numbers.

1. October 2019: ServiceNow announced it hired a new CEO, Bill McDermott. Ostensibly this was to improve the competitiveness of the company in the marketplace.

2. 15 November 2019: ServiceNow released an 8(k) – also featured in our DATAbase product – in which it discussed the Board’s changes to the management structure of the company. In addition to discussing the hiring of McDermott, the company disclosed the hiring of a new CFO, Gina Mastantuono. This 8(k) was assessed by us as having a DATA Score* of +100%, or having very low risk. This is not surprising given that most of what is conveyed in the 8(k) are the facts of the management transition.

3. 4 December 2019: Just after Bill McDermott onboarded as CEO, Lara Caimi, Chief Customer Officer of ServiceNow spoke at the Wells Fargo Tech Summit, a transcript that is on our DATAbase platform. An excerpt of her overall remarks, and about her feelings surrounding the hiring of McDermott, were assessed with a DATA Score of -11.123%, or -66.82%-ile deceptive. Here is what Ms. Caimi said and that we assess as deceptive:

“Yes. Obviously, it’s been a big month. Everyone, I think, was a little bit shocked in the beginning, but it’s been a very smooth transition. And I think on balance, everyone’s really excited about Bill’s experience, his expertise and what he can bring. And so what was beautiful was to watch the transition the week of, the way that — seeing 2 CEOs of their caliber kind of onstage together, whether it was in the press, with investors and then in an all hands with all of our employees and how quickly people transitioned from being sad about John and shocked maybe, to real excitement about Bill. I think he has built and seen where we want to go as a company. And so his expertise in really scaling a software business, drive the evolving go to market to the next level is what we need. And everyone is super excited about what he brings to the table. So he’s indicated that he wants to run with the existing management team, maintain our strategy. Of course, he’ll evolve it and tweak it as he gets on board, but everyone’s super excited about him.”

Thus, Clients of DATA are able to see that there is likely some unease about the hiring of McDermott even two months on from his hiring. By contrast, when she talks about revenue opportunities later at the event at Wells Fargo, her remarks have a DATA Score of +100%.

4. 29 January 2020: ServiceNow hosts its Q4 2019 earnings call in which McDermott lays out his new approach to leading ServiceNow. These comments in its transcript were assessed by us in three separate fragments. DATA Scores were +46.848%, +63.256%, and +54.950%. Each of these statements is at least in the 91st %-ile of truthful, or low risk. In other words, McDermott absolutely believes what he is saying.

5. 11 February 2021: ServiceNow releases its 10(k). Its aggregate DATA Score is +9.67%, or in the 50.09th %-ile of truthful, or low risk. The document has a year-over-year change of -14.68%. In other words, the company has become more deceptive in the last year.

This is shown in the picture below on row 5, column 6. Also, you can see below the trajectory of Year-over-Year DATA Score changes in its regulatory filings is that from this document forward for a year it continues to register DATA Score declines starting with this particular annual 10(k) report. Specifically, their Year over Year DATA Score changes were -14.68% (4Q 2020), -11.07% (Q1 2021), -19.19% (Q2 2021), and -15.44% (Q3 2021). 

Last, the numbers in the red boxes to the right of the year-over-year changes show the number of high risk fragments in ServiceNow’s documents. One column over from that, the last column on the right, the % of the document’s fragments assessed by us as high risk is shown. Note, the increase in the % of high risk fragments at the same time the aggregate DATA Score is falling year-over-year. This is a clear indication that investors need to dig deeper as the company’s language is shifting from the low risk side of the ledger to high risk.

  

ServiceNow’s most high risk fragment is Fragment 29, with a DATA Score of -39.884%, or -97.947th %-ile, indicating very high risk. The language contained in this fragment, their most deceptive, features as very high risk this statement:

“Cost of subscription revenues increased by $181 million for the year ended December 31, 2020, compared to the prior year, primarily due to increased headcount and increased costs to support the growth of our subscription offerings. Personnel-related costs including stock-based compensation and overhead expenses increased by $80 million and depreciation expense related to data center hardware and software and maintenance costs to support the expansion of our data center capacity increased by $88 million as compared to the prior year. In addition, amortization of intangibles increased by $12 million as a result of acquisitions.”

This fragment directly addresses declines in the company’s gross margins. Given that it is flagged by DATA as very high risk in a document whose Year-over-Year DATA Score shows a significant decline in the company’s truthfulness deserves additional scrutiny. Investors would be wise to discuss the company’s gross margins with its investor relations team. Further, if they were modeling ServiceNow’s future performance they probably would want to play around with their assumptions for the company’s gross margins. At the very least, this should be something that is watched by thoughtful investors.

NOW’s stock price at this moment is $594.47 per share.

6. 27 April 2021: NOW hosts its first quarter 2021 earnings call. Its DATA Score is 54.07%, or 96.25th %-ile, or very low risk, with only one fragment scoring on the high risk side of the ledger, Fragment #2, with a DATA Score of -4.918%, or Moderately High Risk. Discussed in this Fragment is high demand; ServiceNow’s teams’ innovations; new products; and a regional breakdown of sales performance.

Gross margins, and margins in general, are mentioned in Fragment 4 and assessed with a DATA Score of 5.653%, or on the truthful side of the ledger as Moderately Low Risk:

“We continue to expect subscription gross margins of 85% and operating margin of 23.5%. Finally, we expect recapture margin of 30% and 202 million diluted weighted outstanding shares for the year.”

Noteworthy on this call is that management does not dwell on gross margins, nor does the investment community on this earnings call.

NOW’s stock price is $562.63 per share on this date.

7. 28 April 2021: NOW’s first quarter 10(q) issued. Its most deceptive fragment is #13, assessed with a DATA Score of -32.965%, or -95.37th %-ile, indicating very high risk. This fragment discusses, you guessed it, the company’s gross margins and their expectations for gross margins going forward. Note how much more high risk the DATA Score in the regulatory filing around their complete gross margin picture is, as compared with the disclosure in the earnings call the day prior.

NOW’s stock price is $557.24 on this date.

8. 27 July 2021: ServiceNow hosts its second quarter earnings call. The overall transcript is assessed with a DATA Score of +45.21%, or 90.42nd %-ile, or very low risk. Margins were very specifically discussed in Fragment 4 of the document when Gregg Moskowitz of Mizuho Securities asked the following question:

“I think your subscription gross margins were a bit lower than they’ve been in a while. Is there anything that you would call out here?”

To which Gina Mastantuono responded:

“Yes. The margins we talked about there, we’re keeping them flat for our guide. But they are impacted versus the prior year a bit. And we talked about this earlier in the year. We’re making increased investments in our data centers and customer support to serve customers impacted by the new data residency regulations as well as serving our customers who require additional security measures such as IL5 for our Fed customers. So those are big-ticket items that are impacting our margin. That was included in our original guide for the year, and we are achieving exactly what we set out to do on both those fronts.”

The fragment, of which just a small portion is excerpted above, was assessed by us with an overall DATA Score of 100%, or very low risk. But the excerpted paragraph from that same fragment quoted above from Ms. Mastantuono is actually the highest risk portion of that fragment.

NOW’s stock price is $582.29 on this date.

9. 28 July 2021: NOW releases its second quarter 10(q) as per usual, the day after its earnings call. By contrast with the gross margin story told above in ServiceNow’s earnings call for its second quarter is the one contained in its regulatory filing. First, their second quarter 10(q)’s aggregate DATA Score was 5.22% vs. its earnings transcript DATA Score of +45.21%. In other words, the 10(q) is assessed as having more risk than the earnings call. Furthermore, this 10(q) showed a Year-over-Year DATA Score change of -19.19%. In fact, this marks the third quarter in a row that a regulatory filing from the company shows a double-digit decline in its truthfulness.

In this 10(q) what do you suppose is their most deceptive fragment (see below)?

It is Fragment 14, unequivocally that stands out because it is assessed with a DATA Score of -42.32%, or -98.49th %-ile deceptive, and very high risk. This section features the following two paragraphs from the company regarding, you guessed it, their gross margins:

“Cost of subscription revenues increased by $76 million and $144 million for the three and six months ended June 30, 2021 compared to the three and six months ended June 30, 2020, respectively, primarily due to increased headcount and increased costs to support the growth of our subscription offerings. Personnel-related costs including stock-based compensation and overhead expenses increased by $33 million and $61 million for the three and six months ended June 30, 2021, respectively, compared to the same periods in the prior year. In addition, depreciation expense related to data center hardware, software and maintenance costs to support the expansion of our data center capacity including public cloud service costs increased by $42 million and $77 million for the three and six months ended June 30, 2021, respectively, compared to the same periods in the prior year.

[The above paragraph is a portion of the company’s most risky section, but is less deceptive than the one immediately below which discusses the company’s expectations for gross profit margins going forward.]

“We expect our cost of subscription revenues to increase in absolute dollars as we provide subscription services to more customers and increase usage within our customer instances. Our subscription gross profit percentage was 81% and 82% for the three and six months ended June 30, 2021, respectively, compared to 83% for each of the three and six months ended June 30, 2020. We expect our subscription gross profit percentage to slightly decrease for the year ending December 31, 2021 compared to the year ended December 31, 2020 primarily due to incremental costs to acquire customers in regulated markets by adopting public cloud offerings as well as increased support for customers impacted by new and evolving data residency requirements. To the extent future acquisitions are consummated, our cost of subscription revenues may increase due to additional non-cash charges associated with the amortization of intangible assets acquired.”

Let’s review what DATA has revealed so far:

1. DATA is able to demonstrate that for three quarters in a row there is a double-digit decline in the company’s truthfulness.
2. In these filings that show increasing amounts of high risk behaviors, their most deceptive, or high risk, sections for three quarters in a row has to do with their cost of subscription revenues, as well as their expectations for gross margins going forward.

That this should be a concern for investors is unequivocal, because as you can see in the above image, it shows that Fragment 14 is assessed as having the most risk in the document and the language flagged as very high risk is also shown and quoted above.our assessment of NOW’s 10(q), as well as the associated language with the section.

Stock price on 28 July 2021: $582.29

10. 26 October 2021: ServiceNow hosts its third quarter earnings call. This transcript is assessed with a DATA Score of 34.03%, or 74.67th %-ile, or low risk. There is no mention of actual margin compression on this call. Instead in section 5 of the transcript Gina Mastantuono, CFO, says regarding margins:

“That being said, at our Financial Analyst Day in May, I talked about committing to 26.5% margin by 2024. We absolutely remain committed to that and on that trajectory. But I don’t think that, that increase is going to be linear, right? As we think about offices reopening and travel becoming more consistent and in-person events happening more, we definitely don’t believe that it’s going to be a linear path to 26.5%. I’m not actually in a position at this point to guide to 2022, but we are 100% committed to, over that 3-year period, getting to 26.5% because we absolutely believe that we’ll have savings that we’ll be able to take from these learnings and efficiencies from the pandemic and redeploy them on growth opportunities that we see absolutely in front of us today.”

Stock price on 26 October 2021: $676.71, up 16.22% from 28 July 2021.

11. 27 October 2021: NOW releases its third quarter 10(q) and it is assessed with a DATA Score of 4.08%, or -20.46th %-ile, or moderately high risk. Compare this with the DATA Score for its earnings call transcript immediately above. Again, the company’s most high risk paragraph, #14, is assessed with a DATA Score of -41.517%, or -98.33rd %-ile, or very high risk. The section again discusses its cost of subscription revenues and the company’s expectations for them. See excerpt, below:

“Cost of subscription revenues increased by $75 million and $219 million for the three and nine months ended September 30, 2021 compared to the three and nine months ended September 30, 2020, respectively, primarily due to increased headcount and increased costs to support the growth of our subscription offerings. Personnel-related costs including stock-based compensation and overhead expenses increased by $31 million and $92 million for the three and nine months ended September 30, 2021, respectively, compared to the same periods in the prior year. In addition, depreciation expense related to data center hardware, software and maintenance costs to support the expansion of our data center capacity including public cloud service costs increased by $34 million and $111 million and amortization of intangible assets increased by $8 million and $15 million for the three and nine months ended September 30, 2021, respectively, compared to the same periods in the prior year.

“We expect our cost of subscription revenues to increase in absolute dollars as we provide subscription services to more customers and increase usage within our customer instances. Our subscription gross profit percentage was 81% and 82% for the three and nine months ended September 30, 2021, respectively, compared to 83% for each of the three and nine months ended September 30, 2020. We expect our subscription gross profit percentage to slightly decrease for the year ending December 31, 2021 compared to the year ended December 31, 2020 primarily due to incremental costs to acquire customers in regulated markets by adopting public cloud offerings as well as increased support for customers impacted by new and evolving data residency requirements. To the extent future acquisitions are consummated, our cost of subscription revenues may increase due to additional non-cash charges associated with the amortization of intangible assets acquired.”

ServiceNow’s stock price on this day is $664.76.

12. 4 November 2021: Stock price hits an all-time high of $701.73, up 20.51% from 28 July 2021.

13. 25 January 2022: NOW’s annual earnings call is held. The transcript of this event was assessed with a DATA Score of +10.62%, or 3.73rd %-ile, or moderately low risk; whereas their 10(k) of 2/2/2023 had a DATA Score of 8.14%, or -5.52nd %-ile, or moderately high risk. Note that the DATA Score for the earnings calls gradually fell into line with the company’s regulatory filings. Interesting, but not necessarily significant. In Fragment 3, assessed with a DATA Score of 40.196%, or 84.75th %-ile, or low risk, the company states something incredible and very important to our overall Case Study because they finally acknowledge publicly that their gross margin pressure has led to an accounting change to compensate:

“I would also note that in January, we completed an assessment of the useful life of our data center equipment and determined that we could extend their estimated life from 3 to 4 years [emphasis: ours]. As a result, we expect a reduction in depreciation expense, which will contribute approximately 100 basis points to gross margin in 2022, trending down to just 50 basis points in 2024.”

This is an accounting change done to increase gross margin, and was absolutely telegraphed to the Clients of Deception And Truth Analysis by our highlighting of ServiceNow’s gross margins having been under pressure for all of 2021.

Why did this change need to happen? Later in Fragment 7 of this earnings call, assessed by us with a DATA Score of -13.328%, or -71.71st %-ile, or high risk, Gina Mastantuono, CFO, states:

“And then, Keith, on your question with respect to labor inflation rates. Enterprise software’s talent has been in high demand for some time now. So competitive compensation has been on the rise, and it’s not new for us. We continue to monitor it. We definitely anticipate some continued pressure in the coming quarters, but we remain very committed to our margin guidance that we’ve given you for 2022 and beyond.”

Fragment 8 of the ECT features a great question from Michael Turits of KeyBanc Capital Markets, in which he asks about that extension of useful lives:

“Great. And then, Gina, just some clarifications on the extension of the useful life. So you got essentially to flattish op margins but down a bit, like 70 bps on free cash flow. So just making sure that I understand it, and then obviously, you get that benefit to EBIT margins, but that flows through, and we see it on free cash flow because this is just a noncash, and you’re actually including expenses. So I want to make sure those mechanics are right. And then also, do you get in year 2 of this — do you get that tailwind to EBIT and accrual? Does that flip around to a headwind on EBIT margins and gross margins?”

Gina responds:

“Great question, Michael. So you’re absolutely right, the benefit that you’re seeing this year hit EBIT and operating margins, but not free cash flow, right, because it’s just a change in depreciation, which is a noncash, right? So that’s the reason why you’re seeing the benefit in operating income, but not as free cash flow. What you are seeing is free cash flow is the increased cost that I’ve been talking about as the COVID savings start to fade in ’22 as we come back in person for digital events and in-person events in [indiscernible]. With respect to the longer tail of this change in depreciation, it definitely tapers off and get smaller, right? So it’s a 100 basis point benefit this year by 2024. It goes down to just 50 points, right? So — and that’s only on the EBIT margins, not on free cash flow.”

By contrast, in the company’s 10(k), released the very next day, in Fragment 30, scoring -41.988%, or -98.43rd %-ile, or very high risk, it has to do with that perennial ServiceNow issue:

“Cost of subscription revenues increased by $291 million for the year ended December 31, 2021, compared to the prior year, primarily due to increased headcount and increased costs to support the growth of our subscription offerings including costs to support customers in regulated markets. Personnel-related costs including stock-based compensation and overhead expenses increased by $123 million as compared to prior year. Depreciation expense related to data center hardware and software and maintenance costs to support the expansion of our data center capacity including public cloud service costs increased by $141 million and amortization of intangibles increased by $29 million as a result of acquisitions as compared to the prior year.”

Stock price on 27 January 2022: $528.69, down 24.66% from all-time high.

14. 22 September 2022: Stock price is $554.09, down 18.12% from its all-time high of almost two years earlier, and is down 4.84% from 28 July 2021. In other words, NOW has been dead money for two years.

In case this is not clear, the crucial information in assessing ServiceNow, that its gross margins were under pressure, as represented by its very high risk language around them, was contained within its regulatory filings for most of 2021. By contrast, the company did not disclose its issues around gross margins on an earnings call until late October 2021. When ServiceNow did disclose its issue, it floated this into its earnings call ever so subtly by disclosing a change of the useful lives of its datacenters from 3 years to 4 years. This is the sort of mundane, but impactful language that DATA highlights every single day for its Clients. In other words we provide a daily assist to our Clients and not just with the rare, but colossal fraud.

NOW hit an all-time high just after its October 2021 call, but subsequently experienced a precipitous fall of over 20%, and its stock has been dead money for over two years. Deception And Truth Analysis provided rapid, actionable insights of the entirety of this picture to its Clients by using a combination of our innovative assessments on its regulatory filing assessments and transcripts.

---

* DATA Scores range between -100% and +100%, with any negative score being indicative of deceptiveness, or high risk; and any positive score being indicative of truthfulness, or low risk. DATA Scores are roughly normally distributed with, at the time of this Case Study, a mean of 9.62% and a standard deviation of 21.37%.

One of the questions we are frequently asked when our Clients utilize our platform is: What language has your algorithm flagged as either deceptive or truthful? Previously we were reluctant to open our Intellectual Property with such granularity, anxious that it might aid one of our competitors. However, within the next month we will disclose categorically the more than 30 behavioral differences between deceivers and truthtellers identified by deception scientists that we rely upon.* Internally we refer to these as the DATA Dimensions.

The Dimensions of Deception And Truth Analysis are something that decision makers are sure to find insightful as they use our innovative platform to make better decisions. Furthermore, knowledge of our DATA Dimensions can aid you in your interpersonal and business communications. That is, knowledge of the Dimensions means that you can begin to listen to the people whose representations you rely upon in your work with new meaning and clarity.

Without further adieu…here are the Dimensions of Deception And Truth Analysis…

Deceivers tend to obfuscate their information, including answers to questions. Sometimes this is done by being overly complex. Truth tellers are forthright and do not usually withhold information.

Deceivers and truthtellers act differently in social situations, such as in-person, on conference calls, when giving speeches, and so on. One such behavioral difference is that deceivers try and ingratiate themselves to others in a social situation to a greater degree than truthtellers.

Deceivers and truthtellers behave differently with regards to establishing status and power. As an example, deceivers usually try and control a conversation more than a truthteller.

Deceivers typically have a preferred and singular narrative that they want their audience to believe. Whereas, truthtellers are more openminded and willing to consider other points of view.

Truthtellers when recalling an event or a story are much more granular as compared with deceivers. 

Deceivers like to talk about the past and the future and are uncomfortable in the present moment. It is believed that this is because they are trying to distract their audience from present moment awareness. Truthtellers tend to be more present moment focused. 

There are miscellaneous other language differences between deceivers and truth tellers.

---

* Note 1: Our Intellectual Property and analyses rely on other key insights and not just a knowledge of the differences in behavioral tendencies of deceivers and truthtellers.

** Note 2: A previous version of this article used different names for the DATA Dimensions. The change was motivated by an internal discussion about how to communicate the Dimensions more clearly. A map of the old language for Dimensions to the revised Dimensions is: Clarity => Clarity; Authenticity => Authenticity; Influence => Tolerance; Opinions => Flexibility; Descriptions => Precision; Nowness => Focus. An advantage of the new Dimensions language is that new continuum labels allow for opposite ends of the continuum to be true antonyms. For example, Unfocused vs. Focused.