Abseil Blog & Tips

Performance Tip of the Week #97: Virtuous ecosystem cycles

2025-11-27T00:00:00-05:00

Originally posted as Fast TotW #97 on August 21, 2025

Updated 2025-11-27

Software ecosystems aim to maximize qualities like efficiency, correctness, and reliability while minimizing the costs of achieving these properties. Improving a single service through customization can help build an optimization more expediently, but it has an inherent limited scope to its upside and increases technical debt. A point solution fails to provide the full benefits from applying features horizontally. In this episode, we discuss how lessons learned from partnerships to improve individual applications can improve efficiency for everyone.

Case studies

We look at several case studies where we could take a feature that showed benefits for a single team and rolled it out to provide benefits for all of Google.

SwissMap

Abseil’s hash table implementation, SwissMap, originated out of a partnership between our C++ core libraries team and two Zurich-based engineers, Alkis Evlogimenos and Roman Perepelitsa, on our search indexing team. They had set out to make an improved hash table that used open addressing for fewer data indirections, improved the design to reduce memory usage, and added API features to avoid unnecessary copies. Jeff Dean and Sanjay Ghemawat had suggested a SIMD-accelerated control block to speed up comparison, allowing the table to efficiently run at a higher load factor.

While hash tables are a key tool in a programmer’s toolbox, they tend to not span API boundaries to nearly the same degree as other vocabulary types like std::string and std::vector. For the indexing team’s purposes, most of the benefits for their application could be obtained by replacing their hottest hashtables and declaring victory.

But rather than stopping, the team opted to drive widespread adoption of the superior hashtable. Over tens of thousands of changes later, nearly every hash table in Google’s codebase is a SwissMap. The early years of the rollout saved a substantial amount of CPU and RAM. These savings were only unlocked because we pursued a scaled migration.

The broad rollout produced ongoing ecosystem dividends:

“Just use SwissMap” was good advice, but for an engineer trying to develop a new feature, it is easier to use the existing types, even if SwissMap would be a better fit for performance. The widespread usage of SwissMap made it easier for new code to reach for it.
Drawing on lessons from prior hash table changes, randomization was introduced to prevent code from relying on the order of iteration. This made it easier to land subsequent optimizations under the hood, allowing us to iteratively improve the hash function by changing its algorithm.
Because of our implementation freedom, we were able to add telemetry features like built-in profiling that work across the fleet. This has allowed us to find and fix bad hash functions, identify optimizations for small tables, and proactively reserve containers to their final sizes.

Size class-driven allocation

Modern memory allocators like TCMalloc use “size classes” to efficiently cache small objects. Rather than fit each allocation precisely, the allocator determines which size class a request falls into and checks the freelist for that size class. When an object is freed, the allocator determines which size class the memory belongs to and puts the object on the appropriate freelist for future reuse. This design simplifies bookkeeping and reduces the cost of allocating and deallocating memory.

Sized deallocation

Using size classes requires a memory allocator to determine which freelist to put freed objects onto. TCMalloc’s original mapping approach required several pointer dereferences. In 2012, Google’s compiler optimization team identified that the object’s size is frequently already known outside of the allocator. Passing this size as an argument to the allocator avoids the expensive lookup.

Once sized deallocation was implemented, several teams were able to adopt the feature to reduce allocation costs. This optimization can exacerbate existing undefined behavior or introduce bugs in the rare case of tail-padded structures, requiring would-be adopters to clean up their code and transitive dependencies. Because of these lurking issues, the default remained “unsized” delete.

Motivated by fleetwide profiles showing the horizontal cost of TCMalloc, work began to enable sized deallocation for the entire codebase.

Runtime assertions were added to TCMalloc to catch bugs in debug builds.
Test failures were investigated and numerous fixes submitted.
The feature was progressively rolled out to Address Sanitizer, debug, and optimized builds, preventing backsliding.

Infrastructure improvements, like the runtime assertions themselves, cost the same to develop for one team as they do would for widespread deployment. For other steps, like enabling sized deallocation in tests, operating centrally on all tests was cheaper and easier to do than asking each team to set up a parallel set of tests for themselves and their dependencies.

While the project delivered on its original goal of reducing TCMalloc costs in the fleet, the central effort strengthened the ecosystem’s reliability as a whole by creating a sturdier foundation:

By enabling sized deallocation for all tests, teams already relying on the optimization benefited from the prevention of new bugs in their transitive dependencies. In the past, bugs would manifest as production crashes requiring teams to play whack-a-mole to set up extra testing. Moving the entire platform to sized delete foreclosed this entire class of bugs while avoiding the cost of duplicative and ad hoc testing setups.
The added information allows TCMalloc to occasionally double-check the purported size, allowing further memory corruption bugs to be uncovered. This was only possible because size arguments are routinely passed to deallocations.
The centralized burndown uncovered a relatively rare, but important, pattern of tail padded structures not working well with sized deallocation. This spurred C++20’s destroying delete feature, which may not have been standardized and adopted universally without a central vantage point of its value.

Had we eschewed centralized adoption, individual teams would have faced similar challenges, more bugs in production, and not benefitted from the flywheel of follow-on improvements.

Size class improvements

Size classes simplify the logic required to allocate and deallocate memory, but they introduce a new optimization problem: which sizes should we select as our buckets? For many years, these partitions were chosen to minimize the amount of memory lost by rounding up the request to its bucket (“internal fragmentation”) based on fleetwide profiles.

Internal fragmentation is just one metric we can look at when making these choices. Merging two size classes would mean some smaller sizes grow, increasing roundup overhead, while being offset by reducing the amount of memory left in TCMalloc’s caches at peak demand (“external fragmentation”). Having too few size classes would put more stress on TCMalloc’s backend and global locks.

One partnership with the Search retrieval team produced a series of optimizations. The experiments showed improved performance with two sets of size classes, “power-of-2 below 64” and “power-of-2 only.” Rather than stop with just that service, we opted to evaluate the impact for all workloads.

The former struck a tradeoff between maximizing CPU performance while modestly increasing RAM usage. Fleetwide A/B experimentation allowed this set to be evaluated and rolled out to improve aggregate CPU performance, a result comparable to the initial load tests motivating the work.

Continued investigation of the “power-of-2 only” result pointed towards reducing the amount of time an object might sit on one of TCMalloc’s caches before the next allocation that uses it. By choosing size classes that were likely to see high allocation rates rather than solely minimizing internal fragmentation, we were able to further improve fleetwide CPU performance.

We can contrast this with the hypothetical counterfactual where we made size class customizability a full-fledged feature of TCMalloc and individual teams tuned for their workloads.

For the teams with the time and inclination to customize their configuration, they might have seen even larger wins for themselves than what they obtained because of the global rollout, but over time, these settings might have become stale. The second round of global optimizations underscores the risk that individual teams might have stopped at a suboptimal point.
More configurations make it harder to maintain TCMalloc, and hamper implementation freedom.
The tightly controlled configuration space made it possible to land improvements centrally, without having to navigate the existing customizations or risks from breaking someone’s use case.

Turbo troubles

Early work to use AVX instructions in general purpose workloads faced the headwind of downclocking on the first AVX-capable platforms. In a math-intensive workload, a modest frequency reduction would be a tolerable tradeoff for a doubling in floating point throughput. In other workloads, the compiler’s occasional use of them in mostly scalar code produced performance regressions, not wins, which deterred adoption.

Changing the compiler to constrain autovectorization broke the logjam. It was possible to make this change because:

Most math-intensive code already used intrinsics to ensure high-quality vectorization, rather than relying on autovectorization. This was unaffected by the compiler change.
Scalar code could unlock and leverage new instruction sets like BMI2 from subsequent microarchitectures.
In places where mostly scalar code could be vectorized somewhat, the compiler was constrained from introducing heavyweight vector instructions that introduced downclocking.

Being able to leverage the performance benefits of these new instructions without downsides spurred broader adoption, creating a flywheel for further horizontally and vertically-driven optimizations to be built on top of it.

Lessons

Don’t assume the status quo is fixed

Teams working on a single application sometimes treat the underlying ecosystem and platform as immutable, and solve problems within that constraint. The platform itself is rarely immutable if there’s sufficient value to be gained from an ecosystem-wide change.

Even if it makes sense to work around a problem for reasons of expediency, surfacing these pain points can raise awareness of frequently encountered pain points. A frequent problem might look rare if everyone works around it rather than seeing it get fixed in the platform once and for all.

Focus on problems and outcomes, not exact solutions

Prototypes are an invaluable resource for assessing how to solve a problem and determining what strategies actually work. It can help to step back to think about requirements and desired outcomes. We can say “yes, this is an important problem worth solving,” while simultaneously saying “but in a different way.”

When charting a path for scaled adoption of a prototype, we might want to use a different implementation strategy to target a broader addressable market, take advantage of existing use cases, or avoid long-term technical debt. Rarely is “deploy exactly this prototype” the outcome we care about, so flexibility towards solving the problem in different ways can let us grow our impact.

Create and use leverage

As we take vertically-inspired solutions and deploy them horizontally, we are often looking for leverage. If there is already a broadly used library or feature that we can improve, we get a larger benefit than if we created an entirely new thing that needed adoption. Rather than workaround a problem, we should see if we can push changes further down the stack to fix issues more generally and for everyone.

At other times, we might need to create a lever for ourselves. As seen with SwissMap and AVX, driving adoption helped us to bootstrap. It created usage where there was none previously, and then used that lever to motivate further centralized optimizations.

Understand what is seen and unseen

Opinionated libraries that might offer relatively few configuration options may appear to be an impediment to optimization, but it is easy to miss the opportunity costs of catering too much to specific use cases or flexibility. The costs of the former tend to be visible, while the latter is not.

A healthier, more sustainable core library can deliver other optimizations that help our service of interest. For example, we can see the costs of SwissMap’s per-instance randomization when copying.

The direct savings are clear: Removing this randomization would help improve performance for copying-intensive users.
The indirect costs are less so: Loss of randomization would make it harder to add optimizations to common operations like lookup and insertion. This would hurt the broad ecosystem, likely including copy-intensive users as well.

Closing words

Deep work with individual workloads can produce novel optimization insights. Finding ways to scale these to the broader ecosystem can increase our impact, both by removing bottlenecks for that workload and by addressing the problem systematically.

Performance Tip of the Week #99: Illuminating the processor core with llvm-mca

2025-10-07T00:00:00-04:00

Originally posted as Fast TotW #99 on September 29, 2025

By Chris Kennelly

Updated 2025-10-07

Quicklink: abseil.io/fast/99

The RISC versus CISC debate ended in a draw: Modern processors decompose instructions into micro-ops handled by backend execution units. Understanding how instructions are executed by these units can give us insights on optimizing key functions that are backend bound. In this episode, we walk through using llvm-mca to analyze functions and identify performance insights from its simulation.

Preliminaries: Varint optimization

llvm-mca, short for Machine Code Analyzer, is a tool within LLVM. It uses the same datasets that the compiler uses for making instruction scheduling decisions. This ensures that improvements made to compiler optimizations automatically flow towards keeping llvm-mca representative. The flip side is that the tool is only as good as LLVM’s internal modeling of processor designs, so certain quirks of individual microarchitecture generations might be omitted. It also models the processor behavior statically, so cache misses, branch mispredictions, and other dynamic properties aren’t considered.

Consider Protobuf’s VarintSize64 method:

size_t CodedOutputStream::VarintSize64(uint64_t value) {
#if PROTOBUF_CODED_STREAM_H_PREFER_BSR
  // Explicit OR 0x1 to avoid calling absl::countl_zero(0), which
  // requires a branch to check for on platforms without a clz instruction.
  uint32_t log2value = (std::numeric_limits<uint64_t>::digits - 1) -
                       absl::countl_zero(value | 0x1);
  return static_cast<size_t>((log2value * 9 + (64 + 9)) / 64);
#else
  uint32_t clz = absl::countl_zero(value);
  return static_cast<size_t>(
      ((std::numeric_limits<uint64_t>::digits * 9 + 64) - (clz * 9)) / 64);
#endif
}

This function calculates how many bytes an encoded integer will consume in Protobuf’s wire format. It first computes the number of bits needed to represent the value by finding the log2 size of the input, then approximates division by 7. The size of the input can be calculated using the absl::countl_zero function. However this has two possible implementations depending on whether the processor has a lzcnt (Leading Zero Count) instruction available or if this operation needs to instead leverage the bsr (Bit Scan Reverse) instruction.

Under the hood of absl::countl_zero, we need to check whether the argument is zero, since __builtin_clz (Count Leading Zeros) models the behavior of x86’s bsr (Bit Scan Reverse) instruction and has unspecified behavior if the input is 0. The | 0x1 avoids needing a branch by ensuring the argument is non-zero in a way the compiler can follow.

When we have lzcnt available, the compiler optimizes x == 0 ? 32 : __builtin_clz(x) in absl::countl_zero to lzcnt without branches. This makes the | 0x1 unnecessary.

Compiling this gives us two different assembly sequences depending on whether the lzcnt instruction is available or not:

bsr (-march=ivybridge):

orq     $1, %rdi
bsrq    %rdi, %rax
leal    (%rax,%rax,8), %eax
addl    $73, %eax
shrl    $6, %eax

lzcnt (-march=haswell):

lzcntq  %rdi, %rax
leal    (%rax,%rax,8), %ecx
movl    $640, %eax
subl    %ecx, %eax
shrl    $6, %eax

Analyzing the code

We can now use Compiler Explorer to run these sequences through llvm-mca and get an analysis of how they would execute on a simulated Skylake processor (-mcpu=skylake) for a single invocation (-iterations=1) and include -timeline:

bsr (-march=ivybridge):

Iterations:        1
Instructions:      5
Total Cycles:      10
Total uOps:        5

Dispatch Width:    6
uOps Per Cycle:    0.50
IPC:               0.50
Block RThroughput: 1.0

Timeline view:
Index     0123456789

[0,0]     DeER .   .   orq      $1, %rdi
[0,1]     D=eeeER  .   bsrq     %rdi, %rax
[0,2]     D====eER .   leal     (%rax,%rax,8), %eax
[0,3]     D=====eER.   addl     $73, %eax
[0,4]     D======eER   shrl     $6, %eax

lzcnt (-march=haswell):

Iterations:        1
Instructions:      5
Total Cycles:      9
Total uOps:        5

Dispatch Width:    6
uOps Per Cycle:    0.56
IPC:               0.56
Block RThroughput: 1.0

Timeline view:
Index     012345678

[0,0]     DeeeER  .   lzcntq    %rdi, %rax
[0,1]     D===eER .   leal      (%rax,%rax,8), %ecx
[0,2]     DeE---R .   movl      $640, %eax
[0,3]     D====eER.   subl      %ecx, %eax
[0,4]     D=====eER   shrl      $6, %eax

This can also be obtained via the command line

$ clang file.cpp -O3 --target=x86_64 -S -o - | llvm-mca -mcpu=skylake -iterations=1 -timeline

There’s two sections to this output, the first section provides some summary statistics for the code, the second section covers the execution “timeline.” The timeline provides interesting detail about how instructions flow through the execution pipelines in the processor. There are three columns, and each instruction is shown on a separate row. The three columns are as follows:

The first column is a pair of numbers, the first number is the iteration, the second number is the index of the instruction. In the above example there’s a single iteration, number 0, so just the index of the instruction changes on each row.
The third column is the instruction.
The second column is the timeline. Each character in that column represents a cycle, and the character indicates what’s happening to the instruction in that cycle.

The timeline is counted in cycles. Each instruction goes through several steps:

D the instruction is dispatched by the processor; modern desktop or server processors can dispatch many instructions per cycle. Little Arm cores like the Cortex-A55 used in smartphones are more limited.
= the instruction is waiting to execute. In this case, the instructions are waiting for the results of prior instructions to be available. In other cases, there might be a bottleneck in the processor’s backend.
e the instruction is executing.
E the instruction’s output is available.
- the instruction has completed execution and is waiting to be retired. Instructions generally retire in program order, the order instructions appear in the program. An instruction will wait to retire until prior ones have also retired. On some architectures like the Cortex-A55, there is no R phase in the timeline as some instructions retire out-of-order.
R the instruction has been retired, and is no longer occupying execution resources.

The output is lengthy, but we can extract a few high-level insights from it:

The lzcnt implementation is quicker to execute (9 cycles) than the “bsr” implementation (10 cycles). This is seen under the Total Cycles summary as well as the timeline.
The routine is simple: with the exception of movl, the instructions depend on each other sequentially (E-finishing to e-starting vertically aligning, pairwise, in the timeline view).
Bitwise-or of 0x1 delays bsrq’s input being available by 1 cycle, contributing to the longer execution cost.
Note that although movl starts immediately in the lzcnt implementation, it can’t retire until prior instructions are retired, since we retire in program order.
Both sequences are 5 instructions long, but the lzcnt implementation has higher instruction-level parallelism (ILP) because the mov has no dependencies. This demonstrates that counting instructions need not tell us the cycle cost.

llvm-mca is flexible and can model other processors:

AMD Zen3 (Compiler Explorer), where the cost difference is more stark (8 cycles versus 12 cycles).
Arm Neoverse-V2 (Compiler Explorer), a server CPU where the difference is 7 vs 9 cycles.
Arm Cortex-A55 (Compiler Explorer), a popular little core used in smartphones, where the difference is 8 vs 10 cycles. Note how the much smaller dispatch width results in the D phase of instructions starting later.

Throughput versus latency

When designing microbenchmarks, we sometimes want to distinguish between throughput and latency microbenchmarks. If the input of one benchmark iteration does not depend on the prior iteration, the processor can execute multiple iterations in parallel. Generally for code that is expected to execute in a loop we care more about throughput, and for code that is inlined in many places interspersed with other logic we care more about latency.

We can use llvm-mca to model execution of the block of code in a tight loop. By specifying -iterations=100 on the lzcnt version, we get a very different set of results because one iteration’s execution can overlap with the next:

Iterations:        100
Instructions:      500
Total Cycles:      134
Total uOps:        500

Dispatch Width:    6
uOps Per Cycle:    3.73
IPC:               3.73
Block RThroughput: 1.0

We were able to execute 100 iterations in only 134 cycles (1.34 cycles/element) by achieving high ILP.

Achieving the best performance may sometimes entail trading off the latency of a basic block in favor of higher throughput. Inside of the protobuf implementation of VarintSize (protobuf/wire_format_lite.cc), we use a vectorized version for realizing higher throughput albeit with worse latency. A single iteration of the loop takes 29 cycles to process 32 elements (Compiler Explorer) for 0.91 cycles/element, but 100 iterations (3200 elements) only requires 1217 cycles (0.38 cycles/element - about 3x faster) showcasing the high throughput once setup costs are amortized.

Understanding dependency chains

When we are looking at CPU profiles, we are often tracking when instructions retire. Costs are attributed to instructions that took longer to retire. Suppose we profile a small function that accesses memory pseudo-randomly:

unsigned Chains(unsigned* x) {
   unsigned a0 = x[0];
   unsigned b0 = x[1];

   unsigned a1 = x[a0];
   unsigned b1 = x[b0];

   unsigned b2 = x[b1];

   return a1 | b2;
}

llvm-mca models memory loads being an L1 hit (Compiler Explorer): It takes 5 cycles for the value of a load to be available after the load starts execution. The output has been annotated with the source code to make it easier to read.

Iterations:        1
Instructions:      6
Total Cycles:      19
Total uOps:        9

Dispatch Width:    6
uOps Per Cycle:    0.47
IPC:               0.32
Block RThroughput: 3.0

Timeline view:
                    012345678
Index     0123456789


[0,0]     DeeeeeER  .    .  .   movl    (%rdi), %ecx         // ecx = a0 = x[0]
[0,1]     DeeeeeER  .    .  .   movl    4(%rdi), %eax        // eax = b0 = x[1]
[0,2]     D=====eeeeeER  .  .   movl    (%rdi,%rax,4), %eax  // eax = b1 = x[b0]
[0,3]     D==========eeeeeER.   movl    (%rdi,%rax,4), %eax  // eax = b2 = x[b1]
[0,4]     D==========eeeeeeER   orl     (%rdi,%rcx,4), %eax  // eax |= a1 = x[a0]
[0,5]     .DeeeeeeeE--------R   retq

In this timeline the first two instructions load a0 and b0. Both of these operations can happen immediately. However, the load of x[b0] can only happen once the value for b0 is available in a register - after a 5 cycle delay. The load of x[b1] can only happen once the value for b1 is available after another 5 cycle delay.

This program has two places where we can execute loads in parallel: the pair a0 and b0 and the pair a1 and b1 (note: llvm-mca does not correctly model the memory load uop from orl for a1 starting). Since the processor retires instructions in program order we expect the profile weight to appear on the loads for a0, b1, and b2, even though we had parallel loads in-flight simultaneously.

If we examine this profile, we might try to optimize one of the memory indirections because it appears in our profile. We might do this by miraculously replacing a0 with a constant (Compiler Explorer).

unsigned Chains(unsigned* x) {
   unsigned a0 = 0;
   unsigned b0 = x[1];

   unsigned a1 = x[a0];
   unsigned b1 = x[b0];

   unsigned b2 = x[b1];

   return a1 | b2;
}

Iterations:        1
Instructions:      5
Total Cycles:      19
Total uOps:        8

Dispatch Width:    6
uOps Per Cycle:    0.42
IPC:               0.26
Block RThroughput: 2.5

Timeline view:
                    012345678
Index     0123456789

[0,0]     DeeeeeER  .    .  .   movl    4(%rdi), %eax
[0,1]     D=====eeeeeER  .  .   movl    (%rdi,%rax,4), %eax
[0,2]     D==========eeeeeER.   movl    (%rdi,%rax,4), %eax
[0,3]     D==========eeeeeeER   orl     (%rdi), %eax
[0,4]     .DeeeeeeeE--------R   retq

Even though we got rid of the “expensive” load we saw in the CPU profile, we didn’t actually change the overall length of the critical path that was dominated by the 3 load long “b” chain. The timeline view shows the critical path for the function, and performance can only be improved if the duration of the critical path is reduced.

Optimizing CRC32C

CRC32C is a common hashing function and modern architectures include dedicated instructions for calculating it. On short sizes, we’re largely dealing with handling odd numbers of bytes. For large sizes, we are constrained by repeatedly invoking crc32q (x86) or similar every few bytes of the input. By examining the repeated invocation, we can look at how the processor will execute it (Compiler Explorer):

uint32_t BlockHash() {
 asm volatile("# LLVM-MCA-BEGIN");
 uint32_t crc = 0;
 for (int i = 0; i < 16; ++i) {
   crc = _mm_crc32_u64(crc, i);
 }
 asm volatile("# LLVM-MCA-END" : "+r"(crc));
 return crc;
}

This function doesn’t hash anything useful, but it allows us to see the back-to-back usage of one crc32q’s output with the next crc32q’s inputs.

Iterations:        1
Instructions:      32
Total Cycles:      51
Total uOps:        32

Dispatch Width:    6
uOps Per Cycle:    0.63
IPC:               0.63
Block RThroughput: 16.0

Instruction Info:

[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)

[1]    [2]    [3]    [4]    [5]    [6]    Instructions:
1      0     0.17                        xorl   %eax, %eax
1      3     1.00                        crc32q %rax, %rax
1      1     0.25                        movl   $1, %ecx
1      3     1.00                        crc32q %rcx, %rax
...

Resources:
[0]   - SKLDivider
[1]   - SKLFPDivider
[2]   - SKLPort0
[3]   - SKLPort1
[4]   - SKLPort2
[5]   - SKLPort3
[6]   - SKLPort4
[7]   - SKLPort5
[8]   - SKLPort6
[9]   - SKLPort7


Resource pressure per iteration:
[0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]
-      -     4.00   18.00   -     1.00    -     5.00   6.00    -

Resource pressure by instruction:
[0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    Instructions:
-      -      -      -      -      -      -      -      -      -     xorl       %eax, %eax
-      -      -     1.00    -      -      -      -      -      -     crc32q     %rax, %rax
-      -      -      -      -      -      -      -     1.00    -     movl       $1, %ecx
-      -      -     1.00    -      -      -      -      -      -     crc32q     %rcx, %rax
-      -      -      -      -      -      -     1.00    -      -     movl       $2, %ecx
-      -      -     1.00    -      -      -      -      -      -     crc32q     %rcx, %rax
...
-      -      -      -      -      -      -      -     1.00    -     movl       $15, %ecx
-      -      -     1.00    -      -      -      -      -      -     crc32q     %rcx, %rax
-      -      -      -      -     1.00    -     1.00   1.00    -     retq


Timeline view:
                    0123456789          0123456789          0
Index     0123456789          0123456789          0123456789

[0,0]     DR   .    .    .    .    .    .    .    .    .    .   xorl    %eax, %eax
[0,1]     DeeeER    .    .    .    .    .    .    .    .    .   crc32q  %rax, %rax
[0,2]     DeE--R    .    .    .    .    .    .    .    .    .   movl    $1, %ecx
[0,3]     D===eeeER .    .    .    .    .    .    .    .    .   crc32q  %rcx, %rax
[0,4]     DeE-----R .    .    .    .    .    .    .    .    .   movl    $2, %ecx
[0,5]     D======eeeER   .    .    .    .    .    .    .    .   crc32q  %rcx, %rax
...
[0,30]    .    DeE---------------------------------------R  .   movl    $15, %ecx
[0,31]    .    D========================================eeeER   crc32q  %rcx, %rax

Based on the “Instruction Info” table, crc32q has latency 3 and throughput 1: Every clock cycle, we can start processing a new invocation on port 1 ([3] in the table), but it takes 3 cycles for the result to be available.

Instructions decompose into individual micro operations (or “uops”). The resources section lists the processor execution pipelines (often referred to as ports). Every cycle uops can be issued to these ports. There are constraints - no port can take every kind of uop and there is a maximum number of uops that can be dispatched to the processor pipelines every cycle.

For the instructions in our function, there is a one-to-one correspondence so the number of instructions and the number of uops executed are equivalent (32). The processor has several backends for processing uops. From the resource pressure tables, we see that while crc32 must execute on port 1, the movl executes on any of ports 0, 1, 5, and 6.

In the timeline view, we see that for our back-to-back sequence, we can’t actually begin processing the 2nd crc32q for several clock cycles until the 1st crc32q hasn’t completed. This tells us that we’re underutilizing port 1’s capabilities, since its throughput indicates that an instruction can be dispatched to it once per cycle.

If we restructure BlockHash to compute 3 parallel streams with a simulated combine function (the code uses a bitwise or as a placeholder for the correct logic that this approach requires), we can accomplish the same amount of work in fewer clock cycles (Compiler Explorer):

uint32_t ParallelBlockHash(const char* p) {
 uint32_t crc0 = 0, crc1 = 0, crc2 = 0;
 for (int i = 0; i < 5; ++i) {
   crc0 = _mm_crc32_u64(crc0, 3 * i + 0);
   crc1 = _mm_crc32_u64(crc1, 3 * i + 1);
   crc2 = _mm_crc32_u64(crc2, 3 * i + 2);
 }
 crc0 = _mm_crc32_u64(crc0, 15);
 return crc0 | crc1 | crc2;
}

Iterations:        1
Instructions:      36
Total Cycles:      22
Total uOps:        36

Dispatch Width:    6
uOps Per Cycle:    1.64
IPC:               1.64
Block RThroughput: 16.0

Timeline view:
                    0123456789
Index     0123456789          01

[0,0]     DR   .    .    .    ..   xorl %eax, %eax
[0,1]     DR   .    .    .    ..   xorl %ecx, %ecx
[0,2]     DeeeER    .    .    ..   crc32q       %rcx, %rcx
[0,3]     DeE--R    .    .    ..   movl $1, %esi
[0,4]     D----R    .    .    ..   xorl %edx, %edx
[0,5]     D=eeeER   .    .    ..   crc32q       %rsi, %rdx
[0,6]     .DeE--R   .    .    ..   movl $2, %esi
[0,7]     .D=eeeER  .    .    ..   crc32q       %rsi, %rax
[0,8]     .DeE---R  .    .    ..   movl $3, %esi
[0,9]     .D==eeeER .    .    ..   crc32q       %rsi, %rcx
[0,10]    .DeE----R .    .    ..   movl $4, %esi
[0,11]    .D===eeeER.    .    ..   crc32q       %rsi, %rdx
...
[0,32]    .    DeE-----------R..   movl $15, %esi
[0,33]    .    D==========eeeER.   crc32q       %rsi, %rcx
[0,34]    .    D============eER.   orl  %edx, %eax
[0,35]    .    D=============eER   orl  %ecx, %eax

The implementation invokes crc32q the same number of times, but the end-to-end latency of the block is 22 cycles instead of 51 cycles The timeline view shows that the processor can issue a crc32 instruction every cycle.

This modeling can be evidenced by microbenchmark results for absl::ComputeCrc32c (absl/crc/crc32c_benchmark.cc). The real implementation uses multiple streams (and correctly combines them). Ablating these shows a regression, validating the value of the technique.

name                          CYCLES/op    CYCLES/op    vs base
BM_Calculate/0                   5.007 ± 0%     5.008 ± 0%         ~ (p=0.149 n=6)
BM_Calculate/1                   6.669 ± 1%     8.012 ± 0%   +20.14% (p=0.002 n=6)
BM_Calculate/100                 30.82 ± 0%     30.05 ± 0%    -2.49% (p=0.002 n=6)
BM_Calculate/2048                285.6 ± 0%     644.8 ± 0%  +125.78% (p=0.002 n=6)
BM_Calculate/10000               906.7 ± 0%    3633.8 ± 0%  +300.78% (p=0.002 n=6)
BM_Calculate/500000             37.77k ± 0%   187.69k ± 0%  +396.97% (p=0.002 n=6)

If we create a 4th stream for ParallelBlockHash (Compiler Explorer), llvm-mca shows that the overall latency is unchanged since we are bottlenecked on port 1’s throughput. Unrolling further adds additional overhead to combine the streams and makes prefetching harder without actually improving performance.

To improve performance, many fast CRC32C implementations use other processor features. Instructions like the carryless multiply instruction (pclmulqdq on x86) can be used to implement another parallel stream. This allows additional ILP to be extracted by using the other ports of the processor without worsening the bottleneck on the port used by crc32.

Limitations

While llvm-mca can be a useful tool in many situations, its modeling has limits:

Memory accesses are modeled as L1 hits. In the real world, we can have much longer stalls when we need to access the L2, L3, or even main memory.
It cannot model branch predictor behavior.
It does not model instruction fetch and decode steps.
Its analysis is only as good as LLVM’s processor models. If these do not accurately model the processor, the simulation might differ from the real processor.

For example, many ARM processor models are incomplete, and llvm-mca picks a processor model that it estimates to be a good substitute; this is generally fine for compiler heuristics, where differences only matter if it would result in different generated code, but it can derail manual optimization efforts.

Closing words

Understanding how the processor executes and retires instructions can give us powerful insights for optimizing functions. llvm-mca lets us peer into the processor to let us understand bottlenecks and underutilized resources.

Performance Tip of the Week #98: Measurement has an ROI

2025-10-03T00:00:00-04:00

Originally posted as Fast TotW #98 on August 23, 2025

By Chris Kennelly

Updated 2025-10-03

Quicklink: abseil.io/fast/98

Effectively measuring optimization projects is an important part of the lifecycle of an optimization. Overlooking a large positive (or negative) externality can cause us to make the wrong decisions for choosing our next steps and future projects. Nevertheless, this quest for accuracy needs to be balanced against the ROI from a better measurement. In this episode, we discuss strategies for deciding when to invest more time in measuring a project and when to move on.

Getting the big picture right

In choosing a measurement strategy, we want to get the big picture right. We want to look for where a difference changes our decisions for a project: Do we roll back because something wasn’t really helpful, the complexity unwarranted, or do we double down and keep working in the same space following success?

Spending a small amount of effort to realize a 2x difference in our measured results can have a tremendously positive ROI: The extra measurement time is easier than finding, developing, and landing another equally-sized optimization from scratch. Conversely, spending twice the effort to refine the measurement of a much smaller optimization has a poor ROI, as the added precision is lost in the noise of larger effects.

It is easy to fixate on the numbers that we have on the page while overlooking the numbers that are off the page. It is more important that we don’t overlook large positive (or negative) externalities that change our conclusions than to eke out a 7th digit of precision.

Significant figures

Optimization outcomes are quantitative, which can make them an attractive nuisance for believing they are overly precise and accurate. For many techniques, we can only measure 1 or 2 significant digits.

With a tool like Google-Wide Profiling, we can represent the fraction of Google fleet cycles in TCMalloc as a 17 decimal digit, double-precision floating point number. Writing out this many digits doesn’t actually mean the number has that precision. Spending a few additional milliseconds in TCMalloc on a single server will change some of the digits in that number. While the overall trend is stable, there is a day-to-day standard deviation in the total. A second-order effect with small magnitude relative to that variation might be entirely lost in the noise.

Aggregating more days can give us more samples, but longer longitudinal studies can experience confounding factors from unrelated changes in the environment if the true effect size is small. We may have achieved a more precise measurement, but changes to the environment will negatively impact our accuracy beyond the added precision.

Precision and accuracy

We want to avoid confusing precision for accuracy, and vice-versa. Similar to how a long duration analysis might be stymied by confounding factors, a carefully controlled experiment might claim a high precision result without being accurate.

Many load tests repeatedly replay traffic and compare the performance between a modified and baseline configuration. These repeated runs can deliver large sample sizes to produce low standard deviation estimates, instilling a (potentially false) belief that the result is accurate. By construction, load tests are simplifications of production and so they may omit certain workload types, platforms, or environment features that would be seen in widespread deployment.

Even with tools like production A/B experiments, which can help give us accurate results by testing against the full variety of workloads, we still must account for nuances in the technique:

Our sample has to be representative. For example, an A/B test in a single data center will encounter a different subset of workloads than the fleet as a whole.
Our control and experiment groups have to be sufficiently isolated from each other.
Our experimental setup needs to avoid statistical pitfalls.

Spending time to account for these factors makes sense if we’re making real gains in accuracy: Large scale data locality and contention effects can be challenging to measure by other means. For small changes without these externalities, we may expend a lot of effort to measure without obtaining a better measurement.

Increasing statistical power through aggregation

Small changes create agility by breaking them into manageable, focused chunks, which aids testing, debugging, and review. This can complicate a measurement story, since it adds more pieces that we need to track.

Rather than allow the measurement tail to wag the dog by forcing us to combine unrelated optimizations or forgo them entirely, we may opt to track the aggregate benefit from a series of optimizations made over a brief period of time.

As we iteratively optimized TCMalloc’s fast path, we could track the overall trend and the performance impact against the baseline. Each individual change could be evaluated through inspection of assembly or microbenchmarks.
During the SwissMap migration, we could track the overall trend in associative container costs. Based on the overwhelming performance results for early deployments, each individual change could be primarily evaluated for correctness with careful benchmarking reserved for only a fraction of the changes in the migration.

Even if we’re pursuing a general strategy of aggregating similar changes, we may want to separate out the effects of more tradeoff-heavy optimizations. A change that creates subtle edge cases or long-lived technical debt might be worth it if it’s a sufficiently large optimization. If we don’t end up realizing the benefits, we might prefer to roll it back in favor of looking for simpler approaches.

Balancing complexity and simplicity

Aiming for greater accuracy can tempt us to pursue more complex methodologies. An approach that is easier to implement, but captures most of the impact and regression signals from our project, can afford us more time to work on the actual optimizations, look for externalities, and be just as accurate in the end.

Importantly, a simple measurement should not be confused with an incomplete one. The goal is to avoid needless precision, not to overlook significant impacts like transitive costs on other systems or shifts between different resource dimensions.

The more complex approach that captures small effects can draw attention to its details, causing us to overlook the larger effects that we missed. Even though we explicitly made a good faith attempt to quantify everything, it is easier to focus on the words on the page than the words absent from it.
While we might capture additional phenomena relevant to performance, the additional factors we consider introduce new sources of error for us. Joining multiple data sources can be tempting; but if we are not familiar with their idiosyncrasies, we might end up with surprising (or wrong) results.
Simple is explainable. “This performance optimization only has an effect during Australian business hours” might be completely correct due to diurnal effects, but it is harder to see the immediate connection between what our approach left implicit and a causal explanation.

When we consider additional factors for measuring a project, we should try to gauge the contribution of each relative to the significant figures provided by the others. If one signal contributes 1% +/- 0.1pp, it will overwhelm another term that contributes 0.01% +/- 0.001pp.

Conclusion

In choosing a measurement strategy, we want to strike a balance between precision, accuracy, and effort. Accurate measurements can help us to continue pursuing fruitful areas for adjacent optimizations, but we should be mindful where increasing effort produces diminishing returns.

Performance Tip of the Week #26: Fixing things with hashtable profiling

2025-07-16T00:00:00-04:00

Originally posted as Fast TotW #26 on May 20, 2020

By Chris Kennelly and Zie Weaver

Updated 2025-07-16

Quicklink: abseil.io/fast/26

As discussed by Matt Kulukundis in his 2019 CppCon talk, identifying a “slow” hashtable from a pure CPU profile can be challenging. Abseil’s C++ hashtables have a built-in profiler. In this episode we describe what insights about the hash function quality and hash collisions it can provide, making them discernible at scale. We also look at a couple of case studies where this information was used to improve Google’s production fleet efficiency.

Starting the analysis

We can gather hashtable profiles by collecting them from a server in production and loading them into pprof.

The data can be used to identify problematic hashtables. Bad hashtables usually come from a low-entropy hash function or using the same hash function for both the table and sharding across instances. These can be addressed by:

Using absl::Hash to provide a well-mixed hash function.
Using a distinct hash function by salting the hash.

Finding stuck bits

Stuck bits are bits that never change across every hash. Naturally, this increases the probability of a collision exponentially in the number of stuck bits.

We can look at a few examples to demonstrate causes of stuck hash functions.

Case study: `InternalSubscriber::MessageReceived`

This function showed up multiple times when searching for stuck bits. It’s in a client library used in many servers.

We see an insertion into an indexed array called pending_ack_callbacks:

pending_ack_callbacks_[i].shard.insert(ack_cb, ack_callback);

Its definition is an array of hashtables:

struct {
  ABSL_CACHELINE_ALIGNED absl::flat_hash_map<Closure*, Closure*> shard;
} pending_ack_callbacks_[kNumPendingAckShards];

where kNumPendingAckShards = 256. Suspiciously, our stuck bits were 0xff =

We choose our shard based on the hash:

/* static */
int InternalSubscriber::ClosureAddressToShard(Closure *address) {
  absl::flat_hash_set<Closure*>::hasher h;
  return h(address) % kNumPendingAckShards;
}

For the hashtable of any shard, all the bottom bits will be the same by definition. The hashtable uses these same bits to determine where to insert new values, so we see worse performance due to collisions.

This was fixed by salting the hash that determines sharding:

absl::Hash<std::pair<Closure*, int>> h;
return h(std::pair(address, 42)) % kNumPendingAckShards;

This salting technique provides distinct hash functions for shard lookup and table lookup. This halved the cost of insertions in this library.

Case study: `ProcessWatchDone`

In a different library, we found an unusual pattern of stuck bits: 0x4030802800000000. Before it was fixed, we found the declarations:

typedef absl::node_hash_map<NamespaceKey, int64, NamespaceKeyHash>
    NsKeyRequestIdMap;

Since it the code previously used std::unordered_map and NamespaceKey wasn’t hashable with std::hash, it specified a custom hasher, NamespaceKeyHash. It used using GoodFastHash:

size_t NamespaceKeyHash::operator()(const NamespaceKey& ns_key) const {
  return GoodFastHash<NamespaceKey>()(ns_key);
}

NamespaceKey is a pair of Cords. But GoodFastHash does not properly mix bits for std::pair:

typedef std::pair<absl::Cord, absl::Cord> NamespaceKey;

Migrating this to use absl::Hash fixed this by improving mixing on the std::pair.

Finding tables with many collisions

When a hashtable has more collisions, it has to do more probes to find a particular element for a lookup. With more collisions, lookup performance scales with O(n) rather than O(1). The required memory loads might be uncached, hurting performance.

A perfect probe length is 0, which means we found the object right where we first looked. If the average probe length is greater than 0.1, then the table is probing more often than should be necessary as ~10% of keys are encountering collisions. We can calculate average probe length by dividing the total_probe_length by the sum of the num_erases and size (these two numbers capture the total number of elements that have been inserted into the table).

Case study: `PortMapperImpl::UpdateActiveMap`

This one showed a max probe length of 133 and an average probe length of 8.4. This is effectively performing a linear search on many lookups.

We can look at where we are mutating the table:

// Allocate new ConnLabel element and update counters
iter = active_map_->find_or_insert(*flow);
iter->second = new ConnLabel;

active_map_’s definition points us to:

typedef priority_hash_map<IpFlow,
                          ConnLabel*,
                          IpFlowPrioritizer> ClientPortMap;

priority_hash_map is a custom type wrapping a SwissMap, but with less-than-ideal defaults:

template<class _Key,
         class _Val, // NOLINT - "Failed to find complete declaration of
                     //           class _Val  [build/class] [5]"
         class _PriorityFunc,
         class _PriorityKey = int,
         class _PriorityHash = hash<_PriorityKey>,
         class _PriorityEqual = std::equal_to<_PriorityKey>,
         class _PriorityCompare = std::less<_PriorityKey>,
         class _KeyHash = hash<_Key>,
         class _KeyEqual = std::equal_to<_Key> >
class priority_hash_map {
 public:
  typedef absl::flat_hash_map<_Key, _Val, _KeyHash, _KeyEqual> hash_map_type;
  ...

_PriorityHash and _KeyHash are using hash<>, which provides a custom hash specialization:

template<> struct hash<IpFlow> {
  size_t operator()(const IpFlow& flow) const {
    return flow.remote_ip()
           (flow.proto() << 24) ^ flow.local_port() ^
           (flow.remote_port() << 16);
 }
};

The xor‘d bits can lead to poor mixing. For example, all you’d need for a collision is an IP 10.0.0.33 on port 2038 and another IP 10.0.0.32 on port 2039. To fix this, we implemented AbslHashValue for IpFlow so priority_hash_map could switch to absl::Hash:

// Allow this to be used as a hash map key.
template<typename H>
friend H AbslHashValue(H h, const IpFlow& flow) {
 return H::combine(
     std::move(h),
     flow.remote_ip(),
     flow.proto(),
     flow.local_port(),
     flow.remote_port());
}

Hashtable statistics

For each profiled hashtable, we record statistics in several tags:

capacity tells the exact capacity of the hashtable.
size is the current number of elements in the hashtable.
The probe length statistics (max_probe_length, total_probe_length) tell us how many extra lookups were needed to find an element. In an ideal hashtable, we find the element we are looking for on the first lookup (probe length = 0). If there are collisions, we’ll have a higher probe length. max_probe_length tells us the worst case probe length for any element in the hashtable. total_probe_length is the sum of probe lengths for all elements of the hashtable.
stuck_bits is a bitmask that will indicate any bits for which the hash codes in the table have only seen one value (either zero or one). For a good hash function this should be 0 for any table >10 elements. This is implemented as a (running & of all hashes) | ~(running | of all hashes). A stuck bit indicates that our hash function may not be providing sufficient entropy.
num_rehashes records the number of times the table has been rehashed. Calling reserve with an appropriate size (close to the true size of the table) can avoid reduce hashes.
max_reserve records the maximum size passed to a call to reserve for the instance, or 0 if no such call has occurred. This can be useful for identifying tables that are too large (size << capacity) because they were called with too large a size. Similarly, tables with many rehashes can save on rehashing costs by calling reserve with a sufficient size.
num_erases is the number of elements that have been erased from the hashtable (since the last rehash). The sum of num_erases and size indicates the number of elements added to the table since the last rehash.
inline_element_size is the size of the elements of the flat part of the array. For flat hashtables, this is the size of the key-value pair. For node hashtables, this is the size of the pointer to the key-value pair.
key_size is equal to sizeof(key_type) for the table.
value_size is equal to sizeof(value_type) for the table. On sets, sizeof(key_type) == sizeof(value_type). On maps, value_type holds both key_type and mapped_type with appropriate padding for alignment.
soo_capacity is the number of elements in the table’s small-object optimization (SOO).
table_age reports the age in microseconds of the table.

Detecting bad hashtables in unit tests

One additional safeguard is that we can run the profiler while running our tests. This requires running with the environment variable ABSL_CONTAINER_FORCE_HASHTABLEZ_SAMPLE=1.

Since this triggers a test failure, its threshold for alerting is fairly high. Unit tests may not insert sufficiently large numbers of elements into the hashtables to produce collisions.

Closing words

Finding issues in hashtables from a CPU profile alone is challenging since opportunities may not appear prominently in the profile. SwissMap’s built-in hashtable profiling lets us peer into a number of hashtable properties to find low-quality hash functions, collisions, or excessive rehashing.

Performance Tip of the Week #95: Spooky action at a distance

2025-07-14T00:00:00-04:00

Originally posted as Fast TotW #95 on July 1, 2025

By Chris Kennelly

Updated 2025-07-14

Quicklink: abseil.io/fast/95

Shared resources can cause surprising performance impacts on seemingly unchanged software. In this episode, we discuss how to anticipate these effects and externalities.

Normalization techniques

Workload changes can confound longitudinal analysis: If you optimize a library like protocol buffers, does spending more time in that code mean your optimization didn’t work or that the application now serves more load?

A/B tests can control for independent variables. Nevertheless, load balancing can throw a wrench into this. A client-side load balancing algorithm (like Weighted Round Robin) might observe the better performance of some tasks and send more requests to them.

In absolute terms, the total CPU usage of both experiment arms might be unchanged.
In relative terms, there may be small, second order changes in relative time (for example, %-of-time in Protobuf). This might hide the true significance of the optimization or worse, lead us to believe the change is a regression when it is not.

For individual jobs, normalizing by useful work done can let us compare throughput-per-CPU or throughput-per-RAM to allow us to control for workload shifts.

Choosing the right partitioning scheme

Where we have shared resources at the process, machine, or even data center level, we want to design an experimentation scheme that allows us to treat the shared resources as part of our experiment and control groups too.

Cache locality and memory bandwidth

Cache misses carry two costs for performance: they cause our code to wait for the miss to resolve and every miss puts pressure on the memory subsystem for other code that we are not closely studying.

In a previous episode, we discussed an optimization to move a hot absl::node_hash_set to absl::flat_hash_set. This change only affected one particular type of queries going through the server. While we saw 5-7% improvements for those types of queries, completely unmodified code paths for other query types also showed an improvement, albeit smaller.

Measuring the latter effect required process-level partitioning, which selected some processes to always use absl::node_hash_set and other processes to always use absl::flat_hash_set. Without this technique, all requests handled by the server coexist with a 50:50 mix of modified/unmodified code paths. The modified code path would have shown a performance improvement from fewer cache misses on its lookups. For query types unaffected by the change, their data structures would have seen the same mix of cache pressure from neighboring modified and unmodified requests, rather than ever seeing a 100% modified or 100% unmodified neighbor. Per-request randomization would prevent us from measuring the “blast radius” impact of our change. This happens all the time for any change that uses a shared resource (aka most changes).

This challenge of experiment design can be especially pronounced with load tests. Without studying the right query distribution of different request types concurrently, we won’t see the full impact, both positive and negative, of our change.

Better hugepage coverage lifted all boats

Before Temeraire, TCMalloc’s hugepage-aware allocator, many teams had opted to not periodically release memory from TCMalloc’s page heap to maximize hugepage availability and CPU performance. Other, less CPU-intensive workloads released memory periodically to strike a different balance between CPU usage and RAM usage due to free pages resting on TCMalloc’s heap.

Several additional optimizations have since modified the behavior, but the initial rollout sought to maintain the same “release on request” characteristic of the old page heap to minimize tradeoffs. Ensuring that almost all applications used no more memory than they previously did allowed us to focus on CPU performance and a handful of edge cases in allocation patterns.

When we ran A/B experiments prior to the fleetwide rollout of Temeraire, we saw improvements in hugepage coverage, even for applications that did not periodically release memory.

Managing memory in a hugepage-aware fashion everywhere improved performance even where we did not anticipate substantial benefits. The new allocator allowed whole hugepages to be returned, avoiding physical memory fragmentation altogether, and allowed us to iteratively target already fragmented pages to satisfy requests.

These added benefits were only recognized by A/B testing at the machine level. While we had enabled Temeraire with the help of many eager, early adopters, we could only see the first order effect–direct impact on the modified application–and not the second order effect–better behaved neighbors–during those experiments.

Distributed systems

In larger, distributed serving stacks, we might go further to partition resources to prevent confounding our results.

For example, if we send more requests to server backends during an A/B experiment, the effect of higher load will be smeared across all of the backends, affecting both the experiment and control group’s latency. Several teams use a “stripes” setup for this purpose, partitioning the serving stack so that once a request is routed to one particular partition, it does not cross back into other partitions.
A change to job scheduling might impact how work is distributed to individual machines in a data center. If we were to improve performance on some machines, effectively lowering their utilization, the control plane might shift machine assignments to rebalance. While jobs may still have effectively the same amount of CPU time, a busier machine can translate into lower per-core performance due to memory bandwidth, cache contention, and frequency scaling effects that create a headwind for the experiment group.

Closing words

Managing and reducing contention for shared resources is a large source of performance opportunities. Nevertheless, recognizing these opportunities fully requires consideration of experiment design to ensure confounding factors do not obscure the real benefits from us.

Performance Tip of the Week #94: Decision making in a data-imperfect world

2025-06-27T00:00:00-04:00

Originally posted as Fast TotW #94 on June 26, 2025

By Chris Kennelly

Updated 2025-06-27

Quicklink: abseil.io/fast/94

Profiling tools are vital to narrowing the search space of all possible changes we could make to highlight the best uses of our time. In this episode, we discuss strategies for identifying when we might want to seek out more data and when we are in the midst of diminishing returns.

Focusing on outcomes

The ultimate outcomes that we want from our tools are insights that let us improve performance.

We can almost always do more analysis – increase the time horizon, auxiliary metrics considered, alternative bucketing strategies – to try to make a decision. This can be initially helpful at finding unforeseen situations, but we might mire ourselves in spurious findings or develop analysis paralysis.

Changing decisions

At some point, redundant tools, extra precision, or further experiments add less value than their cost to obtain. While we can seek these things anyway, if they aren’t changing the decisions we make, we could do without and achieve the same outcome.

It’s easy to fall into a trap of looking for one last thing, but this carries an opportunity cost. We will occasionally make mistakes in the form of unsuccessful projects. While some of these could be avoided in hindsight with extra analysis or a small “fail fast” prototype, these are tolerable when the stakes are low.

When considering an experiment to get additional information, ask yourself questions like “can the possible range of outcomes convince me to change my mind?” Otherwise, constructing additional experiments may merely serve to provide confirmation bias. The following are examples where the extra analysis seems appealing, but is ultimately unnecessary:

We might try out a new optimization technique with minimal rollout scope and complexity. If it works, we’ve gained confidence. If it fails, we should change our plans. If either of those possible outcomes might be insufficient to persuade us, we should instead develop a plan that will provide a definitive next step.

If we plan to iteratively move from microbenchmark to loadtest to production, we should stop or at least reconsider if the early benchmarks produce neutral results rather than moving to production. The fact that a benchmark can give us a result different from production might motivate us to move forward anyway, but tediously collecting the data from them is pointless if our plan is to unconditionally ignore them.
Estimates only need to be good enough to break ties. If our plan is to look at the top 10 largest users of a library for performance opportunities, improved accuracy might adjust the order somewhat, but not dramatically. Historical data might give us slightly different orders for rankings chosen using yesterday’s data versus last week’s, but the bulk of the distribution is often roughly consistent.

Failing fast

Consider a project where we’re optimistic it will work, but we have a long slog ahead of ourselves to get it over the finish line. There are a couple of strategies we can use to quickly gain confidence in the optimization or abandon it if it fails to provide the expected benefits:

A new API might be more expressive and easier to optimize, but we need to migrate our existing users to it. Testing and canarying a single user is easier to attempt (and rollback, if unsuccessful) than to deploy it everywhere before switching implementations.
We might be interested in deploying a new data layout, either in-memory or on-disk. A simple prototype that simulates an approximation of that layout can tell us a lot about the probable performance characteristics. Even before we can handle every edge case, the “best case” scenario not panning out gives us a reason to stop and go no further.

For example, as we attempt to remove data indirections, we might microbenchmark a few candidate layouts for a data structure, starting from the status quo std::vector<T*> to std::deque<T>, absl::InlinedVector<T*, 4>, and std::vector<T>. Each of these solutions has its merits based on the constraints of T and the access pattern of the program. Having a sense of where the largest opportunity is for the current access pattern can help us focus our attention and avoid a situation where we sink time into a migration before we can derisk anything.

Getting things over the finish line may still be 80% of the work, but our initial work will have derisked the outcome. Investing time in polish for a project that ultimately fails carries a high opportunity cost.

Updating our priors

The analysis we do is just to determine the course of action we should take. Ultimately, we care about the impact of the action, not how elegant our plans were. A promising change backed by good estimates and preliminary benchmarks has to be successful when deployed to production to actually be a success. Good benchmarks alone are not the outcome we are after. Sometimes we’ll find that promising analysis or benchmarks do not turn into benefits in production, and the outcome is instead an opportunity for learning.

If we chose among several candidate solutions to a problem, we should confirm that their qualities held up. For example, we might have picked a strategy that was more complex but perceived to be more reliable than an alternative. Even if the project was a success otherwise, but reliability instead suffered, we should reconsider if the alternatives are worth revisiting.

Data censorship

Our tools sometimes have blind spots that we need to consider when using them. Simplifications to get a “good enough” result can help our priors, but we should be cautious about extrapolating too broadly from them.

More data points with the same caveats merely make us overconfident, not more accurate. When the stakes are higher, cross-validation against other information can help uncover gaps. More data points from distinct vantage points are more valuable than more data points from the same ones. We should prime ourselves to consider what new evidence would cause us to reconsider our plans.

Profiler limitations

Many profilers are cost-conscious to minimize their impact. To do this, they employ sampling strategies that omit some data points.

Our hashtable profiler makes its sampling decisions when tables are first mutated. Avoiding a sampling decision in the default constructor keeps things efficient, but means that empty tables are not represented in the statistics. Using other profilers, we can determine that many destroyed tables are in fact empty.

Historically, TCMalloc’s lifetime profiler had a similar caveat. To simplify the initial implementation, it only reported objects that had been both allocated and deallocated during a profiling session. It omitted existing objects (left censorship) and objects that outlived the session (right censorship). This profiler has since been improved to include these, but understanding a profiler’s limitations is crucial to avoiding drawing the wrong conclusions from biased data.

Profilers tracking CPU cycles are often measuring how long it took for an instruction to retire. The profile hides the cost of instructions that come after a high latency one. In other situations, diffusely attributed costs may obscure the actual critical path of the function found only by careful analysis or tools like llvm-mca.

These examples illustrate how not everything can be measured with a sampling-based profiler, but there are often different approaches.

Partial populations

Running a load test or even canarying a change in production for a single application can increase our confidence that something will work. Nevertheless, the limited scope doesn’t assure us that the effect will be the same on a wider population.

This pitfall cuts in both positive and negative directions. If we have an optimization for a library that isn’t used by an application, no amount of testing it with that application is likely to produce a real effect. A negative result where there is no opportunity for the optimization to show benefit should not deter us, but we might abandon a good idea because of the spurious result. Conversely, an optimization might falter as it is brought to a broader scale, as our initial data points were biased by streetlamp effects.

We can avoid this by measuring the effect on the wider population or by choosing a broadly representative set of data points, instead of just one. Minimally we should be confident that the method we have chosen to evaluate the optimization has the potential to show a positive or negative impact.

Closing words

As we work to gather data to guide our decisions, we should work to ensure we’re looking for features that could lead us to change our plans. It is easy to fall into the trap of seeking additional data points to increase confidence, but we may merely be falling into a trap of confirmation bias.

Performance Tip of the Week #93: Robots never sleep

2025-06-03T00:00:00-04:00

Originally posted as Fast TotW #93 on May 27, 2025

By Chris Kennelly

Updated 2025-06-03

Quicklink: abseil.io/fast/93

Techniques like presubmits are essential tools for effective software engineering. For performance optimization, changing observable but unspecified behaviors can be a large source of opportunities. In this episode, we discuss strategies for leaning on automation and additional tools to make it easier to evolve software.

Prevent problems via technical means

Use technical means, rather than (eventually fallible) humans, to prevent problems. Humans can be a valuable line of defense for unknown unknowns, but they cannot be the only line of defense for everything. While human-executed checklists are great, they also have to be short and that makes them unsuitable for encoding a bunch of knowledge about a problem domain. It can be tempting to add to the checklist after every incident and never subtract from it, but this leads to inevitable toil or skipped steps. Robots never get tired.

When we introduced GoodFastHash as a faster alternative to std::hash, it was intended that we could change the algorithm from time to time. This property was documented in a comment. Over time, we accreted code depending on its existing behavior. This ranged from brittle tests relying on the stable order of iteration of std::unordered_map paired with GoodFastHash to production designs that relied on it for stable cache keys. This stymied performance and hash quality improvements to GoodFastHash.

Automated testing is a well-known software technique, but it can be worth bringing the automation to bear in new ways on new problems to shift-left. Consider an example outside of performance: Many outages have been caused by large, unanticipated changes in configurations. For example, a 100.0% reduction in configuration size is probably a bug (and an outage waiting to happen). An empty configuration file can be detected mechanically, leaving humans to worry about all of the much more subtle gotchas. Presubmit checks that detect and block large diffs without an explicit exception can stop many issues in their tracks.

We can apply automation towards keeping code flexible over the long run, enabling us to make it more efficient while simultaneously ensuring reliability.

Strategies for taming subtlety

As the complexity of our software stack grows, tending to every corner reliably and manually is challenging. An API might promise one thing, but the software built on top of it relies on its implementation details much farther down the stack.

Compile-time hardening

The very first C++ TotW was for string_view, which can improve performance by avoiding string copies. This makes it attractive for regular usage, but it is bug-prone due to dangling references.

Reference-like types like absl::string_view and absl::Span allow us to write code agnostic to the underlying data storage. A string could be backed by constant data in a global or a heap allocated copy. A container of N elements might be stored as an absl::InlinedVector, removing a heap indirection in the common case.

As with other references, these types come with downsides: The usage of the reference might not coincide with the lifetime of the underlying storage, leading to use-after-free/use-after-return bugs. While sanitizers can detect these bugs, they can have false negatives depending on test coverage. Fortunately, source code annotations like ABSL_ATTRIBUTE_LIFETIME_BOUND can find many of them at compile time. These annotations allow us to shift-left on finding bugs, allowing the engineer actively writing code to fix the issue right away, rather than waiting for a production outage with its commensurately higher costs.

Similarly, lock annotations allow us to programmatically convey thread safety requirements so they can be checked at compile time. Rather than having to remember to hold a lock on every access of a member variable by hand, we can transform these into build failures when we access it without holding the lock. While these annotations can’t express subtle, but correct, locking strategies, they can allows us budget more time for those subtle cases, saved by not having to carefully audit even mundate concurrency patterns.

Runtime hardening

Over time, we’ve developed several techniques for warding off Hyrum’s Law. While there are any number of implementation details we can obfuscate, the following techniques have proven useful:

Memory addresses for low-cost entropy: Taking the address of a global is extremely efficient (a rip-relative lea on x86) and powers absl::Hash’s randomization. Having learned from the lessons of GoodFastHash, absl::Hash deliberately used a random number to seed its hash computation. This ensured tests were more robust because they could not rely on fragile implementation details. Consequently, it has been possible to land several optimizations to replace its implementation wholesale without being blocked by brittleness.

This introduces just enough entropy from ASLR and linker orderings to make it hard to rely on its values across tests and production tasks. Similarly, heap allocations can provide instance-specific entropy. We use this in SwissMap to ensure different table instances have different iteration orders.
Extra checks in debug builds / sampling: We want to maintain implementation freedom to ensure we can improve performance in future. However, expensive checks would reduce performance if done in optimized builds, defeating part of the purpose. For these, we can put most of our checks in debug builds.

Sized delete from C++14 allows our deallocation path to avoid a costly radix tree walk, but incorrect sizes would lead to data corruption. In debug builds, TCMalloc checks the provided size against its internal metadata groundtruth. TCMalloc already samples periodically to drive heap profiling, allowing it to provide extra checks on these sampled objects to proactively find further issues in production.
Counterfactual checks: In sanitizer builds, we check that SwissMap’s iterators remain valid after any potential invalidations. Even if a particular insert doesn’t cause a rehash but could have caused one, the iterator is considered invalid. This allows us to prevent dependencies on SwissMap’s growth rate, iterator invalidation behaviors, and so forth, even if the present implementation used in day-to-day builds overdelivers against its guarantees.

Enabling these defenses widely means we get benefits from everyone’s tests. These approaches ensure that code does not depend on characteristics that are sensitive to implementation details, so we can change implementations quickly without breaking code.

Static analysis

Tools like Clang-Tidy can spot problematic patterns and flag them at code review time. This can shift left on preventing problems, rather than waiting for a failed runtime check in production.

For example, some protobuf optimizations are sensitive to misuse of const_cast. This misuse is partly stopped by placing default instances in global, read-only storage, causing the program to crash if the instance is mutated. Since it isn’t ideal to crash on potentially under-tested code paths and because this technique doesn’t cover all types of misuse, a Clang-Tidy check can flag const_cast where the type is a protobuf instance.

Ratchet-and-pawl

Matt Kulukundis popularized the phrase “ratchet-and-pawl” to describe incremental migrations that prevented backsliding. This allows us to make progress where we can, secure in the knowledge that the problem will not get worse.

During the SwissMap migration, it was infeasible to randomize all existing unordered containers due to failing tests and actual production dependencies on the current behavior. Individual instances could be migrated where tests passed (or were made to pass). As the migration progressed, visibility allowlists on lesser used containers (dense_hash_map) and Clang Tidy on more common ones (std::unordered_map) reduced new usages of the legacy containers. An allowlist with a thousand entries might seem inelegant, but it’s a powerful tool for preventing backsliding. The list can be ratcheted down over time by hand or by automated cleanups as uses are removed.

Costs of avoided guarantees

Preserving implementation freedom carries costs, whether due to performance overheads when we need to work around it, opportunity costs, or simply worse ergonomics.

With hashtables, changing hash algorithms, initial sizes, and growth rates had come up time and time again, stymied by brittle tests, prior to the introduction of SwissMap and Abseil Hash. SwissMap using its heap allocation’s address as an entropy source makes copies more expensive on primitive types: Rather than simply memcpy‘ing the raw data, we need to rehash our keys. Making the order more deterministic could let us improve performance, but the benefits outweigh these small costs.

As useful as it is to maintain as much implementation flexibility, it is important to focus on the implementation details most likely to change. Just because we can obscure an implementation detail doesn’t necessarily mean it’s worth the runtime and engineering cost if it’s unlikely to ever change or valuable to do so.

Validating policy choices

For large, performance-critical optimizations, we may want to carefully test that the desired performance characteristics remain, even where the code would be functionally correct with or without the optimization.

TCMalloc’s tests often relied on making lots of actual allocations to exercise edge cases, but with the development of Temeraire, tests could exercise the policies to simulate gigabytes of memory usage without actually using that much physical memory. Leveraging this design choice allowed a wider variety of edge cases to be tested and regressions avoided as new improvements are brought online.
Protobuf elides copies in several situations. Tests ensure that this implementation detail is preserved to avoid substantial regressions.

If we like our optimizations, we should put a test on it. Automation prevents regressions upfront, rather than waiting for after-the-fact debugging.

Closing words

Shifting problem-finding away from humans and onto automated mechanisms lets us focus on the bigger picture and improve our velocity. When things do break, we save human time in the long-run because we can spend less effort figuring out what caused a regression.

Performance Tip of the Week #90: How to estimate

2025-02-06T00:00:00-05:00

Originally posted as Fast TotW #90 on February 6, 2025

By Chris Kennelly

Updated 2025-08-11

Quicklink: abseil.io/fast/90

Estimating savings can be useful for improving our decision making. In this episode, we discuss how to make and use estimates in the optimization lifecycle.

Why estimate?

While looking for and developing optimizations, we use performance estimates frequently to decide how to approach problems and spend time:

A sense of how large a problem domain is might tell us where to look for opportunities in the first place.
Given finite time and multiple projects to work on, we will prioritize those with the highest return on investment. Estimation lets us fill in a guess for “return,” before we have it in hand. Our goal is to make better decisions, not perfect-in-hindsight ones.
Within a specific optimization project, an estimate of the benefit might inform us how much complexity (fiddly edge cases, technical debt, etc.) we might be willing to tolerate.

The outcome of better estimates is better decisions, which informs us how much precision we might need. If we’re considering project A that will make things 5% faster and project B that will make things 0.1% faster, we don’t need to worry about additional significant figures for project A’s estimate. A more precise estimate for project A of 5.134% won’t change our prioritization all things being equal (effort, complexity, etc.). In that situation, we should instead prefer to focus on information that could change our decision rather than gathering unneeded precision that won’t.

How big is it?

Profiles are our jumping-off point for sizing how large an opportunity might be. When we’re deciding between competing choices, this information can give us a high-order bit to winnow the set for further investigation.

The best-case scenario is that we completely eliminate a bottleneck: If we drove something’s cost all the way to zero, would it be a worthwhile opportunity, or are there larger ones to pursue? This can be a handy heuristic as we’re filling in blanks for where an optimization might not be applicable. We can assume the best case scenario–an optimization applies everywhere, to the fullest degree–and see if the result is still meaningful. If it’s not, a more accurate estimate won’t change the outcome.

Exploring profiles

The most common starting point is a CPU or heap profile. We can refine this information to identify a rough upper bound for the opportunity. To illustrate this, consider a few examples:

As of February 2025, the protobuf implementation checks the_repeated_field.size() != 0 for deciding whether a repeated field is present and in need of further recursion. While there’s nothing inherently wrong with this, a message with many absent fields might be able to check their presence more efficiently with “has bit”-like auxiliary data rather than touching many cachelines.

In the very best case, we can make two assumptions. First, the added cost of toggling a bit every time we create a mutable reference to the field is free. Second, every call to size() internal to the protocol buffer implementation can be elided with this because the fields are all absent. This is a gross simplifying assumption (and not true), but it provides an upper bound.
The C++ protocol buffer parsing implementation uses a series of tail-called functions for parsing each type of field. Prior to the preserve_none calling convention added in LLVM and adopted by protobuf, these function bodies would preserve registers–adding extra push/pops and code size pressure–that would be unneeded.

We could use profiles to identify parsing functions with push/pop instructions setting up frames, sum their total cost, and scale based on past experience from optimizations that had eliminated stack frames in other libraries to produce an estimate.
Copies and destructors consume a large fraction of compute. While we cannot necessarily eliminate all of those copies and their corresponding destructors, some can be elided by using a const reference or a move. Tools allow us to join profiles against a specific C++ pattern to estimate the potential savings.

Leveraging other data sources

Other PMU events and server facts can give us more information about the size and scope of a problem.

Events like page table walks or what fraction of effective pages were small can tell us about the total size of the opportunity from huge pages.
Topdown analysis to break down whether a piece of code is frontend or backend-bound can tell us how much we can expect to save. If a particular function is largely backend-bound, reducing instruction count, branches, or applying SIMD might not be helpful. Conversely, if a function is frontend-bound, optimizing the memory accesses it makes might not be helpful.
Allocation size information from TCMalloc profiles can help tell us about the size of a particular container. If all of the memory requests coming from a std::vector<MyType> instance are for exactly sizeof(MyType) bytes, the vector’s size is likely 1.

While proving a strict bound may be difficult, distributions from the heap or container-specific profiles can give us a sense of what is most likely to be encountered and worth optimizing for.

How much can we change?

For identifying high-value projects, we can winnow many by making best-case assumptions for the project: We can completely eliminate the bottleneck and so on. While this is a helpful simplifying assumption, we often need to estimate how much we can move the needle by to get more precision when needed.

Past performance

Analogous past projects can help form a baseline for the estimates. For example, if job reshaping improved performance for a variety of workloads by 15% or more, we might apply a similar rule of thumb to our own reshaping project.

Temporarily disabling an optimization can give us a sense of how sensitive a workload is to a particular parameter. Ablating an existing optimization can be easier to implement than a prototype.

Adaptive Prefetching modulates HW prefetchers based on the machine’s memory bandwidth usage. While beneficial in aggregate, slicing by function allowed us to identify functions that were more sensitive to prefetching that were good candidates for software prefetching.

Speed of light

Comparing a simpler implementation that isn’t fit for purpose can tell us how close our current implementation is to the speed of light of the hardware. This can tell us about the cost of factors like data movement that we cannot make any faster without reworking data structures or the problem altogether.

memcpy or memset can be a ready stand-in for many data processing functions, since they simply move data without performing computation. If an existing implementation is already close to these substitutes, the headroom for optimization might be small.

Latency rules of thumb or known hardware characteristics can give us another lower bound.

Analytical methods

We may be able to deduce a fraction based on the known properties of a library.

SwissMap doubles its capacity on resize and supports a maximum load factor of 87.5%, so its minimum load factor is typically 43.75%.
A std::vector typically doubles when full, so size()/capacity() will be at least 50%.

A simple mean of the two extremes might be a good enough approximation. When profiles are available for the container, we can be even more precise.

Combining different pieces of information may also help us estimate headroom. While job reshaping has substantial tried-and-true savings results to draw from, plotting memory usage against load can let us deduce the fixed overheads of a workload. An actual job reshape might save even more by improving cache locality, but the estimate might be sufficient to guide prioritization.

Refactoring

We may want to refactor the code to make the opportunity more clear in our profiles. A small inline function adds no costs at runtime, but the debug information that we add at build time allows us to disambiguate the costs in our profile more easily.

Consider a function that does some work and may take a lock to update shared state afterwards. If we want to optimize lock contention, knowing where we spend time under that lock can help us identify the functions we want to prioritize for optimization.

Partial prototyping

A prototype that does not handle all edge cases can refine our estimates of what a complete implementation will provide.

Microbenchmarks can provide an estimate of how much faster we can make a library. While we can make them arbitrarily complex and realistic, we are likely better off moving to other methods or even production. We may want to ask ourselves how a result will change our decision path:

Is a neutral result likely to suspend our work, or would we try to make the microbenchmark “more realistic” by adding complexity?
Will we use the benchmark to fine tune the implementation? This can be helpful for improving our OODA loop, but we risk overfitting due to its pitfalls.

Another scenario we might encounter is that we have an implementation, but it does not support all edge cases yet. These might lead to performance regressions, or could be hard blockers due to correctness. With an implementation that covers common use cases, we can run load tests to get a sense of where an optimization works and by how much.

A partial prototype can let us test our hypothesis. If we don’t see the expected performance delta despite shortcuts, our initial estimate may have been overly optimistic. If we do see a performance benefit (and it is correct), we may be able to land it and iterate from there.

Partial deployment and counterfactuals

Counterfactual deployments can give us information in a lower-risk setting. Consider a situation where we can use multiple compression algorithms. Making a wholesale change to which algorithm we use in a system could be risky: We don’t know whether the new ratio would be better or what its performance characteristics are like.

By choosing an alternative algorithm on a small sample of data and discarding the actual compressed bytes, we can collect data about the performance of the new algorithm. While this comes at a small runtime cost–double compression for perhaps 1-in-1000 samples–that cost is low and only temporary.

Things to look out for

We want to avoid analysis paralysis. Carefully and precisely estimating every possible idea before commencing work could keep us busy indefinitely.

Many of these techniques are geared towards optimistic assumptions: We assume we can address the entire market. While the “worst case” might be no improvement that we land, we may want to scope where these scenarios can arise as well.

Using multiple strategies for producing an estimate can give us increased confidence in it. While we need to be cautious about anchoring bias and making a catastrophic, common assumption, having two (or three) distinct approaches land in the same ballpark can tell us that it might be a reasonable one.

It is valuable to understand where numbers in prior estimates came from. For example, if someone estimated a bottleneck could be improved by “10%” out-of-thin-air, we shouldn’t anchor on that assumption indefinitely. The improvement could be entirely correct–or entirely wrong–but we should challenge the assumptions when comparing against our own analysis.

Calibrate

After a project concludes, it’s useful to understand why our estimates were high or low compared to how a project actually came in. There’s no right or wrong answer here, so we don’t want to penalize ourselves for being wrong. Our goal is to learn from our misses to improve our thought processes in the future.

Closing words

Judicious use of estimates can guide optimization work, allowing us to identify the most impactful projects and design decisions.

Performance Tip of the Week #88: Measurement methodology: Avoid the jelly beans trap

2024-11-18T00:00:00-05:00

Originally posted as Fast TotW #88 on November 7, 2024

By Patrick Xia and Chris Kennelly

Updated 2024-11-18

Quicklink: abseil.io/fast/88

Measurement of performance optimizations often requires in-depth analysis due to the inherent stochasticity of workloads. We need to gather data to calculate the metrics we’ve selected. In this episode, we discuss the importance of defining your measurement methodology ahead of time rather than fishing for possible sources of significance after the data has been collected.

Choose a methodology, without peeking

Choose and publish a methodology before looking at the data. Ideally, this process is completed before any changes land. It is otherwise easy to pick the methodology that tells the best story (the “biggest number”) unwittingly after the fact. Preregistration of experiment methodology also helps avoid false positives in statistical analysis.

The Jelly Beans XKCD comic offers a humorous take on this situation, which is referred to alternately as data dredging or p-hacking. Even keeping this sort of situation in mind, data dredging remains a pernicious problem. Less obvious data dredging can occur in many ways.

Prefer simpler methodologies. Simpler methodologies offer fewer parameters to abuse and thus result in sounder analyses. Filters quickly combine to form a similar effect as the green jelly beans from the comic. A statement like “this performance optimization only has an effect during Australian business hours” would be generally suspect.
Identify which metrics are primary and which are secondary proxies. For example, if we ran an A/B experiment, we might have multiple performance metrics that we could consider. If we deferred making the choice of which metric to use, we might find post hoc reasons for preferring one metric or the other.
Reuse measurement techniques when appropriate. While we shouldn’t shy away from improving our techniques over time, reusing an approach reduces the risk that we’ve overfit our analysis to the results.

If we are making a number of similar changes, we should generally strive for measurement consistency across them.
Publish the observation interval of the analysis. Optional stopping of experiments increases the risk of false positives. A sufficiently random experiment that is run for an arbitrary amount of time will explore the entire outcome space. Stopping the experiment when we reach that outcome biases us towards finding a signal where there is none.
Precommit to a launch bar. For changes that add complexity, we want to be cautious that the results merit the long-term cost. The error bars for a very weakly positive result might overlap with zero.

If we’re removing complexity–deleting an unused calculation, cleaning up dead flags, etc., we would likely be inclined to do so anyway independent of the efficiency savings that are merely a nice-to-have bonus.

Publishing the methodology also means that we have fewer chances to add post-collection changes to analysis. Trivial-looking changes (“clearly errant outlier data points can be deleted”) may negatively affect the soundness of experiments.

Anchoring bias

Estimation is an important part of our project planning process. We worked on particular optimizations because we think it was important enough to be worth our time.

When we read out the result of our hard work, we need to put our estimate aside and not anchor on it. The estimate was intentionally rough. It only needed to be “good enough” to break ties in our decision making to prioritize project A over project B over project C.

We should avoid scrutinizing a result simply because it differs from our prior estimates and results.

A lower-than-expected result might be from optimistic assumptions that we made when estimating. Our “speed of light” for optimization might be that we would fully eliminate the cost of something, but the result in practice could be entirely different. To realize an earlier landing, we might have carved out exceptions and edge cases, rather than fully tackling them.
A higher-than-expected result might come about from a missed consideration. For example, when we enabled huge page-backed text, most of the upfront focus was on iTLB pressure because the optimization had “text” in the name. In practice, we also moved most of the executable’s constant global data onto huge pages as well, producing a drop in dTLB misses as well.

While we want to have a good calibration for our estimates, we can learn from the estimation mistakes as well. A “surprise” in either direction might indicate an opportunity that we did not notice upfront.

Nevertheless, extraordinary claims in either direction require extraordinary evidence. Sometimes an optimization appears to have an impact many times larger than we expect, or even than the apparent opportunity. Sometimes these are real due to step functions in cost or non-linear behavior around lock contention. More often, it is mismeasurement, and unexpectedly fortuitous results deserve the extra scrutiny and cross-validation.

Follow-through on the published methodology

After we have published a methodology, we should actually follow that methodology when reporting results. When going through the process of data analysis, the data shape may look completely different than expectations. It is important to obtain the results using the original methodology for posterity’s sake even when this occurs. Future adjustments may refine a readout of the impact of the project, but the number from the original methodology should still be produced.

It’s important to re-emphasize that we don’t anchor on the original estimate of impact. The shape of the data may look different because previous estimates were not accurate.

Post-measurement adjustments

After analyzing the data, it may become clear that some part of the methodology was lacking; for example, a confounding variable that was earlier not considered may become apparent in the analyzed data. Adjustments at this stage are incredibly prone to bias and statistical error, so additional caution is mandatory. The null hypothesis of a neutral result is also a perfectly acceptable result as well.

When considering a potential confounder, we want to have a scientifically sound mechanism to explain why we should include it. Suppose after we changed function A, we saw an unexpected improvement in function B. We could update our analysis to include the impact on function B, but we should be able to explain why we’re not including functions C or D as well.

The simplest criteria–all functions that use a particular data structure that we modified–is a more sound one than cherry-picking an arbitrary set of functions (e.g. all const functions affecting mutable members). “We included B because it improved and excluded C and D because they regressed” would be suspicious. When adding more criteria after-the-fact, we should be able to define them without peeking at the results from doing so.

While convenient, longitudinal analysis can pick up truly unrelated confounders from changes in usage:

Did another major optimization land to an application in the same release?
Did usage patterns for a library we were optimizing change?

Tossing data points after the fact because they “seem irrelevant” or “due to mix-shift” has a high bar. It’s very easy to bias our analysis by disproportionately suppressing negative outliers.

A/B experimentation takes more effort, but it can control for many of these variables and mix-shifts by isolating a single independent variable. Ablating or rolling back a change can also provide a compelling case for causality: If it did cause the unanticipated changes we saw, those should revert back as well with similar but reversed magnitude.

Unanticipated improvements can be a source of future opportunities by highlighting promising areas to look for more ideas, but this only works if the improvements are real. If we fool ourselves into believing there was an oversized effect, our follow-on projects may fail to pan out.

Conclusion

Committing to a measurement methodology ahead of time helps us preserve rigor when we might otherwise fish for possible sources of significance after the data has been collected.

Performance Tip of the Week #87: Two-way doors

2024-11-08T00:00:00-05:00

Originally posted as Fast TotW #87 on October 16, 2024

By Chris Kennelly

Updated 2025-11-17

Quicklink: abseil.io/fast/87

Jeff Bezos divides decisions between “one-way doors”–ones that are hard to reverse–and “two-way doors”–those that are easy to reverse. Different optimizations fall on each side of this divide. In this episode, we discuss patterns common to two-way doors to reduce risk without exhaustively analyzing the situation. Good decisions endure, while missteps can be corrected along the way.

Assessing reversibility

As we explore a new optimization idea, we want to prioritize blockers to landing and ignore (for now) less important details. This is easier said than done, since we need to figure out which subproblems are actually on our critical path and those which can be ignored.

Outside of major technical blockers–does a proposed optimization work at all, or to a sufficient degree to be meaningful–an important consideration is which decisions we need to make upfront because they are hard to change later. We approach this with a strong prior that most software decisions are easy to reverse, but discuss the hallmarks of those that are harder to.

Feature flags

Feature flags are a common technique for gating new functionality so that it can be developed and gradually tested before being rolled out. If an issue is recognized with a release, the flag update can be rolled back and the system restored to the previous, good known state.

Broadly speaking, fine-grained flag changes are easy to undo and thus two-way doors. The system was working before and we can go back to the drawing board, if and when a problem arises, to remedy it if we need to rollback.

Flags aren’t all equally important: A flag to launch (or turn down) a major product feature is far more weighty than a flag that controls the size of a cache as part of an optimization. Flags gating major product features, however, undergo far more consideration as part of the launch process.

APIs

Unlike flags where we can make a central decision to roll forward or roll back a particular setting and even adopt application-specific values (at least temporarily), new API surfaces can prove to be more of a one-way door at times.

While a new library is being developed, a build visibility rule might help constrain adoption to a manageable set of users. This can provide invaluable feedback on the ergonomics of the library and further battletesting while leaving things in a tractable enough state to “undo” later and migrate away from it.

For optimization projects, we might consider how impact scales with adoption. For example, adopting RCU might get most of its benefits from tackling a handful of the most contended data structures, but a vocabulary type like SwissMap might need thousands of usages to get meaningful traction. For SwissMap, nearly drop-in API compatibility makes a hypothetical rollback possible, whether by changing usage or by making the implementation a wrapper for std::unordered_map. SwissMap makes strictly fewer guarantees with respect to actually being unordered and iterator/pointer stability, so it could be easily substituted for the existing implementation.

At the other end of the spectrum are new programming paradigms. While a simple API might be possible to decompose in terms of other libraries, moving back and forth between coroutines and other asynchronous primitives might be challenging at the very least.

Similar considerations apply to releasing open source libraries: Having internal experience first, where it’s possible to talk to every user and update them as needed, can provide needed confidence that rough edges have been sanded down. Once released, compatibility guarantees might make it more challenging to make substantial changes without breaking existing users.

Data at rest versus data in flight

Data formats that only live “in flight,” for example, during an active RPC, face far different reversibility considerations than data that lives at rest, for example, if stored to disk.

Protocol buffers serve both of these roles by joining an in-memory representation to a wire format.

If we change an implementation detail of the in-memory format, whether by optimizing our parsing routines or changing how we lay out fields, we can improve efficiency without long-term ramifications. The data is “in flight” and the layout doesn’t have to be consistent or compatible with future (or past) binaries.

When we look at the wire format, some data will be serialized, sent over the network, and immediately decoded by another server. If we were to introduce a new wire format, we might consider negotiating this new version and only if both ends of the connection supported it, use the new format instead. This transition period of limiting ourselves to “in flight” data gives us a series of breadcrumbs to follow for undoing it: We can stop negotiating the new format and phase it out if we need to.

Once this data is persisted to disk where it becomes “at rest” data, we face a different set of considerations: We need to be able to read that data for as long as it is useful. Practically speaking, this might be “forever” if we cannot centrally transcode formats.

Experimentation

In their ideal lifecycle, feature flags aid the rollout of features and then are removed when we get to complete adoption. One challenge might be where a new feature introduces drastically different tradeoffs between applications, for example, one that saves CPU but causes some applications to use more RAM. While it might be best to avoid this altogether or to self-tune, these still might be desirable optimizations.

Experiments can help us dip our toe into the waters of a change while still allowing us to centrally roll things back. By making the change over a small slice of each application, we can avoid “getting stuck” in a half-way state between some users having the new feature and others not. Even if the change is very beneficial for some users who might be hesitant to see it rolled back, the narrow slice minimizes how sticky it might be. This can allow us to fine-tune things to sandblast away the roughest parts of the tradeoff.

Dark and counterfactual launches

A distinct kind of running experimental functionality in production is to enable it in parallel with the status quo in dark launch. For example, to consider different compression algorithms, we can compress a small fraction of the data using experimental settings, discarding it. Comparing this counterfactual data with the still-enabled primary compression algorithm we can choose the best settings for the given application. This approach isn’t always applicable–for example, we can’t use it for decompression–but when it works it is a useful tool in higher-risk scenarios such as experimenting with data-at-rest.

Latency injection and SLO brownouts

Systems sometime underpromise on SLOs and overdeliver on actual performance. This can make it challenging to determine whether higher-level services depend on the better-than-promised performance characteristics.

To mitigate this, one approach is to implement intentional brownouts for a service. For example, Chubby did that in the past for its global instance, ensuring that users experienced the letter of the SLO, allowing infeasible dependencies to be uncovered in a controlled manner.

Similarly, injecting latency can allow us to simulate a slower implementation, which we might consider as part of a tradeoff of compute for other resources. An implementation to add latency is straightforward to build and easy to turn on and off.

By handling this in a controlled setting, we can rapidly turn off the injection experiment or availability brownout if we uncover downstream issues. This can help mitigate the risk of building on the assumption that a particular tradeoff was possible, only to discover a late-arrising issue partway through a large scale rollout.

Handling bugs and regressions

Bugs, regressions, and outages are never fun, but they are inevitable as the system changes, whether intentionally (code updates) or unintentionally (workloads shift). The goal isn’t to get to zero risk, but instead is to manage it. The possibility of a bug does not make the door one-way.

Regardless of the type of change we’re making, we still need to do testing, progressive rollouts, and monitoring to look for issues. When we are trying to decide how to proceed, we should aim for how additional flight miles can let us gain new information to make progress, since no amount of analysis will ever completely derisk things.

Conclusion

Thinking about how reversible decisions are can help avoid analysis paralysis. Many software decisions are easily reversed, allowing us to mitigate the risk of regressions from changes and to shift our focus onto the more consequential decisions.

Tip of the Week #234: Pass by Value, by Pointer, or by Reference?

2024-09-30T00:00:00-04:00

Originally posted as TotW #234 on August 29, 2024

By Steve Wang

Updated 2024-09-30

Quicklink: abseil.io/tips/234

Overview

Many programming languages, such as Java and Python, always access objects via references, and functions that accept objects get their own reference to the caller’s object. Others, such as C and Go, allow you to explicitly specify pointers to objects. C++ further allows you to choose whether to pass by value, giving the called function a copy of the argument, or to pass by reference, giving the called function access to the caller’s object. This tip will illustrate the various ways input-only function parameters are passed in C++ and provide recommendations and caveats.

When we talk about passing by value, we explicitly mean that the language ensures that the scope of the function call has an exclusive copy of its argument[^elision]. Reassigning a new value to this variable does not mutate the corresponding object in the caller’s scope. However, invoking the argument’s methods may still mutate its underlying state.

Meanwhile, when we talk about passing by reference, we effectively bring the object from the caller’s scope into the current function’s scope, and reassignment will mutate the underlying object.

Passing by pointer has some similarities to passing by reference, yet is technically a special case of pass-by-value, as the pointer itself is a value that corresponds to the address of the underlying object (or a null pointer, which refers to no object at all).

Consider the following:

void AddOneToValue(int x) {
  ++x;
}

void AddOneToReference(int& x) {
  ++x;
}

// Here, the pointer points at a "pointee"; we're adding one to the
// pointed-at object.
void AddOneToPointee(int* x) {
  ++*x;
}

...

int x = 5;
AddOneToValue(x);
// x is still 5.
AddOneToReference(x);
// x is now 6.
AddOneToPointee(&x);
// x is now 7.

As a result, when writing functions in C++, the language makes us consider how to pass parameters – should we pass by value, by pointer, by reference (if so, which kind)?

Why Do We Care About Passing by Value?

The astute reader might wonder what the issues are with always passing by reference. First off, having unnecessary const T& (e.g., Add(const int& a, const int& b)) adds visual clutter.

Second, in C++, as mentioned above, a reference is largely syntactic sugar for a pointer[^const_ref], with the associated overhead of a memory lookup when we want to use it unless the compiler is able to optimize that away. By passing small types by value, we can pass them in registers instead of needing to store them on the stack.

// Passing a small value by value, the compiler can avoid a stack allocation and
// pass it in a register.
int foo = 5;
Bar(foo);

// However, passing a small value by reference requires `foo` to be copied
// ("spilled") to the stack since you can't take the address of a register.
int foo = 5;
Bar(&foo);

Of course, if the variable is already on the stack or heap (e.g., it’s part of an array) then this concern is irrelevant, but we should still prefer to pass by value to avoid some cache misses and memory pressure if it’s already loaded in a register.

Regardless, references can further introduce concerns regarding aliasing[^aliasing] – since the function does not have an exclusive copy of the object, we have no guarantees that the object will remain unchanged throughout the lifetime of the function, even if we have a reference-to-const (which is only a promise that we will not mutate it through that particular parameter).

Why Do We Care About Passing by Reference?

On the opposite end of the spectrum, one might wonder why we don’t just pass all input parameters by value.

In C++, if you pass a variable by value, depending on how the function is called the variable’s value may be copied or moved (or neither)[^names]. On the other hand, passing by reference (or pointer) allows you to refer to an existing object and therefore avoid a copy entirely. So, in general, the larger an object, the more you should prefer passing by reference.

Passing by value can have benefits as well as drawbacks from the perspective of memory safety. On one hand, if you have the only copy of an object, then you don’t have to worry about other threads stomping over its state. On the other hand, if you retain a reference to this object, once it goes out of scope, then you have a use-after-free bug (just as for any other local variable).

Guidelines

All of these rules apply equally. If none of them apply, a safe option is to pass by reference-to-const for required parameters, and by pointer for optional ones.

Pass by Value

Passing by value can be more efficient in some cases (when the relevant types are small enough that moving or copying them is more efficient than working via pointers), and is helpful for conveying ownership (typically when the called function wants to own the value, to move from it, or to otherwise modify its own copy).

Specifically, the types listed below should usually be passed by value:

Numeric and enumeration types (including protobuf enums).
Smart pointers, when the called function takes ownership unconditionally.

Some additional types can be efficiently passed by value, as an optimization:

Types that provide an efficient move constructor, only if the called function needs its own copy of the value. Examples include std::vector<T>, std::string, absl::flat_hash_map<T> and other containers that don’t store their contents inline[^proto_move].

In these cases, you should pass by value, and std::move at the callsite when needed (or pass in a temporary which is subject to mandatory copy elision per Tip #166). See Tip #117 for supplemental reading on copy elision and pass-by-value.

Passing by value is especially common in a constructor that stores one of these types in a member variable.
```
class Foo {
 public:
 // Here, we pass bar by reference and copy it into bar_.
 Foo(const std::vector<int>& bar) : bar_(bar) {}

 // But, we can instead use std::vector's move constructor to avoid
 // the expensive copy entirely, in some cases.
 Foo(std::vector<int> bar) : bar_(std::move(bar)) {}
 private:
 std::vector<int> bar_;
};
```
T, where sizeof(T) <= 16¹ and T is either a scalar type such as an integer or a pointer type, or a class[^calls] such that:
- It has a non-deleted copy constructor or move constructor.
- All copy and move operations are trivial – one requirement is that they must either be omitted or explicitly defaulted (Tip #131).
- The destructor is trivial and non-deleted.
For types that your team does not own[^hyrum], you should only rely on this behavior if they explicitly document that they should be passed by value, such as spanner::Database and absl::Duration.
std::optional<T>, where passing T by value applies.

std::optional<T> adds some size overhead compared to T, which further limits the types that can be passed by value efficiently. So, for instance, std::optional<std::span> and std::optional<absl::string_view> are too big, as each of these wrapped types is 16 bytes before accounting for std::optional’s overhead.

If sizeof(std::optional<T>) > 16, or if T has a nontrivial copy constructor, then prefer passing absl::Nullable<const T*> (Tip #163), and use a null pointer to represent the case that would otherwise be captured by std::nullopt.

Note that Tip #163 applies here – if all callers will always have a std::optional<T>, then you may pass by const&.

Do not use this idiom with smart pointers or other types that have a representation for “no value”. For instance, do not write std::optional<std::unique_ptr>; instead, prefer to use std::unique_ptr directly, and pass a null pointer to represent “no value”.

Pass by Reference or by Pointer

Note: If an argument x in a call f(x) is required to outlive the function call, do not pass it by reference.

The types listed below should usually be passed by reference (for required parameters) or by pointer (for optional parameters).

Smart pointers (e.g., std::unique_ptr<T>) where you don’t want to transfer ownership: dereference the smart pointer to pass const T& if the pointed-at value is always known (and required) to be non-null; else pass absl::Nullable<const T*> (Tip #188).

In cases of shared ownership[^shared_ptr] where you only sometimes want to take ownership, you may want to pass a reference to the std::shared_ptr to avoid the slight overhead of updating reference counts.
Containers that store their contents inline, e.g., std::array<T, N> and absl::InlinedVector<T, N>.

While std::array<T, N> can be efficient to pass by value if sizeof(T) * N <= 16, absl::InlinedVector<T, N> has a non-trivial copy constructor and thus will never be passed in a register.
Types with non-trivial copy constructors, where you don’t intend to use move semantics.
Protocol buffers.

You might think that the Duration type defined by
```
edition = "2023";

message Duration {
  int64 seconds = 1;
  int32 nanos = 2;
}
```
only contains an int64 (8 bytes) and an int32 (4 bytes) and is therefore 12 bytes (padded out to 16 bytes), but that’s not correct because protobufs may have a vtable pointer (8 bytes) or other metadata. Additionally, you shouldn’t pass protos by value by default (even if they don’t have many fields) because they do not promise that they can be trivially copied (and in practice they usually cannot).

Pass a View (by Value)

For some types, a corresponding view type – a type that gives read-only access to the underlying data, and might support various different underlying types – can be a good way to accept inputs to a function that does not need its own copy of those inputs.

For functions accepting a string argument, whether as std::string, absl::string_view, or const char*, defining the parameter as an absl::string_view is efficient and supports all of these inputs types (see Tip #179).
For functions accepting a std::vector<T>, defining the parameter as an absl::Span<const T> is more efficient and more flexible (see Tip #93), though using const std::vector<T>& can be a reasonable choice if constraints make absl::Span impractical.
For functions that accept a callable object, such as a lambda, we can choose between defining a function parameter as const Fn& (where Fn is a template parameter) or as a type-erased callable such as absl::FunctionRef (see Tip #145).

Closing Words

While passing function parameters by const& is a good default choice, there are plenty of cases where it’s not the best option. The guidelines in this tip can help to weigh the relevant factors and design safe and efficient APIs.

We want to emphasize that they are just guidelines, though, and if you have good reason to deviate from these (e.g., benchmarking or profiling identifies potential performance gains), we encourage you to do so (and to document your rationale for the next reader).

In typical Google production environments, namely x86-64 Linux. See section 3.2.3 of the ELF x86-64 ABI spec. On Windows, only types that are 8 bytes or fewer are passed in registers. [^aliasing]: In best-case scenarios, pointer aliasing prevents the compiler from making certain optimizations. In worst-case scenarios, pointer aliasing can result in violated preconditions, logic bugs, and buffer overflows. [^calls]: Formally, this class must not be “non-trivial for the purpose of calls”. This is very similar, but not quite identical, to a trivially copyable class in C++. See the ABI specification for the formal definition of this term. [^const_ref]: const-references are somewhat more complex – for one, they can bind to temporaries. Further, references cannot be null, so we generally recommend passing references instead of pointers for required input parameters that don’t need to outlive the function call. [^elision]: This does not necessarily mean that the function creates a separate copy of its argument, as copy elision may have taken place. [^hyrum]: While it can be more efficient to pass small types by value, you may accidentally make it harder to add new fields to those types or otherwise change the internal representation (see go/hyrums-law), since you’re adding an implicit dependency on the size of the type, as well as on the constructors and destructors that it defines. [^names]: To a first approximation, this results in a copy when you pass in a named object (such as a non-reference stack variable or data member). Tip #166 covers this in more detail. [^proto_move]: Protocol buffers also define a move constructor that is usually comparable to a shallow copy – the exception is if you’re moving between two messages that live on different arenas, or between a heap-allocated message and an arena-allocated message, in which case it is comparable to a deep copy. go/proto-cpp-arena-allocation#message-class-methods has more details. [^shared_ptr]: As stated in the style guide, shared ownership should only be used with good reason, and not as a way to avoid thinking about object lifetimes. ↩

Tip of the Week #232: When to Use `auto` for Variable Declarations

2024-09-30T00:00:00-04:00

Originally posted as TotW #232 on June 20, 2024

By Kenji Inoue and Michael Diamond, Google Engineer

Updated 2024-09-30

Quicklink: abseil.io/tips/232

The style guide says in the Type Deduction (including auto) section:

Use type deduction only if it makes the code clearer to readers who aren’t familiar with the project, or if it makes the code safer. Do not use it merely to avoid the inconvenience of writing an explicit type.

Ironically, overuse of auto often leads to code becoming less clear. Over time, however, several patterns have emerged where using auto can improve code clarity and safety, such as:

In situations where specifying the type correctly can be difficult and specifying the wrong type can lead to performance or correctness issues, e.g., range-based for loops over a map.
In situations where the type information is truly redundant and specification of the full type is distracting, e.g., commonly-used templated factory functions and some iterator uses.
In generic code where the type itself is not important as long as it is syntactically correct.

We’ll discuss each of those cases below, with an eye toward clarifying the cases in which auto makes code safer or clearer.

Range-Based For Loops Over a Map

The following code has a problem that each element in the map is unintentionally copied:

absl::flat_hash_map<std::string, DogBreed> dog_breeds_by_name = ...;
// `name_and_breed` is copy-constructed for each element of the map.
for (const std::pair<std::string, DogBreed>& name_and_breed :
     dog_breeds_by_name) {
  ...
}

The unintended copy happens because the value_type of associative containers is std::pair<const Key, Value> and std::pair allows implicit conversions between pair objects if their underlying types can be implicitly converted. Because std::pair::first_type here is std::string and the map entry here has std::pair::first_type of const std::string, the pairs are not the same type and an implicit conversion occurs, copying the contents of the pair despite name_and_breed being declared as a reference.

Using auto, possibly in conjunction with structured bindings (Tip #169), can make the code safer and more performant:

absl::flat_hash_map<std::string, DogBreed> dog_breeds_by_name = ...;

// `auto` with structured bindings - if the element types are clear from local
// context.
for (const auto& [name, breed] : dog_breeds_by_name) {
  ...
}

Sometimes, the element types are not obvious from local context. In that case, you can do this:

// `auto` without structured bindings - allows specifying the element types.
for (const auto& name_and_breed : dog_breeds_by_name) {
  const std::string& name = name_and_breed.first;
  const DogBreed& breed = name_and_breed.second;
  ...
}

Iterators

The names of iterator types are verbose and often provide redundant type information when the type of the container is visible nearby.

Here is an example code snippet that assigns an iterator to a local variable.

std::vector<std::string> names = ...;
std::vector<std::string>::iterator name_it = names.begin();
while (name_it != names.end()) {
  ...
}

All containers expose begin() and end() functions which return iterators, and these iterators have type ContainerType::iterator or ContainerType::const_iterator.

When the type of the container is visible nearby, calling out these types would only have a small benefit of differentiating iterator and const_iterator because the container type part (e.g., std::vector<std::string>) is the same as that of the container. In this case, we can use auto to remove redundancy without hiding helpful information:

std::vector<std::string> names = ...;
auto name_it = names.begin();
while (name_it != names.end()) {
  ...
}

When the container type is not visible locally, prefer to spell out the full iterator type or element type:

std::vector<std::string>::iterator name_it = names_.begin();
while (name_it != names_.end()) {
  ...
}

auto name_it = names_.begin();
while (name_it != names_.end()) {
  const std::string& name = *name_it;
  ...
}

`std::make_unique` and Other Google-wide Factory Functions

In the following code snippet, std::make_unique and proto2::MakeArenaSafeUnique specify the types to be instantiated.

std::unique_ptr<MyFavoriteType> my_type =
    std::make_unique<MyFavoriteType>(...);

proto2::ArenaSafeUniquePtr<MyFavoriteProto> my_proto =
    proto2::MakeArenaSafeUnique<MyFavoriteProto>(arena);

It is widely known throughout Google that std::make_unique<T> returns std::unique_ptr<T> and proto2::MakeArenaSafeUnique<T> returns proto2::ArenaSafeUniquePtr<T>. In particular, the important part of the resulting type T is specified on the right-hand side (RHS) expression, and it is company-wide knowledge rather than project-specific knowledge. We can use auto here to remove redundancy without hiding helpful information:

auto my_type = std::make_unique<MyFavoriteType>(...);

auto my_proto = proto2::MakeArenaSafeUnique<MyFavoriteProto>(arena);

Generic Code

In some circumstances when writing generic code, such as templates or GoogleTest matchers, the type may be impossible or very difficult to specify (e.g., a type written with template metaprogramming or decltype). In these cases auto may also be appropriate. However, these situations should be rare.

Otherwise: Avoid Using `auto`

While it can be tempting to use auto in situations where the type is long and seems obvious to you, remember that future readers of the code may not be familiar with your project and the types it uses (why). For example, consider a common pattern of nested proto access.

// Of course `breed` has type `const DetailedDomesticCatBreed&`!
const auto& breed = cat.pedigree().detailed_breed();

auto may also hide basic semantics like constness, whether a type is a pointer, and whether a copy is being made (Tip #44).

// Did the author mean to make a copy here?
// It is not obvious to all readers that `breed` is not a reference even though
// `detailed_breed()` returns a reference!
auto breed = cat.pedigree().detailed_breed();

// Type and semantics are clear.
const DetailedDomesticCatBreed& breed = cat.pedigree().detailed_breed();

Summary of Recommendations

Use auto when manually writing out a more specific type would incur a high risk of correctness or performance problems.
Use auto to remove redundancy without hiding helpful information when the useful type information is visible locally.
For some generic code where the type is impossible or very difficult to specify, auto may be appropriate; these situations should be rare.
Avoid using auto in other situations: while it may make it easier for you to write the code or allow you to avoid a line-break, it probably makes the code harder to understand for someone unfamiliar with your project.

Tip of the Week #231: Between Here and There: Some Minor Overlooked Algorithms

2024-09-30T00:00:00-04:00

Originally posted as TotW #231 on March 7, 2024

By James Dennett

Updated 2024-09-30

Quicklink: abseil.io/tips/231

Overview

In recent C++ versions the Standard Library has added a few functions whose sole job is to supply some (specific!) point somewhere between two other points x and y: std::clamp (from C++17), and std::midpoint and std::lerp (from C++20).

Adding these function templates to the Standard Library serves two main purposes. First, it establishes common terminology (vocabulary) for these operations, that is likely to be widely recognized. Second, and particularly in the case of std::midpoint and std::lerp, it ensures the availability of high quality implementations that avoid common pitfalls.

All of these operations are constexpr, meaning that they can be used both at runtime and at compile time. The types that can be passed to them depend on the specific operation; they all support floating point types, and std::midpoint and std::clamp offer additional flexibility. Read on for the details.

`std::clamp`

std::clamp(x, min, max) “clamps” x to the range [min, max]. More explicitly, if x is already in the range min to max (inclusive) then std::clamp(x, min, max) returns x, and for x outside of that range std::clamp returns whichever of min or max is closest to x. This is equivalent to std::max(std::min(x, max), min) except for being a more direct way to express the intent (and much less of a puzzle for readers).

Warning: While std::clamp returns a reference, code depending on that is subtle and unusual, and would warrant a comment to alert readers. It is easy to accidentally create a dangling reference by passing a temporary to std::clamp and binding the result to a temporary:
// `std::clamp(1, 3, 4)` returns a reference to a temporary int initialized
// from `3`, which must not be used beyond the lifetime of the temporary.
// See [Tip #101](/tips/101).
const int& dangling = std::clamp(1, 3, 4);

std::clamp works for any type that can be compared with < (or with a user-supplied comparator passed to std::clamp(x, min, max, cmp).

`std::midpoint`

There are no surprises in what std::midpoint does: std::midpoint(x, y) returns a point halfway between x and y (rounding towards x when x and y are of an integral type).

std::midpoint(x, y) works for values x, y of any floating point or integral type (not including bool). As a bonus, std::midpoint(p, q) also works for pointers p, q into an array.

`std::lerp`

The lerp in std::lerp is short for “linear interpolation”, and std::lerp(x, y, t) returns a value some fraction t of the way from x to y. For example, std::lerp(x, y, 0) returns x, std::lerp(x, y, 1) returns y, and std::lerp(x, y, 0.5) can be simplified to std::midpoint(x, y).

Note: In spite of the name, std::lerp can also be used for extrapolation if we pass a value of t outside of the range [0, 1]. For example, std::lerp(100, 101, -2) evaluates to 98, and std::lerp(100, 101, +2) is 102.

std::lerp works on floating point types.

Recommendations

One of the main benefits of these library functions is that they provide a common vocabulary. As always, prefer to use these standard facilities instead of writing them from scratch.
Prefer std::midpoint(x, y) over std::lerp(x, y, 0.5) when applicable.
Avoid declaring a reference to the result of std::clamp.

Performance Tip of the Week #83: Reducing memory indirections

2024-09-04T00:00:00-04:00

Originally posted as Fast TotW #83 on June 17, 2024

By Chris Kennelly

Updated 2025-08-23

Quicklink: abseil.io/fast/83

Memory indirections are a frequent cause for latency and memory bandwidth bottlenecks. By forcing the processor to follow pointers to get to the useful data it needs, our programs incur slowdowns and require more memory bandwidth than they might need from a more efficient layout. In this episode, we discuss tools for identifying inefficient data structures and improving them.

Latency and throughput

In Google’s fleet, our processors spend 40-50% of their time waiting for data coming from their caches or main memory. Memory access latency and throughput are inextricably linked:

As the software makes more accesses, it essentially rolls the dice each time that it will incur cache misses. Further, as the rate of accesses over time increases, the core sees higher access latencies due to memory bandwidth saturation.
For every access, the program incurs more latency if the processor needs to access data from an outermost cache or main memory.

We can tackle both of these factors as we think about how to lay out data in our programs. We can work to reduce accesses or choose layouts that increase the likelihood of something already being reused. Improving this allows us to do more useful work with fewer CPUs.

Helping the hardware help us

PMU events are proxies that we use for understanding accesses and identifying changes to code to make it friendlier to the cache hierarchy. Consider a linked list: The individual nodes are unlikely to be physically contiguous since they are allocated on the heap. Without this correlation, the hardware prefetcher can’t help us hide access latency (at best) and might even issue accesses to (logically) unrelated, neighboring cachelines that our program won’t need, wasting bandwidth.

While Arenas are an incredibly useful tool, it’s best to apply this after optimizing the data structure, not before. A more convoluted, but Arena-allocated data structure is likely worse than using the best one in the first place.

Surveying data structures

Consider the problem of finding an element in a std::list, a std::set, an absl::node_hash_set, or absl::flat_hash_set. We can construct the benchmark in two ways: doing lookups one after another with no dependency, or chaining the output of one lookup to influence the next to suss out latency characteristics.

template<typename T>
void BM_LookupSpeed(benchmark::State& state) {
  using ValueType = T::value_type;

  const int size = std::max<int>(1, state.range(0) / sizeof(ValueType));

  // Choose size random integers, then populate our container with it.
  absl::BitGen rng;
  std::vector<std::remove_cv_t<typename ValueType::first_type>> v;
  v.reserve(size);
  for (int i = 0; i < size; ++i) {
    v.push_back(absl::Uniform(rng, std::numeric_limits<typename ValueType::first_type>::min(),
                              std::numeric_limits<typename ValueType::first_type>::max()));
  }

  T container;
  for (auto u : v) {
    if constexpr (std::is_same<T, std::list<ValueType>>::value) {
      container.emplace_back(u, u);
    } else {
      container.emplace(u, u);
    }
  }

  const auto begin = container.begin();
  const auto end = container.end();

  // Use a small state PRNG for selecting indices to avoid confounding access
  // counts that might occur with larger state RNGs (like Mersenne Twister).
  std::minstd_rand lcg;
  uint32_t last = 0;
  uint64_t sum = 0;
  for (auto _ : state) {
    const size_t index = (lcg() + last) % size;
    const auto value = v[index];

    typename T::const_iterator it;
    if constexpr (std::is_same<T, std::list<ValueType>>::value) {
      it = std::find(begin, end, std::make_pair(value, value));
    } else {
      it = container.find(value);
    }
    ABSL_DCHECK(it != end);
    auto found = it->second;
    sum += found;

    // Introduce a deliberate dependency from this lookup on the key for the
    // next value.
    last = found;
  }

  ABSL_CHECK_NE(sum, 0);
}

// std::list is O(N), so reduce the data set size 100x for reasonable running
// time.  The reported numbers are per operation, so they are still comparable.
BENCHMARK_TEMPLATE(BM_LookupSpeed, std::list<std::pair<uint32_t, uint32_t>>)
   ->Range(1, benchmark::CPUInfo::Get().caches.back().size / 100);
BENCHMARK_TEMPLATE(BM_LookupSpeed, std::map<uint32_t, uint32_t>)
   ->Range(1, benchmark::CPUInfo::Get().caches.back().size);
BENCHMARK_TEMPLATE(BM_LookupSpeed, absl::node_hash_map<uint32_t, uint32_t>)
   ->Range(1, benchmark::CPUInfo::Get().caches.back().size);
BENCHMARK_TEMPLATE(BM_LookupSpeed, absl::flat_hash_map<uint32_t, uint32_t>)
   ->Range(1, benchmark::CPUInfo::Get().caches.back().size);

We can run this benchmark gathering PMU counters concurrently.

The full output from the benchmark:

name                                                              CYCLES/op
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/1          44.8 ±27%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/8          39.5 ±22%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/64         71.4 ± 6%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/512         181 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/4096      1.08k ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/32768     13.2k ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/262144     102k ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/403701     164k ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/1                      44.2 ±12%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/8                      47.1 ±21%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/64                     84.1 ± 8%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/512                     126 ± 3%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/4096                    162 ± 2%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/32768                   247 ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/262144                  426 ± 2%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/2097152                 780 ± 2%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/16777216              1.93k ± 2%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/40370176              2.53k ± 3%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/1           0.25 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/8           0.25 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/64          81.0 ± 9%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/512         76.8 ± 4%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/4096        77.0 ± 1%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/32768        103 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/262144       118 ± 1%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/2097152      289 ± 3%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/16777216     572 ± 2%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/40370176     871 ± 2%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/1           0.25 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/8           0.25 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/64          71.3 ± 9%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/512         72.1 ± 2%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/4096        71.7 ± 1%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/32768       86.5 ± 1%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/262144       102 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/2097152      210 ± 3%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/16777216     331 ± 3%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/40370176     546 ± 9%

name                                                              MEM_INST_RETIRED:ALL_LOADS/op
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/1          3.00 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/8          3.00 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/64         10.0 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/512        66.0 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/4096        514 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/32768     4.10k ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/262144    32.7k ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/403701    50.5k ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/1                      5.00 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/8                      5.00 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/64                     9.60 ± 7%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/512                    15.5 ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/4096                   21.5 ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/32768                  27.7 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/262144                 33.7 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/2097152                39.8 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/16777216               45.9 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/40370176               48.5 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/1           0.00 ±NaN%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/8           0.00 ±NaN%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/64          8.20 ± 4%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/512         8.00 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/4096        8.01 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/32768       8.01 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/262144      8.01 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/2097152     8.01 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/16777216    8.01 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/40370176    8.01 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/1           0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/8           0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/64          7.00 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/512         7.00 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/4096        7.01 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/32768       7.00 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/262144      7.00 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/2097152     7.00 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/16777216    7.00 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/40370176    7.01 ± 0%

name                                                              MEM_LOAD_RETIRED.L1_MISS/op
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/1          0.00 ±NaN%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/8          0.00 ±NaN%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/64         0.00 ±NaN%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/512        0.00 ±NaN%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/4096       0.01 ±22%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/32768       109 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/262144      705 ± 0%
BM_LookupSpeed<std::list<std::pair<uint32_t, uint32_t>>>/403701    1.04k ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/1                      0.00 ±NaN%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/8                      0.00 ±NaN%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/64                     0.00 ±NaN%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/512                    0.00 ±NaN%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/4096                   0.23 ± 4%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/32768                  5.52 ± 1%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/262144                 9.45 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/2097152                12.9 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/16777216               16.3 ± 0%
BM_LookupSpeed<std::map<uint32_t, uint32_t>>/40370176               17.7 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/1           0.00 ±NaN%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/8           0.00 ±NaN%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/64          0.00 ±NaN%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/512         0.00 ±NaN%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/4096        0.00 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/32768       2.30 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/262144      3.58 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/2097152     3.75 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/16777216    3.77 ± 0%
BM_LookupSpeed<absl::node_hash_map<uint32_t, uint32_t>>/40370176    3.77 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/1           0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/8           0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/64          0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/512         0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/4096        0.00 ±NaN%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/32768       1.20 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/262144      2.55 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/2097152     2.74 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/16777216    2.76 ± 0%
BM_LookupSpeed<absl::flat_hash_map<uint32_t, uint32_t>>/40370176    2.77 ± 0%

While there are clear differences in these containers due to their asymptotic complexity, the memory subsystem adds an appreciable scaling factor to their performance as well.

The microbenchmark ALL_LOADS data correlates with our expectations. A 256KB list of uint32_t’s has 64K elements. The benchmark needs to scan half of those on average to do a lookup, but the scan also requires reading the pointer to find the next node, doubling the number of accesses. This control data is cached but not intrinsically useful to our program. We need a different number of memory accesses (MEM_INST_RETIRED:ALL_LOADS) per lookup, depending on our data structure. For the hashtables, each lookup in our microbenchmark makes 4 accesses: 1 for the vector of values (part of our harness), 1 for the SwissMap’s control bytes, 1 for checking the key match, and 1 for finally retrieving the value. node has an additional indirection here.
L1_MISS lets us differentiate on the efficiency of accesses in terms of how many cachelines the program accesses. Many of the program’s loads are to the same cacheline–it needs to access both the metadata of pointers to the next node and a value, or it need to access both the key and the mapped value when the program finally looks up the value.

Finding unneeded indirections

To find unneeded indirections, we focus on heap allocations and their access patterns, since many accesses are to these data structures. Globals tend to have less complexity: A command line flag holding a value is fairly common, but a complex, allocation-free data structure is comparatively rare.

We are also looking for allocations that are accessed frequently. This is a qualitative, rather than quantitative threshold, depending on the circumstances. Flattening a data structure that is too cold might prove to be too costly in terms of RAM, or even harmful to cache efficiency.

Combining these two factors, we are often looking for places where the lifetime of one is a superset of the other. For example, a std::unique_ptr member variable might be initialized at construction and freed in the destructor. In contrast, something held in a std::shared_ptr might have an indeterminate lifetime, making it less suitable for coupling the two.

Allocations

Small, frequently made allocations are good candidates for optimization.

A small allocation is cacheline-inefficient, since we need storage for both the pointer to the allocation and the object itself (and potentially unrelated, neighboring data too).
Short lifetimes bound our error if we don’t use all of the data. They also give us a rough proxy for access frequency–the bytes of every allocation should generally be touched once, otherwise we could make the allocation smaller. Containers with amortized growth (like std::vector) might be larger than needed by a small constant factor (typically 2x), but a large constant factor would only occur if we called reserve with too large an argument.

We can find these by using TCMalloc’s deallocation profiler to explicitly filter on lifetime, or combine its allocation profile with heap profiles.

Examining frequent allocations can also help find opportunities for eliminating the allocation altogether.

  rpc::StreamStatsInfo* stats_info = new rpc::StreamStatsInfo;
  // ...populate stats_info...
  data.stats_info = stats_info;
  // ...populate data...
  GetStreamMultiplexer()->HandleBytes(&data);
  delete stats_info;

  rpc::StreamStatsInfo stats_info;
  // ...populate stats_info...
  data.stats_info = &stats_info;
  // ...populate data...
  GetStreamMultiplexer()->HandleBytes(&data);
  // stats_info destroyed when it goes out of scope

stats_info was only 120 bytes, so accessing its fields from data.stats_info might entail 4 cachelines: data.stats_info itself, and 2-3 for *stats_info depending on how it was aligned. Using the stack doesn’t zero out the cache footprint of the object, but it is far more likely that those cachelines will already be resident and their contiguous nature makes it far more prefetchable (if not).

In another case, we replaced a std::vector with an absl::InlinedVector:

  std::vector<const Table*> new_tables(count_of_new_objects);
  const Table* new_table = to;
  for (int i = count_of_new_objects - 1; i >= 0; i--) {
    new_tables[i] = new_table;
    new_table = new_table->parent();
  }

  absl::InlinedVector<const Table*, 4> new_tables(count_of_new_objects);
  const Table* new_table = to;
  for (int i = count_of_new_objects - 1; i >= 0; i--) {
    new_tables[i] = new_table;
    new_table = new_table->parent();
  }

The vector was only used for the duration of the function call and allocation profiles showed that all allocations were 32-bytes or smaller. While we can’t statically guarantee that, absl::InlinedVector gracefully handles the situation where we need more elements.

Summary

Data indirections are a frequent source of cache misses that can hurt performance. Using profiling tools, we can identify allocations that cause these misses and choose better data structures to reduce them.

Performance Tip of the Week #79: Make at most one tradeoff at a time

2024-09-04T00:00:00-04:00

Originally posted as Fast TotW #79 on January 19, 2024

By Chris Kennelly and Matt Kulukundis

Updated 2025-06-20

Quicklink: abseil.io/fast/79

Developing and enabling optimizations can often involve tradeoffs: using more RAM and less CPU, choosing which problems to solve right away and which to defer, and so on. In this episode, we discuss examples and strategies for breaking down projects into smaller steps to increase velocity and maximize area under the curve.

Step-by-step migrations: SwissTable

Hash tables have many different implicit and explicit properties that affect their contracts and behaviors. In designing SwissTables and planning for their associated migrations, we made careful choices to defer, avoid, or consciously embrace a great number of tradeoffs when modifying implementation contracts.

The SwissTable migration primarily focused on iteration order, deferring several other valuable changes. In the SwissMap implementation, we introduced code that deliberately randomized hashtable iteration order so that future improvements would not have to deal with brittle tests relying on iteration order again.

Because of the randomization we had placed into both SwissTables and absl::Hash, we were able to make multiple optimizations to the underlying structure of both of these, changing the behavior for small sizes, adjusting the way windowing works, and completely replacing the core hash function. All of these optimizations were made after launch, because every step in the process was a net positive.

TIP: When fixing Hyrum’s Law issues, look for ways to deliberately perturb the behavior so future changes are easier.

Pointer stability of entries is extremely visible in a Hyrum’s Law sense. The SwissTable migration decided to deliberately defer this choice, by providing a pointer stable variant, absl::node_hash_map, that we directly migrated users to. We published guidance encouraging users to migrate themselves from absl::node_hash_map to absl::flat_hash_map, but focused our own efforts on getting people to take the first step. Users found the secondary step, migrating to absl::flat_hash_map, significantly easier because of the randomization we had already in place for SwissTable.

TIP: Stable intermediate states can form good stopping points for a migration to allow progress to be made without jumping all the way to the final state. Even when you cannot move people to the final state immediately, be clear in your written guidance what the best case should be.

When we started the SwissTable migration, we knew that we would likely want to release it in Abseil and that we wanted to build a new hashing framework (now absl::Hash) for it. But those steps weren’t ready. Rather than delay the launch until we had all the parts in place, we began the SwissTable migration without a custom hashing framework, and later migrated the spelling from a different namespace to its final home in absl.

TIP: Shipping early allows you to capture wins earlier and get important feedback from users.

Once absl::Hash was ready, we made sure to have built-in randomization and switched the default hasher for SwissTable to it. Because it worked by default and made the well lit path easier for customers, its adoption went incredibly smoothly.

TIP: Prefer switching defaults to migrating code if you can.

When we introduced hashtable profiling for monitoring tables fleet wide, some users were surprised that tables could be sampled (triggering additional system calls). If we had tried to have sampled monitoring from the start, the migration would have had a new class of issues to debug. This also allowed us to have a very clear opt-out for this specific feature without delaying the entire rollout. Additionally, the folks doing the migrations didn’t have to debug as many distinct types of failures, so each launch could be handled much faster!

TIP: Separate changes into distinct launches to isolate debugging to a distinct class of issues at a time.

Iterative improvement: Deploying TCMalloc’s CPU caches

When TCMalloc was first introduced, it used per-thread caches, hence its name, “Thread-Caching Malloc.” As thread counts continued to increase, per-thread caches suffered from two growing problems: a per-process cache size was divided over more and more threads, making each cache on average smaller, and more idle threads (threads » cores) meant more RAM was effectively inaccessible.

Initially, Andrew Hunter added support for per-CPU caches. Rather than cache memory for each thread, the implementation had a cache for each physical core on the machine. If a thread was descheduled, another thread would be able to reuse that same cache. Over time, organic adoption brought per-CPU cache usage to roughly half of the fleet’s CPU usage and memory allocations.

After extensive early adoption, TCMalloc’s default changed: unless otherwise requested, per-CPU caches were used. Due to the extra metadata per-CPU caches, this made a tradeoff of RAM for CPU. Rather than completely eliminate this cost, we opted to make this intentional tradeoff. While later optimizations minimized the RAM overhead of per-CPU caches, they would not materialize for several years, so this strategy allowed us to realize incremental benefits years earlier.

TIP: Identify ways to iteratively land improvements. This allows optimizations to be deployed when ready, without the R&D of implementing all anticipated optimizations upfront. This can help maximize the savings area-under-curve.

As years went by, though, core counts for the typical server had increased dramatically. Since the per-CPU cache uses an array–indexed by physical CPU ID–of caches, more metadata had to be allocated even though the number of cores used by a typical application had not grown commensurately. Additionally, since a job configured to use 16 cores might move around across a socket with 128 cores, we could populate caches on each of these cores even though the application might not actively run on them. These observations motivated development of several optimizations. TCMalloc includes extensive telemetry that enabled us to calculate the amount of memory being used for per-vCPU caches which provided estimates of the potential opportunity - to motivate the work - and measure the final impact - for recognising the benefit.

TIP: Tracking metrics that we intend to optimize later, even if not right away, can help identify when an idea is worth pursuing and prioritizing. By monitoring metadata memory usage and the number of active caches, we were able to identify when the problem had grown to be worth solving compared to other opportunities.

Decoupled rollouts: Limoncello

Experiments to switch-off hardware prefetchers under high system memory bandwidth usage showed significant performance improvements for the fleet. Analysis of the data showed that most workloads showed broad improvements, but a handful saw regressions.

By exploring the data at a per-function granularity, we were able to recognize the functions that most significantly regressed in A/B testing. Many of these regressions were in core libraries with streaming access patterns. By adding software prefetches to them, we were able to recover their performance when HW prefetchers were disabled.

TIP: A regression in one dimension or slice of data can sometimes become the kernel of an idea for a new opportunity.

While these prefetches primarily recover performance when the hardware stream prefetchers are turned off, they also help performance even when HW prefetchers are turned on. Hardware prefetchers require a warmup period to learn about the access pattern, and unlike higher-level C++ code, lack knowledge about when to stop prefetching when streams are short. Software prefetches can avoid a warmup period and maintain high precision. While turning off the HW prefetchers had illuminated the opportunity, the instantaneous warmup period and high precision meant that SW prefetchers were deployable right away without consideration for whether the HW prefetchers were on or off.

Once several prefetches were in place, the project was reevaluated with several teams that had previously seen regressions. Thanks to the SW prefetches, these gaps had narrowed dramatically. Since these were located in a handful of carefully studied and frequently optimized core libraries, maintaining the prefetches is feasible, even over successive hardware generations.

TIP: Rolling out changes independently from one another avoids unneeded couplings and simplifies rollout strategies.

Closing words

Structuring how we identify and roll out optimizations can help us move faster. By keeping our tradeoffs as simple and straightforward as possible, or better yet, avoiding tradeoffs altogether, we can iteratively improve performance without hitting stumbling blocks. Exporting metrics along the way allows us to identify the biggest opportunities as workloads and hardware evolve.

Performance Tip of the Week #72: Optimizing optimization

2024-09-04T00:00:00-04:00

Originally posted as Fast TotW #72 on August 7, 2023

By Chris Kennelly

Updated 2025-08-23

Quicklink: abseil.io/fast/72

Finding optimizations is increasingly crucial as Moore’s Law ends and we can no longer expect a free lunch from continued hardware innovation. In this episode, we discuss the process of finding and developing optimizations to work on.

Planning

“Plans are worthless, but planning is everything”

Just as efficiency projects are trying to maximize the productivity of our hardware, the projects themselves can be optimized by effective planning. Project selection makes a huge difference in outcomes and impact.

Realized impact of a project is the product of several factors. When we are first starting to think about an optimization, we estimate all of these factors. While it’s rewarding for our estimates to be correct, the primary goal is to have just enough information to make a better decision and set priorities–to preferentially work on optimization “A” over optimization “B” because “A” has a larger expected ROI. Oftentimes, we only need a single significant figure to do so: Spending more time making a better estimate or gathering more data does not make things more efficient by itself. When new information arrives, we can update our priors accordingly.

Once we have identified an area to work in, we can shift to thinking about ways of tackling problems in that area.

What is the largest potential win?

For example, we have limited headroom to optimize a function using a single CPU core out of entire warehouse-scale computers. Even if we can drive its cost to zero, the impact is low. In contrast, the opportunity cost of not working on more important areas is high.

How much can we do about it?

It is not enough to observe “X is expensive”: We need to be able to change X too.

“Speed of light” analysis might provide a helpful upper bound here. If an operation needs to read data from memory, process it, and write it out, it’s hard for it to be substantially faster than memcpy. Performance does not need to ultimately converge with our rough analogue, but if we’re already close, it may be hard to meaningfully affect the performance of X.

An operation may already be as simplified as it can be–for example, it is hard to speed up adding two integers. We may need to look further up the stack–add less often–to realize performance gains.

Can we measure what changed, if so, how?

An unmeasured optimization is a tree that fell in the woods with no one around.

For example, some changes can have highly non-local effects that can make accurate measurement more challenging. TCMalloc has an “expensive” prefetch which makes TCMalloc look more expensive, but improves topline application performance. In a distributed system, changing the requests sent by a client to a server can dramatically impact the server’s costs, but not necessarily affect the metrics the client sees.

What is our likelihood of success?

Changing a minor implementation detail can be straightforward, or we can uncover Hyrum’s Law dependencies on existing quirks. A brand new API may be easy to implement, but challenging to grow adoption for. If others have looked at the same problem before, things might be different this time, but we may want to temper our optimism.

How long will it take?

This estimate should include preliminary analysis, implementation, code review, debugging, rollout, measurement, and a period of refinement after launch.

We want to save several multiples in resource costs as our time investment, both to ensure a positive return on investment and to balance against errors in our estimates.

How long will it live?

Optimizing code that is there to stay for years is a safer bet. When the system in question is likely to be replaced soon, we should consider the expected lifespan of our work.

Sometimes it makes sense the leave the idea alone and move on to the next item in the stack rank.

In other cases the optimization work is transferrable or it raises the efficiency bar for the new system so the economic effect of the optimization outlives the original implementation.

Standing on the shoulders of giants

“Great artists steal”

Frequently, we can find opportunities by growing adoption or enhancing existing optimizations.

Historically, “hugepage text” was used to reduce iTLB misses for only a portion of servers in the fleet. For the remainder, realizing an improvement in application productivity was a single flag-flip away. A lot of factors that go into planning were already de-risked: We knew it would appreciably move the needle for a large portion of the fleet if enabled, leaving us to focus on how to get there.
AutoFDO uses production profiles to provide feedback to the compiler to optimize program performance. Once the initial infrastructure was built, we were able to get further savings with a multi-pronged approach: increasing adoption, improving profile quality, and building further optimizations like FS-AFDO, cmov-vs-branch, and function splitting on top of it.

Borrowing ideas, even if seemingly simple, from another area, can go a long way. “Don’t do extra work” sounds obvious, but bringing fresh eyes to a problem and cross-pollinating ideas can be very effective.

Once we have some ideas to try out, we want to shift towards evaluating them systematically.

Optimization lifecycle

“If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.”

Derisking an idea

In our initial planning, we made a few assumptions to estimate the probable return on investment from a project. Our first priority is to de-risk those assumptions by prototyping our change and evaluating it.

Benchmark to improve estimates. This lets us iteratively improve our estimates, with more effort to achieve higher fidelity.
Evaluate impact on secondary metrics. Even if we can’t see the end-to-end, fleetwide impact immediately, proxies–PMU events, profiles, or even individual applications–can tell us if we’re on the right track.
Cut every corner. Making simplifying assumptions or bypassing edge cases can get us data sooner. Even if we’re not in a necessarily optimal place, we can confirm that our hunch about an opportunity was valid. This exercise also helps with differentiating which project dependencies are truly critical and those that are merely nice to have.

If we try out an idea and cannot move the metrics we expected to, our hypothesis might be wrong. We might realize that something couldn’t be optimized after all, or it wasn’t on the critical path as expected. These explanations can let us tweak our plans, scrub things altogether, or inform future areas of investigation.

Once we’ve demonstrated that we have a viable optimization in-hand, we can focus on the steps towards landing it.

Building the idea out

Test failures can point us to edge cases that constrain our implementation. This may require tweaking our implementation to retain behaviors or clean up usage that is no longer permitted. The choice is largely a judgment call: Having a simpler implementation will be easier to optimize further in the long run, but a herculean cleanup may prevent us from landing anything at all. This may also indicate a good time to insert randomization and other defenses to make future optimizations in this space smoother.

With benchmarks, we can fine-tune our idea to try out different parameters and make sure that it improves performance in a variety of situations.

We need not find the precisely optimal parameters, since we’re liable to simply overfit to our model or loadtest. Optimality might be an illusion.
Making an initial change to a library can often be the hardest as we uncover unknown unknowns, especially when we move forward with it. Optimizing every cycle or byte of RAM doesn’t matter until we actually overcome the hurdles of turning it on.

For example, moving from GoodFastHash and std::hash to absl::Hash improved performance, but required taming Hyrum’s Law by introducing a randomized hash algorithm and identifying brittle tests. Once those issues were surmounted, we were able to iterate from there, replacing absl::Hash’s algorithm with a different, more efficient one.

Once we have something that works well and has sufficient polish, we can move forward with it.

Launching

Production is ultimately what matters and our goal is to bring the optimization there for evaluation. Launching and iterating on an optimization helps in several ways.

It delivers value sooner. The area under the curve of efficiency over time is larger by having small, successive landings than if we tried to realize the “better” optimization all at once, but took much longer to deliver it. We doubled the impact of TCMalloc’s “Temeraire” optimization in following years, but launching early provided benefits years earlier than if we had waited.
Things may perform differently, for better or worse, than we expected. This can inform next steps for where to look for further ideas, allowing us to stand on our own shoulders so to speak.

For an optimization enabled broadly, opt-outs can tell us about what edge cases have proven to be truly important and need more attention, leading to further optimization insights. For example, a single, long-standing opt-out from Temeraire informed two distinct optimizations.

Success in one area brings opportunities for cross-pollination. We can take the same solution, an algorithm (pages on huge pages) or data structure, and apply the idea to a related but different area (objects on pages, or “span” prioritization). Without the original landing, though, we might have never realized this.
Circumstances are continually changing. The assumptions that started a project years ago may be invalid by the time the project is ready. Intermediate milestones allow us to validate assumptions along the way.
We may have bypassed non-critical dependencies for the sake of expediency. When those dependencies are ready, we can revisit our choices to enhance our optimization. For example, a library might have required extensive refactoring to fully enable a particular optimization, so we opted to get most of the benefit upfront and realize the rest later when the refactoring was completed.

Landing

Once we’ve launched an optimization, we want to land it and for that we must measure it. We are primarily interested in measuring our primary metric, for example, seeing queries-per-CPU go up. Our secondary, proxy metrics–CPU time in a particular function, PMU events like cache or TLB misses–help to confirm and support the claim that the optimization had the intended effect. This distinguishes the outcome from mere coincidence–did we actually speed something up, or did another change somewhere else happen to make the application faster? While we relied on estimates to guide starting to work in an area, we also don’t want those expectations to bias our measurements either: The actual data might be better (or worse) than what we anticipated and reality is what matters.

Summary

Project selection and project execution can greatly impact optimization outcomes. By judiciously selecting the areas we dig and adopting a strategy of launching-and-iterating, we can unlock significant savings.

Performance Tip of the Week #62: Identifying and reducing memory bandwidth needs

2024-09-04T00:00:00-04:00

Originally posted as Fast TotW #62 on July 7, 2022

By Chris Kennelly, Luis Otero and Carlos Villavieja

Updated 2025-09-15

Quicklink: abseil.io/fast/62

To accomplish useful work with processors, programs need to access main memory. The rate that programs can transfer data to/from main memory has not grown as fast as processors getting greater per-core performance and more cores. The Memory Wall, long predicted, is here. Memory bandwidth is increasingly the bottleneck for how much useful work modern data centers can accomplish with its CPUs. Optimizing for memory bandwidth productivity can make code faster–by reducing memory stalls–and RAM footprints smaller. In this episode, we discuss techniques for identifying hotspots and reducing their impact.

Memory bandwidth basics

Reads and writes while a program is running trigger accesses to memory. Accessing RAM is slow–approximately 100 nanoseconds–so processors have added layers of caches in between the execution units and the physical modules of RAM. On x86, this started with a single kilobyte-sized cache on the 386, but modern processors have several layers of caches (L1/L2/L3). The closer to the processor, the smaller but faster the caches typically are.

The processor’s caches work at a granularity of a cache line, typically a 64-byte chunk of data. This simplifies the hardware–which can track whether a cache line is dirty and needs to be written to main memory at the cache line level–rather than say tracking byte-by-byte. This abstraction is also useful in practice–common primitive data types like int and T* all span several bytes. Accessing the cache line from main memory is an all-or-nothing affair: Loading any one byte means the whole line is read from memory.

In addition to a program’s loads and stores, the processor can issue prefetches for data at the hardware level. These prefetches need to be timely, the data needs to be ready before the program actually needs it, but not so early that it is evicted from the cache before it can be used. Prefetches also need to be useful: Pulling in a cache line that ends up being unaccessed consumes memory bandwidth without the benefit of avoiding a cache miss on the critical path.

The processor’s memory bus can handle thousands of transfers per second, limited by technology, power requirements, and physics. While cache sizes and the memory bus differ across different architectures, the same general principles apply. At the socket level, this can give the processor several GB/second of memory bandwidth. These windows to transfer data are akin to seats on an airplane, though: If the airline oversells a flight and everyone shows up at the airport, several passengers will be forced to wait for the next flight (in the case of CPUs there aren’t any refunds). Bursts in memory accesses, especially across many cores, can saturate our memory bus, even if the average over time is relatively low. Similarly, once a plane takes off, any unsold seats are wasted and cannot aid future bursts.

This perspective suggests two paths to reducing how much data needs to move to and from main memory: 1) access fewer things; 2) increase the reuse of the data the program accesses.

Making a program’s working set more cacheable allows it to be less bottlenecked on memory bandwidth and fewer cache misses reduce memory latency, allowing the program to speed up.

Finding bottlenecks

We can use data collected with the perf tool on a locally executed process.

While LLC misses are only an approximation–this event only tracks misses on loads–it can be quite effective for finding and fixing bottlenecks. Hardware performance counter limitations largely restrict to load events while simultaneously having precise attribution. Attribution to functions is what allows us to find source code-level optimizations. A cache miss on a load is most likely a read transaction on the memory bus and also probably corresponds to a previous cache eviction at some earlier point in time. The absolute numbers of misses are less important for finding places to look for optimizations than their relative rankings.

How to measure

A key part of optimization work is measuring the outcomes. While production is ultimately what matters, smaller scale proxies such as microbenchmarks can help with exploring the solution space.

One challenge for optimizing for cache misses with a microbenchmark is that the working set may be too small and its data will fit neatly into the processor’s caches. One strategy used for Abseil’s Hashtables is to have “hot” and “cold” microbenchmarks. The “hot” versions stress a single instance, whose data footprint fits into the processor’s L1 cache. The “cold” versions work with many instances, ensuring the working set does not fit into the cache. On modern machines, the cache hierarchy can be several hundred MBs. While an individual service on a multitenant machine may have to contend with neighbors also using cache space, a microbenchmark run on an otherwise idle machine can use the processors caches more completely. While neither of these extremes exactly represents production, they show the contrasts of the computational cost (arithmetic) and memory access costs required for hashtable operations.

perf stat -d path/to/benchmark can provide a quick summary of important performance counters, including cache misses.

$ bazel build -c opt //fleetbench/swissmap:all

$ perf stat -d bazel-bin/fleetbench/swissmap/hot_swissmap_benchmark \
    --benchmark_filter=BM_FindMiss_Hot.*flat.*64.*16.*0

-----------------------------------------------------------------------------------------------------------
Benchmark                                                                 Time             CPU   Iterations
-----------------------------------------------------------------------------------------------------------

BM_FindMiss_Hot<::absl::flat_hash_set, 64>/set_size:16/density:0       1.36 ns         1.36 ns    518389760

 Performance counter stats for 'bazel-bin/fleetbench/swissmap/hot_swissmap_benchmark...':

          1,271.48 msec task-clock                #    0.984 CPUs utilized
                13      context-switches          #   10.224 /sec
                 0      cpu-migrations            #    0.000 /sec
               435      page-faults               #  342.120 /sec
     5,299,599,301      cycles                    #    4.168 GHz                      (49.96%)
    22,698,862,872      instructions              #    4.28  insn per cycle           (62.86%)
     2,843,135,458      branches                  #    2.236 G/sec                    (63.17%)
         2,161,178      branch-misses             #    0.08% of all branches          (63.20%)
     5,548,863,804      L1-dcache-loads           #    4.364 G/sec                    (63.19%)
         1,304,138      L1-dcache-load-misses     #    0.02% of all L1-dcache accesses  (62.31%)**
           153,564      LLC-loads                 #  120.775 K/sec                    (49.10%)
            90,977      LLC-load-misses           #   59.24% of all LL-cache accesses  (49.07%)**

$ perf stat -d bazel-bin/fleetbench/swissmap/cold_swissmap_benchmark \
    --benchmark_filter=BM_FindMiss_Cold.*flat.*64.*16.*0

------------------------------------------------------------------------------------------------------------
Benchmark                                                                  Time             CPU   Iterations
------------------------------------------------------------------------------------------------------------

BM_FindMiss_Cold<::absl::flat_hash_set, 64>/set_size:16/density:0       **22.5 ns         22.4 ns**     37748952

 Performance counter stats for 'bazel-bin/fleetbench/swissmap/cold_swissmap_benchmark...':

          1,502.60 msec task-clock                #    0.984 CPUs utilized
                17      context-switches          #   11.314 /sec
                 3      cpu-migrations            #    1.997 /sec
           111,346      page-faults               #   74.102 K/sec
     4,051,264,336      cycles                    #    2.696 GHz                      (50.00%)
     4,765,057,295      instructions              #    1.18  insn per cycle           (62.51%)
       567,678,472      branches                  #  377.799 M/sec                    (62.51%)
         6,199,555      branch-misses             #    1.09% of all branches          (62.50%)
       982,319,844      L1-dcache-loads           #  653.749 M/sec                    (62.78%)
       157,288,870      L1-dcache-load-misses     #   16.01% of all L1-dcache accesses  (62.51%)**
        19,126,293      LLC-loads                 #   12.729 M/sec                    (49.96%)
        15,576,980      LLC-load-misses           #   81.44% of all LL-cache accesses  (49.73%)**

The benchmark’s large working set made its workload harder to fit into cache, leading to corresponding increases in L1 and LLC misses. Due to the presence of cache misses, the same operation–looking up an absent key from a hashtable–went from ~1.36 nanoseconds to ~22.5 nanoseconds. Worse cache locality doesn’t just consume more memory bandwidth, it also hampers the performance and CPU efficiency of programs.

Additionally, Google’s C++ microbenchmark tool (https://github.com/google/benchmark/blob/main/docs/user_guide.md) supports collecting hardware performance counters. This can be helpful for isolating the counters to when the benchmark is actually running and to individual benchmark operations. In contrast, perf stat captures the benchmark harness’ own setup and its numbers cover the entirety of all microbenchmark cases being run.

Optimization strategies

Our goal is to maximize the productivity of our accesses to cache lines, making sure our programs accomplish as much useful work as possible for each memory access. This is not just useful for reducing memory bandwidth needs, but it can improve CPU and RAM productivity as well. Accessing more data and cachelines than we need is a source of false demand that we can optimize.

Avoid unnecessary indirections

Consider a simple linked list containing integers:

struct Node { Node* next; int value; }

In memory, this will fill part of a cache line (16 bytes out of 64) and each node is likely to fall on its own cache line, separate from other nodes. Over time, a program’s memory allocations will trend towards being random due to the mix of allocations and deletions that have left gaps.

This is inefficient to access in several ways:

The next pointers, while important to the linked list, aren’t holding useful data.
Accessing a cache line pulls in only a single 4-byte integer at a time. If the program were to use a contiguous data structure like std::vector, accessing the first element means it likely also accessed the cache line for the second and so on.
Due to alignment requirements, each node has 4 bytes of padding going unused.

While std::list is quite rare, many containers have similar implicit linked-list like layouts. Matt Kulukundis discussed this in his 2017 CppCon presentation on Abseil’s hashtables: Containers like std::unordered_map and to a lesser extent, absl::node_hash_map achieve pointer stability by separately allocating each key-value pair. This guarantee comes at a cost, since the data structure requires an extra memory access to get to the logical value the program is accessing.

Similarly, one common pattern is std::vector<std::unique_ptr<T>> where there is an indirection that makes logically adjacent objects not colocated in memory. It is important to look with a critical eye at whether this indirection is necessary. Sometimes it exists because T is not copyable/movable. In that case it may be possible to remove the indirection by fixing the type semantics or by using a container type that allows non-movable content like absl::FixedArray or std::deque. In other cases the indirection may be used to exercise polymorphic behavior, and techniques like tag-based dispatch may help. There is no one size fits all solution for this but practical options do exist.

Minimize copies

Copies are sometimes needed for API or correctness reasons, but extraneous ones require the processor to do more work. For example:

void DoStuff(Protobuf);

void Process(const std::vector<Protobuf>& elements) {
  for (Protobuf element : elements) { // Creates a copy of each element.
    DoStuff(element);
  }
}

void DoStuff(Protobuf);

void Process(const std::vector<Protobuf>& elements) {
  for (const Protobuf& element : elements) { // Avoids a copy.
    DoStuff(element);
  }
}

The program is copying each element from the vector and then processing that copy, when no copy would have sufficed. This increases its working set–it has all of elements and element–and increases pressure on the processor’s cache, making it more likely that the program will trigger evictions and misses. In addition to the demands on the memory system, the program is also less productive in terms of its CPU usage–copying isn’t helping us process faster–and RAM usage–its extraneous copy isn’t needed.

While this is just an example–and Clang Tidy can recommend fixes for this one automatically–copies of much larger data structures such as protobufs can commonly arise.

void Populate(MyProto* msg, int n) {
  for (int i = 0; i < n; ++i) {
    MySubmessage s;
    s.set_value(i);
    s.mutable_foo()->set_bar(i);
    *msg->add_submessage() = s; // Copies, including complex heap allocated structure.
  }
}

Populating data in its final destination can minimize the working set size and improve efficiency:

void Populate(MyProto* msg, int n) {
  for (int i = 0; i < n; ++i) {
    MySubmessage& s = *msg->add_submessage();  // Places s in its destination.
    s.set_value(i);
    s.mutable_foo()->set_bar(i);
  }
}

Similarly, many containers guarantee amortized growth, growing by a multiplicative factor to achieve this guarantee. With each step in growth, we allocate another buffer, move elements over, and deallocate the old one. Where we know the final size, a call to reserve can avoid these copies and reduce our working set size.

Densify data

As explained above, allocating objects contiguously can be particularly beneficial for memory bandwidth, especially if the access patterns will require us to traverse objects in sequence. However, the benefits will exist to the extent that the traversed objects are sufficiently small and can be packed neatly into cache lines.

In general, densifying data in a way that goes in accordance with its access patterns can cause improvements to the memory bandwidth consumption of an application. This lets us use each cache line we access to its fullest. Also, it helps as prefetchers tend to bring continuous or easy to predict cache lines into the cache.

Based on this principle and its implications, consider the following guidelines when writing an application:

Compact data structures as much as possible by reducing padding and eliminating fields which are never accessed.
When processing large amounts of data, narrower data types can increase data density when values have constrained ranges. Integers over a small range might fit in int8_t or int16_t, rather than an int64_t. Representing a small set of values is best done with a narrow enum, rather than a std::string.
If some fields are accessed more frequently than others, consider breaking down the objects into two distinct classes (i.e. “hot” and “cold”) such that arrays of hotter fields/objects are closer in memory.
In the extreme of this technique, prefer using Structure-of-Arrays (SoA) than Array-of-Structures (AoS) to define the layout of the data structures in your program.

As a concrete example of SoA vs. AoS, consider the following code snippet (AoS):

struct Point3D {
  float x;
  float y;
  float z;
};

float AddAllY(const std::vector<Point3D>& elements) {
  float result = 0;
  for (const auto& element : elements) {
    result += element.y;
  }
  return result;
}

A more optimal alternative could be the following (SoA):

struct PointList3D {
  std::vector<float> x;
  std::vector<float> y;
  std::vector<float> z;
};

float AddAllY(const PointList3D& elements) {
  float result = 0;
  for (auto element : elements.y) {  // Traverse the array of "y" coordinates.
    result += element;
  }
  return result;
}

Summary

As with CPU and RAM usage, memory bandwidth is another dimension that we can optimize for, maximizing the amount of useful work we can do with the minimal amount of memory bandwidth. Reducing cache misses and improving data locality can also bring improvements to CPU performance and reducing required RAM footprints.

Additional reading

Chandler Carruth’s 2017 CppCon talk “Going Nowhere Faster”
Matt Kulukundis’s 2017 CppCon talk “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step”

Tip of the Week #197: Reader Locks Should Be Rare

2024-06-25T00:00:00-04:00

Originally posted as TotW #197 on July 29, 2021

By Titus Winters

Updated 2024-04-01

Quicklink: abseil.io/tips/197

“Ah, how good it is to be among people who are reading.” - Rainer Maria Rilke

The absl::Mutex class has supported two styles of locking for many years now:

Exclusive locks, in which exactly one thread holds the lock.
Shared locks, which have two modes. If they are held “for writing” they use an exclusive lock, but they also have a different mode in which many threads can hold the lock “for reading.”

How can a shared lock be acceptable? Isn’t the whole point of having a lock to gain exclusive access to an object? The perceived value in shared locks is when we need read-only access to the underlying data/objects. Remember that we get data races and API races when two threads access the same data without synchronization, and at least one of those accesses is a write. If we use a shared-lock when many threads only need to read data, and always use exclusive locks when writing data, we can avoid contention among the readers and still avoid data and API races.

To support this, absl::Mutex has both Mutex::Lock() (and Mutex::WriterLock(), an alternate name for the same exclusive behavior) as well as Mutex::ReaderLock(). From reading through those interfaces, you might think that we should prefer ReaderLock() when we’re only reading from the data protected by the lock.

In many cases you’d be wrong.

ReaderLock Is Slow

ReaderLock inherently does more bookkeeping and requires more overhead than a standard exclusive lock. As a result, in many cases using the more specialized form (shared locks) is actually a performance loss, as we have to do quite a bit more work in the lock machinery itself. This cost is minor in the absence of contention, but ReaderLock underperforms Lock under contention for short critical sections. Without contention, the value ReaderLock provides is less significant in the first place.

Consider the logic in an exclusive lock vs. a shared lock. A shared lock generally must also have an exclusive lock mode - if there are no writers, no data race can occur, and thus there is no need for locking in the first place. Shared locking is therefore inherently more complex, requiring checks on whether other readers hold locks, or modifications to the (atomic) count of readers, etc.

When are Shared Locks Useful?

Shared locks are primarily a benefit when the lock is going to be held for a comparatively long time and it’s likely that multiple readers will concurrently obtain the shared lock. For example, if you’re going to do a lot of work while holding the lock (e.g. iterating over a large container, not just doing a single lookup), then a shared locking scheme may be valuable. The dominant question is not “am I writing to the data”, it’s “how long do I expect the lock to be held by readers (compared to how long it takes to acquire the lock)?”

// This is bad - the amount of work done under the lock is insignificant.
// The added complexity of using reader locks is going to cost more in aggregate
// than the contention saved by having multiple threads able to call this
// function concurrently.
int Foo::GetElementSize() const {
  absl::ReaderMutexLock l(&lock_);
  return element_size_;
}

Even when the amount of computation performed under a lock is larger, and reader locks become more useful, we often find we have better special-case interfaces to avoid contention entirely - see https://abseil.io/fast and https://abseil.io/docs/cpp/guides/synchronization for more. RCU (“Read Copy Update”) abstractions provide a particularly common solution here, making the read path essentially free.

What Should We Do?

Be on the lookout for use of ReaderLock - the overwhelming majority of uses of it are actually a pessimization … but we can’t statically determine that definitively to rewrite code to use exclusive locking instead. (Reasoning about concurrency properties in C++ is still too hard for most refactoring work.)

If you spot ReaderLock, especially new uses of it, try to ask “Is the computation under this lock often long?” If it’s just looking up a value in a container, an exclusive lock is almost certainly a better solution.

In the end, profiling may be the only way to be sure - contention tracking is particularly valuable here.

Tip of the Week #229: Ranked Overloads for Template Metaprogramming

2024-03-21T00:00:00-04:00

Originally posted as TotW #229 on February 5, 2024

By Miguel Young de la Sota and Matt Kulukundis

Updated 2024-02-20

Quicklink: abseil.io/tips/229

Warning: This is an advanced tip for folks doing template metaprogramming. In general, avoid template metaprogramming unless you have a very, very good reason. If you are reading this, it’s because you need to do some template metaprogramming or just want to learn something nifty.

One Cool Trick

Ordinarily, C++ requires every function invocation to resolve to a single “best” function or it produces an ambiguity error. The exact definition of “best” is more complex than we want to go into, but involves things like implicit conversions and type qualifiers.

In situations that would produce ambiguity errors, we can use explicit class hierarchies to force the definition of “best” to be what we prefer. This “ranked overloads” technique uses structures with a class hierarchy so they have a priority ordering and the compiler will select the highest priority method first. We’ll define a family of empty tag types Rank0, Rank1, etc., that are related by inheritance, and use those to guide the overload resolution process.¹

// Public API with good comments.
template <typename T>
size_t Size(const T& t);

// Everything below here is a working example of ranked overloads, that
// you can copy and paste to get you started!
namespace internal_size {

// Use [Tip #229](/tips/229) for dispatching.
struct Rank0 {};
struct Rank1 : Rank0 {};
struct Rank2 : Rank1 {};
struct Rank3 : Rank2 {};

template <typename T>
size_t SizeImpl(Rank3, const std::optional<T>& x) {
  return x.has_value() ? Size(*x) : 0;
}

template <typename T>
size_t SizeImpl(Rank3, const std::vector<T>& v) {
  size_t res = 0;
  for (const auto& e : v) res += Size(e);
  return res;
}

template <typename T>
size_t SizeImpl(Rank3, const T& t)
  requires std::convertible_to<T, absl::string_view>
{
  return absl::string_view{t}.size();
}

template <typename T>
size_t SizeImpl(Rank2, const T& x)
  requires requires { x.length(); }
{
  return x.length();
}

template <typename T>
size_t SizeImpl(Rank1, const T& x)
  requires requires { x.size(); }
{
  return x.size();
}

template <typename T>
size_t SizeImpl(Rank0, const T& x) { return 1; }

}  // namespace internal_size

template <typename T>
size_t Size(const T& t) {
  // Start with the highest rank, Rank3.
  return internal_size::SizeImpl(internal_size::Rank3{}, t);
}

auto i = Size("foo");                      // hits the string_view overload
auto j = Size(std::vector<int>{1, 2, 3});  // hits the vector overload
auto k = Size(17);                         // hits the lowest rank "catch all"

Note that the absl::string_view, std::optional, and std::vector overloads use Rank3. When overloads are mutually incompatible (the call can’t be ambiguous by construction), the same rank type can be used. You can think of all overloads with the same rank as being tried in parallel.

NOTE: The astute reader may wonder why the absl::string_view overload is declared as a template. Doing so ensures that no implicit conversions will take place in the signature other than the one for the rank structs. If this overload were declared with an absl::string_view parameter then the call would be ambiguous: Rank2{} -> Rank0{} would count as a conversion but const char[] to absl::string_view would also and the call would become ambiguous again.

Detailed Example

Let’s suppose we want Size(x) to return x.length(), x.size(), or 1 depending on what the passed type x implements. The naive approach does not work:

template <typename T>
size_t Size(const T& x)
  requires requires { x.length(); }
{
  return x.length();
}

template <typename T>
size_t Size(const T& x)
  requires requires { x.size(); }
{
  return x.size();
}

template <typename T>
size_t Size(const T& x) { return 1; }

auto i = Size(std::string("foo"));  // Ambiguous.

Because the size and length overloads are of equal rank, they are equally good matches for the call. Because overload resolution does not eliminate all but one candidate, the compiler declares the callsite ambiguous. There are clever tricks using variadic functions or int/long promotion to create an ordering for two options, but these do not scale to having N descending ranks of overloads.

Using ranked overloads as we’ve suggested attaches explicit ranks in the form of inheritance to specific overloads. This rank is based on the following rule: overloads with more derived classes have higher rank than overloads with less specific ones. That is, if two overloads differ by a single argument’s type, and both are bases of the argument, the type closest in the inheritance hierarchy has higher rank and is a better match.

This means we can build a tower of empty structs, each deriving from the previous, to put an explicit, numeric rank on each overload. Using this trick, we would write Size like this:

// Public API with good comments.
template <typename T>
size_t Size(const T& t);

namespace internal_size {

// Use go/ranked-overloads for dispatching.
struct Rank0 {};
struct Rank1 : Rank0 {};
struct Rank2 : Rank1 {};

template <typename T>
size_t SizeImpl(Rank2, const T& x)
  requires requires { x.length(); }
{
  return x.length();
}

template <typename T>
size_t SizeImpl(Rank1, const T& x)
  requires requires { x.size(); }
{
  return x.size();
}

template <typename T>
size_t SizeImpl(Rank0, const T& x) { return 1; }

}  // namespace internal_size

template <typename T>
size_t Size(const T& t) {
  // Start with the highest rank
  return internal_size::SizeImpl(internal_size::Rank2{}, t);
}

auto i = Size(std::string("foo"));  // 3

The overloads can now be read as an if/else chain. First we try the Rank2 overload; if substitution fails, we fall back to the next rank, Rank1, and then Rank0. Of course, this particular method will treat Size(std::string("foo")) differently from Size("foo"). This highlights some of the dangers of generic programming, though the fix is relatively straightforward: add an explicit rank to handle strings, as below.

// Public API with good comments.
template <typename T>
size_t Size(const T& t);

namespace internal_size {
// Use go/ranked-overloads for dispatching.
struct Rank0 {};
struct Rank1 : Rank0 {};
struct Rank2 : Rank1 {};
struct Rank3 : Rank2 {};

template <typename T>
size_t SizeImpl(Rank3, const T& t)
  requires std::convertible_to<T, absl::string_view>
{
  return absl::string_view{t}.size();
}

template <typename T>
size_t SizeImpl(Rank2, const T& x)
  requires requires { x.length(); }
{
  return x.length();
}

template <typename T>
size_t SizeImpl(Rank1, const T& x)
  requires requires { x.size(); }
{
  return x.size();
}

template <typename T>
size_t SizeImpl(Rank0, const T& x) { return 1; }

}  // namespace internal_size

template <typename T>
size_t Size(const T& t) {
  // Start with the highest rank
  return internal_size::SizeImpl(internal_size::Rank3{}, t);
}

auto i = Size("foo");  // 3

Now extending this to vector and optional is quite straightforward!

// Public API with good comments.
template <typename T>
size_t Size(const T& t);

namespace internal_size {
// Use go/ranked-overloads for dispatching.
struct Rank0 {};
struct Rank1 : Rank0 {};
struct Rank2 : Rank1 {};
struct Rank3 : Rank2 {};

template <typename T>
size_t SizeImpl(Rank3, const std::optional<T>& x) {
  return x.has_value() ? Size(*x) : 0;
}

template <typename T>
size_t SizeImpl(Rank3, const std::vector<T>& v) {
  size_t res = 0;
  for (const auto& e : v) res += Size(e);
  return res;
}

template <typename T>
size_t SizeImpl(Rank3, const T& t)
  requires std::convertible_to<T, absl::string_view>
{
  return absl::string_view{t}.size();
}

template <typename T>
size_t SizeImpl(Rank2, const T& x)
  requires requires { x.length(); }
{
  return x.length();
}

template <typename T>
size_t SizeImpl(Rank1, const T& x)
  requires requires { x.size(); }
{
  return x.size();
}

template <typename T>
size_t SizeImpl(Rank0, const T& x) { return 1; }

}  // namespace internal_size

template <typename T>
size_t Size(const T& t) {
  // Start with the highest rank
  return internal_size::SizeImpl(internal_size::Rank3{}, t);
}

auto i = Size("foo");                      // hits the string_view overload
auto j = Size(std::vector<int>{1, 2, 3});  // hits the vector overload
auto k = Size(17);                         // hits the lowest rank "catch all"

Parting Thoughts

Now that you have learned this awesome power, please remember to use it sparingly. As we saw with the absl::string_view overload, generic programming is subtle and can lead to unexpected results.

Note that “rank” here is used in a colloquial sense and is unrelated to conversion ranks for integers and floating point values. ↩

Tip of the Week #227: Be Careful with Empty Containers and Unsigned Arithmetic

2024-03-21T00:00:00-04:00

Originally posted as TotW #227 on November 16, 2023

By James Dennett

Updated 2024-03-11

Quicklink: abseil.io/tips/227

Index-Based Loops (Still Have Their Uses)

Modern C++ code uses index-based for loops much less often now that range-based for loops are available, but there are still times when we need to use an index while we iterate. Parallel iteration over multiple containers is one such case; another is when we want to process multiple adjacent elements from a single container. In this tip we’ll look at a pitfall for the second of these.

Plausible Code

Let’s start by looking at some code that might be correct:

for (int64_t i = 0; i < v.size() - 1; ++i) {
  ProcessPair(v[i], v[i+1]);
}

This code wisely takes some care to check for valid indexes before calling ProcessPair(), so why do we say that it only “might” be correct? A careful unit test (or almost any fuzz test) will cover the case where v is empty. If the code surrounding our for loop ensures that the for loop is never reached in that case, all is well. But if we execute our loop with an empty v, C++ makes trouble for us.

Unsigned Types to the Unrescue

Recall that our style guide warns against use of unsigned types in C++. The style guide also says

Because of historical accident, the C++ standard also uses unsigned integers to represent the size of containers - many members of the standards body believe this to be a mistake, but it is effectively impossible to fix at this point.

(emphasis added)

Looking carefully, we can see that our example falls afoul of the exact problem discussed in the style guide. When checking whether we have valid v[i] and v[i+1] elements, we are seemingly correct in checking whether i is less than v.size() - 1 given that both elements need to be valid. However, for an empty container v.size() is zero (so far, so good!), but because the type of v.size() is unsigned, when we subtract one from that zero, we don’t get the value -1, but instead we get the maximum value of the given type. Then the check for whether i is less than v.size() - 1 evaluates as true for any small value of i, and so the code will use out-of-bounds indexes for v - yielding undefined behavior.

How Should We Fix This?

Interestingly, if we make the code express its intent a little more directly, our problem goes away.

What do we mean by “express its intent a little more directly”? What is the intent here? The purpose of the loop condition here is to ensure that the indexes i and i + 1 used to index into v are valid.

Given that indexes in C++ are zero-based, we test whether a (non-negative) index i into a container is valid by checking i < v.size(). It would be redundant to check validity of both indexes (though we could do so if we wished): if i + 1 is valid then we know that i is (because i is never negative here). “i + 1 is valid” translates directly into C++ as i + 1 < v.size(). Our original code i < v.size() - 1 does not have such a direct translation as a statement about the validity of an index.

The rewritten code i + 1 < v.size() looks almost the same as i < v.size() - 1, but it is crucially different in that we never subtract, so we avoid the danger of wrapping around to a huge positive value. Did we swap this for a risk of overflowing when we calculate i + 1? Only if i is already the largest value of its type (int64_t) – so we are safe. This difference is sometimes characterized by noting that the common, useful values of int64_t are a long way away from overflowing, whereas with unsigned types such as uint64_t, the very common value 0 is the smallest value of the type, so it’s much easier to unintentionally wrap around.

Fixed Code, Fuzz Free

With this one small change, our now-robust code looks like this:

for (int64_t i = 0; i + 1 < v.size(); ++i) {
  ProcessPair(v[i], v[i+1]);
}

The indexes into v are clearly safe, without a need to look further afield to know whether v might be empty.

Now we can let our fuzzer loose on the fixed code, and feel warm fuzzy feelings that our for loop is bug-free and reviewer-friendly.

Note: This is just one (robust) way to write this loop; there are many others.

Summary

While our fix doesn’t change many bytes of source code, it touches on a number of ideas:

As the style guide says, be wary of arithmetic on unsigned types in C++.
Remember that container.size() yields an unsigned type.
Prefer code where correctness can be verified as locally as possible.
Try to make code correspond as directly as possible to the underlying intent.

Tip of the Week #224: Avoid `vector.at()`

2024-03-21T00:00:00-04:00

Originally posted as TotW #224 on August 24, 2023

By Titus Winters

Updated 2024-01-24

Quicklink: abseil.io/tips/224

There is no good use of vector<T>::at() in google3, and fairly few good uses in other C++ environments. The same reasoning applies to at() on other random-access sequences like RepeatedPtrField in protobuf, as well as to value() on wrapper types like optional<T> and absl::StatusOr<T>.

What Does `at()` Do?

The specification of at(size_type pos) is as follows:

Returns a reference to the element at specified location pos, with bounds checking. If pos is not within the range of the container, an exception of type std::out_of_range is thrown.

This means we could view the contract of this method as two distinct behaviors:

Check whether pos >= size(), and if so then throw a std::out_of_range exception.
Otherwise, return the element at index pos.

Note: The specification does not directly address the case of code passing a negative index, but std::out_of_range will be thrown for that case too – because size_type is an unsigned integral type, a call to at(-5) will yield a very large positive value for pos.

When Would We Use `at()`?

Since the contract of at() depends on the bounds-checking logic, we can break this into two cases: either we know by construction that the index is valid, or we don’t.

If we already know that the sequence is sufficiently large and the lookup will succeed, the extra bounds check is overhead. Most vector accesses, for instance, are as part of a loop from 0 to size() and we already know the operation will succeed. Therefore, in cases where we already know the bounds check will be successful, it’s likely that we want the more common operator[]().

 {.bad}
for (int i = 0; i + 1 < vec.size(); ++i) {
  ProcessPair(vec.at(i), vec.at(i + 1));
}

becomes

 {.good}
for (int i = 0; i + 1  < vec.size(); ++i) {
  ProcessPair(vec[i], vec[i + 1]);
}

If we do not know that the sequence is sufficiently large, is throwing an exception the right way to handle that? Usually not. In google3 builds, throwing an exception will terminate the program, messily. Many (perhaps most) readers won’t necessarily spot an innocuously named method like at() as a process termination risk.

 {.bad}
std::vector<absl::string_view> tokens = absl::StrSplit(user_string, ByChar(','));
LOG(INFO) << "Got leading token " << tokens.at(0);

is probably better as

 {.good}
std::vector<absl::string_view> tokens = absl::StrSplit(user_string, ByChar(','));
if (tokens.empty()) {
  return absl::InvalidArgumentError("Invalid user_string, expected ','");
}

or if aborting the program is preferable

 {.good}
std::vector<absl::string_view> tokens = absl::StrSplit(user_string, ByChar(','));
CHECK(!tokens.empty()) << "Invalid user_string "
                       << std::quoted(user_string)
                       << ", expected at least one ','";

So at least in a google3 context, none of the uses of at() are really useful — for any given use case, there is a more preferred alternative.

What About UB?

Unfortunately, reality is hardly so clean as “we know or we don’t”: we make mistakes and code can change over time, invalidating originally correct assumptions. Given that humans are fallible, we can imagine a use-case for at(). Specifically, if we are completely consistent in using at() instead of operator[], we might ensure that even if we’re crashing messily (bad), we don’t trigger undefined behavior (UB) (worse).

While we believe “avoid UB” is a very legitimate goal, we still don’t endorse the use of at(), specifically, because of its exception-entangled semantics, discussed above. The ideal future solution is a hardened-by-default operator[], with compiler optimizations to remove bounds checking, when provably safe. The at() method is a bad approximation of this solution.

Instead, we encourage users to stick with operator[] and reduce exposure to UB by other means, including:

If your project can afford it, we recommend also enabling bounds check in production in other libraries where available.
If you run your code with ASAN you’ll also get diagnostics if you access an element out of range.

In fact, your project is likely already relying on some of these protections!

What About Maps?

In Tip #202 we discussed the use of at() on associative containers like maps and sets. In general, the error-handling logic above applies: it’s likely the case that a missing key should be handled by logging or returning an error, rather than messily crashing the process.

However, the “bounds checking” overhead logic is different for these containers. In the std::vector case, the compute cost of doing the bounds check is similar to the cost of doing the actual work (returning the indicated reference). For associative containers, the “bounds check” equivalent is doing the (necessary) lookup, whether that is tree traversal, hashing, etc.

Following that reasoning, we might use at() when we know the key is present already (no exception throwing) but were unable to keep an iterator or reference, so it is necessary to perform the lookup again. This is pretty rare: see Tip #132 for ways to avoid redundant map lookups.

In the end, there’s some minor room for usage of at() in associative containers. There is more room for nuance in those cases than there is for vector.

What About C++ With Exceptions?

In an exceptions-enabled environment, opinions may differ a bit more when it comes to at(). It’s still broadly the case that explicit bounds checking is likely better performance (and harder to mess up) than relying on exceptions. An argument could be made for defense-in-depth prevention of UB, but it’s fairly clear that the idiom is (and will continue to be) operator[]() rather than at().

Ideally, code should make as few assumptions as it can about the environment in which it will work. Reasoning about code based on which toolchains will be used to compile it is often fragile. For code that uses at() (or another exception-based API) to be correct, it needs to be correct for two different build modes: it must be acceptable to terminate the entire process and it must be acceptable for code at a higher level to catch the exception and continue execution, so the library code must preserve all invariants. In practice that means that the code must be exception-safe and that it must be OK for any out-of-bounds use of at() to terminate the process.

The best advice we can give about use of at() in an exception-enabled environment is perhaps that it trades a reduction in potential UB for hidden and often unnecessary error handling. That isn’t always a clear tradeoff, but it still seems unlikely to be commonly worth the cost.

Closing Thoughts

When indexing into a container, be mindful of which case we are in: is the index “correct by construction”, or does the code need to detect and handle invalid indexes? In both cases we can do better than using the exception-based std::vector<T>::at() API.

Similar thinking applies to other exception-based APIs such as std::optional<T>::value() and absl::StatusOr<T>::value() (See Tip #181). For error handling in non-concurrent C++ code, prefer to “look before you leap” – and then, having checked that things are in order, avoid APIs that include their own checking.

Performance Tip of the Week #75: How to microbenchmark

2023-11-10T00:00:00-05:00

Originally posted as Fast TotW #75 on September 29, 2023

By Chris Kennelly

Updated 2025-10-03

Quicklink: abseil.io/fast/75

Imagine having access to two Oracles. One gives 90% accurate predictions and responds in 10 minutes and one that gives 99% accurate predictions and responds in 1 month. With access to these secret seers, which will let us be best able to make good decisions?

“Production is ultimately what matters” might obviate the need for microbenchmarks altogether, but they are a useful tool in the performance optimization toolbox. While reduced fidelity doesn’t sound desirable at face value, it comes with an important tradeoff: speed.

In this episode, we discuss techniques for using microbenchmarks as an important tool for decision-making. Full documentation for the microbenchmark framework is available at https://github.com/google/benchmark/blob/main/docs/user_guide.md.

Preliminaries

Eli Bendersky has written about a number of pitfalls that can come up when using microbenchmarks. In writing our own series of microbenchmarks to evaluate a varint parser, we’ll apply several of these techniques to get useful answers.

template<bool DoBMI = false>
const uint8_t* ParseVarint32(const uint8_t* buf, uint32_t* value) {
  uint8_t byte;
  int count = 0;
  uint32_t tmp = 0;
  if (DoBMI) {
#ifdef __BMI2__
#ifndef ABSL_IS_LITTLE_ENDIAN
#error "Only supports little endian architectures"
#endif
    // Read 8 bytes.   This requires our input buffer be appropriately padded to
    // permit an overread.
    uint64_t r;
    memcpy(&r, buf, sizeof(r));

    int length = (absl::countr_zero(~r & uint64_t{0x8080808080}) >> 3) + 1;
    buf = buf + length;
    *value = _bzhi_u64(_pext_u64(r, uint64_t{0x7F7F7F7F7F}), 7 * length);
    return buf;
#endif  // __BMI2__
  }
  do {
    byte = buf[0];
    tmp |= (byte & 0x7F) << (7 * count);
    ++buf;
    ++count;
  } while (byte & 0x80);
  *value = tmp;
  return buf;
}

void BM_OneshotVarintParsing(benchmark::State& state) {
  absl::BitGen rng;
  const uint32_t target = absl::Uniform(rng, 0u, std::numeric_limits<uint32_t>::max());
  uint8_t buf[10];
  const uint8_t* end_buf = WriteVarint32(buf, target);
  ABSL_CHECK(end_buf <= buf + sizeof(buf));

  for (auto s : state) {
    uint32_t read;
    ParseVarint32(buf, &read);
  }
}

BENCHMARK(BM_OneshotVarintParsing);

Running this benchmark shows us spending 0 nanoseconds parsing. This result is implausible and should give cause for concern.

------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_OneshotVarintParsing      0.000 ns        0.000 ns   1000000000

Annotating read with benchmark::DoNotOptimize produces more realistic results. This prevents the compiler from recognizing that the return value is unused and eliminating the loop body. We also add ABSL_DCHECK_EQ to verify the result is correct. While this will be a no-op in optimized builds, this is a lightweight way to double-check the calculation to make sure our microbenchmark is functionally correct.

Nevertheless, the benchmark varies from run-to-run:

------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_OneshotVarintParsing       5.31 ns         5.29 ns    131846276
...
BM_OneshotVarintParsing       4.37 ns         4.36 ns    132147859

Since we are targeting a variable length integer, chosen via a random number generator, we aren’t doing a consistent amount of work. The vast majority of integers in the range [0, UINT_MAX] will be encoded as several bytes, but sometimes we’ll choose a small integer and skew our benchmark results. Separately benchmarking by varint size lets us differentiate these cases.

void BM_OneshotVarintParsingSingleWidth(benchmark::State& state) {
  const int shift = state.range(0);
  absl::BitGen rng;
  const uint64_t lo = 1u << (7 * shift);
  const uint64_t hi = std::min<uint64_t>((lo << 7) - 1u, UINT_MAX);
  const uint32_t target = absl::Uniform<uint32_t>(rng, lo, hi);

  uint8_t buf[10];
  const uint8_t* end_buf = WriteVarint32(buf, target);
  ABSL_CHECK(end_buf <= buf + sizeof(buf));

  for (auto s : state) {
    uint32_t read;
    ParseVarint32(buf, &read);
    benchmark::DoNotOptimize(read);
    ABSL_DCHECK_EQ(read, target);
  }
}

BENCHMARK(BM_OneshotVarintParsingSingleWidth)->DenseRange(0, 4);

BM_OneshotVarintParsingSingleWidth/0              0.955 ns        0.955 ns    749769481
BM_OneshotVarintParsingSingleWidth/1               1.61 ns         1.61 ns    434548050
BM_OneshotVarintParsingSingleWidth/2               2.12 ns         2.12 ns    333038803
BM_OneshotVarintParsingSingleWidth/3               2.80 ns         2.80 ns    250300587
BM_OneshotVarintParsingSingleWidth/4               3.27 ns         3.27 ns    215507384

Gauging latency

Real-world parsing does not parse a single varint at a time like the previous benchmark does, and that can have some profound effects on what we measure. For example, there, since we’re iterating over the same buffer, and there’s no dependency on the last value, the processor is very likely to be able to speculatively start the next iteration and won’t need to undo the work. This converts a benchmark that we’d like to measure as a chain of dependencies into a measurement of the number of pipelines that the processor has (or the duration of the dependency chain divided by the number of parallel executions).

To make the benchmark more realistic, we can instead parse from a larger buffer of varints serialized end-on-end:

void BM_VarintParsingEndOnEnd(benchmark::State& state) {
  const int shift = state.range(0);
  absl::BitGen rng;
  const uint64_t lo = 1u << (7 * shift);
  const uint64_t hi = std::min<uint64_t>((lo << 7) - 1u, UINT_MAX);
  const uint32_t target = absl::Uniform<uint32_t>(rng, lo, hi);

  const int kNumVarints = 10000;
  std::vector<uint8_t> v(5 * kNumVarints, 0);
  uint8_t* buf = &v[0];
  for (int i = 0; i < kNumVarints; ++i) {
    buf = WriteVarint32(buf, target);
  }
  const uint8_t* const end_buf = buf;

  // Use KeepRunningBatch to get an accurate, per-item timing.
  while (state.KeepRunningBatch(kNumVarints)) {
    // Start from the top, so we don't just run a single meaningful
    // iteration and stop.
    const uint8_t* read_buf = &v[0];
    while (read_buf < end_buf) {
      uint32_t read;
      read_buf = ParseVarint32(read_buf, &read);
      benchmark::DoNotOptimize(read);
      ABSL_DCHECK_EQ(read, target);
    }
  }

  state.SetItemsProcessed(state.iterations() * kNumVarints);
}

We use KeepRunningBatch since each iteration is processing kNumVarint elements at a time, rather than 1 element.

BM_VarintParsingEndOnEnd/0                        0.933 ns        0.933 ns    749030000 bytes_per_second=1022.04Mi/s items_per_second=1.07169G/s
BM_VarintParsingEndOnEnd/1                         1.64 ns         1.64 ns    432610000 bytes_per_second=1.13844Gi/s items_per_second=611.197M/s
BM_VarintParsingEndOnEnd/2                         2.16 ns         2.16 ns    331480000 bytes_per_second=1.2962Gi/s items_per_second=463.928M/s
BM_VarintParsingEndOnEnd/3                         3.11 ns         3.11 ns    249780000 bytes_per_second=1.19976Gi/s items_per_second=322.058M/s
BM_VarintParsingEndOnEnd/4                         3.27 ns         3.27 ns    216500000 bytes_per_second=1.42589Gi/s items_per_second=306.208M/s

This approach lets us gauge how quickly we figure out the next place to deserialize and there’s a dependency between the end of the previous iteration and the start of the next. Still note, however, that since the varint sizes are invariant, the processor can likely speculate on the next iteration due to successful branch prediction, so this is by no means perfect.

This technique can also be employed for other algorithms, such as hashing, hashtable lookups, and SIMD problems. It can also be helpful for understanding when the processor’s execution backends are port-bound or otherwise lead to a long-critical path.

When applying this technique, we may need to look at the disassembly for our benchmark to ensure that we carry a data dependency across loop iterations. Control flow can be subject to speculation that defeats our attempt at constructing a latency-focused benchmark. This can often arise when working with booleans, for example, a benchmark that exercises contains on a set should have similar performance to one that does find. While this can be sometimes fixed through heavy application of DoNotOptimize and ClobberMemory, the cure may be worse than the disease.

Understanding the speed of light

Before embarking too far on optimizing the ParseVarint32 routine, we might want to identify the “speed of light” of the hardware. For varint parsing, this is approximately memcpy, since we are reading serialized bytes and writing the (mostly expanded) bytes into the parsed data structure. While this is not quite the operation we’re interested in, it’s readily available off the shelf without much effort.

void BM_Memcpy(benchmark::State& state) {
  const int shift = state.range(0);
  const int kNumVarints = 10000;
  // Simulate the skeleton of BM_VarintParsingEndOnEnd by memcpy'ing, to
  // understand the approximate "speed of light" of the operation.
  std::vector<uint8_t> src(5 * kNumVarints, 0);
  std::vector<uint8_t> dst(5 * kNumVarints, 0);

  while (state.KeepRunningBatch(kNumVarints)) {
    memcpy(&dst[0], &src[0], (shift + 1) * kNumVarints);
    benchmark::DoNotOptimize(&src[0]);
    benchmark::DoNotOptimize(&dst[0]);
  }

  state.SetBytesProcessed(state.iterations() * (shift + 1));
}

BENCHMARK(BM_Memcpy)->DenseRange(0, 4);

BM_Memcpy/0                               0.027 ns        0.027 ns   1000000000 bytes_per_second=34.9201Gi/s
BM_Memcpy/1                               0.053 ns        0.053 ns   1000000000 bytes_per_second=35.3181Gi/s
BM_Memcpy/2                               0.080 ns        0.079 ns   1000000000 bytes_per_second=35.1449Gi/s
BM_Memcpy/3                               0.106 ns        0.106 ns   1000000000 bytes_per_second=35.308Gi/s
BM_Memcpy/4                               0.134 ns        0.133 ns   1000000000 bytes_per_second=34.9284Gi/s

It’s also important to do a rough double-check of these numbers. 0.027ns per “operation” should normally seem implausibly fast and set off alarm bells, but on a 2Ghz processor we might be issuing a 32-byte load and store every clock cycle. Each “varint” here is a single byte in the fastest case and 0.027ns is approximately ~36 bytes per nanosecond or ~18 bytes per clock, which aligns with the processor’s abilities.

Increasing representativeness

This particular benchmark works by parsing a single, varint width at a time. Once we shifted to the end-on-end buffer style, we can actually choose a random distribution of varint sizes. Varints shine when values are small and we can roughly assume that the distribution is accordingly small-skewed. Production profiles largely validate this assumption.

Using a synthetic distribution (like absl::LogUniform) can help us simulate this. This helps fight one pitfall of microbenchmarks by using more realistic data and avoiding “training” the branch-predictor too well.

NOTE: We want to strike a balance between improving representativeness enough to avoid chasing dead ends while simultaneously not adding so much complexity that our time would be better spent using more accurate methods like loadtests or production evaluation. Some of the pitfalls of microbenchmarks are inescapable, so the solution is to recognize the benchmark’s limitations rather than try to fully eliminate them.

void BM_VarintParsingSkewed(benchmark::State& state) {
  absl::BitGen rng;

  const int kNumVarints = 10000;
  std::vector<uint8_t> v(5 * kNumVarints, 0);
  uint8_t* buf = &v[0];
  for (int i = 0; i < kNumVarints; ++i) {
    // Initialize using a small-skewed value to create a synthetic distribution
    // of non-uniform varint sizes that roughly resemble real world ones.
    buf = WriteVarint32(
        buf, absl::LogUniform(rng, 0u, std::numeric_limits<uint32_t>::max()));
  }
  const uint8_t* const end_buf = buf;

  // Use KeepRunningBatch to get an accurate, per-item estimate of cost,
  // invariant from kNumVarints value.
  while (state.KeepRunningBatch(kNumVarints)) {
    // Start from the top every time, otherwise we run the loop once and stop,
    // so per-item speeds are infinitely fast.
    const uint8_t* read_buf = &v[0];
    while (read_buf < end_buf) {
      uint32_t read;
      read_buf = ParseVarint32(read_buf, &read);
      benchmark::DoNotOptimize(read);
    }
  }

  state.SetItemsProcessed(state.iterations());
  state.SetBytesProcessed(state.iterations() * (end_buf - &v[0]) / kNumVarints);
}

We can use the benchmark library’s built-in performance counter feature to capture per-benchmark statistics around branch mispredictions (--benchmark_perf_counters=BR_MISP_RETIRED:ALL_BRANCHES). This confirms that we processed enough varints (kNumVarints = 10000) to overwhelm the branch predictor’s ability to memorize the input pattern.

BM_VarintParsingEndOnEnd/0                        0.933 ns        0.933 ns    749030000 BR_MISP_RETIRED:ALL_BRANCHES=100.032u bytes_per_second=1022.04Mi/s items_per_second=1.07169G/s
BM_VarintParsingEndOnEnd/1                         1.64 ns         1.64 ns    432610000 BR_MISP_RETIRED:ALL_BRANCHES=100.999u bytes_per_second=1.13844Gi/s items_per_second=611.197M/s
BM_VarintParsingEndOnEnd/2                         2.16 ns         2.16 ns    331480000 BR_MISP_RETIRED:ALL_BRANCHES=8.9521m bytes_per_second=1.2962Gi/s items_per_second=463.928M/s
BM_VarintParsingEndOnEnd/3                         3.11 ns         3.11 ns    249780000 BR_MISP_RETIRED:ALL_BRANCHES=328.881u bytes_per_second=1.19976Gi/s items_per_second=322.058M/s
BM_VarintParsingEndOnEnd/4                         3.27 ns         3.27 ns    216500000 BR_MISP_RETIRED:ALL_BRANCHES=112.993u bytes_per_second=1.42589Gi/s items_per_second=306.208M/s
BM_VarintParsingSkewed                             5.66 ns         5.66 ns    123440000 BR_MISP_RETIRED:ALL_BRANCHES=0.762816 bytes_per_second=465.11Mi/s items_per_second=176.563M/s

We can back out the approximate item size from these throughputs as roughly 465/176, or ~2.6 bytes/varint on average. Because we incur more branch mispredictions, performance is worse than any of the end-on-end inputs.

For more complex benchmarks, using a randomly seeded benchmark can introduce run-to-run instability. While this can be remedied by using fixed seeds and deterministic distributions, this strategy risks using an aberrant input for our benchmark. Deterministically generating a set of varied inputs and then randomly shuffling them can help combat this.

Intentionally leaning into non-representativeness

With our current microbenchmark, we uniformly fit inside the processor’s L1 cache. When this tip was written, a processor with a 32KB L1 data cache was used for the benchmark result examples. In comparison, we’re parsing a series of 1K varints, whose length is at most 5 bytes each. This ensures that the data can always fit into cache.

The microbenchmark library framework includes helper methods (like benchmark::CPUInfo) to access information about the hardware we’re running on. This allows us to sweep through the parameter space of cache sizes, from a single varint (at most 5 bytes) to many multiples of the last cache’s size. We’ll also evaluate an optimization using BMI2 (implementation omitted for brevity).

template<bool DoBMI>
void BM_VarintParsingSkewedCaches(benchmark::State& state) {
  const int kNumVarints = state.range(0);
  absl::BitGen rng;

  // Pad input vector to permit an overread with the DoBMI implementation.
  std::vector<uint8_t> v(size_t{5} * kNumVarints + (DoBMI ? 8 : 0), 0);
  uint8_t* buf = &v[0];
  for (int i = 0; i < kNumVarints; ++i) {
    buf = WriteVarint32(buf, absl::LogUniform(rng, 1u, std::numeric_limits<uint32_t>::max()));
  }
  const uint8_t* const end_buf = buf;

  // Use KeepRunningBatch to get an accurate, per-item estimate of cost,
  // invariant from kNumVarints value.
  const uint8_t* read_buf;
  while (state.KeepRunningBatch(kNumVarints)) {
    // Start from the top every time, otherwise we run the loop once and stop,
    // so per-item speeds are infinitely fast.
    read_buf = &v[0];
    while (read_buf < end_buf) {
      uint32_t read;
      read_buf = ParseVarint32<DoBMI>(read_buf, &read);
      benchmark::DoNotOptimize(read);
    }
  }

  state.SetItemsProcessed(state.iterations());
  state.SetBytesProcessed(state.iterations() * (end_buf - &v[0]) / kNumVarints);
}

BENCHMARK_TEMPLATE(BM_VarintParsingSkewedCaches, false)
    ->Range(1, 4 * benchmark::CPUInfo::Get().caches.back().size);
BENCHMARK_TEMPLATE(BM_VarintParsingSkewedCaches, true)
    ->Range(1, 4 * benchmark::CPUInfo::Get().caches.back().size);

BM_VarintParsingSkewedCaches<false>/1               3.35 ns         3.34 ns    457036460 BR_MISP_RETIRED=6.03891u CYCLES=11.0191 INSTRUCTIONS=50 bytes_per_second=1.11618Gi/s items_per_second=299.622M/s
BM_VarintParsingSkewedCaches<false>/8               2.13 ns         2.12 ns    268979016 BR_MISP_RETIRED=9.79259u CYCLES=7.01295 INSTRUCTIONS=31.75 bytes_per_second=1.20814Gi/s items_per_second=471.718M/s
BM_VarintParsingSkewedCaches<false>/64              2.05 ns         2.05 ns    321649792 BR_MISP_RETIRED=22.9504u CYCLES=6.73112 INSTRUCTIONS=30.5938 bytes_per_second=1.23747Gi/s items_per_second=488.727M/s
BM_VarintParsingSkewedCaches<false>/512             2.05 ns         2.05 ns    313550336 BR_MISP_RETIRED=1.31374m CYCLES=6.75802 INSTRUCTIONS=30.6074 bytes_per_second=1.24299Gi/s items_per_second=488.45M/s
BM_VarintParsingSkewedCaches<false>/4096            6.37 ns         6.36 ns    106352640 BR_MISP_RETIRED=0.652793 CYCLES=20.8777 INSTRUCTIONS=31.0552 bytes_per_second=417.736Mi/s items_per_second=157.355M/s
BM_VarintParsingSkewedCaches<false>/32768           7.97 ns         7.95 ns     89980928 BR_MISP_RETIRED=0.902907 CYCLES=26.1535 INSTRUCTIONS=30.8263 bytes_per_second=330.981Mi/s items_per_second=125.816M/s
BM_VarintParsingSkewedCaches<false>/262144          8.66 ns         8.63 ns     83361792 BR_MISP_RETIRED=0.886097 CYCLES=25.8864 INSTRUCTIONS=30.7883 bytes_per_second=304.299Mi/s items_per_second=115.85M/s
BM_VarintParsingSkewedCaches<false>/2097152         7.88 ns         7.86 ns     90177536 BR_MISP_RETIRED=0.886966 CYCLES=25.9592 INSTRUCTIONS=30.8173 bytes_per_second=334.775Mi/s items_per_second=127.303M/s
BM_VarintParsingSkewedCaches<false>/16777216        7.91 ns         7.89 ns    100663296 BR_MISP_RETIRED=0.885126 CYCLES=26.0696 INSTRUCTIONS=30.8228 bytes_per_second=333.415Mi/s items_per_second=126.759M/s
BM_VarintParsingSkewedCaches<false>/134217728       8.52 ns         8.50 ns    134217728 BR_MISP_RETIRED=0.884259 CYCLES=26.0602 INSTRUCTIONS=30.8234 bytes_per_second=309.466Mi/s items_per_second=117.651M/s
BM_VarintParsingSkewedCaches<false>/161480704       8.06 ns         8.04 ns    161480704 BR_MISP_RETIRED=0.885522 CYCLES=26.1663 INSTRUCTIONS=30.8221 bytes_per_second=327.339Mi/s items_per_second=124.452M/s

We see inflections in performance as the working set increases beyond each cache tier. The benchmark framework can concurrently collect performance counters and we also asymptote to 0.88 branch predictions per varint that we parse on average.

Since our branches determine how quickly we move through the stream, we can see an improvement by switching to a branchless implementation using BMI2. Our performance is a bit worse for some of the small cases (where we’re already in L1 and can memorize the branches) but asymptote to a faster per-varint speed when working with large buffers. The PMU events show very few (<5e-6 per varint) branch mispredictions.

BM_VarintParsingSkewedCaches<true>/1                1.68 ns         1.68 ns    414588127 BR_MISP_RETIRED=31.3564n CYCLES=5.2722 INSTRUCTIONS=23 bytes_per_second=1.66702Gi/s items_per_second=596.649M/s
BM_VarintParsingSkewedCaches<true>/8                2.32 ns         2.32 ns    301227112 BR_MISP_RETIRED=4.35884u CYCLES=7.43719 INSTRUCTIONS=16 bytes_per_second=1.30667Gi/s items_per_second=431.699M/s
BM_VarintParsingSkewedCaches<true>/64               3.83 ns         3.82 ns    184510144 BR_MISP_RETIRED=11.3761u CYCLES=12.7085 INSTRUCTIONS=15.125 bytes_per_second=667.478Mi/s items_per_second=261.951M/s
BM_VarintParsingSkewedCaches<true>/512              4.24 ns         4.23 ns    174177280 BR_MISP_RETIRED=1.954m CYCLES=13.4285 INSTRUCTIONS=15.0156 bytes_per_second=613.728Mi/s items_per_second=236.535M/s
BM_VarintParsingSkewedCaches<true>/4096             4.04 ns         4.03 ns    173842432 BR_MISP_RETIRED=245.849u CYCLES=13.4066 INSTRUCTIONS=15.002 bytes_per_second=646.705Mi/s items_per_second=248.375M/s
BM_VarintParsingSkewedCaches<true>/32768            4.06 ns         4.05 ns    173244416 BR_MISP_RETIRED=44.8961u CYCLES=13.4526 INSTRUCTIONS=15.0003 bytes_per_second=646.09Mi/s items_per_second=246.625M/s
BM_VarintParsingSkewedCaches<true>/262144           4.40 ns         4.39 ns    173277184 BR_MISP_RETIRED=5.36135u CYCLES=13.4397 INSTRUCTIONS=15 bytes_per_second=598.464Mi/s items_per_second=227.561M/s
BM_VarintParsingSkewedCaches<true>/2097152          4.07 ns         4.06 ns    142606336 BR_MISP_RETIRED=1.66192u CYCLES=13.4542 INSTRUCTIONS=15 bytes_per_second=647.626Mi/s items_per_second=246.183M/s
BM_VarintParsingSkewedCaches<true>/16777216         4.07 ns         4.06 ns    184549376 BR_MISP_RETIRED=671.907n CYCLES=13.4525 INSTRUCTIONS=15 bytes_per_second=647.685Mi/s items_per_second=246.264M/s
BM_VarintParsingSkewedCaches<true>/134217728        4.05 ns         4.04 ns    134217728 BR_MISP_RETIRED=1.09524u CYCLES=13.4582 INSTRUCTIONS=15 bytes_per_second=651.6Mi/s items_per_second=247.741M/s
BM_VarintParsingSkewedCaches<true>/161480704        4.01 ns         4.00 ns    161480704 BR_MISP_RETIRED=829.821n CYCLES=13.4539 INSTRUCTIONS=15 bytes_per_second=658.024Mi/s items_per_second=250.184M/s

This pattern is also used in fleetbench. For example, it has a “hot” SwissMap benchmark that performs its operations (lookups, etc.) against a single instance and a “cold” SwissMap benchmark where we randomly pick a table on each iteration. The latter makes it more likely that we’ll incur a cache miss. Hardware counters and the benchmark framework’s support for collecting them can help diagnose and explain performance differences.

Even though the extremes are not representative, they can help us frame how to tackle the problem. We might find an optimization for one extreme and then work on it to hold performance for the other extremes constant.

Summary

In conclusion, microbenchmarks can be a useful technique for quickly exploring the performance landscape. Unlike macrobenchmarks which might take hours to complete to get reliable signals, microbenchmarks can give us information in a minute or two to make a better decision for what to try next while working on a problem.

While microbenchmarks are no substitute for using progressively more realistic benchmarks and evaluating production, imperfect information lets us iterate more quickly to achieve a better solution to perform a final, production evaluation and landing.

Performance Tip of the Week #74: Avoid sweeping street lights under rugs

2023-11-10T00:00:00-05:00

Originally posted as Fast TotW #74 on September 29, 2023

By Chris Kennelly and Matt Kulukundis

Updated 2025-10-03

Quicklink: abseil.io/fast/74

While their issues go by multiple names (i.e. the streetlight effect, the McNamara fallacy), proxy metrics present a seductive danger. Simply put, people often focus on improving the things they can measure and neglecting the things they cannot even if those other things are important. In this episode, we explore multiple stories of how this can go wrong to help ground ourselves in the real world failure modes that proxy metrics present.

Most folks are familiar with the streetlight effect’s titular story:

A police officer sees a drunkard searching for something under a streetlight and asks what the drunk has lost. They says they lost their keys and the two look under the streetlight together. After a few minutes the policeman asks if they is sure they lost them here, and the drunk replies, no, and that they lost the keys in the park. The policeman asks why they are searching here, and the drunk replies, “this is where the light is”.

But there is an analog that is equally dangerous.

A trash collector assigned to keep the streets free of litter may simply sweep the litter away from the street lights, thereby leaving the street visibly clean.

The “Data Center Tax”

In “Profiling a Warehouse-Scale Computer”, Svilen Kanev, et. al., coined the term, the “data center tax.” While a small part of every single application, the commonly used libraries could add up at scale when viewed horizontally. At the time in 2015, TCMalloc was as large as the biggest single application. This view on the data helps make these horizontal overheads “legible,” bringing an order to the chaos of understanding where fleet costs go. This analysis motivated a strategy of reducing these overheads as a relative portion of the fleet.

Nevertheless, there are places where externalities, both positive and negative, get overlooked. Focusing on the easily observed costs can cause us to miss opportunities for greater impact. We’ll start with several examples of where trying to reduce easily understood costs would have led to a worse outcome.

Artificial costs in TCMalloc

Starting from 2016, work commenced to reduce TCMalloc’s cost. Much of this early work involved making things generally faster, by removing instructions, avoiding cache misses, and shortening lock critical sections.

During this process, a prefetch was added on its fast path. Our fleet-wide profiling even indicates that 70%+ of cycles in the malloc fastpath are spent on that prefetch! Guided by the costs we could easily understand, we might be tempted to remove it. TCMalloc’s fast path would appear cheaper, but other code somewhere else would experience a cache miss and application productivity would decline.

To make matters worse, the cost is partly a profiling artifact. The TLB miss blocks instruction retirement, but our processors are superscalar, out-of-order behemoths. The processor can continue to execute further instructions in the meantime, but this execution is not visible to a sampling profiler like Google-Wide Profiling. IPC in the application may be improved, but not in a way immediately associated with TCMalloc.

Hidden context switch costs

We can observe the apparent cost of context switches by running perf and collecting PMU data from the processor. We end up running into two problems.

Running perf to collect PMU events perturbs the system. Collecting PMU events increases the overhead of context switches, to maintain the bookkeeping required for it.

The kernel cost is legible, and has been subject to substantial optimization. Changing from one process to another invalidates caches and the TLB. These will manifest as misses and stalls for ordinary user code, completely disconnected from the context switch itself.

We see a similar effect from changing scheduling parameters. Preferring to keep threads on the same core will improve cache locality, even though it may increase apparent kernel scheduler latency.

Sweeping away protocol buffers

Consider an extreme example. When our hashtable profiler for Abseil’s hashtables indicates a problematic hashtable, a user could switch the offending table from absl::flat_hash_map to std::unordered_map. Since the profiler doesn’t collect information about std containers, the offending table would no longer show up, although the fleet itself would be dramatically worse.

While the above example may seem contrived, an almost entirely analogous recommendation comes up with some regularity: migrate users from protos to structs. Due to its ubiquity, Protobuf has a large cost across the fleet.

It is true the structs have benefits in terms of simplicity. They have fewer features, are generally lower cost to the compiler, and lack hasbits. But they also have drawbacks, they lack support for arena allocation strategies, debugging affordances, and centralized optimizations.

Rewriting protobufs to use hand-written struct’s completely eliminates their contribution to our horizontal data, but still shifts the costs elsewhere. Migrations from internal protos to structs can produce very large performance wins, but these usually come by simplifying the surrounding code and dropping unused things. In other words, these migrations are usually catalysts for identifying room-at-the-middle of the stack for changing how APIs are used to make code more efficient.

Using protos alone or using structs alone will likely produce better wins than mixing and matching as it exposes the code to fewer copies and a smaller set of vocabulary types. For example, converting back and forth between absl::Cord and std::string can prove to be costly. While hand-rolled data structures might afford better optimizations upfront, these can often be brought back directly into the proto implementation–or where that is not practical, suggests a copy might be needed to interconvert.

Optimizing for externalities

We can use the problems and pitfalls of legible indicators to find opportunities to surface more data and uncover new optimizations. Thinking about the next metric to improve can help us understand the full scope of a single optimization and help us see the overall picture.

Seeing through reference counting and allocations

TCMalloc’s heap profiling feature tracks where memory is allocated. For long-lived, reference-counted objects, this view of the data is misleading. The code responsible for holding the last reference to the object may be completely unrelated to where it is allocated.

For example, our RPC framework is one of the largest sources of memory allocations in the fleet. It allocates memory that is held in absl::Cord. Since the framework does the allocation it looks like it is to blame for the memory usage. In reality, the application that holds onto the memory and identifying why/how the application holds the memory is the key to reducing memory footprint.

This problem motivated building a profiler for absl::Cords. Rather than solely track where data is allocated, we track where references occur. This allows us to ignore the false positive from Stubby and instead focus on code that holds long-lived Cords.

Similarly, consider when we embed type A as a data member in type B. Changing sizeof(A) indirectly changes sizeof(B) and the memory we allocate when we type new B() or std::vector. Small types and memory paddings are peanut-buttered across the codebase, but in aggregate can consume large amounts of memory for commonly used types.

Improving data placement

The cost of memory allocation is not simply the cost of the memory allocator itself, but also the TLB and cache misses that happen elsewhere in the program. With nearly ~40% of cycles backend bound, waiting on caches or memory, the apparent scope for allocator changes is much larger.

With the Temeraire optimization, TCMalloc changed how it placed allocations onto pages in memory. By reducing TLB misses and stalls, this sped up the other parts of the application, leading to application throughput, latency, and RAM usage broadly improved because of the better placement decisions. Even though this is an across-the-board win, applications tended to spend more time in TCMalloc in relative terms, making this an apparent regression.

Reducing calling convention costs

libc functions like memcpy, memset, and memcmp are frequently used. Every core in Google’s fleet makes several calls to these functions every second. During the development of implementations now in llvm-libc, we saw that implementations that were “slower” in microbenchmarks produced better application performance. Smaller implementations have less impact on instruction cache misses, both for the memcpy itself and for evicting other useful code.

With the advent of optimized instructions like rep movsb, we can inline the implementation to a single, two-byte instruction. This instruction uses 3 registers, containing source, destination, and size, akin to the arguments to memcpy. X86_64 callers maintain the stack and preserve several registers over calls. Even though rep movsb can be outperformed by hand-optimized implementations in microbenchmarks, this strategy can reduce code cache pressure and external overheads.

Making effective prefetches

During the A/B experiment evaluation of adaptive prefetching, we observed that while topline, application-performance broadly improved, individual functions sometimes regressed. If a function generally benefited from the HW prefetcher, it often regressed. If a function were antagonized by memory bandwidth saturation, it could improve.

This data identified an opportunity to add software prefetches to functions with simple access patterns. This allowed us to optimize for the best of both worlds: keeping effective prefetches intact, while ablating often ineffective hardware prefetchers under saturation.

Closing thoughts

When improving anything (performance, quality, even reliability) do not mistake the measures you have for the actual thing you want to improve. All metrics are proxy metrics for some deeper goal, and we as engineers must always be careful to avoid externalizing costs into areas where we simply don’t measure them. Metrics are not a replacement for good judgment–they are a tool to help us sharpen our judgment.

Performance Tip of the Week #70: Defining and measuring optimization success

2023-10-20T00:00:00-04:00

Originally posted as Fast TotW #70 on June 26, 2023

By Chris Kennelly

Updated 2025-10-03

Quicklink: abseil.io/fast/70

Engineers optimizing performance are ultimately trying to maximize the things Google does (serve search queries, videos on YouTube, etc.) and minimize the things Google buys (CPUs, RAM, disks, etc.). In this episode, we discuss how to choose metrics that help us influence the optimizations we work on, make effective decisions, and measure the outcome of projects.

Economic value

Things like search queries and video playbacks on YouTube represent economic value. Useful work is happening to deliver an experience to an end-user which then translates into Google revenue.

This isn’t simply a matter of adding up all of the requests that happen though: Some requests are more important than others:

A search query from a human is valuable. A search query from a prober used for monitoring is important insofar as it helps with monitoring the service’s reliability, but search exists for humans, not for monitoring.
A YouTube session might continue to play for several minutes after someone has left their computer. The automatic playback pause after inactivity cuts off a flow of unimportant video plays.

Things we make	Things we buy
Searches	CPUs
Driving directions	RAM
Cat videos	TPUs
Email messages	Disks
Cloud compute VM time	Electricity
Starcraft

Maximizing value and minimizing costs are the outcomes we are ultimately after.

In the course of handling requests across servers, there’s work that happens independently from user requests:

Above, we touched on probing for monitoring.
Request hedging helps give users consistent latency by sending duplicative requests: One of the requests and the work done might prove to be unnecessary. Taken to an exaggerated extreme, sending multiple hedged requests unconditionally is wasteful: Users don’t get meaningfully better latency and production uses resources profligately. Choosing the right time to hedge can let us strike a balance.
Deadline propagation allows child requests to backends to time out early, rather than continue to perform work believing the requestor will still use it. This is an important technique for minimizing unproductive work, especially during overload.
Load testing a machine or cluster to the point of overload produces valuable knowledge for capacity planning and validates that load shedding mechanisms work. The requests do not directly answer user requests; but without this knowledge, the user experience would eventually suffer as regressions go unnoticed.

Many of these activities are effectively a cost of doing business. It’s hard to run a reliable, low-latency service distributed across numerous servers without them. Nonetheless, it’s important to not let the tail wag the dog here: Monitoring or load testing are not an end unto themselves.

Selecting good proxies

Along the way, there are proxy metrics that can help with telling us that our optimization idea is on the right track, or help to explain the causal connection to top-level metrics. We want to align with the business problem, without boiling the ocean every time we make a small change and want to assess it. Measurement has its own return on investment too and the benefits of additional precision is quickly outweighed by the cost of obtaining it.

Goodhart’s Law reminds us that “when a measure becomes a target, it ceases to be a good measure.” Escaping this completely is challenging, but analysis is easier the more closely aligned the metric is with what we’re optimizing.

One common leap that we might need to make is connecting an abstract or harder to measure end goal such as business value or user satisfaction to more easily measured metrics. Totals of RPC requests made or their latency are common proxies for this.

In working on optimizations, we also need to optimize our feedback loop for developing optimizations. For example,

Microbenchmarks provide a much shorter feedback loop than measuring application performance in macrobenchmarks or production. We can build, run, and observe a microbenchmark in a manner of minutes. They have limitations, but as long as we’re mindful of those pitfalls, they can get us directional information much more quickly.
PMU counters can tell us rich details about bottlenecks in code such as cache misses or branch mispredictions. Seeing changes in these metrics can be a proxy that helps us understand the effect. For example, inserting software prefetches can reduce cache miss events, but in a memory bandwidth-bound program, the prefetches can go no faster than the “speed of light” of the memory bus. Similarly, eliminating a stall far off the critical path might have little bearing on the application’s actual performance.

If we expect to improve an application’s performance, we might start by taking a large function in the CPU profile and finding an optimization for it–say by changing to a more cache-friendly data structure. The reduction in cache misses and improvement in microbenchmark times help validate that the optimization is working according to our mental model of the code being optimized. We avoid false positives by doing so: Changing the font color of a webpage to green and running a loadtest might give a positive result purely by chance, not due to a causal effect.

These development-time proxies help us get an understanding of bottlenecks and performance improvements. We still need to measure the impact on application and service-level performance, but the proxies help us hone in on an optimization that we want to deploy faster.

When we are considering multiple options for a project, secondary metrics can give us confirmation after the fact that our expectations were correct. For example, suppose we chose option A over option B because both provided comparable performance but A would not impact reliability. We should measure both the performance and reliability outcomes to support our engineering decision. This lets us close the loop between expectations and reality.

Aligning with success

The metrics we pick need to align with success. If a metric tells us to do the opposite of a seemingly good thing, the metric is potentially flawed.

For example, Flume tracks useful work done by workers in terms of records processed. While no default is ever perfect–this elides that records can be of varying workloads, shapes, and sizes–it better aligns with other optimizations than bytes processed. With static and dynamic field tracking, Flume can read a subset of fields from every record. The total number of records is unchanged, but the total number of bytes goes down and total pipeline costs fall as well. Comparing the two possible metrics:

With records as the denominator, we see a drop in resources required per record processed with this optimization. ie the compute per record decreases with optimization.
With bytes as the denominator, absolute resource usage does go down, but fixed overheads stay the same. As a proportion of the total pipeline’s cost, fixed costs grow and are amortized over fewer bytes. ie the compute per byte may actually increase with optimization. This might make an optimization–skipping unnecessary data–look like an apparent regression.

In other cases, we want to normalize for the amount of work done. For example, a service where the size of the response depends on the request will likely want to track the number of returned bytes as its work done. A video transcoding service similarly needs to count the number of pixels: Videos at higher resolution require more processing time than lower resolution ones, and this roughly normalizes the higher difficulty per frame.

Pitfalls

Instructions per clock (IPC) is a challenging metric to use as a proxy. While executing more instructions in less time is generally good–for example, because we reduced cache misses by optimizing things–there are other times where it is worse. A thread spinning for a locked SpinLock is not making forward progress, despite having high IPC. Similarly, using vector instructions, rep movsb, or differences in microarchitecture allows us to accomplish more useful work in a single instruction. Optimizing for IPC or instructions can lead us to prefer behaviors that are worse for application performance.

Similarly, relative time in libraries is a useful yardstick for finding places to optimize and tracking their costs. In the long run, though, optimizations that speed up the whole program might come at the “cost” of this proxy.

Distributed systems complicate metrics as well. We still want to make sure the high level goal–business value per TCO (total cost of ownership) is maximized, but we may be able to put more precise calipers on subsystems to detect improvements (or regressions). Task-level metrics such as throughput and latency may not translate to the overall system’s performance: Optimizing the latency of a task off the critical path may have little impact on the overall, user-facing latency of a service. On the other hand, throughput improvements, allowing us to do more work with less CPU time and memory per request, allow us to use fewer resources to handle the same workload.

Summary

The goal of optimization projects is to maximize value–serving search queries, videos on YouTube, etc–and minimize costs–CPUs, RAM, disks, etc. Similar to how we can carefully use microbenchmarks to predict macrobenchmarks to predict production, we can select proxy metrics to measure success. This lets us align with business goals, especially harder to measure ones, while still having an effective yardstick for day-to-day work.

Performance Tip of the Week #60: In-process profiling: lessons learned

2023-10-15T00:00:00-04:00

Originally posted as Fast TotW #60 on June 6, 2022

By Chris Kennelly

Updated 2025-09-29

Quicklink: abseil.io/fast/60

Google-Wide Profiling collects data not just from our hardware performance counters, but also from in-process profilers. These have been covered in previous episodes covering hashtables.

In-process profilers can give deeper insights about the state of the program that are hard to observe from the outside, such as lock contention, where memory was allocated, and the distribution of collisions on a hashtable. In this tip we discuss how to determine that a new profiler is necessary, and the best practices for producing one.

Overview

“The purpose of computing is insight, not numbers.” – Richard Hamming

Developing a new profiler and augmenting existing ones allows us to have more information to make optimization decisions and aid debugging. The goal isn’t to have perfect information and to make perfect decisions, but to make better decisions faster, shortening our “OODA loop” (Observe Orient Decide Act). The value is in pulling in the area-under-curve and landing in a better spot. An “imperfect” profiler that can help make a decision is better than a “perfect” profiler that is unwieldy to collect for performance or privacy reasons. Extra information or precision is only useful insofar as it helps us make a better decision or changes the outcome.

For example, most new optimizations to TCMalloc start from adding new data points to TCMalloc’s statistics that are collected by visiting malloc profile handlers across the fleet. This information helps with understanding the scope of a particular phenomenon. After landing an optimization, these metrics can help provide indicators that we changed what we set out to change, even if the actual CPU and RAM savings might be measured by other means. These steps didn’t directly save any CPU usage or bytes of RAM, but they enabled better decisions. Capabilities are harder to directly quantify, but they are the motor of progress.

Leveraging existing profilers: the “No build” option

Developing a new profiler takes considerable time, both in terms of implementation and wallclock time to ready the fleet for collection at scale. Before moving to implement one, it is valuable to consider whether we can derive the necessary information from existing profilers and tools we already have.

For example, if the case for hashtable profiling was just reporting the capacity of hashtables, then we could also derive that information from heap profiles, TCMalloc’s heap profiles of the fleet. Even where heap profiles might not be able to provide precise insights–the actual “size” of the hashtable, rather than its capacity–we can make an informed guess from the profile combined with knowledge about the typical load factors due to SwissMap’s design.

It is important to articulate the value of the new profiler over what is already provided. A key driver for hashtable-specific profiling is that the CPU profiles of a hashtable with a bad hash function look similar to those with a good hash function. The added information collected for stuck bits helps us drive optimization decisions we wouldn’t have been able to make. The capacity information collected during hashtable-profiling is incidental to the profiler’s richer, hashtable-specific details, but wouldn’t be a particularly compelling reason to collect it on its own given the redundant information available from ordinary heap profiles.

Sampling strategies

A key design aspect of a profiler is deciding when and how to collect information. Most profilers do some kind of sampling to provide an estimate of the total without the overhead of recording every event. Collecting some data, even if heavily sampled, can be useful for gauging behaviors of a library at scale. There are two aspects to a profiler that need to be decided up front:

Duration versus duration-less: Several of our profilers track sampled events over a period of time. Other profilers capture an instantaneous snapshot of the program’s state.

Duration-less handlers are profiling during the entire program lifetime, which imposes a higher bar on the stability and overhead that can be required. In contrast, a profiler that is only active during collection can be more expensive, as collecting itself is rare.
Sampling strategy: The overheads of capturing data about every instance of a class can be prohibitive, so it’s important to determine a strategy for sampling - and figure out how that sampling strategy can be scaled to provide information representative of the entire fleet.

Sampling operations can make-or-break the feasibility of a profiler: Our compression profiler originally recorded statistics about every compression operation during its collection window, but the high overhead caused major services to turn off the profiler altogether. It was fixed by moving to a sampling strategy to only record statistics on a subset of compression operations during the profiling window. This allowed the profiler to be reenabled.

While knobs are often undesirable, allowing applications to tune their sampling rate (or whether sampling occurs at all) can be helpful for balancing the information gained against the overheads imposed.

Unless there is a justified exception, we require that the profiler applies the sampling factor back to the data to “unsample” it before returning it. This allows consumers to easily use the data without having to deal with the sampling arithmetic themselves. This is especially important as sampling rate can be variable–either automatically adjusted or tunable via a configuration knob such as a flag. This step can also help with validation via cross-checking with other profilers. For example, SwissMap’s total memory allocations seen by TCMalloc’s heap profiles are consistent with the total capacity seen by the hashtable profiler.

Choosing the right sampling approach (and the unsampling counterpart) needs to carefully balance accuracy vs. overhead. For example, with the heap profiler in TCMalloc one might decide to simply pick every Nth allocation. But that would not work well: in a typical app memory profiles are dominated by many small allocations and sampling those with reasonable overhead would require high sampling factor. It is also likely to miss more rare large allocations. Interestingly, the next obvious improvement of sampling an allocation every N bytes would “almost work” but is subject to statistical bias. This was fixed by introducing Poisson sampler which is used to date.

For libraries with significant global process state, the threads running in a process or the state of malloc, we may use a more exhaustive strategy. For example, a profiler could snapshot and summarize the state of the library without further sampling.

What data to record

In addition to choosing a sampling strategy, we need to decide what data to collect. We want to choose data that will influence an optimization decision. Just as different optimization projects have varying returns on investment, we want to strike a balance between the cost of implementing our profiler, of running it, and implementing the optimizations it motivates.

Mutation operations can be an excellent place to record additional statistics on sampled instances. These are frequently heavyweight for unrelated reasons–they trigger copies and reallocations–so checking whether to record statistics has minimal added performance penalty. This is the strategy we use for many of our existing profilers. In contrast, non-mutating operations, such as hashtable lookups, can be prohibitively expensive as we use these operations frequently and rely on them being fast.

There is a cost-benefit tradeoff for having more information. Sampling more frequently or collecting more data with each sample can paint a richer picture, but this increases the runtime cost of profiling. TCMalloc’s heap profiling has low, but non-zero costs, but it more than pays for itself by allowing us to look at where much of our RAM usage goes. Increasing the sampling rate would give us extra precision, but it wouldn’t materially affect the optimizations we can uncover and deploy. The extra overheads would negatively impact performance.

More practically, a minimal set of information can be a good starting point for getting a new profiler up and running to start debugging it. While obvious in hindsight, several new profilers have hit issues with their stack trace collection and filtering. While collecting more data can give additional insights, implementations that compute too many statistics or add contended locks may simply be infeasible. A profiler that is too expensive to leave enabled may be worse than no profiler at all: We spend time implementing it and rolling it out, but we lose the visibility into the library usage that we were after in the first place.

Summary

Profilers are a useful tool for probing the internal state of a program to answer questions during debugging and optimization. The types of questions posed can greatly influence the design and architecture of a profiler.

While a particular design may not be able to answer all questions, all at once, the goal is ultimately to make better decisions faster, shortening our “OODA loop” (Observe Orient Decide Act). Just as optimization projects are framed in terms of return-on-investment, we can frame how additional information influences or changes course of a decision.

Performance Tip of the Week #64: More Moore with better API design

2023-10-10T00:00:00-04:00

Originally posted as Fast TotW #64 on October 21, 2022

By Chris Kennelly

Updated 2025-09-29

Quicklink: abseil.io/fast/64

Optimizing library implementations only carries us so far in making software more efficient. In this episode, we discuss the importance of good APIs and the right abstractions for finding optimization opportunities. As we can make the hardware–especially with the end of Moore’s Law–and software run only so fast, the right abstractions give us continued optimization opportunities.

Correctness is paramount

We can simplify an implementation down to return 42; regardless of the input to see blazing fast results, but an API that doesn’t work correctly isn’t doing its job.

“Subtle” and “clever” code has costs for both maintainers and users alike. Today’s tricky edge cases can be tomorrow’s headaches when we try to optimize an implementation. Threading the needle of preserving explicitly (or implicitly) promised quirks makes the optimization process slower and more fragile over time. Being able to iterate faster helps with exploring more of the design space to find the best minima.

At times, we may need to break abstraction boundaries or have complex preconditions to unlock the best possible performance. We need to document and test these sharp edges. Future debugging has an opportunity cost: When we spend time tracking down and fixing bugs, we are not developing new optimizations. We can use assertions for preconditions, especially in debug/sanitizer builds, to double-check contracts and enforce them. Testing robots never sleep, while humans are fallible. Randomized implementation behaviors provide a useful bulwark against Hyrum’s Law from creeping in to implicitly expand the contract of an interface.

Express intents

Small, composable operations give users flexibility to express their intents more clearly. We can find optimizations by combining high-level but related concepts.

Consider memcpy and a hypothetical memcpy_but_faster API that we could build. They both express the same intent, but presumably with different tradeoffs around performance.

Users need to think about which one to call. This adds a cognitive cost to every call site. They cannot quickly reach for precisely one to realize their desired functionality. When in doubt, typing fewer characters is faster. Over time, choices made will be incorrect, either because they were suboptimal from the start or circumstances changed.
Bifurcating the API gives us two implementations, each with less usage. This lowers the leverage from developing optimizations to one, unless its maintainers can reliably cross-pollinate ideas from one to the other. Actively maintaining two implementations requires a larger investment, reducing the RoI from having two in the first place. Engineers may give the more commonly used implementation more care and attention, leading it to eventually outstrip the “faster” implementation.
Data structures and types can be especially costly to duplicate, due to the “impedance mismatch” of having a library that works solely with one type (say std::string) and another that needs a different one (absl::my_fast_string). In order for the two to interoperate, the interfaces will require expensive copies–a single type would not require such conversions.

While this hypothetical might seem far-fetched, this is precisely what happened with the predecessor implementation to absl::popcount. We had two implementations, but the “better” one was ultimately outstripped by the “worse” one because engineers optimized the one with the wider usage instead.

In terms of API design around intents, we can consider:

void* memcpy(void* dest, const void* src, size_t count);
crc32c_t absl::ComputeCrc32c(absl::string_view buf);
crc32c_t absl::MemcpyCrc32c(void* dest, const void* src, size_t count);

With the first two primitives, we can build a trivial, but non-optimal implementation for the third. Combining the concepts makes sense when it is a common operation where finer-grained operations might leave performance on the table. Knowing we are going to both copy and checksum the bytes allows us to read data once, rather than twice. We can decompose the implementation into its components, as well, if that ever became more efficient.

The extra output of the operation (the crc32c_t) distinguishes it from just being a memcpy with different performance characteristics. We would recommend using the combined operation when we need to both copy data and checksum it. MemcpyCrc32c isn’t a suitable replacement for calls to memcpy without a need for a checksum, which removes the cognitive cost of considering it solely for performance reasons.

The explicit function calls can also help with understanding the purpose of the code when we are looking at profiles later. For example, we can compare protobufs for equality in two ways:

By serializing and comparing the bytes, which is unfortunately both common and unsound.
Field-by-field directly, which is faster.

While reading a profile, we might see the individual calls to serialize and memcmp, but it is harder to ascertain the intended semantics later. We may be tempted to optimize the discrete functions–the process of serializing and subsequent the process of comparing the resulting string. Understanding the high-level intent and data flow gives us opportunities to optimize further up the stack to find the “Room at the Middle”, optimizing the direct comparison. At a minimum, an optimized version could avoid holding the serialized versions in memory.

Avoid unnecessarily strong guarantees

There are situations where the benefits of duplicate APIs outweigh the costs.

The Abseil hash containers (SwissMap) added new hashtable implementations to the codebase, which at first glance, appear redundant with the ones in the C++ standard library. This apparent duplication allowed us to have a more efficient set of containers which match the standard library API, but adhere to a weaker set of constraints.

The Abseil hash containers provided weaker guarantees for iterator and pointer stability, allowing them to improve performance by reducing data indirections. It is difficult to implement std::unordered_map’s guarantees without resorting to a node-based implementation that requires data indirections and constrains performance. Given std::unordered_map’s widespread usage, it was not feasible to relax these guarantees all at once.

Node-based containers necessitate implementation overheads, but they come with a direct benefit: They actively facilitate migration while allowing weaker containers to be available. Making a guarantee stronger without an accompanying benefit is undesirable.

The migration was a replacement path for the legacy containers, not an alternative. The superior performance characteristics meant that users could “just use SwissMap” without tedious benchmarking on a case-by-case basis. There’s little need for a user to revisit their decision to migrate to SwissMap with the passage of time. This meant that usage could be actively driven towards SwissMap: Two types would be a temporary (albeit long) state, rather than one where every individual usage had to be carefully selected.

Years after SwissMap’s development, there are far fewer–but non-zero–uses of std::unordered_map. Blocking the improvement on the complete cleanup means no benefit would have accrued. We were able to migrate instance-by-instance, realizing incremental benefits over time.

It’s important to avoid ascribing intent–even with expressive APIs–to use of a previously predominant one. A use of std::map might require keys to be ordered, but the more likely explanation might be that it is older code in need of updating.

Avoid leaking implementation details

Hyrum’s Law reminds us that observable behaviors will be relied upon, but sometimes our API design choices constrain our implementation details. These often arise from returning references to data or giving fine-grained control in APIs. This can help performance in the short-term, but care is required to make sure it allows long-term evolution to continue to improve performance over time.

Consider protocol buffers for a simple message.

message MyMessage {
  optional string foo = 1;
  repeated string bar = 2;
}

As of October 2023, the accessor .foo() returns a const std::string&. This requires that we have an in-memory representation of a std::string instance that can be returned. This approach has two problems:

std::string encodes a specific allocation strategy (std::allocator). If we change the allocation strategy, for example wrapping Arena, we change the type.
Individual fields can have a wide range of sizes (or likelihoods of presence) that we can determine from profiling, which could benefit from variable small string object buffer sizes. Returning const std::string& constrains the implementation to that particular size of buffer.

In contrast, by returning std::string_view (or our internal predecessor, StringPiece), we decouple callers from the internal representation. The API is the same, independent of whether the string is constant data (backed by the .rodata section), allocated on the heap by a std::string instance, or allocated by an Arena. We’ve abstracted away the implementation detail from our user, giving us more optimization freedom.

Similarly, consider the allocation-aware APIs in protobuf, add_allocated_..., release_..., and unsafe_arena_.... Fine-grained control over when and where allocations occur can offer significant performance benefits, but they also constrain future implementations by creating sharp performance edges.

release_... allows us to remove a submessage and return ownership to the caller. Subobjects were heap allocated and the operation was fast–it’s hard to beat swapping two pointers. When Protobuf Arenas became available, release_... created a new copy of the underlying message on the heap, so it could release that. The API couldn’t convey that the returned pointer was owned by the Arena, not caller, so making a full copy was required to keep code working. As a result, code that calls release_... may be O(1) or O(n) based on non-local information (whether the source object was constructed on an arena)!
With Arenas, unsafe_arena_... gives us the raw hooks we need to add or remove fields from a message without making the copy mentioned above , with “unsafe” in the name conveying the subtlety and gravitas of what we’re doing. These APIs are tricky to use correctly, though, as today’s tested combination of arena and heap ownership may change over time and assumptions break. The APIs are also extremely fine-grained, but do not convey the higher-level intent–transferring pointer ownership, “lending” a submessage to another one, etc.

Concluding remarks

Good performance should be available by default, not an optional feature. While feature flags and knobs can be useful for testing and initial rollout, we should strive to make the right choices for users, rather than requiring users adopt the improvement on a case-by-case basis.

Developing an optimization for an existing implementation can provide a larger return-on-investment by targeting widespread, current usage upfront. Adding a new API or optimization knob can be expedient, but without widespread usage and adoption, the benefit is far more limited.

Optimization of existing code can hit stumbling blocks around unnecessarily strong guarantees or APIs that constrain the implementation–and thus the optimization search space–too much to find improvements.

Performance Tip of the Week #52: Configuration knobs considered harmful

2023-09-30T00:00:00-04:00

Originally posted as Fast TotW #52 on September 30, 2021

By Chris Kennelly

Updated 2025-10-03

Quicklink: abseil.io/fast/52

Flags, options, and other mechanisms to override default behaviors are useful during a migration or as a short-term mechanism to address an unusual need. In the long term they go stale (not providing real benefit to users), are almost always haunted (in the haunted graveyard sense), and prevent centralized consistency/optimization efforts. In this episode, we discuss the tradeoffs in technical debt and optimization velocity for adding configurability.

The ideal flag lifecycle

When developing a new feature, it’s straightforward and often recommended to guard it behind a flag. This approach of using feature flags makes it possible to decouple pushing the code changes to production from turning on a new feature, which might have undiscovered correctness bugs or different resource requirements.

For a commonly-used library, flags also allow early opt-ins from users. When the default is changed, the flag also provides an escape hatch to revert to the old behavior.

For example, this was employed successfully for the rollout of TCMalloc’s Huge Page Aware Allocator optimization: many applications opted-in early, but even with extensive testing, a few applications saw changes in their resource requirements. These could be opted-out while deeper investigation occurred without rolling back the efficiency gains seen by most other users of TCMalloc.

These experiences suggest flags are an unalloyed good in theory, but practice is wholly different. Whether flags are considered good or not is dependent on what percentage of users will use the feature:

If the number of users of a flag is always expected to be small, its existence hampers future evolution.
If that number is mid-ranged, this can be a well-justified level of complexity but it can be challenging to set the flags optimally. Some teams have observed that, often, only the authors of features have the necessary context to set the flag appropriately - either to know when to set it at all or to which value it should actually be set.
If that number is near 100% – then probably we’re transitioning to a new default and the flag exists to provide an opt-out - this can be a good use of the flag. Nonetheless, it is important to clean that flag up after the rollout is complete so it doesn’t linger indefinitely. Without the cleanup, this becomes technical debt that hinders future changes or becomes a “standard knob with a weird name.”

Flags failing to convey intent

The units for many flags are entirely opaque and often have second or third order effects that may not be immediately intuitive.

In his 2021 CppCon talk, Titus Winters makes a real-world note of this phenomenon: The “popcorn button” of microwaves should not be used for microwave popcorn, as the button does not align with the settings required.

Moving to Google’s C++ codebase, SwissMap, Abseil’s high performance hashtables, does not provide an implementation of the flag max_load_factor. The low utility of max_load_factor was uncovered during the migration to SwissMap. Even worse, in many of the situations where max_load_factor was set, it was set incorrectly.

Even when the role of max_load_factor was correctly understood, its value was often misconfigured to achieve a desired goal. While max_load_factor(0.25) might convey an intent to “trade RAM for speed,” such a setting can make CPU performance worse while simultaneously using more RAM, defeating the intent of its user.

In other situations, different implementations can be API-compatible, but their behaviors do not transfer effectively between implementations. Open addressing hashtables have typical load factors <1, while chained hashtables have load factors typically ≥1. Changing between these implementations would cause the max_load_factor to have a surprisingly different effect.

This experience led the SwissMap authors to make max_load_factor a no-op, providing it only for API compatibility.

Stale configuration parameters

Tuning a configuration is another optimization that does not age well.

For flags defined in commonly used libraries, the defaults themselves have probably evolved: a feature was launched or an optimization landed. The nature of Google’s production configuration languages often means that once a service has hard-coded a flag’s value, it takes precedence over the default. This was the whole reason for choosing a non-default value in the first place; but with the codebase evolving at a high rate, it’s easy to overlook that the underlying infrastructure has improved and that overriding value now is worse than the default.

The key action here is to use customized flags lightly and regularly reconsider their use. When designing new options, prefer good defaults or make parameters self-tune if possible. Self-tuning may come in the form of adapting automatically to workloads, rather than requiring careful tuning through flags.

Reduced long-term velocity

Titus Winters notes that “If 99% of your users understand an API’s behavior through the lens of the default setting, the 1% of users that change that setting are at risk: APIs built at a higher level have a good chance of assuming the default behavior, leaving your 1% semi-supported.”

Configurability can be a great short-term boon; but long-term, configurability is a double edged sword. Options increase the state-space that has to be considered with every future change, making it more difficult to reason about, test, and successfully land new features in production. Beyond just optimizing costs, this complexity also hampers achieving better business objectives: Extra complexity that delays an improvement to product experiences is a non-obvious externality.

For example, TCMalloc has a number of tuning options and customization points, but ultimately, several optimizations came from sanding away extra configuration complexity. The rarely used malloc hooks API required careful structuring of TCMalloc’s fast path to allow users who didn’t use hooks–most users–to not pay for their possible presence. In another case, removing the sbrk allocator allowed TCMalloc to structure its virtual address space carefully, enabling several enhancements.

Beyond knobs

While this discussion has largely focused on knobs and tunables, APIs and libraries have the same challenges.

An existing library, X, might be inadequate or insufficiently expressive, which can motivate building a “better” alternative, Y, along some dimensions. Realizing the benefit of using Y is dependent on users both discovering Y and picking between X and Y correctly–and in the case of a long-lived codebase, keeping that choice optimal over time.

For some uses, this strategy is infeasible. my::super_fast_string will probably never replace std::string because the latter is so entrenched and the impedance mismatch of living in an independent string ecosystem exceeds the benefits. Multiple vocabulary types suffer from impedance mismatch–costly interconversions can overwhelm the overall benefits. The costs of migrating the world also need to be considered upfront. Without active migration, we end up with two things.

There are times where a new library or API is truly needed – SwissMap needed to break stability guarantees provided by std::unordered_map on an instance-by-instance basis to avoid waiting for every problematic usage to be fixed. In that case, however, the performance benefits it provided were only realized by active migration. Being able to aim for a complete migration eases maintenance and educational burdens as well. A compelling performance case simplified to “just use SwissMap” avoids the need for painstaking benchmarking with every use where the optimal choice could get out of date.

Best practices

When adding new customization points, consider how they’ll evolve over the long-term.

When using flags to gate new features that will be enabled by default, make a plan for removing any opt-outs so the flag itself can be removed, rather than end up as technical debt.
Flags are a powerful tool for tuning and optimization, but the author of a customization point has the most context for how to use it effectively. Choosing good defaults or making features self-tune is often better for the codebase as a whole.

Discoverability, let alone optimal selection, is challenging.

Performance Tip of the Week #7: Optimizing for application productivity

2023-09-14T00:00:00-04:00

Originally posted as Fast TotW #7 on June 6, 2019

By Chris Kennelly

Updated 2025-10-03

Quicklink: abseil.io/fast/7

Overview

Google manages a vast fleet of servers to handle search queries, process records, and transcode cat videos. We don’t buy those servers to allocate memory in TCMalloc, put protocol buffers into other protocol buffers, or to handle branch mispredictions by our processors.

To make our fleet more efficient, we want to optimize for how productive our servers are, that is, how much useful work they accomplish per CPU-second, byte-second of RAM, disk operation, or by using hardware accelerators. While measuring a job’s resource consumption is easy, it’s harder to tell just how much useful work it’s accomplishing without help.

A task’s CPU usage going up could mean the task has suffered a performance regression or that it’s simply busier. Consider a plot of a service’s CPU usage against time, breaking down the total CPU usage of two versions of the binary. We cannot determine from casual inspection what caused the increase in CPU usage, whether this is from an increase in workload (serving more videos per unit time) or a decrease in efficiency (some added, needless protocol conversion per video).

To determine what is really happening, we need a productivity metric which captures the amount of real work completed. If we know the number of cat videos processed, we can easily determine whether we are getting more, or less, real work done per CPU-second (or byte-second of RAM, disk operation, or hardware accelerator time). These metrics are referred to as application productivity metrics, or APMs.

If we do not have productivity metrics, we are faced with entire classes of optimizations that are not well-represented by existing metrics:

Application speedups through core infrastructure changes:

As seen in our 2021 OSDI paper, “one classical approach is to increase the efficiency of an allocator to minimize the cycles spent in the allocator code. However, memory allocation decisions also impact overall application performance via data placement, offering opportunities to improve fleetwide productivity by completing more units of application work using fewer hardware resources.”

Experiments with TCMalloc’s hugepage-aware allocator, also known as Temeraire, have shown considerable speedups by improving application performance, not time spent in TCMalloc.

We spend more relative time in TCMalloc but greatly improve application performance. Focusing just on relative time in TCMalloc would produce an error in sign: We’d deprioritize (or even rollback) a strongly positive optimization.
Allocating more protocol buffer messages on Arenas speeds up not just the protocol buffer code itself (like message destructors), but also in the business logic that uses them. Enabling Arenas in major frameworks allowed them to process 15-30% more work per CPU, but protobuf destructor costs were a small fraction of this cost. The improvements in data locality could produce outsized benefits for the entire application.
New instruction sets: With successive hardware generations, vendors have added new instructions to their ISAs.

In future hardware generations, we expect to replace calls to memcpy with microcode-optimized rep movsb instructions that are faster than any handwritten assembly sequence we can come up with. We expect rep movsb to have low IPC (instructions per cycle): It’s a single instruction that replaces an entire copy loop of instructions!

Using these new instructions can be triggered by optimizing the source code or through compiler enhancements that improve vectorization.

Focusing on MIPS (millions of instructions per second) or IPC would cause us to prefer any implementation that executes a large number of instructions, even if those instructions take longer to execute to copy n bytes.

In fact, enabling the AVX, FMA, and BMI instruction sets by compiling with --march=haswell shows a MIPS regression while simultaneously improving application productivity. These instructions can do more work per instruction, however, replacing several low latency instructions may mean that average instruction latency increases. If we had 10 million instructions and 10 ms per query, we may now have 8 million instructions taking only 9 ms per query. QPS is up and MIPS would go down.

Since Google’s fleet runs on a wide variety of architectures, we cannot easily compare instructions across platforms and need to instead compare useful work accomplished by an application.
Compiler optimizations: Compiler optimizations can significantly affect the number of dynamically executed instructions. Techniques such as inlining reduce function preambles and enable further simplifying optimizations. Thus, fewer instructions translate to faster, more productive code.
Kernel optimizations: The kernel has many policies around hugepages, thread scheduling, and other system parameters. While changing these policies may make the kernel nominally more costly, for example, if we did more work to compact memory, the application benefits can easily outweigh them.

Availability of these metrics help infrastructure and efficiency teams guide their work more effectively.

Abseil Breaking Change Policy Update

2023-08-28T00:00:00-04:00

Abseil Breaking Change Policy Update

By Derek Mauro

Today we are announcing that Abseil is adopting the Google OSS Library Breaking Change Policy. What does this mean for Abseil users? Let’s take a look.

Abseil’s original compatibility policy made this statement:

We will always strive to not break API compatibility. If we feel that we must, we will provide a compiler-based refactoring tool to assist in the upgrade … any time we are making a change publicly we’re also doing the work to update the 250MLoC+ internal Google codebase — we very rarely do such refactoring that cannot be automated.

In the 6 years since Abseil has been available to the open source community, we have released only one tool, and we believe that there is a good chance that no one ever needed to use that tool.

There are a few reasons why we never really needed to make use of our policy of providing an automated update tool.

First, Abseil only ships production-ready APIs. While these APIs did evolve in their early life, APIs must be considered stable and have thousands of usages in Google’s internal codebase before they are eligible for inclusion in Abseil.

Second, as stated in the original policy, when we do change an API, we also have to update Google’s internal code base. While we do use automated tooling to do this, the fact that Google’s internal code base is so large constrains the types of changes that we can make.

The policy of releasing a tool for automated upgrades has not worked out in practice. Not only have we not made changes that would have made upgrading difficult without the use of a tool, but the policy of requiring releasing a tool has also made it harder for us to release fixes that are technically API breaks but that impact very few callsites.

Under the new policy, despite claiming the right to make breaking changes without shipping an automated upgrade tool, we still commit to making upgrading as easy as possible, and only making breaking changes when we believe they will provide benefit to our users. We will announce these changes prominently in our commit messages as well as in the release notes for LTS releases, and will provide guidance on how to resolve issues.

Please read our updated compatibility guidelines for more information about what users must (and must not) do to successfully update to newer versions of Abseil.

Perf Tips

2023-03-02T00:00:00-05:00

Performance Tips of the Week

By Alexey Alexandrov, Google Engineer

We are pleased to announce that we’ve started publishing some of Google’s internal “Performance Tips of the Week” externally on abseil.io!

These tips form a sort of “Effective analysis and optimization of production performance and resource usage”: a gallery of “do”s and “don’t”s gathered from the hard-learned lessons of optimizing performance of production systems running on machines in Google data centers.

This set of tips started as an internal series at Google and it has been hugely popular, with thousands of monthly readers and dozens of episodes published. We are making some of these episodes available to a wider external audience as they discuss topics that are common to the industry. We hope you find these tips valuable and welcome your feedback!

–Alexey

Performance Tip of the Week #9: Optimizations past their prime

2023-03-02T00:00:00-05:00

Originally posted as Fast TotW #9 on June 24, 2019

By Chris Kennelly

Updated 2025-10-03

Quicklink: abseil.io/fast/9

Overview

Optimizations don’t always age gracefully. Faster yesterday might mean slower today.

Benchmarks citing performance on Intel Pentium 3’s or AMD Opterons may have been meaningful several years ago, but optimization equilibria, originally chosen for long-unplugged platforms, may have changed since. Let’s look at a couple examples where well-intended optimizations ultimately hurt performance in the long run.

Popcount

In 2008, Intel introduced the popcnt instruction to determine the number of set bits in a 32- or 64-bit integer. This is of interest for computing hamming distances and a bunch of other things. Without the instruction, we can use a slightly more complex, longer sequence of shifts, bitwise ands, and adds to achieve the same.

Google’s predecessor library to C++20’s bit manipulation functions offered two population count routines. CountOnes64 uses popcnt if the compiler knows it’s going to be available and a fallback otherwise. CountOnes64withPopcount unconditionally uses popcnt on x86_64 machines.

When this instruction first started rolling out, it made sense to test for the availability of the instruction at runtime before choosing which one to call. This would be faster for crunching data on the machines with the instruction and we’d avoid SIGILL‘ing on machines without it.

Years later, every x86_64 machine in the fleet supported this instruction, so CountOnes64 and CountOnes64withPopcount should be the same, right?

Unfortunately, we’re still paying for all of the runtime dispatch machinery to check availability of the instruction, even though the answer is always “yes.” We could make this a compile time constant, but this is akin to picking up rocks faster when we should have instead just left them on the ground. The branch cost might seem trivial, but there’s actually more to be concerned about here.

Prior to cleanups, the implementations weren’t the same.

CountOnes64withPopcount used inline assembly to unconditionally emit the popcnt instruction. The inline assembly prevented the compiler from working around a false dependency bug in some processors.
When the compiler built-in is used (the “slow” version), we actually end up with a better sequence of machine code and can perform stronger optimizations at compile-time around constant folding.

Once the compiler began emitting the popcnt instruction for __builtin_popcount, CountOnes64 was the unconditionally better implementation.

Ironically, code using runtime dispatch to select a “fast” implementation is (unintentionally) preferring an actually slower implementation (CountOnes64withPopcount) and paying for the privilege.

`CHECK_EQ`

In 2005, Google implemented its CHECK logging macros in terms of CheckOpString, a thin wrapper around a std::string*. This later underwent further optimizations, adding hints to the compiler optimizer that the comparison would likely be true.

As of early 2019, a simplified implementation for optimized builds for CHECK_EQ(a, b), after preprocessing looked like:

template<typename T1, typename T2>
std::string* MakeCheckOpString(const T1& a, const T2& b, const char*);

std::string* Check_EQ_Impl(const T1& a, const T2& b, const char* error) {
  if (ABSL_PREDICT_TRUE(a == b))
    return nullptr;
  else
    return MakeCheckOpString(a, b, error);
}

struct CheckOpString {
  CheckOpString(std::string* str) : str_(str) {}
  operator bool() const { return ABSL_PREDICT_FALSE(str_ != nullptr); }
  std::string* str_;
};

#define CHECK_EQ(a, b)                                                \
  while (CheckOpString _result = Check_EQ_Impl(a, b, "...error..."))  \
    ...log failure...

When LLVM generated assembly for this code, it made two checks:

a == b: We predict that this is typically true.
str_ != nullptr: We predict that this is typically false.

…but the second check is redundant. Once we’ve determined that a == b, str_ is always nullptr. This is a missed compiler optimization, but the optimizer faces the challenge of layers of complexity and hand-tuning added to this code over a decade.

Removing CheckOpString completely removes the extra branch: We compare a and b, but the optimizer does not need to reason about CheckOpString. Working directly with std::string* for the comparison leads to better code. Ironically, this optimization had already been applied to debug builds, added in 2008.

Best practices

Prefer writing clear, idiomatic code whenever possible. It is not only easier to read and debug, but in the long run, also easier for the compiler to optimize.
Whenever you find a low-level performance optimization that requires fancy bit-twiddling, intrinsics code, or inline assembly, consider first whether this is something the compiler could do.
If the code is hot, and the optimization is not something the compiler can be taught to perform, then: prefer portable code, possibly using hwy to generate efficient and portable vector code, failing that use intrinsics, failing that use inline asm (this should be extremely rare). Avoiding inline assembly makes the code more portable across microarchitectures.
Keep the “naive” code you are replacing. If you are optimizing ComputeFoo, consider keeping the simple implementation in a REFERENCE_ComputeFoo function. This makes it easy to write a unit-test for the new implementation that ensures the two functions are equivalent; it makes it easier to write a microbenchmark; and it makes it easier to revert to the reference code when (not if) the machine-dependent implementation outlives its usefulness.
Include a microbenchmark with your change.
When designing or changing configuration knobs, ensure that the choices stay optimal over time. Frequently, overriding the default can lead to suboptimal behavior when the default changes by pinning things in a worse-than-out-of-the-box state. Designing the knobs in terms of the outcome rather than specific behavior aspects can make such overrides easier (or even possible) to evolve.

Performance Tip of the Week #53: Precise C++ benchmark measurements with Hardware Performance Counters

2023-03-02T00:00:00-05:00

Originally posted as Fast TotW #53 on October 14, 2021

By Mircea Trofin

Updated 2025-09-03

Quicklink: abseil.io/fast/53

Use performance benchmarks as the first line of defense in detecting costly regressions, and as a way to guide performance improvement work. Getting to the root cause of a change in performance can be time consuming and full of “false leads”, because on modern architectures program execution is influenced by many factors.

In this episode, we present a productivity tool that helps lower the cost of performance investigations by leveraging Hardware Performance Counters to surface low-level architectural metrics. The tool is available for C++ benchmarks running on Linux, on GitHub.

What are Hardware Performance Counters?

Hardware Performance Counters are a hardware feature where you can request precise counts of events such as: instructions retired, load or store instructions retired, clock cycles, cache misses, branches taken or mispredicted, etc. See https://perf.wiki.kernel.org/index.php/Tutorial for more information.

With performance counters you get less noisy measurements when compared to time-based ones. CPU timer-based measurements are noisier, even on isolated machines, because:

performance counter measurements can be isolated to the benchmark process, and, thus, not account context switching time when the process is preempted - which is otherwise measured by the CPU timer,
we can further isolate counter increments to user mode executed instructions only, which further reduces noise due to context switching
specific counters (e.g. instruction-counting ones) inherently produce measurements that are almost noise-free (variations under 0.01%). This is because the value of such counters is independent of systemic sources of noise like frequency throttling.
finally, depending on the setup of the benchmarking machine, time-based measurements suffer from noise introduced by thermal effects, hyperthreading, or shared resource (like memory bus) access. Some counters will also suffer from noise due to these, but others - like instructions retired counters - won’t.

By selecting appropriate performance counters you can get nuanced insight into the execution of a benchmark. For instance, a measurement using CPU time that points to a regression may be caused by subtle changes in executable layout, which increases branch mispredictions. This is generally not actionable and considered acceptable. Identifying this is the case, when only looking at time measurements, is not very productive and not scalable over a large benchmark suite corpus. With performance counter-based measurements, it is immediately apparent by observing branch mispredict variations and instruction count variations, and the detection is easily scriptable.

How-to gather performance counter data

The Google Benchmark project simplifies the process of writing a benchmark. An example of its use may be seen here

The benchmark harness support for performance counters consists of allowing the user to specify counters in a comma-separated list, via the --benchmark_perf_counters flag, to be measured alongside the time measurement. Just like time measurement, each counter value is captured right before the benchmarked code is run, and right after. The difference is reported to the user as per-iteration values (similar to the time measurement).

Basic usage

Note: counter names are hardware vendor and version specific. The example here assumes Intel Skylake. Check how this maps to other versions of Intel CPUs, other vendors (e.g. AMD), or other architectures (e.g. ARM); also refer to perfmon2 which we use for counter name resolution, and/or perf list.

Build a benchmark executable - for example, let’s use “swissmap” from fleetbench:

bazel build -c opt //fleetbench/swissmap:swissmap_benchmark

Run the benchmark; let’s ask for instructions, cycles, and loads:

bazel-bin/fleetbench/swissmap/swissmap_benchmark \
  --benchmark_filter='.*Cold.*::absl::flat_hash_set.*64.*set_size:64.*density:0' \
  --benchmark_perf_counters=INSTRUCTIONS,CYCLES,MEM_UOPS_RETIRED:ALL_LOADS

The output looks like:

Running ./swissmap_benchmark
Run on (8 X 4667.91 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 256 KiB (x4)
  L3 Unified 8192 KiB (x1)
Load Average: 2.31, 2.08, 1.95
---------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                             Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------------------------------------------
BM_FindMiss_Cold<::absl::flat_hash_set, 64>/set_size:64/density:0                  18.4 ns         18.4 ns     39048136 CYCLES=82.9019 INSTRUCTIONS=35.7284 MEM_UOPS_RETIRED:ALL_LOADS=6.05507
BM_FindHit_Cold<::absl::flat_hash_set, 64>/set_size:64/density:0                   33.3 ns         33.3 ns     20600490 CYCLES=152.156 INSTRUCTIONS=55.0354 MEM_UOPS_RETIRED:ALL_LOADS=15.0034
BM_InsertHit_Cold<::absl::flat_hash_set, 64>/set_size:64/density:0                 34.8 ns         34.8 ns     19004416 CYCLES=157.956 INSTRUCTIONS=59.0354 MEM_UOPS_RETIRED:ALL_LOADS=16.0013
BM_Iterate_Cold<::absl::flat_hash_set, 64>/set_size:64/density:0                   33.5 ns         33.5 ns     25444389 CYCLES=152.431 INSTRUCTIONS=57.9225 MEM_UOPS_RETIRED:ALL_LOADS=13.3892
BM_InsertManyOrdered_Cold<::absl::flat_hash_set, 64>/set_size:64/density:0         54.9 ns         54.8 ns     14141958 CYCLES=242.373 INSTRUCTIONS=111.455 MEM_UOPS_RETIRED:ALL_LOADS=33.1838
BM_InsertManyUnordered_Cold<::absl::flat_hash_set, 64>/set_size:64/density:0       50.0 ns         50.0 ns     14234753 CYCLES=227.516 INSTRUCTIONS=111.415 MEM_UOPS_RETIRED:ALL_LOADS=33.1781

So we can see that BM_FindMiss_Cold took approximately 83 cycles, 36 instructions, and 6 memory ops per iteration.

Limitations

Number of counters: At most 32 events may be requested for simultaneous collection. Note however, that the number of hardware counters available is much lower (usually 4-8 on modern CPUs, see PerfCounterValues::kMaxCounters) – requesting more events than the hardware counters will cause multiplexing and decreased accuracy.
Visualization: There is no dedicated visualization UI available, so for complex analysis, users may need to collect JSON result files and summarize the results.
Counting vs. Sampling: The framework only collects counters in “counting” mode – it answers how many cycles/cache misses/etc. happened, but not does not associate them to the code location causing the events. For that, you’d need a sampling profiler like Linux perf.

Summary

Use the --benchmark_perf_counters flag in https://github.com/google/benchmark benchmarks to quickly drill into the root cause of a performance regression, or to guide performance optimization work.

Performance Tip of the Week #39: Beware microbenchmarks bearing gifts

2023-03-02T00:00:00-05:00

Originally posted as Fast TotW #39 on January 22, 2021

By Chris Kennelly and Alkis Evlogimenos

Updated 2025-09-29

Quicklink: abseil.io/fast/39

Benchmarks are only a tool for debugging efficiency: Production is ultimately what matters. Benchmarks analyze the performance of code under the specific circumstances created and maintained by the benchmark. They cannot perfectly predict the performance of code in the real world. In this episode, we discuss some of the pitfalls of microbenchmarks and mitigation strategies.

For example, we can use the following series of benchmarks for evaluating changes to search query performance:

Posting List iteration benchmarks
Single task loadtests
Cluster loadtests of many tasks
Production

Each benchmark is increasingly more complicated and increasingly more accurate but takes more time to run. It is not uncommon for the first and last to disagree. It is less common for single task loadtests to disagree with cluster loadtests but it happens. The reason to have the hierarchy of benchmarks is to optimize the iterative process of developing performance optimizations themselves. If we are able to observe and iterate faster (a shorter “OODA loop”), we explore more ideas more quickly.

There is a parallel here to testing, starting from narrow scope (unit tests) to increasing scope (integration and system tests) to production itself. Like correctness testing, performance testing is complicated. Unfortunately, performance testing is less predictive than correctness testing, especially at the micro scale. Surrounding code or competing processes might interfere with network resource, memory bandwidth, CPU instruction decoding, branch predictor, processor cache utilization, flash i/o, mutexes. If, for example, we have medium confidence that a passing unittest suggests correctness in prod, then we should have relatively low confidence in benchmark results materializing the same way in prod.

Where benchmarks mislead

There are situations where the results from benchmarks can be misleading when applied to full applications. The following are some examples of situations where the results from benchmarks are not applicable at application scale.

`memcmp` bloat

Google’s software tends to be instruction footprint-heavy, challenging our ability to effectively cache it, which leads to frontend stalls on our processors. The functions provided by libc are no exception.

In “AsmDb: Front-End Stalls in WSC” (Figure 13 / Section 4.4), the glibc implementation for memcmp was ~6KB, leading to icache misses of its own and evictions of other functions. The implementation for glibc is in hand-written assembly, using a variety of techniques based on comparison size to obtain “good” performance. 99% of cycles in the function span 41 cache lines, or about 2.6KB of cache capacity.

Individually, these small micro-optimizations were entirely justifiable with microbenchmarks. We saw this in greater detail while developing a Google workload-tuned memcpy implementation: “Faster” implementations on microbenchmarks had larger code footprints and produced worse macrobenchmark results than seemingly slower implementations that had smaller code footprints.

This is undoubtedly a place where hardware acceleration can shine to deliver its best possible performance consistently, rather than having programmers optimize their implementations for each generation (only to make things worse in real workloads).

Arithmetic versus load

Consider a simple masking function, to get the lower bits of a value:

uint32_t getLowerBits(uint32_t value, int bits) {
  return value & ((uint64_t{1} << bits) - 1);
}

This requires several instructions and processor uops to shift, subtract, and finally mask. This might motivate loading a precomputed array of masks.

constexpr uint32_t kMasks[] = {0x1, 0x3, 0x7, 0xf, ...};

uint32_t getLowerBits(uint32_t value, int bits) {
  return value & kMasks[bits];
}

This might appear to be a profitable optimization in microbenchmarks. kMasks can be consistently cached in L1. Modeling cache behavior in microbenchmarks is challenging: Microbenchmarks tend to have small working sets that tend to be cache resident. Real code, particularly Google C++, is not.

In production, the cacheline holding kMasks might be evicted, leading to much worse stalls (hundreds of cycles to access main memory). Additionally, on x86 processors since Haswell, this optimization can be past its prime: BMI2’s bzhi instruction is both faster than loading and masking and delivers more consistent performance.

When developing benchmarks for SwissMap, individual operations were benchmarked in two ways: always triggering a cache hit and always triggering a cache miss. The latter can be achieved by having enough hashtables that their working set will not fit in cache, then picking a hashtable to lookup at random on every iteration of the benchmark.

Having explicit benchmarks for the boundary conditions - always-cache-hit and always-cache-miss gives more insight on how changes to the code affect its operations under different conditions and help develop intuition on which operations are important to optimize to reach our goal: improve production performance.

TCMalloc’s suspicious prefetch

TCMalloc has a suspicious prefetch on its allocation path. By the time we’ve computed which object to return, prefetching would be too late: User code could begin using the object within a few cycles. Instead, it prefetches the next object of the same size class that would be returned. This object will not be used by the application until the next time we allocate another object of the same size.

This prefetch appears to be extraordinarily costly: Microbenchmarks measuring allocation performance show potential savings if it were removed and Google-Wide Profiling shows 70%+ of cycles in new’s fastpath on the prefetch. Removing it would “reduce” the data center tax, but we would actually hurt application productivity-per-CPU. Time we spend in malloc is less important than application performance.

Trace-driven simulations with hardware-validated architectural simulators showed the prefetched data was frequently used. Additionally, it is better to stall on a TLB miss at the prefetch site–which has no dependencies, than to stall at the point of use.

Pitfalls

There are a number of things that commonly go wrong when writing benchmarks. The following is a non-exhaustive list:

Data being resident. Workloads have large footprints, a small footprint may be instruction bound, whereas the true workload could be memory bound. There’s a trade-off between adding instructions to save some memory costs vs placing data in memory to save instructions.
Small instruction cache footprint. Google codes typically have large instruction footprints. Benchmarks are often cache resident. The memcmp and TCMalloc examples go directly to this.
Sensitivity to function and branch alignment. Small changes to code layout and branch alignment can have large effects on microbenchmark performance. Code being changed can also affect neighboring code that is actually benchmarked, leading to paradoxical speedups and slowdowns. For example, memcpy has 20% swings from this effect. For several years, snappy had “load bearing nops” to perturb branch alignment of an inner loop to an optimal state. Stabilizer (by Berger, et. al.) deliberately perturb these parameters to improve benchmarking statistical quality.
Sensitivity to stack alignment. Changes anywhere in the stack–added/removed variables, better (or worse) spilling due to compiler optimizations, etc.–can affect the alignment at the start of the function-under-test. This has been seen to produce 20% performance swings.
Representative data. The data in the benchmark needs to be “similar” to the data in production - for example, imagine having short strings in the benchmark, and long strings in the fleet. This also extends to the code paths in the benchmarks being similar to the code paths that the application exercises. This is a common pain point for macrobenchmarks too. A loadtest may cover certain request types, rather than all of those seen by production servers.
Benchmarking the right code. It’s very easy to introduce code into the benchmark that’s not present in the real workload. For example, using a random number generator’s cost for a benchmark could exceed the cost of the work being benchmarked.
Being aware of steady state vs dynamic behaviour. For more complex benchmarks it’s easy to produce something that converges to a steady state - for example if it has a constant arrival rate and service time. Production workloads may demonstrate more variability.

Leveraging benchmarks fully

Focus on writing microbenchmarks that test individual properties of the component under test. Using Search as an example, consider the compute kernel of search intersects posting lists (PLs). A posting list is a sorted sequence of hits, where a hit is an occurrence of a term in a document. When a query like “dog AND cat” is executed, the PLs for terms “dog” and “cat” are intersected to find the documents that contain both “dog” and “cat”.

For PLs we benchmark the basic operations of a PL:

PL iterator creation/destruction
PL iterator iteration
PL iterator advance (jump forward)

Observe production behaviour to determine the appropriate code and functionality to optimize:

Profile production jobs to identify which operation is critical to performance.

In some cases, PL iterator advance is where the majority of CPU cycles are spent. In others, PL iterator creation is as critical as PL iterator advance.
After identification of the above, one can start looking for opportunities to:
- improve PL iterator creation/destruction without pessimizing PL iterator advance
- improve PL iterator advance without pessimizing PL iterator creation/destruction
Note: PL iterator iteration is largely irrelevant - at this particular time and shape of the system

After coming up with a strategy and prototype to improve the two microbenchmarks in isolation, run a more accurate test (single task) to validate that this results in meaningful improvements. The next steps depend on whether this, more accurate, benchmark also produces favorable results:

If not, it is very important to understand why and fix the microbenchmark: it is likely not measuring what is intended!
If yes, then proceed with higher accuracy tests (full cluster/prod) and gather artifacts measuring the impact of the change.

One might wonder why writing the PL iterator iteration benchmark is useful if it does not show up in production profiles. One useful piece of information we can extract from this benchmark is the relative performance of iteration vs advance. With such knowledge at hand, one can look into opportunities to use iteration more prominently in the intersection kernel code. For example, it may be more efficient to replace the advance operation with multiple iteration actions. Pragmatically, even though iteration contributes few cycles, we don’t want its performance to regress to the point where it does take significant time.

Summary

Benchmarks are a useful way to prototype the impact of code changes on production systems. However, writing benchmarks that accurately reflect production performance is hard. There are multiple pitfalls where the size of the benchmark, or other of its characteristics, may lead to incorrect estimates of production impact. One approach to solving this is to have multiple benchmarks, with increasing fidelity and complexity. Understanding why a particular benchmark does not produce representative results is a critical step in improving benchmark fidelity, and can even produce insights into production behavior.

Performance Tip of the Week #21: Improving the efficiency of your regular expressions

2023-03-02T00:00:00-05:00

Originally posted as Fast TotW #21 on January 16, 2020

By Paul Wankadia and Darryl Gove

Updated 2025-09-03

Quicklink: abseil.io/fast/21

Regular expressions are used, misused and abused nearly everywhere. Google is no exception, alas, and at our scale, even a simple change can save hundreds or thousands of cores. In this tip, we describe ways to use RE2 more efficiently.

NOTE: This tip is specifically about RE2 and C++. A number of the ideas below are universally applicable, but discussion of other libraries and other languages is out of scope.

Using regular expressions: a representative sample

As a prelude, let’s consider an example of how regular expressions are often used. This snippet looks for a zone ID at the end of the zone_name string and extracts its value into the zone_id integer:

int zone_id;
if (RE2::FullMatch(zone_name, R"(.*\.zone(\d+))", &zone_id)) {

This tip describes several techniques for improving efficiency in situations such as this. These fall into two broad categories: improving the code that uses regular expressions; and improving the regular expressions themselves.

Writing more efficient code

A few words about `RE2` objects

In order to understand why the following techniques matter, we need to talk briefly about RE2 objects. In the initial example, we passed a pattern string to RE2::FullMatch(). Passing a pattern string instead of an RE2 object implicitly constructs a temporary RE2 object. During construction, RE2 parses the pattern string to a syntax tree and compiles the syntax tree to an automaton. Depending on the complexity of the regular expression, construction can require a lot of CPU time and can build an automaton that will have a large memory footprint.

Use Abseil functions for literal strings

In many situations, regular expressions are unnecessary because simple string operations will suffice. For exact matching, absl::string_view defines operator==(). For substring, prefix and suffix matching, Abseil provides absl::StrContains(), absl::StartsWith() and absl::EndsWith(), respectively, in absl/strings/match.h. These are much faster than regular expressions and more readable, so using them where possible is recommended.

For example:

const RE2 re(absl::StrCat(row_key, ":.*"));
for (const auto& row : rows) {
  if (RE2::FullMatch(row, re)) {

could be rewritten as:

const std::string prefix = absl::StrCat(row_key, ":");
for (const auto& row : rows) {
  if (absl::StartsWith(row, prefix)) {

Minimise `RE2` object churn

As discussed above, constructing RE2 objects can be expensive, so as a rule of thumb, they should be long-lived. Precompiling or caching them where possible is recommended.

Use `LazyRE2` for static or global regular expressions

The RE2 class is unsafe for direct use with static or global regular expressions. Use LazyRE2 instead because it lazily constructs the underlying RE2 object and never destructs it.

The initial example could be rewritten as:

static constexpr LazyRE2 kZoneRe = {R"(.*\.zone(\d+))"};
int zone_id;
if (RE2::FullMatch(zone_name, *kZoneRe, &zone_id)) {

Writing more efficient regular expressions

Use `RE2::PartialMatch()` to avoid leading or trailing `.*`

Using RE2::FullMatch() with leading or trailing .* is an antipattern. Instead, change it to RE2::PartialMatch() and remove the .*. RE2::PartialMatch() performs an unanchored search, so it is also necessary to anchor the regular expression (i.e. with ^ or $) to indicate that it must match at the start or end of the string. Let’s look at some examples.

Replacing leading `.*`

For a regular expression with a leading .*, the leading .* should be removed and the regular expression should be anchored at the end with $. For example, .*-(?:bar|qux)-foo should become -(?:bar|qux)-foo$.

The leading .* prevents RE2 from terminating reverse execution (i.e. backwards from the end of the input string) after matching the last byte of interest. When the remainder of the input string is relatively large, RE2 has to do a lot more work for no benefit. More about that shortly…

The initial example could be rewritten further as:

static constexpr LazyRE2 kZoneRe = {R"(\.zone(\d+)$)"};
int zone_id;
if (RE2::PartialMatch(zone_name, *kZoneRe, &zone_id)) {

Replacing trailing `.*`

For a regular expression with a trailing .*, the trailing .* should be removed and the regular expression should be anchored at the start with ^. For example, foo-(?:bar|qux)-.* should become ^foo-(?:bar|qux)-. RE2 will detach the ^foo- prefix and match it with memcmp(3). (Note that this optimisation applies when the regular expression has ^ plus some literal string as its prefix–even when using RE2::FullMatch()!)

The trailing .* prevents RE2 from terminating execution after matching the last byte of interest. When the remainder of the input string is relatively large, RE2 has to do a lot more work for no benefit. More about that very shortly…

Replacing leading and trailing `.*`

For a regular expression with both a leading and trailing .*, both the leading and trailing .* should be removed. For example, .*-(?:bar|qux)-.* should become -(?:bar|qux)-.

The leading .* prevents RE2 from using memchr(3) to find the first byte of interest. (Note that this optimisation applies when the regular expression has one distinct first byte such as the f in foo\d+, but not when there are two or more possible first bytes such as the \d ≡ [0-9] in \d+bar.)

What `.*` really means

The problem with .* is that it doesn’t mean “match anything” by default. In fact, . ≡ [^\n] by default, so it matches any character that isn’t newline. RE2 defaults to UTF-8 encoding, so it builds an automaton that handles multibyte characters. Consequently, the default meaning of .* is “match zero or more multibyte characters that aren’t newline”. An automaton steps over the input string one byte at a time, so executing .* is much slower than using memchr(3) (which is typically implemented using SIMD) and it is infinitely slower than terminating execution as soon as the regular expression has been matched.

Minimise capturing groups and submatches

In situations where parentheses are necessary for grouping, use a non-capturing group (i.e. (?: … )) unless a submatch is being extracted. Moreover, extract as few submatches as possible because execution engine selection depends in part on how many submatches the caller wants.

Passing nullptr for a no-op submatch is an antipattern. Instead, change the capturing group to a non-capturing group.

Use `absl::string_view` for submatches

When extracting a submatch, use absl::string_view. Extracting to std::string necessarily involves a string copy.

The same advice applies even when extracting a submatch with numeric conversion. Extracting to a numeric type also involves a string copy because strtol(3) et al. require NUL-terminated strings. Extracting to absl::string_view and calling absl::SimpleAtoi() et al. avoids the string copy.

The initial example could be rewritten even further as:

static constexpr LazyRE2 kZoneRe = {R"(\.zone(\d+)$)"};
absl::string_view zone_id_str;
int zone_id;
if (RE2::PartialMatch(zone_name, *kZoneRe, &zone_id_str) &&
    absl::SimpleAtoi(zone_id_str, &zone_id)) {

Minimise ambiguity

The “cost” of executing a regular expression depends greatly on ambiguity. See the comments in re2/onepass.cc for details, but the rules can be summarised as:

it must be immediately obvious which branch of an alternation to take; and
it must be immediately obvious when a repetition ends.

For example, (.+)/ and (.+?)/ are ambiguous because . and / aren’t disjoint–a '/' byte could be consumed by either. In contrast, ([^/]+)/ is unambiguous because [^/] and / are disjoint. It isn’t always possible to eliminate ambiguity, but it is often possible to reduce it.

Exception: common prefixes of alternations

RE2 will factor common prefixes of alternations. For example, abacus|abalone parses as aba(?:cus|lone). This optimisation permits the regular expression to be written in a more readable manner. Note that it applies to contiguous runs, so lexicographically sorting the subexpressions is recommended.

Avoid counted repetition

A subexpression of the form x{n}, x{n,} or x{n,m} results in x being duplicated at least n or m times in the automaton. It’s bad enough (particularly when using Unicode character classes) for anchored matching, but it’s dangerous for unanchored matching because it’s easy to blow up the number of DFA states and their sizes. Each RE2 object has a memory budget (8 MiB by default) mostly for caching DFA states; exhausting the memory budget incurs a considerable performance hit.

For example, constructing the entire DFA needs 22 KiB for the anchored ^[a-q][^u-z]{13}x versus 6 MiB for the unanchored [a-q][^u-z]{13}x. The former isn’t useful for searching due to being anchored, but the latter isn’t efficient due to the exponential blowup in DFA states from being unanchored. Matching [a-q][^u-z]+x instead (i.e. using uncounted repetition) may be possible, but checking the length of the match in a subsequent step might not be acceptable.

Bonus: Matching multiple regular expressions

To finish, let’s touch on RE2::Set and FilteredRE2. These are two very different approaches to matching multiple regular expressions, which is useful in situations where performing multiple passes over the input string would be prohibitively expensive. Such situations include keyword matching and logs processing.

RE2::Set combines the regular expressions into one “many match” automaton. It has strengths and weaknesses similar to those of normal DFA execution and is suited to shorter, less complex regular expressions.

FilteredRE2 identifies the literal string “atoms” within the regular expressions; given which of those “atoms” occur within the input string, it determines which regular expressions potentially match. It requires an initial pass over the input string with something like Aho-Corasick or RE2::Set and is suited to longer, more complex regular expressions.

You might also like to read about lightgrep, Hyperscan and RE-tree, which were designed specifically for matching multiple regular expressions.

Tip of the Week #198: Tag Types

2023-02-15T00:00:00-05:00

Originally posted as TotW #198 on August 12, 2021

By Alex Konradi

Updated 2022-01-24

Quicklink: abseil.io/tips/198

Suppose we have a class Foo:

class Foo {
 public:
  explicit Foo(int x, int y);
  Foo& operator=(const Foo&) = delete;
  Foo(const Foo&) = delete;
};

Foo is neither movable nor copyable, but it is constructible, e.g. Foo f(1, 2);. Since it has a public constructor, we can easily make an instance wrapped in a std::optional:

std::optional<Foo> maybe_foo;
maybe_foo.emplace(5, 10);

That’s great, but what if the std::optional is declared const, so we can’t call emplace()? Luckily for us, std::optional has a constructor for this:

// Pass std::in_place as the first argument, followed by the arguments for Foo's
// constructor.
const std::optional<Foo> maybe_foo(std::in_place, 5, 10);

Wait, what’s going on with std::in_place? If we look at the documentation for std::optional construction, we can see that one of the overloads takes a std::in_place_t as the first argument, along with a list of additional arguments. Since std::in_place is an instance of std::in_place_t, the compiler chooses the emplacing constructor, which instantiates Foo within the std::optional by forwarding the rest of the arguments to Foo’s constructor.

If we look a little closer at the documentation for std::in_place_t, we see that it’s… an empty struct. No special qualifiers or magic declarations. The only thing even mildly special about it is that the standard library includes a named instance of it, std::in_place.

Overload Resolution Via Tag Types

std::in_place_t is a member of a loose category of classes sometimes referred to as “tag types”. The function of these classes is to pass information to the compiler by “tagging” specific overloads in an overload set. By providing an instance of an appropriate tag class to an overloaded function (often a constructor), we can use the compiler’s ordinary resolution rules to make it select the desired overload. In the case of our std::optional construction, the compiler sees that the first argument is of type std::in_place_t and so chooses the matching emplacing constructor.

Though std::in_place_t was introduced in C++17, the use of empty classes for tagging overloads in the standard library has been prevalent since std::piecewise_construct_t, was introduced in C++11 to select the emplacing constructor for std::pair. C++17 significantly expanded the set of tag types in the standard library.

Templates, of Course

Besides disambiguating overloads, another common use case for tag types is to pass type information to templated constructors. Consider these two structs:

struct A { A(); /* internal members */ };
struct B { B(); /* internal members */ };

Let’s try to construct an instance of std::variant<A, B> with each:

// These work if A and B are copy- or move-constructible, but incur the
// performance cost of an extra copy or move construction.
std::variant<A, B> with_a{A()};
std::variant<A, B> with_b{B()};

// These aren't valid C++; the language doesn't support providing constructor
// template parameters explicitly.
std::variant<A, B> try_templating_a<A>{};
std::variant<A, B><B> try_templating_b{};

What we need is a way to tell the std::variant constructor that we want to instantiate A or B without actually providing an instance of either. For that, we can use std::in_place_type!

std::variant<A, B> with_a{std::in_place_type<A>};
std::variant<A, B> with_b{std::in_place_type<B>};

std::in_place_type<T> is an instance of the class template std::in_place_type_t<T>, which is (unsurprisingly, at this point) empty. By passing a value of type std::in_place_type_t<A> to std::variant’s constructor, the compiler can deduce that the constructor template parameter is our class A.

Usage

Tag types show up occasionally when interacting with generic class templates, especially ones from the standard library. One of the shortcomings of tag types is that other techniques, such as factory functions, often result in more readable code. Take the following example:

// This tag spelling requires the reader to know how std::optional interacts
// with std::in_place.
std::optional<Foo> with_tag(std::in_place, 5, 10);

// Here the intent is clearer: make an optional Foo by providing these arguments.
std::optional<Foo> with_factory = std::make_optional<Foo>(5, 10);

These two std::optional<Foo> objects are guaranteed to be constructed identically thanks to C++17’s mandatory copy elision.

So why use tags? Because factory functions don’t always work:

// This doesn't work because Foo isn't move-constructible.
std::optional<std::optional<Foo>> foo(std::make_optional<Foo>(5, 10));

The above example doesn’t compile because Foo’s move constructor is deleted. To make this work, we use std::in_place to select the std::optional constructor that forwards the remaining arguments.

// This constructs everything in place, resulting in a single call to Foo's
// constructor.
std::optional<std::optional<Foo>> foo(std::in_place, std::in_place, 5, 10);

In addition to working in places where factory functions don’t, tag types have some other nice properties:

they are literal types which means we can declare constexpr instances, even in header files, like std::in_place;
because they are empty, the compiler can optimize them out, resulting in zero runtime overhead.

Though they’re used in the standard library, encountering empty tag types in the wild is relatively uncommon. If you find yourself using one, consider adding a comment to help your readers:

std::unordered_map<int, Foo> int_to_foo;
// std::piecewise_construct is a tag for overload resolution of std::pair's
// constructor. Emplaces 100 -> Foo(5, 10) in the map.
int_to_foo.emplace(std::piecewise_construct,
                   std::forward_as_tuple(100),
                   std::forward_as_tuple(5, 10));

Conclusion

Tag types are a powerful way to give additional information to the compiler. While at first glance they might seem magical, they use the same overload resolution and template type deduction rules as the rest of C++. The standard library uses class tags for disambiguating constructor calls, and you can use the same mechanisms to define tags for your own needs.

Tip of the Week #218: Designing Extension Points With FTADLE

2023-01-19T00:00:00-05:00

Originally posted as TotW #218 on January 19, 2023

By Andy Soffer

Updated 2023-01-19

Quicklink: abseil.io/tips/218

Ftadle. It’s a perfectly cromulent word. ~ Unknown

Designing Extension Points

Suppose you work on a library called sketchy, that draws on a canvas. You already provide ways to draw some common things like points, lines, and text, but you want to provide a mechanism where users can specify how to draw their own types. You’re designing an extension point.

Design Goals For Extension Points

C++ provides many mechanisms for defining extension points, each with their own benefits and drawbacks. When defining an extension point in C++ there are several considerations worth weighing:

Readability – How easy is it for an engineer to understand the relationship between your library and the extension?
Maintainability – How easy will it be to change the extension point as the needs of your library and your library’s users change?
Dependency Hygiene – Does your extension point require your library to be linked in to a user’s binary? We want to make sure extension points play nicely with IWYU, so if a header needs to be included for the extension mechanism to work, the extended types should actually use something from that header.
Lack of ODR violations – Some mechanisms make it easy to have different portions of your program have contradictory views about what a program means. ODR violations are always a bug.

FTADLE: A Good Pattern With A Great Name

When defining an extension point, we recommend following a pattern that we lovingly call FTADLE¹ (Friend Template ADL Extension). FTADLE does well on each of the considerations listed above. It relies heavily on a language feature known as ADL, or Argument Dependent Lookup; the process by which the compiler determines what function is intended when a function call appears without namespace qualification (i.e., no ::s). ADL is explained in detail in Tip #49, To write an extension using the FTADLE pattern:

Pick a name for your extension point and prefix it with your project’s namespace. Our extension is for drawing, and our project lives in the sketchy namespace, so we’ll call our extension SketchyDraw.
Design a type to be passed in to SketchyDraw that has all the behavior your users will need. In our case, this is the sketchy::Canvas on which users can draw their types.

Implement your functionality as an overload set. One member of that overload set will be a template and will call your extension point. The non-template functions in the overload set should be the basic building blocks; the primitive types that your API supports. In our running example, that means functions that accept the types sketchy::Point and sketchy::Line.

namespace sketchy {
// Draws the point `p` on the canvas `c`.
void Draw(Canvas& c, const Point& p);

// Draws the line segment `l` on the canvas `c`.
void Draw(Canvas& c, const Line& l);

// For any user-defined type `T` which implements `SketchyDraw` (see
// documentation that I've definitely written), draws `value` on the canvas
// `c`.
template <typename T>
void Draw(Canvas& c, const T& value) {
  // Called without namespace qualifiers. We rely on ADL to find the correct
  // overload. See [Tip #49](/tips/49) for details on ADL.
  SketchyDraw(c, value);
}

}  // namespace sketchy

With this extension-point designed, users will now be able to make their types drawable. How can an unrelated type add this Draw functionality without adding an explicit dependency? We can make use of a friend function to do so. By adding a friend function template in their type named SketchyDraw with the appropriate signature. the template overload above will use ADL to find the SketchyDraw function. For example,

class Triangle {
 public:
  explicit Triangle(Point a, Point b, Point c) : a_(a), b_(b), c_(c) {}

  template <typename SC>
  friend void SketchyDraw(SC& canvas, const Triangle& triangle) {
    // Note: This is a template, even though the only type we ever expect to be
    // passed in for `SC` is `sketchy::Canvas`. Using `sketchy::Canvas` directly
    // works, but pulls in an extra dependency that may not be used by all users
    // of `Triangle`.
    sketchy::Draw(canvas, sketchy::Line(triangle.a_, triangle.b_));
    sketchy::Draw(canvas, sketchy::Line(triangle.b_, triangle.c_));
    sketchy::Draw(canvas, sketchy::Line(triangle.c_, triangle.a_));
  }

 private:
  Point a_, b_, c_;
};

// Usage:
void DrawTriangles(sketchy::Canvas& canvas, absl::Span<const Triangle> triangles) {
  for (const Triangle& triangle : triangles) {
    sketchy::Draw(canvas, triangle);
  }
}

NOTE: Users of the library never call the ADL extension point SketchyDraw directly. Rather, the library should provide a function like sketchy::Draw which invokes the extension point on the user’s behalf.

Other Examples

The FTADLE pattern has been used with several other common libraries.

The AbslHashValue extension point allows you to make your type hashable by any of Abseil’s hash containers. See Tip #152 for details.
The AbslStringify extension point allows you to print your type with many many Abseil libraries, including logging, absl::StrCat, absl::StrFormat, and absl::Substitute.

What To Avoid

Some common extension point mechanisms fall short of our design goals. Virtual functions, checking at compile-time for member functions, and template specialization are each brittle in their own way, as discussed below.

Virtual Functions

Though perhaps the most familiar, virtual functions and class hierarchies are often overly rigid. They are nearly impossible to refactor, because the base class and all derived classes need to be updated in lock-step. We rarely get designs right on the first try, so it’s prudent to have a design that we can change later.

Beyond the rigidity, class hierarchies force a dependency on your users. In the case of sketchy, users are required to depend on sketchy code, even when only some of their binaries want to use the dependency. FTADLE ensures that only binaries that need to do something sketchy pay that cost.

The same is true for non-template friend extension points as well (for example std::ostream’s operator<<). Each class that wishes to implement operator<< must include one of the standard library headers defining std::ostream (e.g., <ostream>, <iostream>). This means that (barring optimizations) the code for std::ostream will be compiled and linked into the binary whether or not operator<< is used, a potential extra cost to compile-times and binary size.

Member Functions

With some template trickery, you could check to see if a class has a particular method by name (or even signature). However, names can be misleading.

// Requires that the image have a `draw()` member function.
template <typename Image>
void DisplayImage(const Image& image) {
  image.draw();
}

class Cowboy {
 public:
  // Draws the gun from the holster.
  void draw();
};

int main() {
  Cowboy c;
  DisplayImage(c);  // Oops, not the "draw" we meant.
}

With the FTADLE pattern, the extension point is prefixed with the project’s namespace, mitigating accidental conformance.

Template Specializations

Another common but dangerous technique is to use template specializations. This is how std::hash and std::less are specialized.

namespace std {

template<>
struct hash<MyType> {
  size_t operator()(const MyType& m) const {
    return HashCombine(std::hash<>()(m.foo()), std::hash<>()(m.bar()));
  }
};

}  // namespace std

Aside from requiring more boilerplate, this technique is ripe for ODR violations. While not terribly common, providing different specializations for this type in different translation units, or even the same definition twice is an ODR violation. More commonly, if such a specialization is available only in some translation units but not others, metaprogramming techniques will produce different answers to the question “is there a hash function available?” which is also an ODR-violation.

Beyond that, it is generally bad practice to open up a namespace you do not own (amongst other reasons, because it leads to ODR violations). We should design our APIs to avoid bad practices so as not to accidentally encourage dangerous practices.

Conclusion

The FTADLE extension point pattern is readable, maintainable, mitigates against ODR violations, and avoids adding dependencies. If your library needs an extension point, FTADLE comes highly recommended.

C++ has a rich tradition of almost-pronounceable acronyms, including RAII, IFNDR, CRTP, and SFINAE. We have been pronouncing FTADLE as “fftah-dill” (similar to “paddle” but with the ‘p’ replaced by the sound at the end of “raft”), but we encourage you to pronounce it in whichever way brings you the most joy. ↩

Tip of the Week #3: String Concatenation and `operator+` vs. `StrCat()`

2022-11-16T00:00:00-05:00

Originally posted as TotW #3 on May 11, 2012

Updated 2022-11-16

Quicklink: abseil.io/tips/3

Users are often surprised when a reviewer says, “Don’t use the string concatenation operator, it’s not that efficient.” How can it be that string::operator+ is inefficient? Isn’t it hard to get that wrong?

It turns out, such inefficiency isn’t clear cut. These two snippets have close to the same execution time, in practice:

std::string foo = LongString1();
std::string bar = LongString2();
std::string foobar = foo + bar;

std::string foo = LongString1();
std::string bar = LongString2();
std::string foobar = absl::StrCat(foo, bar);

However, the same is not true for these two snippets:

std::string foo = LongString1();
std::string bar = LongString2();
std::string baz = LongString3();
std::string foobarbaz = foo + bar + baz;

std::string foo = LongString1();
std::string bar = LongString2();
std::string baz = LongString3();
std::string foobarbaz = absl::StrCat(foo, bar, baz);

The reason these two cases differ can be understood when we pick apart what is happening in the expression foo + bar + baz. Since there are no overloads for three-argument operators in C++, this operation is necessarily going to make two calls to string::operator+. And between those two calls, the operation will construct (and store) a temporary string. So `std::string foobarbaz = foo + bar

baz` is really equivalent to:

std::string temp = foo + bar;
std::string foobarbaz = std::move(temp) + baz;

Specifically, note that the contents of foo and bar must be copied to a temporary location before they are placed within foobarbaz. (For more on std::move, see Tip #77.)

C++11 at least allows the second concatenation to happen without creating a new string object: std::move(temp) + baz is equivalent to std::move(temp.append(baz)). However, it’s possible that the buffer initially allocated for the temporary won’t be large enough to hold the final string, in which case a reallocation (and another copy) will be required. As a result, in the worst case, chains of n string concatenations require O(n) reallocations.

It is better instead to use absl::StrCat(), a nice helper function from google3/third_party/absl/strings/str_cat.h that calculates the necessary string length, reserves that size, and concatenates all of the input data into the output - a well-optimized O(n). Similarly, for cases like:

foobar += foo + bar + baz;

use absl::StrAppend(), which performs similar optimizations:

absl::StrAppend(&foobar, foo, bar, baz);

In addition, absl::StrCat() and absl::StrAppend() operate on types other than just string types: you can use absl::StrCat/absl::StrAppend to convert int32_t, uint32_t, int64_t, uint64_t, float, double, const char*, and string_view, like this:

std::string foo = absl::StrCat("The year is ", year);

For additional information, see absl::StrCat() and absl::StrAppend() for String Concatenation.

Tip of the Week #215: Stringifying Custom Types with `AbslStringify()`

2022-11-16T00:00:00-05:00

Originally posted as TotW #215 on November 2, 2022

By Phoebe Liang

Updated 2022-11-16

Quicklink: abseil.io/tips/215

Abseil now contains a new lightweight mechanism, AbslStringify(), that allows users to format user-defined types as strings. User-defined types that are extended using AbslStringify() work out of the box with absl::StrFormat, absl::StrCat and absl::Substitute.

As with most type extensions, you should own the type you wish to extend.

Let’s say that we have a simple Point struct:

struct Point {
  int x;
  int y;
};

If we want a Point to be formattable with absl::StrFormat(), absl::StrCat() and absl::Substitute(), we add a friend function template named AbslStringify():

struct Point {
  template <typename Sink>
  friend void AbslStringify(Sink& sink, const Point& p) {
    absl::Format(&sink, "(%d, %d)", p.x, p.y);
  }

  int x;
  int y;
};

Note: AbslStringify() utilizes a generic “sink” buffer to construct its string. This sink has an interface similar to absl::FormatSink, but does not support PutPaddedString().

Now absl::StrCat("The point is ", p) and absl::Substitute("The point is $0", p) will just work.

Note: absl::StrFormat() also provides a more customizable extension point AbslFormatConvert() that is not supported by absl::StrCat().

Type Deduction with the `%v` Specifier

But what if we want to format our type using absl::StrFormat()? absl::StrFormat()’s existing type specifiers don’t support user-defined types extended with AbslStringify(), so we would have to do something like this:

absl::StrFormat("The point is (%d, %d)", p.x, p.y)

This is obviously not ideal. It doesn’t make use of the extension at all and it duplicates the format string used in the AbslStringify() definition. Instead, we can use the new type specifier %v:

absl::StrFormat("The point is %v", p)

%v uses type deduction to format an argument. %v supports the formatting of most primitive types, as well as any types extended using AbslStringify(). You can think of %v as a generic way to format a “value” of any type that absl::StrFormat() can deduce. %v can also be used directly within AbslStringify() definitions.

The %v specifier deduces the following types:

d for signed integral values
u for unsigned integral values
g for floating point values
- double
- float
- long double
s for string values
- std::string
- absl::string_view
- std::string_view
- absl::Cord

NOTE: const char* is not supported. See below for more information.

Some examples:

absl::StrFormat("%v", std::string{"hello"})    -> "hello"
absl::StrFormat("%v", 42)    -> "42"
absl::StrFormat("%v", uint64_t{16})    -> "16"
absl::StrFormat("%v", 1.6)  -> "1.6"
absl::StrFormat("%v", true) -> "true"

Types With Special Handling

%v intentionally does not support char and const char* due to ambiguity in the desired output format. Boolean values are printed as "true" and "false" rather than "1" and "0", which is how absl::StrFormat() and absl::StrCat() print booleans otherwise.

Integration With Other Libraries

AbslStringify() has additional support throughout other Abseil libraries.

Logging

Types that define AbslStringify() are directly loggable:

struct Point {
  template <typename Sink>
  friend void AbslStringify(Sink& sink, const Point& p) {
    absl::Format(&sink, "(%v, %v)", p.x, p.y);
  }

  int x = 10;
  int y = 20;
};

Point p;

LOG(INFO) << p;

This code will produce a message in the logs like:

I0926 09:00:00.000000   12345 main.cc:10] (10, 20)

It is recommended that custom types be made loggable by implementing AbslStringify() rather than operator<< as it is a universal stringification extension that also enables absl::StrFormat, absl::StrCat and absl::Substitute support.

Protocol Buffer Types

Protocol buffers are formattable using AbslStringify(). Since AbslStringify() provides an overall smoother user experience over DebugString(), it is recommended that users use AbslStringify() when formatting protos as strings.

message MyProto {
  optional string my_string = 1
}

MyProto my_proto;
my_proto.set_my_string("hello world");

absl::StrCat("My proto is: ", my_proto);
absl::StrFormat("My proto is: %v", my_proto);
LOG(INFO) << my_proto;

Closing Words

Aside from user-defined types where its use is required, the %v type specifier is intended for use in cases where the precise formatting is not important. It is essentially a catch-all “just print it in a human readable manner” specifier. If you require any other guarantees beyond that, please use one of the more specific type specifiers.

Tip of the Week #18: String Formatting with Substitute

2022-11-16T00:00:00-05:00

Originally posted as TotW #18 on October 4, 2012

By Titus Winters

Updated 2022-11-16

Quicklink: abseil.io/tips/18

It happens all the time: you’re writing code and suddenly you need to assemble a new string from a template and some run-time values. Maybe it’s an error message from a failed Stubby call, maybe it’s the body of an email being sent from an internal process. Probably the most common mechanism for string formatting outside of google3 is sprintf/snprintf. But as we wander through the codebase, we see many things that people have done, and many places where C++ engineers are spending too much of their time, too many cycles, and too many lines of code to accomplish this task. In this week’s tip, we will walk through some common options and point out their various drawbacks.

Option 1: String Concatenation (built-in)

Some people still just reach for basic string concatenation:

std::string GetErrorMessage(const std::string& op, const std::string& user,
                            int id) {
  return "Error in " + op + " for user " + user + "(" + std::to_string(id) +
         ")";
}

As we saw back in Tip #3, there are problems with this approach: chains of calls to operator+() wind up making (and wasting) temporaries, and cause needless copies of your data.

Option 2: absl::StrCat() (google3/third_party/absl/strings/str_cat.h)

Just like in Tip #3, absl::StrCat() avoids those copies, handles the numeric conversion for us (go/willitstrcat), and even allows us to operate on string_view efficiently (which is better in cases when we are called with a C-style string):

std::string GetErrorMessage(absl::string_view op, absl::string_view user,
                            int id) {
  return absl::StrCat("Error in ", op, " for user ", user, "(", id, ")");
}

However, it’s still a little difficult to see at a glance what the string actually will look like: Where are the spaces? Do the parentheses in the string match up properly?

Option 3: absl::Substitute (google3/third_party/absl/strings/substitute.h)

However, an even better option exists: absl::Substitute(). Substitute() uses the same techniques as StrCat() to allow you to pass numeric values, std::strings, char*s, and string_views. It doesn’t require you to remember arcane format strings for numeric types (or string_view).

Even though everyone can more-or-less understand printf-style format strings, Substitute format strings are just as hard to misinterpret:

std::string GetErrorMessage(absl::string_view op, absl::string_view user,
                            int id) {
  return absl::Substitute("Error in $0 for user $1 ($2)", op, user, id);
}

Good readability, and better performance? LGTM.

Tip of the Week #124: `absl::StrFormat()`

2022-11-16T00:00:00-05:00

Originally posted as TotW #124 on October 11, 2016

Updated 2022-11-16

Quicklink: abseil.io/tips/124

The `str_format` Library and `absl::StrFormat()`

After a long testing and development period, we’re pleased to announce that the str_format library is now generally available. The str_format library is a very efficient, typesafe, and extensible library that implements all printf formatting syntax. Nearly all printf-style conversions can be trivially upgraded to absl::StrFormat(). For more detailed documentation, see https://abseil.io/docs/cpp/guides/format. It’s the best option for printf-style formatting, but no position is taken here on where printf-style is or isn’t appropriate.

Usage is simple. Add a BUILD dependency on //third_party/absl/strings:str_format, and include the header:

#include "absl/strings/str_format.h"

Most users will interact with the str_format library simply by calling absl::StrFormat() just as they’d call StringPrintf() or util::format::StringF() in the past. There are also StrAppendFormat() and StreamFormat() variants.

std::string s = absl::StrFormat("%d %s\n", 123, "hello");

Unlike the C library’s printf(), the correctness of absl::StrFormat() conversions doesn’t rely on callers encoding the exact types of arguments into the format string. With printf() this must be carefully done with length modifiers and conversion specifiers, such as %llu encoding the type unsigned long long. But absl::StrFormat() is written in C++, so it uses templates and overloading to safely work directly with types in the caller’s argument list. In absl::StrFormat(), a format conversion specifies a broader C++ conceptual category instead of an exact type. For example, %s binds to any string-like argument, so std::string, absl::string_view, Cord, and const char* are all accepted. Likewise, %d accepts an integer-like argument, etc. It can be further extended for basic user-defined types (though we’d like to manage the extensions for now). It ignores length modifiers like ll and formats any usable value. For example, clients do not have to unnecessarily hardcode the types of the data members of x here:

  X x = project_x::GetStats();
  LOG(INFO) << absl::StreamFormat("%s:%08x", x.name, x.size);

The name can be anything string-like, and size can be any integer-like type. This decoupling is great for the maintainers of project_x.

We can also control the destination much more smoothly with the str_format library. In the printf() family, there was fprintf() for FILE* output, sprintf() for writing to a buffer, asprintf() for writing to allocated memory, or the (now deprecated) usage of StringPrintf() (with wasteful multiple calls to vsnprintf()). The str_format library uses an abstract sink, so the destination can be customized without loss of efficiency. As built-ins, we have absl::StrFormat() to produce a new std::string, absl::StrAppendFormat() to append to a std::string, and absl::StreamFormat() to write to a std::ostream (such as for logging).

Under the clang compiler, compile-time checking is performed on literal format strings. In the uncommon case of format strings determined at runtime, the format string must be parsed and checked against an argument list specification for compatibility before it can be used. This eliminates a danger of traditional printf() when using runtime formats. This ability to produce parsed format specifiers (similar to how regular expressions can be compiled into RE objects) can yield a performance boost, so in performance-sensitive code it may be used even with statically-determined format strings.

There are a few notable differences from printf() (see https://abseil.io/docs/cpp/guides/format). We try in the str_format library to be flexible without information loss. If a signed argument is formatted with an unsigned conversion like %u or %x, we convert the argument to the corresponding unsigned integer type before formatting, so printing negatives with %u may work differently from one’s expectations on this previously undefined behavior.

Best of all, it is highly optimized and much faster than sprintf() or the deprecated StringPrintf() (see format-shootout). Please give this library a try anywhere you’d use printf-style formatting.

Examples

I’ll close with a few examples.

#include "absl/strings/str_format.h"

absl::StrAppendFormat(&s, "Also, %s\n", epilogue);

// Logs something like: "billydonahue         12345.67"
// When formatting streamed output, prefer `absl::StreamFormat()` instead of
// stream I/O manipulators like `std::setw`.
// See https://google.github.io/styleguide/cppguide.html#Streams for more information.
for (const auto& g : hard_workers)
  LOG(INFO) << absl::StreamFormat("%-20s %8.2f", g.username, g.bonus);

// POSIX positional specifiers (yields "veni, vidi, vici!").
summary = absl::StrFormat("%2$s, %3$s, %1$s!", "vici", "veni", "vidi");

std::string letter = response.format_string();  // known only at runtime
// Reject unacceptable form letters.
auto format = absl::ParsedFormat<'d', 's', 's'>::New(letter);
if (!format) {
  // ... error case ...
  return;
}
letter = StringF(*format, vacation_days, from, to);

// Precompile for performance. Yields, e.g., rows like:
//   "<tr><td>alice</td><td>00000123</td></tr>\n"
//   "<tr><td>bob</td><td>00004567</td></tr>\n"
static const auto* const pfmt = new absl::ParsedFormat<'s','d'>(
    "<tr><td>%s</td><td>%08d</td></tr>\n");
for (const auto& joe : folks) {
  absl::StrAppendFormat(&output, *pfmt, joe.name, joe.id);
}
// The 'FormatStreamed' adaptor can be used to format any ostream-formattable
// 'x'.
s = absl::StrFormat("[%-12s]", absl::FormatStreamed(x));

Abseil Stringify

2022-11-15T00:00:00-05:00

By Phoebe Liang, Abseil Engineer

We are pleased to introduce an easier way to format user-defined types as strings in Abseil: AbslStringify(). This API allows user-defined types to be printed more easily using absl::StrFormat() and absl::StrCat().

To “stringify” a custom type, define a friend function template named AbslStringify():

struct Point 
  template <typename Sink>
  friend void AbslStringify(Sink& sink, const Point& p) {
    absl::Format(&sink, "(%d, %d)", p.x, p.y);
  }

  int x;
  int y;
}

absl::StrCat() will work right out of the box like so:

absl::StrCat("The point is ", p);

To print custom types with absl::StrFormat(), use the new type specifier %v:

absl::StrFormat("The point is %v", p);

%v uses type deduction to print an argument and supports any user-defined type that provides an AbslStringify() definition. Most types that are supported natively by absl::StrFormat() are also supported by %v. For a full list of supported types, see the StrFormat Guide.

Abseil Logging

2022-09-08T00:00:00-04:00

By Andy Getzendanner, Abseil Engineer

We are pleased to announce, at long last, the initial availability of the Abseil Logging library. This library provides facilities for writing short text messages about the status of a program to stderr, disk files, or other sinks (via an extension API).

The core interface is the LOG macro, which has a streaming interface like std::ostream’s:

absl::StatusOr<absl::Duration> GetUserAge(absl::string_view user_name) {
  int user_id = GetUserId(user_name);
  if (user_id == -1) {
    LOG(ERROR) << "No user found named " << user_name;
    return absl::NotFoundError(absl::StrCat("No user found named ", user_name));
  }
  return age_map[user_id];
}

The library also supports terminating the process via a severity level named FATAL:

absl::Duration GetUserAge(absl::string_view user_name) {
  int user_id = GetUser(user_name);
  if (user_id == -1) {
    LOG(FATAL) << "No user named " << user_name << " found";
    // No need for a return statement; this line is unreachable.
  }
  return age_map[user_id];
}

The CHECK macro family provides assert()-like precondition checking and process termination in all compilation modes with better error messages:

absl::Duration GetUserAge(absl::string_view user_name) {
  int user_id = GetUser(user_name);
  CHECK(user_id != -1) << "No user named " << user_name << " found";
  return age_map[user_id];
}

If you are familiar with the Google Logging (glog) library, these examples will look very familiar; that project and this one share a common ancestor. The core macro APIs are essentially unchanged, however other interfaces like the extension points have changed quite a bit. While those changes aim to improve interfaces, performance, extensibility, etc., the main reason to choose Abseil Logging over glog is for its compatibility and integration with Abseil and with Google’s internal code now and in the future.

For more information, consult the Abseil Logging library documentation.

The Danger of Atomic Operations

2022-01-18T00:00:00-05:00

By Ashley Hedberg, Software Engineer

The C++ Standard provides a library for performing fine-grained atomic operations (e.g. std::atomic). Engineers sometimes reach for these atomic operations in the hope to introduce some lock-free mechanism, or to introduce clever tricks to improve performance. At Google, we’ve found – more often than not – that such attempts lead to code that is difficult to get right, hard to understand, and can often introduce subtle and sometimes dangerous bugs.

We’ve now published a guide on the danger of these atomic operations, and are publishing it to Abseil as a general programming guide. Atomic operations should be used only in a handful of low-level data structures which are written by a few experts and then reviewed and tested thoroughly. Most programmers make mistakes when they attempt direct use of atomic operations. Even when they don’t, the resulting code is hard for others to maintain.

For more information, check out The Danger of Atomic Operations.

SWE Book Freely Available

2021-04-22T00:00:00-04:00

Software Engineering at Google Book Availability

By Titus Winters, Tom Manshreck, and Hyrum Wright

We’re very pleased to announce that the “Software Engineering at Google” book (the Flamingo Book) is now freely available electronically under a Creative Commons license. You can get a PDF at SWE Book.

From the very beginning of this project, about three years ago, our intent has been to describe how Google thinks about Software Engineering, and to get people thinking about the same kinds of big interesting problems. We think that the best way to do that is to make sure that the content is available for everyone, so we’re providing it now, free of charge. We’re very grateful for our partners at O’Reilly for helping to make this possible (and of course we encourage you to support them and buy a physical copy if you can, etc).

Some of you may be aiming to land a job at Google in the future: we’ve heard from dozens of Nooglers that this is the handbook they needed to understand the unique and complex machine that is Google. We hope this book is useful for you.

Some of you may be running your own software companies. You’ll have different scales and problems, but hopefully you can learn from how we think about problems arising from Time and Scale, and how we evaluate the relevant Tradeoffs (these are the main themes threaded through every chapter of the book). You may want to lean on our thinking, or blaze your own trails. We hope the insights we’ve earned the hard way can make your path easier.

Some of you may know more than us. There are several topics in the book where we are still trying to find a good answer. We’re making our thinking public - why not show us where we’re wrong?

Over the past year we’ve heard from hundreds of you, all over the world, with stories about how this material has affected your practice and thinking. We’re also seeing this picked up by colleges and universities looking to modernize their discussion of software engineering topics. We’re thrilled at this reception, and very excited that we can take this next step in sharing this material.

As Nicole Forsgren told us once, “Accountants still have meetings to discuss their practices, and accountancy goes back thousands of years. Software Engineering is barely 50 years old. Give us a minute.” As an industry, and as a discipline, we’re still figuring things out. We hope that this book, in some small way, can help with that.

-Titus, Tom, and Hyrum

Tip of the Week #188: Be Careful With Smart-Pointer Function Parameters

2020-12-10T00:00:00-05:00

Originally posted as TotW #188 on December 10, 2020

By Krzysztof Kosiński

Updated 2020-12-10

Quicklink: abseil.io/tips/188

What is wrong with this code?

bool CanYouPetTheDog(const std::shared_ptr<Dog>& dog,
                     absl::Duration min_delay) {
  return dog->GetLastPetTime() + min_delay < absl::Now();
}

The function CanYouPetTheDog does not affect the ownership of its dog argument, yet its signature demands that it should be stored in a std::shared_ptr. This creates an unnecessary dependency on a specific ownership model, even though nothing in the function requires it. This dependency prevents callers from using other models, such as std::unique_ptr or constructing objects on the stack.

Use References or Pointers When Ownership is Unaffected

By using a reference, we can remove the dependency on a specific ownership model, and allow our function to work with any object of type Dog.

bool CanYouPetTheDog(const Dog& dog, absl::Duration min_delay) {
  return dog.GetLastPetTime() + min_delay < absl::Now();
}

With the above definition, the function can be called regardless of the caller’s ownership model:

Dog stack_dog;
if (CanYouPetTheDog(stack_dog, delay)) { ... }

auto heap_dog = std::make_unique<Dog>();
if (CanYouPetTheDog(*heap_dog, delay)) { ... }

CustomPetPtr<Dog> custom_dog = CreateDog();
if (CanYouPetTheDog(*custom_dog, delay)) { ... }

If the function modifies the passed value, pass a mutable reference or a raw pointer, and use the same idioms as shown above.

Use Smart Pointers When the Function Modifies Ownership

The following code provides several overloads for different smart pointer parameters. The first overload assumes ownership of the passed object and the second one adds a shared reference to the passed object. Both of these operations depend on how the caller handles ownership of the Dog. Adopting a Dog that lives on the stack isn’t possible, as ownership can’t be taken away from the stack.

class Human {
 public:
  ...
  // Transfers ownership of `dog` to this Human.
  // See Tip #117 for the rationale for accepting std::unique_ptr by value.
  void Adopt(std::unique_ptr<Dog> dog) {
    pets_.push_back(std::move(dog));
  }
  // Adds a shared reference to `cat`.
  void Adopt(std::shared_ptr<Cat> cat) {
    pets_.push_back(std::move(cat));
  }

 private:
  std::vector<std::shared_ptr<Pet>> pets_;
  ...
};

Conclusion

If ownership is not being transferred or modified, avoid having smart pointers as function parameters.

Tip of the Week #187: `std::unique_ptr` Must Be Moved

2020-11-11T00:00:00-05:00

Originally posted as TotW #187 on November 5, 2020

By Andy Soffer

Updated 2020-11-05

Quicklink: abseil.io/tips/187

If you say in the first chapter that there is a std::unique_ptr on the wall, in the second or third chapter it absolutely must be moved. If it’s not going to be moved, it shouldn’t be hanging there. ~ With apologies to Anton Chekhov

std::unique_ptr is for expressing transfer of ownership. If you never pass ownership from one std::unique_ptr to another, the abstraction is rarely necessary or appropriate.

What is a `std::unique_ptr`?

A std::unique_ptr is a pointer that automatically destroys whatever it is pointing at when the std::unique_ptr itself is destroyed. It exists to convey ownership (the responsibility to destroy resources) as part of the type system and is one of C++11’s more valuable additions¹. However, std::unique_ptr is commonly overused. A good litmus test is this: If it is never std::moved to or from another std::unique_ptr, it likely should not be a std::unique_ptr. If we do not transfer ownership then there is almost always a better way to express our intent than by using std::unique_ptr.

The Costs of `std::unique_ptr`

There are several reasons for avoiding std::unique_ptr when ownership is not being transferred.

std::unique_ptr conveys transferrable ownership which is unhelpful if ownership isn’t being transferred. We should aim to use the type that most accurately conveys the required semantics.
std::unique_ptr can be in a null state, which gives extra cognitive overhead for readers if the null state is not actually used.
std::unique_ptr<T> manages a heap-allocated T, which comes with performance implications both due to the heap allocation itself, and the fact that the data is spread out across the heap and less likely to be in CPU cache.

Common Anti-Pattern: Avoiding `&`

It is not uncommon to see examples like the following.

int ComputeValue() {
  auto data = std::make_unique<Data>();
  ModifiesData(data.get());
  return data->GetValue();
}

In this example data does not need to be a std::unique_ptr, because ownership is never transferred. The data will be constructed and destroyed exactly at the same instances as if a Data object were declared on the stack. Therefore, as is also discussed in Tip #123, a better option would be:

int ComputeValue() {
  Data data;
  ModifiesData(&data);
  return data.GetValue();
}

Common Anti-Pattern: Delayed Initialization

Because std::unique_ptr is null when default constructed, and can be assigned a new value from std::make_unique, it’s common to see std::unique_ptr used as a delayed initialization mechanism. There is a particularly common pattern with GoogleTest, in which test fixtures can initialize objects in SetUp.

class MyTest : public testing::Test {
 public:
  void SetUp() override {
    thing_ = std::make_unique<Thing>(data_);
  }

 protected:
  Data data_;
  // Initialized in `SetUp()`, so we're using `std::unique_ptr` as a
  // delayed-initialization mechanism.
  std::unique_ptr<Thing> thing_;
};

Once again, we see that ownership of thing_ is never transferred elsewhere, so there is no need to use std::unique_ptr. The example above could have done all of the initialization in the default constructor for MyTest. See the GoogleTest FAQ for details on SetUp versus construction.

class MyTest : public testing::Test {
 public:
  MyTest() : thing_(data_) {}

 private:
  Data data_;
  Thing thing_;
};

In this example, data_ is default constructed as it was before. Afterwards, Thing is constructed with data_. Remember that a class’s constructor initializes fields in the order they are declared, so this approach initializes objects in the same order as they were before, but without the use of std::unique_ptr.

If delayed initialization is really important and unavoidable, consider using std::optional with its emplace() method. Tip #123 discusses delayed initialization in much greater depth.

class MyTest : public testing::Test {
 public:
  MyTest() {
    Initialize(&data_);
    thing_.emplace(data_);
  }

 private:
  Data data_;
  std::optional<Thing> thing_;
};

Caveats

This being C++, there are of course cases where a std::unique_ptr makes sense even if it is never moved. However these situations are uncommon, and any code handling such situations should come with comments explaining the subtleties. Here are two such examples.

Large, rarely used objects.

If an object is only sometimes needed, std::optional is a good default choice. However, std::optional reserves space regardless of whether the object is actually constructed. If this space is important, it may make sense to hold a std::unique_ptr and only allocate it if it is needed.

Legacy APIs

Many legacy APIs return raw pointers to owned data. These APIs often predate the addition of std::unique_ptr to the C++ standard library, and this pattern should not be copied in new code. However, even if the resulting object is never moved, such legacy API calls should be wrapped in a std::unique_ptr to ensure that the memory is not leaked.

Widget *CreateLegacyWidget() { return new Widget; }

int func() {
  Widget *w = CreateLegacyWidget();
  return w->num_gadgets();
}  // Memory leak!

Wrapping the object in a std::unique_ptr solves both of these issues:

int func() {
  std::unique_ptr<Widget> w = absl::WrapUnique(CreateLegacyWidget());
  return w->num_gadgets();
}  // `w` is properly destroyed.

The word “unique” in the name std::unique_ptr was chosen to signify the idea that no other std::unique_ptr should be holding the same non-null value. That is, at any moment during program execution, amongst all the std::unique_ptrs that are not null, the addresses held by all the std::unique_ptrs are unique. ↩

Tip of the Week #186: Prefer to Put Functions in the Unnamed Namespace

2020-11-11T00:00:00-05:00

Originally posted as TotW #186 on November 5, 2020

By James Dennett and Jason Rennie

Updated 2020-11-05

Quicklink: abseil.io/tips/186

“Everything should be made as simple as possible, but no simpler.” ~ Roger Sessions’s interpretation of Einstein

When adding a new function, default to making it a non-member function local to the .cc file where it is called. While there are valid reasons to make another choice, consider writing it in an unnamed namespace (also known as an “anonymous namespace”).

Benefits

Writing a non-member in an unnamed namespace has benefits both by making functions internal to a .cc file (moving them out of header files) as well as by making them non-members (moving them out of classes).

Benefits over functions declared in a header file include:

Making it easy for a reader to look up the definition (since it’s in the same file as usage and above the first usage).
Placing the documentation, declaration, and definition in a single location (versus two locations for functions which are declared in a header file).
Isolating it from other source files, making it easier to refactor.
Removing the need to worry about meaningful const as there is no separate declaration.
Keeping them in the same place as other implementation helpers for a library, such as convenience aliases and local types. See Tip of the Week #119: Using-declarations and Namespace Aliases.
Providing side benefits, such as allowing a type to also be moved to an unnamed namespace (if it is only referenced in a single source file).

Benefits over private methods include:

Inputs and outputs are clear since they are (much more likely to be) specified via arguments or the return value. Note that a method may read any member variable and a non-const method may modify any non-const member. Conversely, a non-member function may only read or modify according to its interface (except for globals).
The class API is simpler and shorter, and hence easier to read—unnecessary private methods may make it difficult to find inheritance-related private declarations or declarations after the class.

Most of these benefits remain even if there is no relevant header file, such as for a *_test.cc or a *_main.cc file.

Reasons to Look Elsewhere

Sometimes a non-member local function does not make sense. For example:

When the function is useful in multiple source files. Declaring it in a header file allows re-use.
When the function has complex interactions with an object or class. For example, a function that reads from multiple fields and needs to modify state in a way that can’t be handled naturally via a return value is likely better written as a method. In particular, logic involving a mutex usually belongs in a member function.
When the function belongs as part of a class’s API.

An Alternative: `static` Non-Member Functions

Marking a non-member function as static has essentially the same effect as placing it inside an unnamed namespace in terms of isolating it from code in other translation units. While unnamed namespaces do this in a uniform manner that covers types as well as functions and objects, some people like to see static written explicitly in the declaration of a function to show that it is local to a translation unit without having to check for an enclosing unnamed namespace. While this tip recommends using an unnamed namespace, using static can be a reasonable choice.

Other References

Parts of the style guide point us in this direction, but they don’t go as far. For example:

The unnamed namespaces section encourages that definitions go in an unnamed namespace (or be declared static), but doesn’t talk about private methods.
The inputs and outputs section encourages using return values, but doesn’t address modification of members (via this); this is essentially an uber-input/output parameter.
The local variables section encourages minimizing variable scope, but doesn’t extend the idea to classes and objects.
The Nonmember, Static Member, and Global Functions section discourages using a class merely to group functions.

Summary

File-local functions simplify dependencies and improve locality. Non-member functions increase encapsulation, simplify class definitions, and make dependencies more explicit. When writing a function, consider making it a file-local non-member function, such as by putting it in an unnamed namespace of a .cc file.

Abseil Platform Support Update

2020-10-01T00:00:00-04:00

Abseil Platform Support Update

By Derek Mauro, Abseil Engineer

In September 2017, Abseil made the following support promise:

We will support our code for at least 5 years. We will support language versions, compilers, platforms, and workarounds as needed for 5 years after their replacement is available, when possible. If it is technically infeasible (such as support for MSVC before 2015, which has limited C++11 functionality), those will be noted specifically. After 5 years we will stop support and may remove workarounds. ABSL_HAVE_THREAD_LOCAL is a good example: the base language feature works on everything except Xcode prior to Xcode 8 ; once Xcode 8 is out for 5 years, we will drop that workaround support.

The ability to evolve over time has long been a goal of the Abseil project, and supporting older platforms comes with the cost of not being able to take advantage of new features, and not being able to get the best performance possible. With this is mind, the September 2020 LTS release branch will be the last to support the following compilers:

GCC 4.9
Clang 3.6

We will soon remove our continuous integration tests for these compilers.

Preparing for the Future

Of our supported platforms and compilers, the last to facilitate support for C++14 was Microsoft Visual C++ 2017, which was released on March 7, 2017. Therefore, Abseil will only support C++11 (and Microsoft Visual C++ 2015) until March 7, 2022. We will do one final Long Term Support release in March of 2022, at which point we will announce that it is the final LTS release to support C++11. We strongly recommend all users begin moving to C++14 or newer as soon as possible and not wait for this LTS release, however.

In addition, the following support changes will occur in the near future:

Clang 3.8, which replaced Clang 3.7, was released March 8, 2016. Therefore, we intend to drop support for Clang 3.7 after March 8, 2021.
GCC 6.1, which replaced the GCC 5 series, was released on April 27, 2016. Therefore, we intend to drop support for the GCC 5 series after April 27, 2021.

Tip of the Week #76: Use `absl::Status`

2020-09-11T00:00:00-04:00

Originally posted as TotW #76 on May 4, 2014

By Titus Winters

Updated 2020-02-06

Quicklink: abseil.io/tips/76

Some folks have questions about when and how to use absl::Status, so here are a few reasons why you ought to use Status, and some things to keep in mind when using it.

Communicate Intent and Force the Caller to Handle Errors

Use Status to force the caller to handle the possibility of errors. Since June 2013, a function returning an Status object cannot simply be ignored. That is, this code produces a compilation error:

absl::Status Foo();

void CallFoo1() {
  Foo();
}

Whereas these calls are fine:

void CallFoo2() {
  Foo().IgnoreError();
}

void CallFoo3() {
  if (!status.ok()) std::abort();
}

void CallFoo4() {
  absl::Status status = Foo();
  if (!status.ok()) LOG(ERROR) << status;
}

Why is it OK for Status to have an IgnoreError() method, while we just went through all this effort to make the compiler check that Status isn’t ignored? Imagine you’re the code reviewer looking at either CallFoo1() or CallFoo2(). The latter code snippets make the reviewer think “This function could have had an error, but the author thinks its OK to ignore it. Is it?” CallFoo1() does not trigger such a response.

Allow the Caller to Handle Errors Where They Have More Context

Use Status when it’s not clear in your code how to handle the error. Instead, return a Status, and let the caller, who may have more appropriate insight, handle the error.

For example, logging locally might impact performance, such as when writing infrastructure code. If your code is called in a tight loop, even a call to LOG(INFO) may be too expensive. In other cases, users may not really care if a call succeeds, and find the log spam intrusive.

Logging is appropriate in many cases, but functions that return Status don’t need to LOG to explain why something failed: they can return the failure code and error string and let the caller decide what the right error handling response should be.

Isn’t This Just Re-Inventing Exceptions?

The Google style guide famously prohibits exceptions (it’s discussed more than any other prohibition). It can be tempting to view absl::Status as a poor- man’s exception mechanism, with a lot more overhead. While there may be similarities on the surface, absl::Status differs in its need to be explicitly handled, rather than just passed up the stack invisibly as an unhandled exception. It forces engineers to decide how to handle errors, and explicitly documents that in compilable code. And finally, returning an error using a absl::Status is orders of magnitude faster than throwing and catching an exception. These features may seem onerous when writing code, but the result is a net win for everyone that has to read that code, and for Google as a whole.

Conclusion

Error handling is one of the easiest things to get wrong: these are inherently the edge cases. A utility like Status that adds consistency to error handling across API boundaries, projects, processes, and languages helps us minimize a large class of “There were issues with our error handling” bugs. If you’re designing an interface that needs to express the possibility of failure, please use Status if you don’t have a pretty strong reason to do otherwise.

Tip of the Week #5: Disappearing Act

2020-09-11T00:00:00-04:00

Originally posted as TotW #5 on June 26, 2012

Updated 2020-06-01

Quicklink: abseil.io/tips/5

“Don’t know what you got till it’s gone.” –Cinderella

Sometimes, in order to use a C++ library the right way, you need to understand both the library and the language. So … what’s wrong with the following?

// DON’T DO THIS
std::string s1, s2;
...
const char* p1 = (s1 + s2).c_str();             // Avoid!
const char* p2 = absl::StrCat(s1, s2).c_str();  // Avoid!

Both (s1+s2) and absl::StrCat(s1,s2) create temporary objects (in both cases, strings, but the same rules apply for any objects). The member function c_str() returns a pointer to data that lives as long as the temporary object. How long does the temporary object live? According to the C++17 standard [class.temporary], “Temporary objects are destroyed as the last step in evaluating the full-expression that (lexically) contains the point where they were created.” (A “full-expression” is “an expression that is not a subexpression of another expression”). In each example above, as soon as the expression on the right side of the assignment operator was completed, the temporary value was destroyed, and the return value from c_str() became a dangling pointer. tl;dr? By the time you hit a semicolon (often sooner), the temporary object is history. Yikes! How can you avoid this kind of problem?

Option 1: Finish Using the Temporary Object Before the End of the full-expression:

// Safe (albeit a silly example):
size_t len1 = strlen((s1 + s2).c_str());
size_t len2 = strlen(absl::StrCat(s1, s2).c_str());

Option 2: Store the Temporary Object.

You’re creating an object (on the stack) anyway; why not hang on to it for a while? This is cheaper than it might first appear. Because of something called “return value optimization,” (and move semantics on many value types, see Tip #77) the temporary object will be constructed in the variable you’re “assigning” it to, and won’t be copied:

// Safe (and more efficient than you might think):
std::string tmp_1 = s1 + s2;
std::string tmp_2 = absl::StrCat(s1, s2);
// tmp_1.c_str() and tmp_2.c_str() are safe.

Option 3: Store a Reference to the Temporary Object.

C++17 standard [class.temporary]: “The temporary to which the reference is bound or the temporary that is the complete object of a sub-object to which the reference is bound persists for the lifetime of the reference.”

Because of return value optimization, this usually isn’t any cheaper than storing the object itself (Option 2), and it is potentially confusing or worrisome (see Tip #101). (Exceptional cases that need to use lifetime extension should be commented!)

// Equally safe:
const std::string& tmp_1 = s1 + s2;
const std::string& tmp_2 = absl::StrCat(s1, s2);
// tmp_1.c_str() and tmp_2.c_str() are safe.
// The following behavior is dangerously subtle:
// If the compiler can see you’re storing a reference to a
// temporary object’s internals, it will keep the whole
// temporary object alive.
// struct Person { string name; ... }
// GeneratePerson() returns an object; GeneratePerson().name
// is clearly a sub-object:
const std::string& person_name = GeneratePerson().name; // safe
// If the compiler can’t tell, you’re at risk.
// class DiceSeries_DiceRoll { `const string&` nickname() ... }
// GenerateDiceRoll() returns an object; the compiler can’t tell
// if GenerateDiceRoll().nickname() is a subobject.
// The following may store a dangling reference:
const std::string& nickname = GenerateDiceRoll().nickname(); // BAD!

Option 4: Design your functions so they don’t return objects???

Many functions follow this principle; but many don’t. Sometimes it really is better to return an object than to require the caller to pass in a pointer to an output parameter. Be aware of when the creation of temporaries can happen. Anything that returns a pointer or reference to an object’s internals is potentially a problem when operating on a temporary object. c_str() is the most obvious culprit, but protobuf getters (mutable or otherwise) and getters in general can be equally problematic.

Tip of the Week #181: Accessing the value of a StatusOr<T>

2020-09-11T00:00:00-04:00

Originally posted as TotW #181 on July 9, 2020

By Michael Sheely

Updated 2020-09-02

Quicklink: abseil.io/tips/181

StatusOr<Readability>: you don’t have to choose!

When the time comes to access the value inside an absl::StatusOr<T> object, we should strive to make that access safe, clear, and efficient.

Note: This tip attempts to highlight well lit paths providing guidance for typical use-cases. It is not intended to be exhaustive. If you encounter edge cases, consider the recommendations and reasoning here and exercise judgement.

Recommendation

Accessing the value held by a StatusOr should be performed via operator* or operator->, after a call to ok() has verified that the value is present.

// The same pattern used when handling a unique_ptr...
std::unique_ptr<Foo> foo = TryAllocateFoo();
if (foo != nullptr) {
  foo->DoBar();  // use the value object
}

// ...or an optional value...
std::optional<Foo> foo = MaybeFindFoo();
if (foo.has_value()) {
  foo->DoBar();
}

// ...is also ideal for handling a StatusOr.
absl::StatusOr<Foo> foo = TryCreateFoo();
if (foo.ok()) {
  foo->DoBar();
}

You can limit the scope of a StatusOr by declaring it in the initializer of the if statement and checking ok() in the condition. If using a StatusOr immediately, you generally should limit scope in this way (see Tip #165):

if (absl::StatusOr<Foo> foo = TryCreateFoo(); foo.ok()) {
  foo->DoBar();
}

Background on `StatusOr`

The absl::StatusOr<T> class is a tagged union with value semantics indicating exactly one of the following situations:

an object of type T is available,
an absl::Status error (!ok()) indicating why the value is not present.

You can read more about absl::Status and absl::StatusOr in Tip #76.

Safety, Clarity, and Efficiency

Treating the StatusOr object as one would treat a smart pointer helps code achieve clarity while remaining safe and efficient. Below, we will consider some of the other ways you might have seen a StatusOr accessed, and why we prefer the approach using the indirection operators.

Alternative Value Accessor Safety Issues

What about absl::StatusOr<T>::value()?

absl::StatusOr<Foo> foo = TryCreateFoo();
foo.value().DoBar();  // Behavior depends on the build mode.

Here, the behavior depends on the build mode – in particular, whether the code was compiled with exceptions enabled.¹ As such, it is not clear to readers if an error status will terminate the program.

The value() method combines two actions: a test for validity followed by an access of the value. It should therefore be used only if both actions are intended (and even then, think twice and consider that its behavior depends on the build mode). If the status is already known to be OK, then your ideal accessor has semantics of simply accessing the value, which is exactly what operator* and operator-> provide. In addition to making the code more precisely indicate your intent, the access will be at least as efficient as value()’s contract to test for validity and then access the value.

Avoiding Multiple Names for the Same Object

Treating absl::StatusOr objects as we would a smart pointer or optional value also allows us to avoid the conceptually awkward situation of having two variables referring to the same value. It also avoids the naming dilemmas and auto overuse which come with this territory.

// Without digging up the declaration of TryCreateFoo(), a reader will not
// immediately understand the types here (optional? pointer? StatusOr?).
auto maybe_foo = TryCreateFoo();
// ...compounded by the use of implicit bool rather than `.ok()`.
if (!maybe_foo) { /* handle foo not present */ }
// Now two variables (maybe_foo, foo) represent the same value.
Foo& foo = maybe_foo.value();

Avoiding the `_or` Suffix

Another benefit of using a StatusOr variable’s intrinsic value type after checking for validity (rather than creating multiple variables for the same value) is that we can use the best name for the StatusOr without the need (or temptation!) to add a prefix or suffix.

There is an analogy here to the avoidance of the “_ptr” suffix when naming pointer variables.

// The type already describes that this is a unique_ptr; `foo` would be fine.
std::unique_ptr<Foo> foo_ptr;

absl::StatusOr<Foo> foo_or = MaybeFoo();
if (foo_or.ok()) {
  const Foo& foo = foo_or.value();
  foo.DoBar();
}

If there is only one variable (the StatusOr, avoiding a second named variable for the unwrapped value), we can drop the suffix, simply naming the variable after its underlying value (just as we do with pointers).

const absl::StatusOr<Foo> foo = MaybeFoo();
if (foo.ok()) {
  MakeUseOf(*foo);
  foo->DoBar();
}

Moving from the Value of an `absl::StatusOr`

We might write code to move from the T of an absl::StatusOr<T>:

absl::StatusOr<Foo> foo = MaybeFoo();
if (foo.ok()) {
  ConsumeFoo(std::move(*foo));
}

It’s a little better, though, to move from the StatusOr itself, indicating to readers (both humans and machines) that the entire StatusOr object should not be used after its value has been moved from:

absl::StatusOr<Foo> foo = MaybeFoo();
if (foo.ok()) {
  ConsumeFoo(*std::move(foo));
}

Solution

Testing the absl::StatusOr object for validity (as you would a smart pointer or optional) and accessing it using operator* or operator-> is readable, efficient, and safe.

It helps you avoid the potential pitfalls of naming ambiguity mentioned above, and does it all without the use of any macros.

Code that accesses values via operator* or operator-> (whether using a pointer, a StatusOr, an optional, or otherwise) must first verify that the value is present. This validation should be close to where the value is accessed so that readers can easily verify that it is present and correct.

Per the documentation of the value() function, if exceptions are enabled, this will throw absl::BadStatusOrAccess (which may be caught, meaning the program may not terminate). If compiled without exceptions, code will crash with LOG(FATAL). ↩

Tip of the Week #165: `if` and `switch` statements with initializers

2020-09-11T00:00:00-04:00

Originally posted as TotW #165 on August 17, 2019

By Thomas Köppe

Updated 2020-01-17

Quicklink: abseil.io/tips/165

Unless you use conditional control flow, you can stop reading now.

A new syntax

C++17 allows if and switch statements to include an initializer:

if (init; cond) { /* ... */ }
switch (init; cond) { /* ... */ }

This syntax lets you keep the scope of variables as tight as possible:

if (auto it = m.find("key"); it != m.end()) {
  return it->second;
} else {
  return absl::NotFoundError("Entry not found");
}

The semantics of the initializer are exactly as in the for statement; details below.

When this is useful

One of the most important ways to manage complexity is to break complex systems down into non-interacting, local parts that can be understood in isolation and ignored in their entirety. In C++, the presence of variables increases complexity, and scopes allow us to limit the extent of this complexity: the less a variable is in scope, the less often a reader needs to remember that the variable exists.

When demanding reader attention, it is thus valuable to limit the scopes of variables to where they are actually needed. The new syntax offers one new tool for this. Contrast then this new syntax with the alternative code one would have written prior to C++17: Either we keep the scopes tight, and thus need to write additional braces:

{
  auto it = m.find("key");
  if (it != m.end()) {
    return it->second;
  } else {
    return absl::NotFoundError("Entry not found");
  }
}

Or, as seems to be the more typical solution, we do not keep the scopes tight and just “leak” the variables:

auto it = m.find("key");
if (it != m.end()) {
  return it->second;
} else {
  return absl::NotFoundError("Entry not found");
}

By contrast, the new style is self-contained: It is not possible to move the if statement without also moving the variable and its scope. The local meaning of the variable remains unchanged as code is moved around or copy-pasted. With the previous styles, code movement could accidentally change the scope of the variable (if the outer braces are not copied), or its meaning (if the variable itself is not copied and a variable with that name is in scope), or introduce a name clash.

The complexity considerations lead to the common adage that variable name length should match the variable scope’s size; that is, variables that are in scope for longer should have longer names (since they need to make sense to a reader that has long moved on). Conversely, smaller scopes permit shorter names. When variable names are leaked (as above), we see regrettable patterns emerge such as: multiple variables it1, it2, … become necessary to avoid clashes; variables are reassigned (auto it = m1.find(/* ... */); it = m2.find(/* ... */); or variables get intrusively long names (auto database_index_iter = m.find(/* ... */)).

Details, scopes, declarative regions

The new, optional initializer in if and switch statements works exactly like the initializer in a for statement. (The latter is essentially a while statement with initializer.) That is, the syntax-with-initializer is mostly just syntactic sugar around the following rewrites:

Sugared form	Rewritten as
`if (init; cond) BODY`	`{ init; if (cond) BODY }`
`switch (init; cond) BODY`	`{ init; switch (cond) BODY }`
`for (init; cond; incr) BODY`	`{ init; while (cond) { BODY; incr; } }`

Importantly, the names declared in the initializer are in scope of a potential else arm of an if statement.

There is one difference, though: In the sugared form, the initializer is in the same scope as the condition and body (of both the if and the else arm), rather than in a separate, larger scope. This means that variable names must be unique across all these parts, though they may shadow earlier declarations. The following examples illustrate the various disallowed redeclarations and allowed shadowing declarations:

int w;

if (int x, y, z; int y = g()) {   // error: y redeclared, first declared in initializer
  int x;                          // error: x redeclared, first declared in initializer
  int w;                          // OK, shadows outer variable
  {
    int x, y;                     // OK, shadowing in nested scope is allowed
  }
} else {
  int z;                          // error: z redeclared, first declared in initializer
}

if (int w; int q = g()) {         // declaration of "w" OK, shadows outer variable
  int q;                          // error: q redeclared, first declared in condition
  int w;                          // error: w redeclared, first declared in initializer
}

Interaction with structured bindings

C++17 also introduces structured bindings, a mechanism to assign names to the elements of a “destructurable” value (such as a tuple, an array, or a simple struct): auto [iter, ins] = m.insert(/* ... */);

That feature plays nicely with the new initializer in the if statement:

if (auto [iter, ins] = m.try_emplace(key, data); ins) {
  use(iter->second);
} else {
  LOG(ERROR) << "Key '" << key << "' already exists.";
}

Another example comes from using C++17’s new node handles that allow true moving of elements between maps or sets without copying. This feature defines an insert-return-type that is destructurable and that results from inserting a node handle:

if (auto [iter, ins, node] = m2.insert(m1.extract(k)); ins) {
  LOG(INFO) << "Element with key '" << k << "' transferred successfully";
} else if (!node) {
  LOG(ERROR) << "Key '" << k << "' does not exist in first map.";
} else {
  LOG(ERROR) << "Key '" << k << "' already in m2; m2 unchanged; m1 changed.";
}

Conclusion

Use the new if (init; cond) and switch (init; cond) syntax when you need a new variable for use within the if or switch statement that is not needed outside of it. This simplifies the ambient code. Moreover, since the variable’s scope is now small, its name can be shorter, too.

Tip of the Week #116: Keeping References on Arguments

2020-09-11T00:00:00-04:00

Originally posted as TotW #116 on May 26, 2016

By Alex Pilkiewicz

Updated 2020-06-01

Quicklink: abseil.io/tips/116

From painting to image, from image to text, from text to voice, a sort of imaginary pointer indicates, shows, fixes, locates, imposes a system of references, tries to stabilize a unique space. — This is Not a Pipe by Michel Foucault

Const References vs. Pointers to Const

Used as arguments of functions, const references have several advantages when compared to pointers to const: they cannot be null and it’s quite clear that the function is not taking ownership of the object. But they have other differences that can sometimes be problematic: they are more implicit (i.e. there is nothing on the call site showing we are taking a reference) and they can be bound to a temporary.

Risk of a Dangling Reference in Classes

Consider the following class as an example:

class Foo {
 public:
  explicit Foo(const std::string& content) : content_(content) {}
  const std::string& content() const { return content_; }

 private:
  const std::string& content_;
};

It looks reasonable. But what happens if we build a Foo from a string literal?

void Func() {
  Foo foo("something");
  LOG(INFO) << foo.content();  // BOOM!
}

When creating foo, the content_ member gets bound to the temporary std::string object that was created from the literal and passed to the constructor. The temporary string goes out of scope at the end of the line where it was created. Now foo.content_ is a reference to an object that no longer exists. Accessing it is undefined behavior and anything can happen, from working fine in tests to going really wrong in production.

A Solution: Use Pointers

In our example, the simplest solution is probably to pass and store the string by value. But let’s assume that we need to refer to the original argument, e.g. because it’s not a string, but some more interesting type. The solution is to pass the argument by pointer:

class Foo {
 public:
  // Do not forget this comment:
  // Does not take ownership of content, which must refer to a valid string that
  // outlives this object.
  explicit Foo(const std::string* content) : content_(content) {}
  const std::string& content() const { return *content_; }

 private:
  const std::string* const content_;  // not owned, can't be null
};

Now the following will simply fail to compile:

std::string GetString();
void Func() {
  Foo foo1(&GetString());  // error: taking the address of a temporary of
                           // type 'std::string'
  Foo foo2(&"something");  // error: no matching constructor for initialization
                           // of 'Foo'
}

And it will be pretty clear at call site that the object might keep the address of the argument:

void Func2() {
  std::string content = GetString();
  Foo foo(&content);
}

One Step Further, One Less Comment: Storing a Reference

You might have noticed that we say twice that the pointer cannot be null and that it is not owned, once in the documentation of the constructor then in the comment for the instance variable. Is this necessary? Consider this:

class Baz {
 public:
  // Does not take any ownership, and all pointers must refer to valid objects
  // that outlive the one constructed.
  explicit Baz(const Arg1* arg1, Arg2* arg2) : arg1_(*arg1), arg2_(*arg2) {}

 private:
  // It is now clear that we do not have ownership and that the references can't
  // be null.
  const Arg1& arg1_;
  Arg2& arg2_;  // Yes, non-const references are style-compliant!
};

One downside of members of reference type is that you cannot reassign them, meaning that your class will not have a copy assignment operator (copy constructors are still OK) but it might make sense to explicitly delete it to respect the rule of 3. If your class should be assignable you’ll need non-const pointers, still potentially to const objects. Tip #177 discusses this in more detail.

If you want defense in depth and think some caller might accidentally pass a null pointer, you can use *ABSL_DIE_IF_NULL(arg1) to cause a crash. Note that just dereferencing the null pointer is not, as is commonly believed, guaranteed to crash; rather it is undefined behavior and should not be relied upon. Here what would probably happen is that since a reference is implemented as a pointer, it will just be copied and the crash will happen later, when someone actually accesses the field.

Conclusion

It is still OK to pass an argument by const reference to a constructor if the argument is copied, or just used in the constructor and no reference to it is kept in the constructed object. In other cases, consider passing arguments by pointers (to const or not). Also remember that if you are actually transferring the ownership of an object, it should be passed as a std::unique_ptr.

Lastly, what is discussed here is not limited to constructors: any function that somehow keeps an alias to one of its arguments, whether by putting a pointer in a cache or binding the argument in a detached function, should take that argument by pointer.

Abseil Status

2020-09-10T00:00:00-04:00

By Xiaoyi Zhang, Google Engineer, Emeritus

The Abseil status library is now available on abseil.io. This library is used within Google for error handling and contains the following two abstractions:

absl::Status
absl::StatusOr<T>

Within Google, absl::Status is the primary mechanism to gracefully handle errors across API boundaries (and in particular across RPC boundaries). Some of these errors may be recoverable, but others may not. Most functions that can produce a recoverable error should be designed to return either an absl::Status (or the similar absl::StatusOr<T>, which holds either an object of type T or an error).

A Status can either return “OK” (represented by the enumumerated value absl::StatusCode::kOk) or one of a number of canonical error codes to indicate certain error conditions:

absl::Status MyFunction(absl::string_view filename, ...) {
  ...
  // encounter error
  if (error condition) {
    // absl::StatusCode::kInvalidArgument
    return absl::InvalidArgumentError("bad mode");
  }
  // else, return OK
  return absl::OkStatus();
}

Abseil also provides an absl::StatusOr<T> class template to represent a union of an absl::Status object and an object of type T. The absl::StatusOr<T> will either contain an object of type T (indicating a successful operation), or an error (of type absl::Status) explaining why such a value is not present. This abstraction is similar to the proposal in C++ for a std::expected type.

For more information, consult the Abseil Status guide and the list of canonical error codes.

Tip of the Week #140: Constants: Safe Idioms

2020-09-01T00:00:00-04:00

Originally posted as TotW #140 on December 8, 2017

By Matt Armstrong

Updated 2020-05-06

Quicklink: abseil.io/tips/140

What are best ways to express constants in C++? You probably have an idea of what the word means in English, but it is easy to express the concept incorrectly in code. Here we’ll first define a few key concepts, then get to a list of safe techniques. For the curious, we then go into more detail about what can go wrong, and describe a C++17 language feature that makes expressing constants easier.

There is no formal definition of a “C++ constant” so let’s propose an informal one.

The value: A value never changes; five is always five. When we want to express a constant, we need a value, but only one.
The object: At each point in time an object has a value. C++ places a strong emphasis on mutable objects, but mutation of constants is disallowed.
The name: Named constants are more useful than bare literal constants. Both variables and functions can evaluate to constant objects.

Putting that all together, let’s call a constant a variable or function that always evaluates to the same value. There are a few more key concepts.

Safe Initialization: Many times constants are expressed as values in static storage, which must be safely initialized. For more on that, see the C++ Style Guide.
Linkage: Linkage has to do with how many instances (or “copies”) of a named object there are in a program. It is usually best for a constant with one name to refer to a single object within the program. For global or namespace-scoped variables this requires something called external linkage (you can read more about linkage here).
Compile-time evaluation: Sometimes the compiler can do a better job optimizing code if a constant’s value is known at compile time. This benefit can sometimes justify defining the values of constants in header files, despite the additional complexity.

When we say we’re “adding a constant” we’re actually declaring an API and defining its implementation in such a way that satisfies most or all of the above criteria. The language doesn’t dictate how we do this, and some ways are better than others. Often the simplest approach is declaring a const or constexpr variable, marked as inline if it’s in a header file. Another approach is returning a value from a function, which is more flexible. We’ll cover examples of both approaches.

A note on const: it isn’t enough. A const object is read-only but this does not imply that it is immutable nor does it imply that the value is always the same. The language provides ways to mutate values we think of as const, such as the mutable keyword and const_cast. But even straightforward code can demonstrate the point:

void f(const std::string& s) {
  const int size = s.size();
  std::cout << size << '\n';
}

f("");  // Prints 0
f("foo");  // Prints 3

In the above code size is a const variable, yet it holds multiple values as the program runs. It is not a constant.

Constants in Header Files

All of the idioms in this section are robust and recommendable.

An inline constexpr Variable

From C++17 variables can be marked as inline, ensuring that there is only a single copy of the variable. When used with constexpr to ensure safe initialization and destruction this gives another way to define a constant whose value is accessible at compile time. See Tip #168 for more information.

// in foo.h
inline constexpr int kMyNumber = 42;
inline constexpr absl::string_view kMyString = "Hello";

An extern const Variable

// Declared in foo.h
extern const int kMyNumber;
extern const char kMyString[];
extern const absl::string_view kMyStringView;

The above example declares one instance of each object. The extern keyword ensures external linkage, while the const keyword helps prevent accidental mutation of the value. This is a fine way to go, though it does mean the compiler can’t “see” the constant values. This limits their utility somewhat, but not in ways that matter for typical use cases. It also requires defining the variables in the associated .cc file.

// Defined in foo.cc
constexpr int kMyNumber = 42;
constexpr char kMyString[] = "Hello";
constexpr absl::string_view kMyStringView = "Hello";

The constexpr keyword ensures each variable is a constant, is compile-time initialized, and has a trivial destructor. This is a convenient way to ensure it meets the style guide rules for globals.

You should define the variables in the .cc file with constexpr, unless you need to support an old toolchain.

NOTE: absl::string_view is a good way to declare a string constant. The type has a constexpr constructor and a trivial destructor, so it is safe to declare instances of it as global variables. Because a string_view knows its length, using them does not require a runtime call to strlen().

A constexpr Function

A constexpr function that takes no arguments will always return the same value, so it functions as a constant, and can often be used to initialize other constants at compile time. Because all constexpr functions are implicitly inline, there are no linkage concerns. The primary disadvantage of this approach is the limitations placed on the code in constexpr functions. Secondarily, constexpr is a non-trivial aspect of the API contract, which has real consequences .

// in foo.h
constexpr int MyNumber() { return 42; }

An Ordinary Function

When a constexpr function isn’t desirable or possible, an ordinary function may be an option. The function in the following example can’t be constexpr because it has a static variable:

inline absl::string_view MyString() {
  static constexpr char kHello[] = "Hello";
  return kHello;
}

NOTE: make sure you use static constexpr specifiers when returning array data, such as char[] strings, absl::string_view, absl::Span, etc, to avoid subtle bugs.

A `static` Class Member

Static members of a class are a good option, assuming you are already working with a class. These always have external linkage.

// Declared in foo.h
class Foo {
 public:
  static constexpr int kMyNumber = 42;
  static constexpr absl::string_view kMyHello1 = "Hello";
  static constexpr char kMyHello2[] = "Hello";  // No longer recommended
};

Prior to C++17 it was necessary to also provide definitions for these static data members in a .cc file, but for data members that are both static and constexpr these are now unnecessary (and deprecated).

// Defined in foo.cc, prior to C++17.
constexpr int Foo::kMyNumber;
constexpr absl::string_view Foo::kMyHello1;
constexpr char Foo::kMyHello2[];

It isn’t worth introducing a class just to act as a scope for a bunch of constants. Use one of the other techniques instead.

Discouraged Alternatives

#define WHATEVER_VALUE 42

Using the preprocessor is rarely justified, see the style guide.

enum : int { kMyNumber = 42 };

The enum technique used above can be justified in some circumstances. It produces a constant kMyNumber that cannot cause the problems talked about in this tip. But the alternatives already listed will be more familiar to most people, and so are generally preferred. Use an enum when it makes sense in its own right (for examples, see Tip #86 “Enumerating with Class”).

Approaches that Work in Source Files

All of the approaches described above also work within a single .cc file, but may be unnecessarily complex. Because constants declared within a source file are visible only inside that file by default (see internal linkage rules), simpler approaches, such as defining constexpr variables, often work:

// within a .cc file!
constexpr int kBufferSize = 42;
constexpr char kBufferName[] = "example";
constexpr absl::string_view kOtherBufferName = "other example";

The above is fine in a .cc file but not a header file (see caveat). Read that again and commit it to memory. I’ll explain why soon. Long story short: define variables constexpr in .cc files or declare them extern const in header files.

Within a Header File, Beware!

Unless you take care to use idioms explained above, const and constexpr objects are likely to be different objects in each translation unit.

This implies:

Bugs: any code that uses the address of a constant is subject to bugs and even the dreaded “undefined behavior”.
Bloat: each translation unit including your header gets its own copy of the thing. Not such a big deal for simple things like the primitive numeric types. Not so great for strings and bigger data structures.

When at namespace scope (i.e. not in a function or in a class), both const and constexpr objects implicitly have internal linkage (the same linkage used for unnamed-namespace variables and static variables not in a function or in a class). The C++ standard guarantees that every translation unit that uses or references the object gets a different “copy” or “instantiation” of the object, each at a different address.

Within a class, you must additionally declare these objects as static, or they will be unchangeable instance variables, rather than unchangeable class variables shared among every instance of the class.

Likewise, within a function, you must declare these objects as static, or they will take up space on the stack and be constructed every time the function is called.

An Example Bug

So, is this a real risk? Consider:

// Declared in do_something.h
constexpr char kSpecial[] = "special";

// Does something.  Pass kSpecial and it will do something special.
void DoSomething(const char* value);

// Defined in do_something.cc
void DoSomething(const char* value) {
  // Treat pointer equality to kSpecial as a sentinel.
  if (value == kSpecial) {
    // do something special
  } else {
    // do something boring
  }
}

Notice that this code compares the address of the first char in kSpecial to value as a kind of magic value for the function. You sometimes see code do this in an effort to short circuit a full string comparison.

This causes a subtle bug. The kSpecial array is constexpr which implies that it is static (with “internal” linkage). Even though we think of kSpecial as “a constant” – it really isn’t – it’s a whole family of constants, one per translation unit! Calls to DoSomething(kSpecial) look like they should do the same thing, but the function takes different code paths depending on where the call occurs.

Any code using a constant array defined in a header file, or code that takes the address of a constant defined in a header file, suffices for this kind of bug. This class of bug is usually seen with string constants, because they are the most common reason to define arrays in header files.

An Example of Undefined Behavior

Just tweak the above example, and move DoSomething into the header file as an inline function. Bingo: now we’ve got undefined behavior, or UB. The language requires all inline functions to be defined exactly the same way in every translation unit (source file) – this is part of the language’s “One Definition Rule.” This particular DoSomething implementation references a static variable, so every translation unit actually defines DoSomething differently, hence the undefined behavior.

Unrelated changes to program code and compilers can change inlining decisions, which can cause undefined behavior like this to change from benign behavior to bug.

Does this Cause Problems in Practice?

Yes. In one real-life bug we’ve encountered, the compiler was able to determine that in a particular translation unit (source file), a large static const array defined in a header file was only partially used. Rather than emit the entire array, it optimized away the parts it knew weren’t used. One of the ways the array was partially used was through an inline function declared in a header.

The trouble is, the array was used by other translation units in such a way that the static const array was fully used. For those translation units, the compiler generated a version of the inline function that used the full array.

Then the linker came along. The linker assumed that all instances of the inline function were the same, because the One Definition Rule said they had to be. And it discarded all but one copy of the function - and that was the copy with the partially-optimized array.

This kind of bug is possible when code uses a variable in a way that requires its address to be known. The technical term for this is “ODR used”. It is difficult to prevent ODR use of variables in modern C++ programs, particularly if those values are passed to template functions (as was the case in the above example).

These bugs do happen and are not easily caught in tests or code review. It pays to stick to safe idioms when defining constants.

Other Common Mistakes

Mistake #1: the Non-Constant Constant

Seen most often with pointers:

const char* kStr = ...;
const Thing* kFoo = ...;

The kFoo above is a pointer to const, but the pointer itself is not a constant. You can assign to it, set it null, etc.

// Corrected.
const Thing* const kFoo = ...;
// This works too.
constexpr const Thing* kFoo = ...;

Mistake #2: the Non-Constant MyString()

Consider this code:

inline absl::string_view MyString() {
  return "Hello";  // may return a different value with every call
}

The address of a string literal constant is allowed to change every time it is evaluated¹, so the above is subtly wrong because it returns a string_view that might have a different .data() value for each call. While in many cases this won’t be a problem, it can lead to the bug described above.

Making the MyString() constexpr does not fix the issue, because the language standard does not say it does². One way to look at this is that a constexpr function is just an inline function that is allowed to execute at compile time when initializing constant values. At run time it is not different from an inline function.

constexpr absl::string_view MyString() {
  return "Hello";  // may return a different value with every call
}

To avoid the bug, use a static constexpr variable in a function instead:

inline absl::string_view MyString() {
  static constexpr char kHello[] = "Hello";
  return kHello;
}

Rule of thumb: if your “constant” is an array type, store it in a function local static before returning it. This fixes its address.

Mistake #3: Non-Portable Code

For the extern const variables declared in header files the following approach to defining their values is valid according to the standard C++, and is generally preferable to C++20’s constinit (or the older ABSL_CONST_INIT), but runs afoul of a bug with at least one common compiler:

// Defined in foo.cc -- valid C++, but not supported by MSVC 19 by default.
constexpr absl::string_view kOtherBufferName = "other example";

Unfortunately, MSVC++19 incorrectly gives a C2370 error for this code unless the /Zc:externConstexpr option is used. If code needs to compile with MSVC++19 and cannot rely on /Zc:externConstexpr, as a workaround you can provide its value to other files through functions instead of as a global variable.

Mistake #4: Improperly Initialized Constants

The style guide has some detailed rules intended to keep us safe from common problems related to run-time initialization of static and global variables. The root issue arises when the initialization of global variable X references another global variable Y. How can we be sure Y itself doesn’t somehow depend on the value of X? Cyclic initialization dependencies can easily happen with global variables, especially with those we think of as constants.

This is a pretty thorny area of the language in its own right. The style guide is an authoritative reference.

Consider the above links required reading. With a focus on initialization of constants, the phases of initialization can be explained as:

Zero initialization. This is what initializes otherwise uninitialized static variables to the “zero” value for the type (e.g. 0, 0.0, '\0', null, etc.).
```
const int kZero;  // this will be zero-initialized to 0
const int kLotsOfZeroes[5000];  // so will all of these
```
Note that relying on zero initialization is fairly popular in C code but is fairly rare and niche in C++. It is generally clearer to assign variables explicit values, even if the value is zero, which brings us to…
Constant initialization.
```
const int kZero = 0;  // this will be constant-initialized to 0
const int kOne = 1;   // this will be constant-initialized to 1
```
Both “constant initialization” and “zero initialization” are called “static initialization” in the C++ language standard. Both are always safe.
Dynamic initialization.
```
// This will be dynamically initialized at run-time to
// whatever ArbitraryFunction returns.
const int kArbitrary = ArbitraryFunction();
```
Dynamic initialization is where most problems happen. The style guide explains why at https://google.github.io/styleguide/cppguide.html#Static_and_Global_Variables.

Note that documents like the Google C++ style guide have historically included dynamic initialization in the broad category of “static initialization.” The word “static” applies to a few different concepts in C++, which can be confusing. “Static initialization” can mean “initialization of static variables,” which can include run-time computation (dynamic initialization). The language standard uses the term “static initialization” in a different, narrower, sense: initialization that is done statically or at compile-time.

Initialization Cheat Sheet

Here is a super-quick constant initialization cheat sheet (not in header files):

constexpr guarantees safe constant initialization as well as safe (trivial) destruction. Any constexpr variable is entirely fine when defined in a .cc file, but is problematic in header files for reasons explained earlier.
constinit (ABSL_CONST_INIT prior to C++20) guarantees safe constant initialization. Unlike constexpr, it does not actually make the variable const, nor does it ensure the destructor is trivial, so care must still be taken when declaring static variables with it. See again https://google.github.io/styleguide/cppguide.html#Static_and_Global_Variables.
Otherwise, you’re most likely best off using a static variable within a function and returning it. See the “ordinary function” example shown earlier.

Conclusion

The inline variable from C++17 can’t come soon enough. Until then all we can do is use the safe idioms that steer us clear of the rough edges.

We conclude that string literals are not required to evaluate to the same object from the following language in [lex.string] in the C++17 language standard. Equivalent language is also present in C++11 and C++14. ↩
There is no language in [lex.string] describing different behavior in a constexpr context. ↩

Tip of the Week #5: Disappearing Act

2020-06-01T00:00:00-04:00

Originally posted as TotW #5 on June 26, 2012

Updated 2020-06-01

Quicklink: abseil.io/tips/5

“Don’t know what you got till it’s gone.” –Cinderella

Sometimes, in order to use a C++ library the right way, you need to understand both the library and the language. So … what’s wrong with the following?

// DON’T DO THIS
std::string s1, s2;
...
const char* p1 = (s1 + s2).c_str();             // Avoid!
const char* p2 = absl::StrCat(s1, s2).c_str();  // Avoid!

Option 1: Finish Using the Temporary Object Before the End of the full-expression:

// Safe (albeit a silly example):
size_t len1 = strlen((s1 + s2).c_str());
size_t len2 = strlen(absl::StrCat(s1, s2).c_str());

Option 2: Store the Temporary Object.

// Safe (and more efficient than you might think):
std::string tmp_1 = s1 + s2;
std::string tmp_2 = absl::StrCat(s1, s2);
// tmp_1.c_str() and tmp_2.c_str() are safe.

Option 3: Store a Reference to the Temporary Object.

// Equally safe:
const std::string& tmp_1 = s1 + s2;
const std::string& tmp_2 = absl::StrCat(s1, s2);
// tmp_1.c_str() and tmp_2.c_str() are safe.
// The following behavior is dangerously subtle:
// If the compiler can see you’re storing a reference to a
// temporary object’s internals, it will keep the whole
// temporary object alive.
// struct Person { string name; ... }
// GeneratePerson() returns an object; GeneratePerson().name
// is clearly a sub-object:
const std::string& person_name = GeneratePerson().name; // safe
// If the compiler can’t tell, you’re at risk.
// class DiceSeries_DiceRoll { `const string&` nickname() ... }
// GenerateDiceRoll() returns an object; the compiler can’t tell
// if GenerateDiceRoll().nickname() is a subobject.
// The following may store a dangling reference:
const std::string& nickname = GenerateDiceRoll().nickname(); // BAD!

Option 4: Design your functions so they don’t return objects???

Tip of the Week #177: Assignability vs. Data Member Types

2020-04-06T00:00:00-04:00

Originally posted as TotW #177 on April 6, 2020

By Titus Winters

Updated 2020-04-06

Quicklink: abseil.io/tips/177

When implementing a type, decide on type design first. Prioritize API over implementation details. One common example of this is the tradeoff between assignability of a type vs. qualifiers for data members.

Deciding how to represent data members

Imagine you are writing a City class, and discussing how to represent its member variables. You know that it is short-lived, representing the city as a snapshot in time, so things like population, name, and mayor could conceivably be const - we aren’t going to use the same object in a given program for years and years, so we don’t need to account for changes in population, new census results, or elections.

Should we have members like this?

 private:
  const std::string city_name_;
  const Person mayor_;
  const int64_t population_;

Why or why not?

The common suggestion for “Yes, make those const” hinges on the idea “Well, those values aren’t going to change for a given City, so since everything that can be const should be const, make them const.” That will make it easier for maintainers of the class to avoid accidentally modifying those fields.

This misses a critically important concern: what sort of type is City? Is this a value? Or a bundle of business logic? Is it expected to be copyable, move-only, or non-copyable? The set of operations you can write efficiently for City (as a whole) may be impacted by the question of whether a single member is made const, and that is often a bad tradeoff.

Specifically, if your class has const members, it cannot be assigned to (whether by copy-assignment or move-assignment). The language understands this: if your type has a const member, copy-assignment and move-assignment operators will not be synthesized. You can still copy (or move) construct such an object, but you cannot change it in any way after construction (even “just” to copy from another object of the same type). Even if you write your own assignment operators, you’ll quickly find that you (obviously) can’t overwrite these const members.

So it is possible that the question becomes “Which should we prefer: const members or assignment operations?” However, even that is misleading, because both are answered by the one important question, “What sort of type is City?” If it is intended to be a value type, that specifies the API (including assignment operations), and API trumps implementation concern in general.

It is important for those API design decisions to take priority over implementation-detail choices: in the general case, there are more engineers affected by the API of a type than implementation of a type. That is, there are more users of a type than maintainers of that type, so priority should go to design choices that affect the user above the implementer. Even if you think the type will never be used by anyone outside of the team that is maintaining it, software engineering is about interface design and abstraction - we should be prioritizing good interfaces.

Reference Members

The same reasoning applies to storing references as data members. Even if we know that the member must be non-null, it is still usually preferable to store T* for value types, because references are not rebindable. That is, we cannot re-point a T& - any modifications of such a member are modifying the underlying T.

Consider the implementation of std::vector<T>. There will almost certainly be a T* data member in any std::vector implementation, pointing to the allocation. We know from the specification of std::vector that such an allocation must usually be valid (except possibly for empty vectors). An implementation that always has an allocation could make that T&, right? (Yes, I’m ignoring arrays and offsets here.)

Clearly not. std::vector is a value type, it is copyable and assignable. If the allocation was stored with a reference-to-the-first-member instead of pointer-to-the-first-member, we wouldn’t be able to move-assign the storage, and it’s unclear how we’d update data when resizing normally. Our clever way of telling other maintainers “This value is non-null” would be getting in the way of providing users the desired API. Hopefully it is clear that this is the wrong tradeoff.

Non-copyable / assignable types

Of course, if your choices about type design suggest that City (or whatever type you are thinking about) should be non-copyable, that leaves far fewer constraints on your implementation. It isn’t right or wrong for a class to hold const or reference members, it’s only a concern when those implementation decisions are constraining or corrupting the interface presented by that class. If you’ve already made a thoughtful and conscious decision that your type need not be copyable, it’s very reasonable for you to make different choices about how to represent the data members of the class. (But see Tip #116 for some more thoughts and pitfalls around argument lifetime and reference storage).

The Unusual Case: immutable types

There is one useful-but-unusual design that may mandate const members: intentionally immutable types. Instances of such a type are immutable after construction: no mutating methods, no assignment operators. These are fairly rare, but can sometimes be useful. In particular, such a type is inherently thread-safe because there are no mutating operations. Objects of such a type can be freely shared among threads with no concern about data races or synchronization. However, in exchange these objects may have significant run-time overhead stemming from the need to copy them constantly. The immutability even prevents these objects from being efficiently moved.

It is almost always preferable to design your type to be mutable but still thread-compatible, rather than relying on thread-safety-via-immutability. Users of your type are usually in a better position to judge the benefits of mutability case-by-case. Don’t force them to work around unusual design choices without very strong evidence showing why your use case is unusual.

Recommendations

Decide on the design of your type before considering implementation details.
Value types are common and recommended. So are business-logic types, which are often non-copyable.
Immutable types are sometimes useful, but the cases where they are justified are fairly rare.
Prioritize API design and the needs of users over the (usually smaller) concerns of maintainers.
Avoid const and reference data members when building value types or move-only types.

Tip of the Week #176: Prefer Return Values to Output Parameters

2020-04-06T00:00:00-04:00

Originally posted as TotW #176 on March 12, 2020

By Etienne Dechamps

Updated 2020-04-06

Quicklink: abseil.io/tips/176

The problem

Consider the following:

// Extracts the foo spec and the bar spec from the provided doodad.
// Returns false if the input is invalid.
bool ExtractSpecs(Doodad doodad, FooSpec* foo_spec, BarSpec* bar_spec);

Using (or implementing) this function correctly requires the developer to ask themselves a surprising number of questions:

Are foo_spec and bar_spec out or in/out parameters?
What happens to pre-existing data in foo_spec or bar_spec? Is it appended to? Is it overwritten? Does it make the function CHECK-fail? Does it make it return false? Is it undefined behavior?
Can foo_spec be null? Can bar_spec? If they cannot, does a null pointer make the function CHECK-fail? Does it make it return false? Is it undefined behavior?
What are the lifetime requirements on foo_spec and bar_spec? In other words, do they need to outlive the function call?
If false is returned, what happens to foo_spec and bar_spec? Are they guaranteed to be unchanged? Are they “reset” in some way? Is it unspecified?

One cannot answer any of these questions from the function signature alone, and cannot rely on the C++ compiler to enforce these contracts. Function comments can help, but often don’t. This function’s documentation, for example, is silent on most of these issues, and is also ambiguous about what “input” means. Does it refer only to doodad, or to the other parameters too?

Furthermore, this approach inflicts boilerplate on every callsite: the caller has to allocate FooSpec and BarSpec objects in advance in order to call the function.

In this case, there’s a simple way to eliminate the boilerplate and encode the contracts in a way that the compiler can enforce.

The solution

Here’s how to make all these questions moot:

struct ExtractSpecsResult {
  FooSpec foo_spec;
  BarSpec bar_spec;
};
// Extracts the foo spec and the bar spec from the provided doodad.
// Returns nullopt if the input is invalid.
std::optional<ExtractSpecsResult> ExtractSpecs(Doodad doodad);

This new API is semantically the same, but it is now much harder to misuse:

It is clearer what the inputs and outputs are.
There are no questions about pre-existing data in foo_spec and bar_spec because they are created from scratch by the function.
There are no questions about null pointers because there are no pointers.
There are no questions about lifetimes because everything is passed and returned by value.
There are no questions about what happens to foo_spec and bar_spec in case of failure because they cannot even be accessed if nullopt is returned.

This in turn reduces the likelihood of bugs and reduces cognitive load on the developer.

There are other benefits, too. For example the function is more easily composable; that is, it can easily be used as part of a wider expression, e.g. SomeFunction(ExtractSpecs(...)).

Caveats

This approach doesn’t work for in-out parameters.
- In some cases it is possible to use a variant of this approach whereby the parameter is taken by value, mutated, and then returned by value. Whether this is a good idea or not depends on how the function is used and whether the value is efficiently movable (Tip #117).
This approach doesn’t let the caller easily customize the creation of the returned objects.
- For example, if FooSpec and BarSpec are protos, the out-parameter approach lets the caller allocate those protos on a particular arena if they wish. In the return-values approach, the arena would need to be specified as an additional parameter or the callee would have to know it already.
Performance might differ depending on which approach you choose and the specifics of the situation.
- In some cases returning values can be less efficient, for example if it leads to repeated allocation in a loop.
- In other cases, returning values might be more efficient than you think, thanks to (N)RVO (Tip #11, Tip #166). It might even be more efficient than output parameters because the optimizer doesn’t have to worry about aliasing.
- As always, avoid premature optimization. Choose the API that makes the most sense, and only cater to performance if you have evidence that it makes a difference.

Recommendations

Prefer return values to output parameters whenever possible. This is consistent with the style guide.
Use generic wrappers like std::optional to represent a missing return value. Consider returning std::variantif you need a more flexible representation with multiple alternatives.
Use a struct to return multiple values from a function.
- Feel free to write a new struct specifically to represent the return value of the function if that makes sense.
- Resist the temptation to use std::pair or std::tuple for this purpose.

Tip of the Week #175: Changes to Literal Constants in C++14 and C++17.

2020-04-06T00:00:00-04:00

Originally posted as TotW #175 on January 30, 2020

By James Dennett

Updated 2020-04-06

Quicklink: abseil.io/tips/175

“The only thing that stays the same is change” – Melissa Etheridge

Overview

C++ now has some features that can make numeric literals more readable.

Integer literals can now be written in binary (0b00101010) as well as the decimal (42), hex (0x2A), and octal (052) formats that have been supported since the dawn of time.

Single quotes (') serve as digit separators, and can be used to group digits in numeric literals of any base (0xDEAD'C0DE).

Floating point literals can be specified in hex (0x2A0p-4).

Binary Literals

Binary literals such as 0b1110'0000 can be more readable than hex (the next best option) when manipulating bit sets or working with low-level protocols.

Digit Separators

C++14 allows a single quote (') to be used to group digits in a numeric literal. These digit separators have no effect on the value of the constant: their only function is to help readers. They can make it easy to see at a glance how many digits are present, and know that none are missing. For example, 1'000'000'000 is more obviously one billion than is 1000000000 (and unlike 1e9 it is an integer, not a floating point value).

There are no restrictions on the number of digits per group, not even a requirement of consistency within a literal: 0b1'001'0001 is a valid way to write the number 145 (and may even make sense for a byte that is interpreted as three separate fields).

Hex Floating Point Literals

While most decimal floating point literals are not exactly representable in the binary floating point formats used by most computers, hex floating literals do map directly to bit patterns in floating point so long as enough bits are available. This avoids rounding errors when converting a literal to floating point format (though truncation errors are still possible if too many hex digits are present).

Hex floating point literals are indicated by writing a p (or P) to separate the significand from the exponent—where decimal floating point literals would use e (or E). For example, 0x2Ap12 is another way to write the value 0x2A << 12, i.e., 0x2A000, except that is a floating point value, not an integer. As a result, our style guide requires it to be written as 0x2A.0p12 to be explicit that it is a floating point value and not just another way to write an integer.

The exponent is always written in decimal, denotes a power of 2, and may be negative: 0x1p-10 is (exactly) 1.0/1024.

Recommendations

Binary literals should be used sparingly, for code which cares about bit manipulation.
Consider using digit separators when a numeric literal is too long to take in at a single glance.
When using digit separators, use conventional group sizes:
- For decimal, use groups of three digits unless there is a conflicting convention (as with some currencies, for example).
- For binary, prefer groups of four bits (nibble) or eight bits (octet/byte) unless the digits have a more semantically significant grouping.
- For hex, use groups of 2, 4 or 8 hex digits.

Tip of the Week #173: Wrapping Arguments in Option Structs

2020-04-06T00:00:00-04:00

Originally posted as TotW #173 on December 19, 2019

By John Bandela

Updated 2020-04-06

Quicklink: abseil.io/tips/173

It came without packages, boxes or bags. And he puzzled and puzzled ‘till his puzzler was sore.

-Dr. Seuss

Designated Initializers

Designated initializers are a C++20 feature that is available in most compilers today. Designated initializers make using option structs easier and safer since we can construct the options object in the call to the function. This results in shorter code and avoids a lot of temporary lifetime issues with option structs.

struct PrintDoubleOptions {
  absl::string_view prefix = "";
  int precision = 8;
  char thousands_separator = ',';
  char decimal_separator = '.';
  bool scientific = false;
};

void PrintDouble(double value,
                 const PrintDoubleOptions& options = PrintDoubleOptions{});

std::string name = "my_value";
PrintDouble(5.0, {.prefix = absl::StrCat(name, "="), .scientific = true});

For more background on why option structs are helpful and the potential pitfalls in using them that designated initializers help avoid, read on.

The Problem of Passing Many Arguments

Functions that take many arguments can be confusing. To illustrate, let us consider this function for printing out a floating point value.

void PrintDouble(double value, absl::string_view prefix,  int precision,
                 char thousands_separator, char decimal_separator,
                 bool scientific);

This function provides us a lot of flexibility because it takes so many options.

PrintDouble(5.0, "my_value=", 2, ',', '.', false);

The above code will print out: “my_value=5.00”.

However, it is hard to read this code and know to which parameter each argument corresponds. For instance, here we have inadvertently mixed up the order of our precision and thousands_separator.

PrintDouble(5.0, "my_value=", ',', '.', 2, false);

Historically, we have used argument comments to clarify argument meanings at call sites to reduce this sort of ambiguity. The addition of argument comments to the above example would allow ClangTidy to detect the error:

PrintDouble(5.0, "my_value=",
            /*precision=*/2,
            /*thousands_separator=*/',',
            /*decimal_separator=*/'.',
            /*scientific=*/false);

However, argument comments still have several drawbacks:

No enforcement: ClangTidy warnings are not caught at buildtime. Subtle errors (e.g. a missing = sign) can disable the check entirely with no warning, providing a false sense of security.
Availability: not all projects and platforms support ClangTidy.

No matter whether your arguments are commented or not, specifying lots of options can also be tedious. Many times there are sensible defaults for the options. To address this concern, we can add defaults to the parameters.

void PrintDouble(double value, absl::string_view prefix = "", int precision = 8,
                 char thousands_separator = ',', char decimal_separator = '.',
                 bool scientific = false);

Now we can call PrintDouble with less boilerplate.

PrintDouble(5.0, "my_value=");

However, if we want to specify a non-default argument for scientific, we would still be forced to specify values for all of the parameters that come before it:

PrintDouble(5.0, "my_value=",
            /*precision=*/8,              // unchanged from default
            /*thousands_separator=*/',',  // unchanged from default
            /*decimal_separator=*/'.',    // unchanged from default
            /*scientific=*/true);

We can address all of these issues by grouping all of the options together in an option struct:

struct PrintDoubleOptions {
  absl::string_view prefix = "";
  int precision = 8;
  char thousands_separator = ',';
  char decimal_separator = '.';
  bool scientific = false;
};

void PrintDouble(double value,
                 const PrintDoubleOptions& options = PrintDoubleOptions{});

Now we can have names for our values, as well as flexibly use defaults.

PrintDoubleOptions options;
options.prefix = "my_value=";
PrintDouble(5.0, options);

Caveats

There are some issues with this solution, though. First is that we now have some extra boilerplate in passing options, though that’s often a minor cost compared to the benefits. But there are a few more things to consider.

Lifetime of Temporaries

For example, when we took all the options as parameters the following code was safe:

std::string name = "my_value";
PrintDouble(5.0, absl::StrCat(name, "="));

In the code above, we are creating a temporary string and binding a string_view to that. The temporary lifetime is the duration of the function call so we are safe, but using an options struct in the same manner, results in a dangling string_view.

std::string name = "my_value";
PrintDoubleOptions options;
options.prefix = absl::StrCat(name, "=");
PrintDouble(5.0, options);

There are two ways we can fix this. The first is to simply change the type of prefix from string_view to string. The downside of doing this is that now the option struct is less efficient than directly passing the arguments. The other way that we can fix this is to add setter member functions.

class PrintDoubleOptions {
 public:
  PrintDoubleOptions& set_prefix(absl::string_view prefix) {
    prefix_ = prefix;
    return *this;
  }

  absl::string_view prefix() const { return prefix_; }

  // Setters and getters for the other member variables.

 private:
  absl::string_view prefix_ = "";
  int precision_ = 8;
  char thousands_separator_ = ',';
  char decimal_separator_ = '.';
  bool scientific_ = false;
};

This can then be used to set the variables in the call.

std::string name = "my_value";
PrintDouble(5.0, PrintDoubleOptions{}.set_prefix(absl::StrCat(name, "=")));

As you can see, the cost is that our option struct became a more complicated class with a lot more boilerplate.

The simpler alternative is to use designated initializers as shown at the top.

Type Deduction

Designated initializers are often used with copy list initialization, where a brace-enclosed list of initializers doesn’t have an explicit type specified nearby.

When directly passing options to a function with the option struct as a parameter, the braced list is immediately turned into an argument of the specific type. So, this works well:

PrintDouble(5.0, {.scientific=true})

But in a function like std::make_unique that uses “perfect forwarding”, where the type of the parameter must be deduced, the braced list’s type cannot be found. So, this does not work:

class DoublePrinter {
  explicit DoublePrinter(const PrintDoubleOptions& options);
  ...
};

auto printer1 = std::make_unique<DoublePrinter>({.scientific=true});

The caller must name the option struct’s type, or there must be a helper factory function that does so.

class DoublePrinter {
  static std::unique_ptr<DoublePrinter> Make(const PrintDoubleOptions& options);
  explicit DoublePrinter(const PrintDoubleOptions& options);
};

auto printer1 = std::make_unique<DoublePrinter>(
    PrintDoubleOptions{.scientific=true});
auto printer2 = DoublePrinter::Make({.scientific=true});

Default Values in a Nested Type

If an option struct is associated with a class, it’s often reasonable to nest the struct within the class.

class DoublePrinter {
  struct Options {
    int precision = 8;
    ...
  };

  explicit DoublePrinter(const Options& options);

  static std::unique_ptr<DoublePrinter> Make(const Options& options);
};

But then if you need to allow the option struct to be skipped entirely, such as when it’s being added to an existing class, and the nested struct has a default member initializer (the = 8 after the field name precision, for example), you cannot have a default argument whose value leaves the field implicit.

In this case, provide another overload instead of using a default argument.

class DoublePrinter {
  struct Options {
    int precision = 8;
    ...
  };

  static std::unique_ptr<DoublePrinter> Make() { return Make({}); }
  static std::unique_ptr<DoublePrinter> Make(const Options& options);

  explicit DoublePrinter() : DoublePrinter({}) {}
  explicit DoublePrinter(const Options& options);

  // Cannot do this:
  //   static std::unique_ptr<DoublePrinter> Make(const Options& options = {});
  //   explicit DoublePrinter(const Options& options = {});
};

Conclusions

For functions which take multiple arguments which may be confused by the caller or where you want to specify default arguments without having to worry about the order, strongly consider using option structs to increase both convenience and code clarity.
When calling functions that take option structs, using designated initializers can result in shorter code as well as potentially avoiding temporary lifetime issues.
Designated initializers by virtue of their conciseness and clarity further tip the balance towards preferring functions that take option structs over those that have many parameters.

Tip of the Week #172: Designated Initializers

2020-04-06T00:00:00-04:00

Originally posted as TotW #172 on December 11, 2019

By Aaron Jacobs

Updated 2020-04-06

Quicklink: abseil.io/tips/172

Designated initializers are a syntax in the C++20 standard for specifying the contents of a struct in a compact yet readable and maintainable manner. Instead of the repetitive

struct Point {
  double x;
  double y;
  double z;
};

Point point;
point.x = 3.0;
point.y = 4.0;
point.z = 5.0;

one can use designated initializers to write

Point point = {
    .x = 3.0,
    .y = 4.0,
    .z = 5.0,
};

This is a little less repetitive, but more importantly, can be used in more contexts. For example, it means the struct can be made const without resorting to awkward workarounds:

// Make it clear to the reader (of the potentially complicated larger piece of
// code) that this struct will never change.
const Point character_position = { .x = 3.0 };

Or can be used directly in a function call without introducing an additional identifier into the scope:

std::vector<Point> points;
[...]
points.push_back(Point{.x = 3.0, .y = 3.0});
points.push_back(Point{.x = 4.0, .y = 4.0});

Semantics

Designated initializers are a form of aggregate initialization, and so can be used only with aggregates. This means approximately “structs or classes with no user-provided constructors or virtual functions”, which in turn is approximately when we use struct (as opposed to class) in typical Google style.

The semantics of C++20 designated initializers are what you might expect given other C++ language features like member initialization lists in constructors. Explicitly mentioned fields are initialized, in order, with the expression provided, and it is permissible to leave out fields that you want to have “default” behavior for:

Point point = {
    .x = 1.0,
    // y will be 0.0
    .z = 2.0,
};

What does “default” mean above? Outside of special cases like unions the answer is:

If the struct definition contains a default member initializer (i.e. the field definition looks like std::string foo = "default value";) then that is used.
Otherwise the field is initialized as if with = {}. In practice this means that for plain old data types you get the zero value, and for more complicated classes you get a default-constructed instance.

This is typically the least surprising behavior. See the standard for details.

Some History and Language Trivia

Designated initializers have been a standard part of the C language since C99, and have been offered by compilers as a non-standard extension since before that. But until recently they were not part of C++: a notable example where C is not a subset of C++. For this reason the Google style guide used to say not to use them.

After two decades the situation has finally changed: designated initializers are now part of the C++20 standard.

The C++20 form of designated initializers has some restrictions compared to the C version:

C++20 requires fields to be listed in the designator in the same order as they are listed in the struct definition (so Point{.y = 1.0, .x = 2.0} is not legal). C does not require this.
C allows you to mix designated and non-designated initializers (Point{1.0, .z = 2.0}), but C++20 does not.
C supports a syntax for sparsely initializing arrays known as “array designators”. This is not part of C++20.

Tip of the Week #171: Avoid Sentinel Values

2020-04-06T00:00:00-04:00

Originally posted as TotW #171 on November 8, 2019

By Hyrum Wright

Updated 2020-04-06

Quicklink: abseil.io/tips/171

Sentinel values are values that have special meaning in a specific context. For example, consider the following API:

// Returns the account balance, or -5 if the account has been closed.
int AccountBalance();

Every value of int is documented to be a valid return value for AccountBalance, except for -5. Intuitively, this feels a bit odd: should callers only check against -5 specifically, or is any negative value a reliable “account closed” signal? What happens when the system supports negative balances and the API needs to be adjusted to return negative values?

Using sentinel values increases the complexity of the calling code. If the caller is rigorous, it explicitly checks against the sentinel value:

int balance = AccountBalance();
if (balance == -5) {
  LOG(ERROR) << "account closed";
  return;
}
// use `balance` here

Some callers may check against a broader range of values than is specified:

int balance = AccountBalance();
if (balance <= 0) {
  LOG(ERROR) << "where is my account?";
  return;
}
// use `balance` here

And some callers may just ignore the sentinel value altogether, assuming that it doesn’t actually occur in practice:

int balance = AccountBalance();
// use `balance` here

Problems with Sentinel Values

The above example illustrates some of the common problems with using sentinel values. Others include:

Different systems may use different sentinel values, such as a single negative value, all negative values, an infinite value, or any arbitrary value. The only way to communicate the special value is through documentation.
The sentinel values are still part of the type’s domain of valid values, so neither the caller nor the callee is forced by the type system to acknowledge that a value may be invalid. When code and comments disagree, both are usually wrong.
Sentinel values limit interface evolution, as the specific sentinel may someday be a valid value for use in that system.
One system’s sentinel value is another’s valid value, increasing cognitive overhead and code complexity when interfacing with multiple systems.

Forgetting to check for specified sentinel values is a common bug. In the best case, the use of an unchecked sentinel value will immediately crash the system during runtime. More frequently, an unchecked sentinel value may continue to propagate through the system, producing bad results as it goes.

Use `std::optional` Instead

Use std::optional to indicate unavailable or invalid information instead of using special values.

// Returns the account balance, or std::nullopt if the account has been closed.
std::optional<int> AccountBalance();

The caller of our new version of AccountBalance() now must explicitly look inside the returned value for a potential balance, signalling that the result might be invalid in the process. Barring additional documentation, the caller can assume that any valid int value can be returned from this function, without excluding specific sentinel values. This simplification clarifies the intent of calling code.

std::optional<int> balance = AccountBalance();

if (!balance.has_value()) {
  LOG(ERROR) << "Account doesn't exist";
  return;
}
// use `*balance` here

Next time you are tempted to use a sentinel value within your system, strongly consider using an appropriate std::optional instead.

Tip of the Week #163: Passing `std::optional` parameters

2020-04-06T00:00:00-04:00

Originally posted as TotW #163 on July 11, 2019

By Ian Eldred Pudney

Updated 2020-04-06

Quicklink: abseil.io/tips/163

Are nulls really a billion-dollar mistake?

The problem

Let’s say you need to implement a function with a parameter that may or may not exist. You might be tempted to use modern, fancy-schmancy std::optional for this. However, if the object is big enough that it should be passed by reference, std::optional is probably not what you want. Consider the following two declarations:

void MyFunc(const std::optional<Foo>& foo);  // May copy by value
void MyFunc(std::optional<const Foo&> foo);  // Doesn't compile

The first option probably doesn’t do what you want. If someone passes a std::optional<Foo> into MyFunc, it is passed by reference, but if someone passes a Foo into MyFunc (for example as a return value), the Foo will be copied by value into a temporary std::optional<Foo>, which will then be passed by reference into the function. If your goal was to avoid copying the Foo, you haven’t.

The second option would be great, but unfortunately is not supported by std::optional.

Recommendation

Avoid function parameters of the form const std::optional&.

If your object is small enough to not need pass-by-reference, you should take the object wrapped in an std::optional by value. For example:

void MyFunc(std::optional<int> bar);
void MyFunc(std::optional<absl::string_view> baz);

If you are intentionally making a copy of the argument, you should also accept the std::optional by value to make that clear:

void MyFunc(std::optional<Foo> foo);

Otherwise, skip the std::optional altogether.

You can pass it by absl::Nullable<const Foo*> and let nullptr indicate “does not exist.”

void MyFunc(absl::Nullable<const Foo*> foo);

This will be just as efficient as passing by const Foo&, but supports null values. See Tip #116 for more on when to use a pointer instead of a const reference.

Then what on Earth is `std::optional` for???

std::optional can be used if you own the thing that’s optional. For example, class members and function return values often work well with std::optional.

Exception

If you expect all callers of your function to already have a std::optional<Foo> and never pass in a Foo, then you may take a const std::optional<Foo>&. However, this is rare; it usually only occurs if your function is private within your own file/library.

What about `std::reference_wrapper`?

The documentation for std::optional points out that you can use a std::reference_wrapper to work around the fact that optional references aren’t supported:

void MyFunc(std::optional<std::reference_wrapper<const Foo>> foo);

However, we don’t recommend this:

std::reference_wrapper has surprisingly subtle semantics, making it difficult to understand and use safely. For instance, various utilities in the standard library special case it in ways that make it act differently from a normal value or reference.
std::optional<std::reference_wrapper<const Foo>> is cumbersome and verbose, compared to absl::Nullable<const Foo*>.

Announcing TCMalloc

2020-02-12T00:00:00-05:00

By Chris Kennelly, Google Software Engineer

We are happy to announce the arrival of TCMalloc, a fast memory allocator with useful profiling and introspection features.

The source code can be found on Github. This is a distinct repository, allowing Abseil to be used independently of TCMalloc. The library includes:

A lockfree, per-CPU cache implementation based on restartable sequences, available on modern Linux kernels, and a per-thread based fallback implementation.
A hugepage-aware backend that reduces TLB stalls by more densely populating hugepage subregions while reducing memory waste.
Optimizations leveraging modern C++ language features, including sized delete from C++14 and overaligned allocation from C++17.
Always-on, sampling-based heap profiling, allowing an application to obtain a heap profile without additional configuration. We also provide a “peak heap” profile, a snapshot of the application’s memory usage near its high watermark of usage.

This blog post covers getting started and a peek at one of our key optimization techniques: per-CPU caches. For a more detailed explanation of the design of TCMalloc and its features, take a look at our design note.

Getting Started

Adding TCMalloc to a binary involves specifying it as the malloc attribute of a Bazel cc_binary or cc_test rule. For example, with Abseil hello:

cc_binary(
  name = "hello_main",
  srcs = ["hello_main.cc"],
  deps = [
    ":hello",
  ],
  malloc = "@com_google_tcmalloc//tcmalloc",
)

We can also leverage the telemetry TCMalloc provides into our heap with MallocExtension.

#include <iostream>
#include “tcmalloc/malloc_extension.h”

int main(int argc, char** argv) { 
  absl::optional<size_t> heap_size =
    tcmalloc::MallocExtension::GetNumericProperty(
      "generic.current_allocated_bytes");
  if (heap_size.has_value()) {
    std::cout << "heap size = " << *heap_size << " bytes" << std::endl;
  }
}

MallocExtension is a separate library from TCMalloc, allowing it to be used when another malloc implementation is linked-in, for example, when using C++ sanitizers. The library is crafted so that although the telemetry and controls it provides will be inoperative, the code using it will still link and compile.

Key Optimizations: Per-CPU Caches

TCMalloc’s name comes from “thread caching” malloc. These caches allow us to allocate memory most of the time without taking locks, falling back to a more centralized cache when the thread cache is exhausted or full.

The cost of per-thread caches has increased as thread counts in applications have grown. Because we do not allow one thread to access another’s thread cache, to avoid locks, we need to carefully balance between ensuring a cache is large enough to hold needed objects and avoiding stranding memory in an idle cache. Objects held by idle threads are effectively wasted.

Our per-CPU caches take advantage of the fact that only one thread can run on a single core at any given time and that pre-emptions during the critical sequence of an allocation are relatively rare. These caches are based on the Linux kernel’s restartable sequences (RSEQ) feature, developed by Paul Turner and Andrew Hunter at Google and Mathieu Desnoyers at EfficiOS. Restartable sequences let us execute a region of code atomically and restart if we are interrupted by the kernel.

RSEQ allows us to share a cache across many interleaving threads of execution, improving the usefulness of the cache. When one thread is idle, another thread can still allocate from the core’s cache. Additionally, because the number of caches is bounded by the total number of cpus, we can allocate an array-based metadata structure for each cache. This allows us to reduce memory indirections, common in the linked-list data structures we use for our thread-cache implementation.

For systems without this feature, we fall back to a per-thread cache implementation.

More Information

For more information, consult the following documentation:

The TCMalloc Quickstart, which covers downloading, installing and running the TCMalloc code. contains updated links to all documentation
The TCMalloc Overview, for a basic overview of TCMalloc usage and configuration
The TCMalloc API Reference, for complete usage information for our implementations of C and C++ API endpoints such as malloc(), free(), and ::operator new
The TCMalloc Tuning Guide, for more complete information on selecting a proper configuration, and tuning certain aspects of TCMalloc.

Tip of the Week #108: Avoid `std::bind`

2019-12-19T00:00:00-05:00

Originally posted as TotW #108 on January 7, 2016

By Roman Perepelitsa

Updated 2020-08-19

Quicklink: abseil.io/tips/108

Avoid `std::bind`

This Tip summarizes reasons why you should stay away from std::bind() when writing code.

Using std::bind() correctly is hard. Let’s look at a few examples. Does this code look good to you?

void DoStuffAsync(std::function<void(absl::Status)> cb);

class MyClass {
  void Start() {
    DoStuffAsync(std::bind(&MyClass::OnDone, this));
  }
  void OnDone(absl::Status status);
};

Many seasoned C++ engineers have written code like this only to find that it doesn’t compile. It works with std::function<void()> but breaks when you add an extra parameter to MyClass::OnDone. What gives?

std::bind() doesn’t just bind the first N arguments like many C++ engineers expect (this is called partial function application). You must instead specify every argument, so the correct incantation of std::bind() is this:

std::bind(&MyClass::OnDone, this, std::placeholders::_1)

Ugh, that’s ugly. Is there a better way? Why yes, use std::bind_front() instead. (For code that cannot yet use C++20, there’s absl::bind_front.)

std::bind_front(&MyClass::OnDone, this)

Remember partial function application – the thing that std::bind() does not do? Well, std::bind_front() does exactly that: it binds the first N arguments and perfect-forwards the rest: std::bind_front(F, a, b)(x, y) evaluates to F(a, b, x, y).

Ahhh, sanity is restored. Want to see something truly terrifying now? What does this code do?

void DoStuffAsync(std::function<void(absl::Status)> cb);

class MyClass {
  void Start() {
    DoStuffAsync(std::bind(&MyClass::OnDone, this));
  }
  void OnDone();  // No absl::Status here.
};

OnDone() takes no parameters, the DoStuffAsync() callback should take a absl::Status. You might expect a compile error here but this code actually compiles without warnings, because std::bind over-aggressively bridges the gap. Potential errors from DoStuffAsync() get silently ignored.

Code like this can cause serious damage. Thinking that some IO has succeeded when it actually hasn’t can be devastating. Perhaps the author of MyClass didn’t realize that DoStuffAsync() may produce an error that needs to be handled. Or maybe DoStuffAsync() used to accept std::function<void()> and then its author decided to introduce the error mode and updated all callers that stopped compiling. Either way the bug made its way into production code.

std::bind() disables one of the basic compile time checks that we all rely on. The compiler usually tells you if the caller is passing more arguments than you expect, but not for std::bind(). Terrifying enough?

Another example. What do you think of this one?

void Process(std::unique_ptr<Request> req);

void ProcessAsync(std::unique_ptr<Request> req) {
  thread::DefaultQueue()->Add(
      ToCallback(std::bind(&MyClass::Process, this, std::move(req))));
}

Good old passing std::unique_ptr across async boundaries. Needless to say, std::bind() isn’t a solution – the code doesn’t compile, because std::bind() doesn’t move the bound move-only argument to the target function. Simply replacing std::bind() with std::bind_front() fixes it.

The next example regularly trips even C++ experts. See if you can find the problem.

// F must be callable without arguments.
template <class F>
void DoStuffAsync(F cb) {
  auto DoStuffAndNotify = [](F cb) {
    DoStuff();
    cb();
  };
  thread::DefaultQueue()->Schedule(std::bind(DoStuffAndNotify, cb));
}

class MyClass {
  void Start() {
    DoStuffAsync(std::bind(&MyClass::OnDone, this));
  }
  void OnDone();
};

This doesn’t compile because passing the result of std::bind() to another std::bind() is a special case. Normally, std::bind(F, arg)() evaluates to F(arg), except when arg is the result of another std::bind() call, in which case it evaluates to F(arg()). If arg is converted to std::function<void()>, the magic behavior is lost.

Applying std::bind() to a type you don’t control is always a bug. DoStuffAsync() shouldn’t apply std::bind() to the template argument. Either std::bind_front() or lambda would work fine.

The author of DoStuffAsync() might even have entirely green tests because they always pass a lambda or std::function as the argument but never the result of std::bind(). The author of MyClass will be puzzled when hitting on this bug.

Is this special case of std::bind() ever useful? Not really. It only gets in the way. If you are composing functions by writing nested std::bind() calls, you really should write a lambda or a named function instead.

Hopefully you are convinced that it’s easy to go wrong with std::bind(). Runtime and compile time traps are easy to fall into both for newcomers and C++ experts. Now I’ll try to show that even when std::bind() is used correctly, there is usually another option that is more readable.

Calls to std::bind() without placeholders are better off as lambdas.

std::bind(&MyClass::OnDone, this)

[this]() { OnDone(); }

Calls to std::bind() that perform partial application are better off as std::bind_front(). The more placeholders you have, the more obvious it gets.

std::bind(&MyClass::OnDone, this, std::placeholders::_1)

std::bind_front(&MyClass::OnDone, this)

(Whether to use std::bind_front() or a lambda when performing partial function application is a judgement call; use your discretion.)

This covers 99% of all calls std::bind(). The remaining calls do something fancy:

Ignore some of the arguments: std::bind(F, _2).
Use the same argument more than once: std::bind(F, _1, _1).
Bind arguments at the end: std::bind(F, _1, 42).
Change argument order: std::bind(F, _2, _1).
Use function composition: std::bind(F, std::bind(G)).

These advanced uses may have their place. Before resorting to them, consider all known issues with std::bind() and ask yourself whether the potential savings of a few characters or lines of code are worth it.

Conclusion

Avoid std::bind. Use a lambda or std::bind_front instead.

Tip of the Week #166: When a Copy is not a Copy

2019-12-12T00:00:00-05:00

Originally posted as TotW #166 on August 28, 2019

By Richard Smith

Updated 2020-04-06

Quicklink: abseil.io/tips/166

“Entia non sunt multiplicanda praeter necessitatem.” (“Entities should not be multiplied without necessity”) – William of Ockham

“If you don’t know where you’re going, you’re probably going wrong.” – Terry Pratchett

Overview

Starting in C++17, objects are created “in place” when possible.

class BigExpensiveThing {
 public:
  static BigExpensiveThing Make() {
    // ...
    return BigExpensiveThing();
  }
  // ...
 private:
  BigExpensiveThing();
  std::array<OtherThing, 12345> data_;
};

BigExpensiveThing MakeAThing() {
  return BigExpensiveThing::Make();
}

void UseTheThing() {
  BigExpensiveThing thing = MakeAThing();
  // ...
}

How many times does this copy or move a BigExpensiveThing?

Prior to C++17, the answer was up to three: one for each return statement, and one more when initializing thing. This makes some sense: each function potentially puts the BigExpensiveThing in a different place, so a move may be needed in order to put the value where the ultimate caller wants it. In practice, however, the object was always constructed “in place” in the variable thing, with no moves being performed, and the C++ language rules permitted these move operations to be “elided” to facilitate this optimization.

Since C++17, this code is guaranteed to perform zero copies or moves. In fact, the above code is valid even if BigExpensiveThing is not moveable. The constructor call in BigExpensiveThing::Make directly constructs the local variable thing in UseTheThing.

So what’s going on?

When the compiler sees an expression like BigExpensiveThing(), it does not immediately create a temporary object. Instead, it treats that expression as instructions for how to initialize some eventual object, but defers creating (formally, “materializing”) a temporary object for as long as possible.

Generally, creation of an object is deferred until the object is given a name. The named object (thing in the above example) is directly initialized using the instructions found by evaluating the initializer. If the name is a reference, a temporary object will be materialized to hold the value.

As a consequence, objects are constructed directly in the right place, instead of being constructed somewhere else and then copied. This behavior is sometimes referred to as “guaranteed copy elision”, but that’s inaccurate: there was never a copy in the first place.

All you need to know is: objects are not copied until after they are first given a name. There is no extra cost in returning by value.

(Even after being given a name, local variables might still not be copied when returned from a function, due to the Named Return Value Optimization. See Tip 11 for details.)

Gritty Details: When Unnamed Objects Are Copied

There are two corner cases where a use of an object with no name results in a copy anyway:

Constructing base classes: in the base class initializer list of a constructor, copies will be made even when constructing from an unnamed expression of the base class type. This is because classes may have a somewhat different layout and representation when used as base classes (due to virtual base classes and vpointer values), so initializing base classes directly may not result in a correct representation.

class DerivedThing : public BigExpensiveThing {
 public:
  DerivedThing() : BigExpensiveThing(MakeAThing()) {}  // might copy data_
};

Passing or returning small trivial objects: when a sufficiently small object that is trivially copyable is passed to or returned from a function, it may be passed in registers, and hence may have a different address before and after it is passed.

struct Strange {
  int n;
  int *p = &n;
};
void f(Strange s) {
  CHECK(s.p == &s.n);  // might fail
}
void g() { f(Strange{0}); }

Gritty Details: Value Category

There are two flavors of expression in C++:

Those that produce a value, such as 1, or MakeAThing() – expressions that you might consider to have a non-reference type.
Those that produce the location of some existing object, such as s or thing.data_[5] – expressions that you might consider to have a reference type.

This division is called the “value category”; the former are prvalues and the latter are glvalues. When we talked about objects without a name above, what we were really referring to was prvalue expressions.

All prvalue expressions are evaluated in some context that determines where they put their value, and the execution of the prvalue expression initializes that location with its value.

For example, in

  BigExpensiveThing thing = MakeAThing();

the prvalue expression MakeAThing() is evaluated as the initializer of the thing variable, so MakeAThing() will directly initialize thing. The constructor passes a pointer to thing into MakeAThing(), and the return statement in MakeAThing() initializes whatever the pointer points to. Similarly, in

  return BigExpensiveThing();

the compiler has a pointer to an object to initialize, and initializes that object directly by calling the BigExpensiveThing constructor.

Tip of the Week #146: Default vs Value Initialization

2019-12-12T00:00:00-05:00

Originally posted as TotW #146 on April 19, 2018

By Dominic Hamon

Updated 2020-04-06

Quicklink: abseil.io/tips/146

“The road to success is always under construction.” – Lily Tomlin

TL;DR

For safety and readability, you should assume scalar objects are not initialized to a reasonable value until they have been explicitly set to a value. Use of initializers can ensure that scalar values are initialized to safe values.

Introduction

When objects are created, they may be initialized or uninitialized. Uninitialized objects are not safe to read, but understanding when the object is uninitialized is not trivial.

The first thing to understand is if the type under construction is scalar, aggregate, or some other type. A scalar type can be thought of as a simple type: an integral or floating point arithmetic object; a pointer; an enum; a pointer-to-member; nullptr_t. An aggregate type is an array or a class with nothing virtual, no non-public fields or bases, and no constructor declarations.

Another factor affecting whether an instance has been initialized to a value that is safe to read is whether it has an explicit initializer. That is, the object name in the statement is followed by (), {}, or = {}.

As these rules are not intuitive, the easiest rule to remember to ensure an object is initialized is to provide an initializer. This is called value-initialization and is distinct from default-initialization, which is what the compiler will perform otherwise for scalar and aggregate types.

User-Provided Constructors

If a type is defined with a user-defined constructor, it is not an aggregate type, and initialization gets much simpler with both value- and default-initialization invoking the constructor:

struct Foo {
  Foo() : v() {}

  int v;
  std::string s;
};

int main() {
  Foo default_foo;
  Foo value_foo = {};
  ...
}

The = {} triggers value-initialization of value_foo, which calls Foo’s default constructor. After, v is safe to read, because the constructor’s initializer list value-initializes it. In fact, as v does not have a class type, this is a special case of value-initialization called zero-initialization and value_foo.v will have the value 0.

Similarly, while default_foo is default-initialized, it calls the same constructor, so default_foo.v is also zero-initialized and is safe to read.

Note that Foo::s has a user-provided constructor, so it is value-initialized in either case, and safe to read.

User-Declared vs User-Provided Constructors

It is possible for the user to declare a constructor while asking the compiler to provide the definition via =default. For example:

struct Foo {
  Foo() = default; // "User-declared", NOT "user-provided".

  int v;
};

int main() {
  Foo default_foo;
  Foo value_foo = {};
}

In this case, Foo defines a user-declared, but not user-provided, constructor. While this type will not be an aggregate, members will be initialized as if for an aggregate. This means that default_foo.v will be uninitialized, while value_foo.v will be zero-initialized. Note that “user-declared” only applies to a default constructor which is defaulted (= default) at its point of declaration. A defaulted out-of-line implementation (Foo::Foo() = default) is considered user-provided and will behave equivalently to a definition of Foo::Foo() {}.

Uninitialized Members in User-Provided Constructors

struct Foo {
  Foo() {}

  int v;
};

int main() {
  Foo foo = {};
}

In this case, although Foo has a user-provided constructor, it fails to initialize v. In this case, v is once more default-initialized, which means its value is undetermined, and it is unsafe to read.

Explicit Value-Initialization

In general, it is a good idea to replace the initializer with an explicit initialization to a value, even if that value is 0, for the benefit of the reader. This is called direct-initialization, which is a more specific form of value-initialization.

struct Foo {
  Foo() : v(0) {}

  int v;
};

Default Member Initialization

A simpler solution than defining constructors for classes, while still avoiding the pitfalls of default- vs value-initialization, is to initialize members of classes at declaration, wherever possible:

struct Foo {
  int v = 0;
};

This ensures that no matter how the instance of Foo is constructed, v will be initialized to a determinate value.

Default member initialization also serves as documentation, especially in the case of booleans, or non-zero initial values, as to what is a safe initial value for the member.

Pro Tip: Scalar Zero-Initialization

The full set of rules for when scalar values are safe to read after initialization:

The type is followed by an explicit (), {}, or = {} initializer.
The instance of the type under construction is an element of an array with an initializer as above. E.g., new int[10]().
The instance of the type under construction is a member of a class with a disabled default constructor, and the instance of the outer object is value-initialized.
The instance of the type under construction is static or thread-local.
The instance of the type under construction is a member of a class with an aggregate type that has an initializer.

Array Types

It’s easy to forget to add an explicit initializer to array declarations, but this can lead to particularly pernicious initialization issues.

int main() {
  int foo[3];
  int bar[3] = {};
  ...
}

Every element of foo is default-initialized, while every element of bar will be zero-initialized.

A Digression Discerning Defaulted Default Constructor Declarations

Pop quiz: Do these stylistically different declarations affect the behaviour of the code?

struct Foo {
  Foo() = default;

  int v;
};

struct Bar {
  Bar();

  int v;
};

Bar::Bar() = default;

int main() {
  Foo f = {};
  Bar b = {};
  ...
}

Many developers would reasonably assume that this may affect code generation quality, but otherwise is a style preference. As you might have guessed, because I’m asking, this is not the case.

The reason goes back to the section above on User-Declared vs User-Provided Constructors. As the constructor for Foo is defaulted on declaration, it is not user-provided (but it is user-declared). This means that while Foo is not an aggregate type, f.v is still zero-initialized. However, Bar has a user-provided constructor, albeit created by the compiler as a defaulted constructor. As this constructor does not explicitly initialize Bar::v, b.v will be default-initialized and unsafe to read.

Recommendations

Be explicit about the value to which scalar types are being initialized instead of relying on zero-initialization.
Treat all instances of scalar types as having indeterminate values until you explicitly initialize or assign to them.
If a member has a sensible default, and the class has multiple constructors, use a default member initializer to ensure it isn’t left uninitialized. Be aware that a member initializer within a constructor will override the default.

Tip of the Week #168: `inline` Variables

2019-11-25T00:00:00-05:00

Originally posted as TotW #168 on September 12, 2019

By James Dennett

Updated 2020-04-06

Quicklink: abseil.io/tips/168

Here’s one safe way to define a string constant in a header file with C++17’s inline variables.:

inline constexpr absl::string_view kHelloWorld = "Hello World.";

Safety of initialization and destruction is ensured by the use of constexpr, and using inline here ensures that there is only one copy of kHelloWorld in the program.

Using the keyword inline here may seem strange at first, particularly if you are used to thinking of inline as being primarily an optimization hint. The use of inline for functions in headers is a close analogy; compare the variable definition above to something like

inline constexpr absl::string_view HelloWorld() {
  return "Hello World.";
}

but with the advantage that the string is guaranteed to be at the same memory address every time.

Almost every global variable defined in a header file should be marked as inline – and should generally be constexpr too. If they are not marked as inline then there will be a separate instance of the variable for each .cc file that includes the header, which can lead to subtle violations of the ODR (one definition rule).

Outside of header files there is no need to mark variables as inline.

Note: A static constexpr data member of a class is implicitly inline from C++17. This special case does not change the semantics of existing code, but means that it is now unnecessary to provide a separate definition for the member in a source file. This applies only to static constexpr data members, not to other constexpr variables, and not to data members that are merely static const.

References:

Google C++ Style Guide - Static and Global Variables

Tip of the Week #161: Good Locals and Bad Locals

2019-11-25T00:00:00-05:00

Originally posted as TotW #161 on April 16, 2019

By James Dennett

Updated 2020-04-06

Quicklink: abseil.io/tips/161

We may freak out globally, but we suffer locally. – Jonathan Franzen

Synopsis

Local variables are great, but can be overused. We can often simplify code by restricting their use to situations in which they provide a specific benefit.

Recommendations

Use local variables only when one or more of the following applies:

Their name adds useful documentation.
They simplify excessively complicated expressions.
They factor out a repeated expression to make it clear to humans (and to a lesser extent compilers) that it’s the same value every time.
An object’s lifetime needs to extend across multiple statements (for example, because references to the object are retained beyond the end of a single statement or because the variable holds a value that is updated during its lifetime).

In other cases, consider removing a layer of indirection by eliminating the local variable and writing the expression directly where it’s used.

Rationale

Naming values adds a level of indirection to code comprehension unless the variable’s name fully captures the relevant aspects of its meaning. Giving a name to a value in C++ also exposes it to the rest of the scope. It also affects the “value category”, as every named variable is an lvalue even if declared as an rvalue reference and initialized from an rvalue. This can require additional uses of std::move, which warrant care during code review to avoid use-after-move bugs. Given these downsides, use of local variables is best reserved for situations in which it provides a specific benefit.

Examples: Bad Uses of Local Variables

Eliminating a Local Variable That’s Immediately Returned

As a simple example of eliminating an unhelpful local variable, instead of

MyType value = SomeExpression(args);
return value;

prefer

return SomeExpression(args);

Inline Expressions Under Test Into GoogleTest’s `EXPECT_THAT`

std::vector<string> actual = SortedAges(args);
EXPECT_THAT(actual, ElementsAre(21, 42, 63));

Here the variable name actual doesn’t add anything useful (EXPECT_THAT always takes the actual value as its first argument), it’s not simplifying a complicated expression, and its value is used once only. Inlining the expression as in

EXPECT_THAT(SortedAges(args), ElementsAre(21, 42, 63));

makes it clear at a glance what’s being tested, and by avoiding giving a name to actual ensures that it cannot be unintentionally re-used. It also allows the testing framework to show the failing call in error output.

Note: the shorter version hides the expected type of SortedAges. If verifying the type is important, consider declaring a variable in order to show its type.

Use Matchers to Eliminate Variables in Tests.

Matchers can help to avoid the need to name local variables in tests by allowing EXPECT_THAT to directly express everything we expect of a value. Instead of writing code like this

std::optional<std::vector<int>> maybe_ages = GetAges(args);
ASSERT_NE(maybe_ages, std::nullopt);
std::vector<int> ages = maybe_ages.value();
ASSERT_EQ(ages.size(), 3);
EXPECT_EQ(ages[0], 21);
EXPECT_EQ(ages[1], 42);
EXPECT_EQ(ages[2], 63);

where we have to be careful to write ASSERT* instead of EXPECT* to avoid crashes, we can express the intent directly in code:

EXPECT_THAT(GetAges(args),
            Optional(ElementsAre(21, 42, 63)));

Examples: Good Uses of Local Variables

Factoring Out a Repeated Expression

myproto.mutable_submessage()->mutable_subsubmessage()->set_foo(21);
myproto.mutable_submessage()->mutable_subsubmessage()->set_bar(42);
myproto.mutable_submessage()->mutable_subsubmessage()->set_baz(63);

Here the repetition makes the code verbose (sometimes requiring unfortunate line breaks), and can require more effort from readers to see that this is setting three fields of the same proto message. Using a local variable to alias the relevant message can clean it up:

SubSubMessage& subsubmessage =
    *myproto.mutable_submessage()->mutable_subsubmessage();
subsubmessage.set_foo(21);
subsubmessage.set_bar(42);
subsubmessage.set_baz(63);

In some cases this can also help the compiler to generate better code as it doesn’t need to prove that the repeated expression returns the same value each time. Beware of premature optimization, though: if eliminating a common subexpression doesn’t help human readers, profile before trying to help the compiler.

Giving Meaningful Names to Pair and Tuple Elements

While it’s usually better to use a struct with meaningfully-named fields than a pair or a tuple, we can mitigate the problems with pair and tuple by binding meaningfully-named aliases to their elements. For example, instead of

for (const auto& name_and_age : ages_by_name) {
  if (IsDisallowedName(name_and_age.first)) continue;
  if (name_and_age.second < 18) children.insert(name_and_age.first);
}

in C++11 we could write

for (const auto& name_and_age : ages_by_name) {
  const std::string& name = name_and_age.first;
  const int& age = name_and_age.second;

  if (IsDisallowedName(name)) continue;
  if (age < 18) children.insert(name);
}

and in C++17, we can more simply use “structured bindings” to achieve the same result of giving meaningful names:

for (const auto& [name, age] : ages_by_name) {
  if (IsDisallowedName(name)) continue;
  if (age < 18) children.insert(name);
}

Abseil Random

2019-11-21T00:00:00-05:00

By Andy Soffer, Engineer

We are happy to announce the arrival of the Abseil Random library, providing simple tools for generating random numbers and testing code that uses random numbers.

Abseil now includes a random number generation library, which can be found in absl/random/. The library includes

Drop-in replacements for some distributions available through the standard library <random> header, with some modest performance improvements.
Two good default bit generators for users without the domain expertise to make meaningful choices amongst the many options available in the standard.
Distribution functions that provide a simple interface for generating random values.
A bit-generator (absl::MockingBitGen) that, with GoogleTest, enables deterministic testing of application code.

This blog post discusses how to get started using this library. For a more detailed explanation on the design of the helper functions and testing facilities provided by this library, take a look at our design note.

Happy sampling!

Bit Generation

We provide several bit generators:

absl::BitGen: A general-purpose random bit generator whose default constructor is seeded correctly.
absl::InsecureBitGen: A fast bit generator for performance-sensitive use cases when the computational cost of absl::BitGen is prohibitively expensive.
absl::MockingBitGen: A bit generator to be used only in unit tests. For details, see the design note.
absl::BitGenRef: A type-erased reference to a bit generator. Used primarily as a means of swapping bit generators at run-time.

For the bit generator, we recommend using absl::BitGen unless you have specific performance requirements and have thoroughly vetted absl::InsecureBitGen for your use case. Neither absl::BitGen nor absl::InsecureBitGen are guaranteed to be cryptographically secure.

Bit generators are well-seeded by default, so default constructing an absl::BitGen is a correct and intended usage. All of our bit generators provide no stability guarantees. The implementation may change at any time, and the sequence of variates produced need not be the same, even between two invocations of the same process.

Distribution Function Templates

Distribution function templates are thin wrappers around distribution objects that provide a straightforward API for getting randomness. A generator should be passed to a distribution function as in the examples below:

// Using the C++ standard <random> facilities
std::random_device rd;
std:mt19937 gen(rd());

int die_roll = std::uniform_int_distribution<>(1, 6)(gen);
bool fair_coin_landed_heads = std::bernoulli_distribution(0.5)(gen);

// Using Abseil Random
absl::BitGen gen;

int die_roll = absl::Uniform(absl::IntervalClosedClosed, gen, 1, 6);
bool fair_coin_landed_heads = absl::Bernoulli(gen, 0.5);

The full list of distribution functions provided can be seen in distributions.h.

Testing Facilities

For testing functions that require randomness, we provide absl::MockingBitGen, a bit generator that enables a test author to specify the result of a call to a distribution. As you will notice in the code-snippet below, this is not done via a deterministic bit generator (for reasons discusesd in the design note). Rather, absl::MockingBitGen allows the user to specify the result not of the bit generator, but of the entire random sample. The design note goes into detail about why we believe this is a maintainable approach to testing code with random behavior. These testing facilities rely on GoogleTest.

using ::testing::_;
using ::testing::Lt;
using ::testing::Return;

int ComputeValue(absl::BitGenRef gen) {
  if (absl::Bernoulli(gen, 0.00001)) {
    return 10
  } else {
    return 0;
  }
}

TEST(ComputeValueTest, UnlikelyCase) {
  absl::MockingBitGen gen;
  ON_CALL(absl::MockBernoulli(), Call(gen, Lt(0.01))
      .WillByDefault(Return(true));

  EXPECT_EQ(ComputeValue(gen), 10);
}

Introducing Abseil Options

2019-11-15T00:00:00-05:00

By Greg Falcon, Abseil Engineer

Abseil now provides an options file, providing a limited means of configuring Abseil’s features. In the initial release of this options mechanism, we have added the ability to override Abseil’s automatic type detection and force (for example) absl::string_view to either always alias to std::string_view, or always remain using our Abseil implementation.

The key word above is “limited”: these options are dangerous if used incorrectly, and can only be safely used in some contexts. This blog post explains this feature and how to correctly use it.

Motivation

While we discourage relying on compiled representations of libraries, we recognize this isn’t always possible for providers of binaries or compiled libraries (such as package managers). This options mechanism is provided to allow those providers to ensure that copies of Abseil use the same implementation. For information on how to use this options mechanism to provide such ABI stability, consult our Options guide.

Caveats

Two copies of Abseil configured with different options are incompatible, in exactly the same way that two different source snapshots of Abseil are incompatible. Abseil options must be set consistently across an entire program. They cannot be changed on a per-library basis.

These options are dangerous if used incorrectly, and can only be safely used in some contexts.

Because of the increased testing burden they impose, we plan to keep the set of options we support small. If you change any options, we recommend that you run the Abseil unit tests, to ensure your settings work with your toolchain. The default options we ship with reflect how we use Abseil internally. This is the configuration that is “battle tested” in Google’s C++ codebase.

Tip of the Week #182: Initialize Your Ints!

2019-10-01T00:00:00-04:00

Originally posted as TotW #182 on July 23, 2020

Updated 2020-07-23

Quicklink: abseil.io/tips/182

“In any moment of decision, the best thing you can do is the right thing, the next best thing is the wrong thing, and the worst thing you can do is nothing.” –Theodore Roosevelt

C++ makes it too easy to leave variables uninitialized. This is scary, because almost any access to an uninitialized object results in Undefined Behavior. Default initialization is the default, among many forms, and occurs when no initial value is specified for a variable, but it is not always initialization.

Default Initialization of Trivial Types

{
  bool bool_one;
  bool bool_two = bool_one;
}

It surprises many to learn that the above code snippet invokes Undefined Behavior. In the first statement, bool_one is default initialized, which (ironically) is not guaranteed to actually initialize the variable. In the example, bool_one is left uninitialized even though it uses “default initialization”. How do we know this?

To understand this phenomenon, let’s first clarify when default initialization does and does not behave in this way. In C++, not all types expose the ability to skip initialization. There are two primary categories that are worth highlighting.

1) For types that have default constructors, including most class types, default initialization will invoke the default constructor in all cases. For example, std::string str; is guaranteed to initialize str as if it had been value initialized as in std::string str{};.

2) For types with no constructors, such as bool, default initialization can exhibit one of two possible behaviors. A) If the variable being initialized is static or defined at namespace scope, so-called “value initialization” will be performed. B) However, for non-static, block-scope variables, default initialization performs no initialization at all for these types, leaving the variable uninitialized with an indeterminate value.

As a result, in the above example, bool_one is uninitialized because bool has no constructors and bool_one is a non-static, block-scope variable. When bool_two’s initialization reads the value of bool_one, the resulting behavior is undefined.

Which types in C++ lack constructors?

C++ inherits types from C, referred to as trivially default constructible (or colloquially “trivial”) types, which are implemented with no constructors. This includes fundamental types like int and double as well as struct types that contain only trivial fields with no member-wise initializers. It also includes all raw pointer types, even when they point to classes as in MyClass*.

Said another way, since C does not have constructors, such types when used in C++ retain that behavior for default initialization.

Why does C++ allow for uninitialized objects?

The ability to leave some objects uninitialized is needed on rare occasion for performance or for providing placeholders where there is truly no initial value. Since most access patterns of uninitialized values are undefined, sanitizers can also use this information to find bugs.

Default Initialization of Potentially-Trivial Types

Like the code snippet before, the following code also uses default initialization:

{
  MyType my_variable;
}

Is it safe to read the value of my_variable?

To answer that question, we must know more about the implementation of MyType. The callsite shown does not have enough information to determine whether reading my_variable is safe. For example, if MyType is a simple struct type with only int fields, no constructors and no member-wise initializers, my_variable will be uninitialized. However, if MyType is a class type with a user-defined implementation for MyType::MyType(), the constructor is responsible for initializing the variable such that immediately reading its value is a safe operation.

Suggestion: Initialize Trivial Objects

In most code, you probably don’t want uninitialized objects. Rarely, it can make sense for reasons of performance or encoding semantics. However, unless in one of these exceptional cases, prefer initializing trivial objects in struct fields and variables as in the following examples:

float ComputeValueWithDefault() {
  float value = 0.0;  // Guarantees initialization by providing a default value.
  ComputeValue(&value);
  return value;
}

struct MySequence {
  // Member-wise initializers guarantee initialization.
  MyClass* first_element = nullptr;
  int element_count = 0;
};

MySequence GetPopulatedMySequence() {
  MySequence my_sequence;  // Made safe by the member-wise initializers.
  MaybePopulateMySequence(&my_sequence);
  return my_sequence;
}

Additionally, where feasible, refrain from making type aliases for trivial types. We want struct and class types to be safely initialized in all cases. Since fundamental types (integers, pointers, etc.) do not guarantee initialization from default initialization, giving names to such types that appear to be safe can make code more difficult to reason about.

Examples of aliases for trivial types are shown below:

{
  using KeyType = float;  // C++-style alias
  typedef bool ResultT;  // C-style alias

  // [Many lines of code...]

  // Surprise! These variables are uninitialized!
  KeyType some_key;
  ResultT some_result;
}

Additional Information

Tip of the Week #180: Avoiding Dangling References

2019-10-01T00:00:00-04:00

Originally posted as TotW #180 on June 11, 2020

By Titus Winters

Updated 2020-06-11

Quicklink: abseil.io/tips/180

Introduction

Unlike many languages, C++ lacks the safety checks necessary to avoid referencing invalid memory (aka “dangling references”). You can easily dereference a pointer to an object that was already delete-ed, or follow a reference to an object that has gone out of scope. Even class types carry this risk. Importantly, we are building naming conventions around the names view and span to signify “This is an object that has reference semantics and may dangle.” These types, like all types with reference semantics, never own the underlying data that they point to. Be mindful whenever you see instances of any of these types stored.

Dangling, And Understanding C++

If you’re coming to C++ from other languages, there are quite a few fundamental surprises. The type system is meaningfully more complicated than most languages, requiring a sometimes-subtle understanding of references, temporaries, shallow-const, pointers, object lifetimes, etc. One of the most uniquely important issues when learning C++ is recognizing that having a pointer or a reference to an object doesn’t mean the object still exists. C++ is not garbage-collected nor reference-counted, and as a result holding a handle to an object isn’t enough to ensure the object stays alive.

Consider:

int* int_handle;
{
  int foo = 42;
  int_handle = &foo;
}
std::cout << *int_handle << "\n";  // Boom

When we dereference int_handle with operator*, we are following a pointer to an object whose lifetime is ended. This is a bug. Formally, this is undefined behavior, and anything can happen.

Distressingly, one of the “anything can happen” options is “this does what you naively think it might” - printing 42. C++ is a language that does not promise to diagnose or react to your bugs. The fact that your program seems to work does not mean it is correct. It means at best that the compiler happened to choose an outcome that worked for you. But make no mistake: this is no less buggy than if int_handle was a pointer to null.

From this we draw two important points:

Unlike most languages we use today, the fact that a program runs to completion or behaves as expected is only weakly correlated with “this is correct.” Other languages would diagnose (at compile time or runtime) our errors, C++ chooses to focus instead on optimization and efficiency: spending extra computing power to check that you didn’t make an error isn’t the C++ way. In most languages “It works” is much better evidence for “It is correct.” C++ requires that we question that evidence.
Holding a handle (a pointer or reference) to an object does not guarantee that the object is alive and valid to access. Other languages have runtime overhead to keep objects alive, or statically constrain the code you can write. C++ focuses instead on optimization and efficiency. Anytime you use a handle to access the underlying object you need a mental proof to understand why you’re sure the underlying object is still alive. It may have gone out of scope, it may have been explicitly delete-ed.

It is critically important to understand that our informal “handle” discussion applies to values of certain class types as well as to the more-obvious pointers and references. Consider iterators:

std::vector<int>::iterator int_handle;
{
  std::vector<int> v = {42};
  int_handle = v.begin();
}
std::cout << *int_handle << "\n"; // Boom

This is morally identical to the previous example. On some platforms, vector iterators may in fact be implemented as pointers. Even if these iterators are class types, the same language rules apply: dereferencing the iterator will (under the hood) eventually be following a pointer or reference to an object that is no longer in scope (in this case, v[0]).

Because C++ does not define what happens when code uses an invalid pointer, reference, or iterator, code that does so is always incorrect (even if it appears to work). This allows debugging tools such as sanitizers and debugging iterators to report bugs with no false positives.

Class Types that May Dangle

Over the past few years, Abseil and the C++ standard library have been introducing additional class types with similar “handle” behavior. The most common of these is string_view, which is a handle to some contiguous buffer of characters (often a string). Holding a string_view is exactly like holding any other handle type: there is no general guarantee that the underlying data lives. It is up the programmer to prove that the underlying buffer outlives the string_view. Importantly the handle that string_view provides does not allow for mutation: a string_view cannot be used to modify the underlying data.

Another handle design that is becoming common is span<T>, which is a contiguous buffer of any type T. If T is non-const, then span allows mutation of the underlying data. If T is const, then the span cannot modify it, in the same fashion that string_view cannot modify the underlying buffer. Thus, span<const char> is similar to string_view. Although the two types have different APIs, reasoning about the handles or underlying buffers works in exactly the same way.

string_view and span tend to be very safe to use as function parameters, abstracting away from a variety of input argument formats. Because of the possibility of a dangling reference, any time that types of this design are stored, they become a significant source of programmer error. Every storage of any handle type requires critical thinking to understand why we are sure the underlying object stays valid for the lifetime of the handle. Using string_view or span in a container is not always wrong but is a subtle optimization that warrants clear comments describing the associated storage. Using these types for data members of a class is rarely the right choice.

It is critically important going forward that C++ programmers understand these design patterns, and how to use these “reference parameter types.” To assist in that understanding, type designers and library providers tend toward the following meaning for types:

view - a reference type that cannot be used to mutate the underlying data
span - a reference type that might be used to mutate the underlying data

Since both of these naming indicators suggest reference types, any storage of a library-provided type called a “view” or a “span” needs to be accompanied by the same logic you would use when thinking about the lifetimes of a pointer or reference: how do I know that the underlying object is still alive?

Caveats and Further Reading

The popular external range_v3 library and the upcoming C++20 ranges library have a different meaning for “view”, although the types described by these definitions overlap. In ranges, “view” means “a range that can be copied in O(1)”. This includes string_view. However, this definition does not preclude mutation of the underlying data. This mismatch is unfortunate, and largely recognized by the C++ standards committee, but nobody could find consensus on any alternative to “view” after the concern was raised.

The C++20 span type and Abseil’s Span type have slightly different interfaces and semantics when it comes to comparability and copying. The most notable difference is with absl::Span::operator==, which we now know to probably be a design mistake.

For more on the design theory underlying modern reference parameter types, see Revisiting Regular Types.

Tip of the Week #158: Abseil Associative containers and `contains()`

2019-10-01T00:00:00-04:00

Originally posted as TotW #158 on January 3, 2019

By James Dennett

Updated 2020-04-20

Quicklink: abseil.io/tips/158

“I cannot contain myself” – Bertrand Russell

Does That Container Contain This Thing or Not?

When checking whether a set contains a value or a map contains a key, C++ has historically forced users to choose between writing the rather verbose

container.find(value) != container.end()

or the arguably obtuse (and sometimes inefficient)

container.count(value) != 0

instead of writing

container.contains(value)

as we’d like to.

`container.contains(value)` to the Rescue

The simpler syntax is part of the C++20 Standard, and Abseil’s (absl::{flat,node}_hash_{map,set}) and btree containers (absl::btree_*) support it today.

contains has the same support for heterogeneous lookup as find, so (for example) it’s possible to check whether an absl::flat_hash_set<std::string> contains an absl::string_view value without paying the costs of converting to a std::string object:

constexpr absl::string_view name = "Willard Van Orman Quine";
absl::flat_hash_set<std::string> names = {std::string(name)};
assert(names.contains(name));  // No dynamic allocation here.

Given that most of our code that needs associative containers (whether sets or maps) should be using the Abseil hashed containers today (see Tip #136), it should rarely be necessary to use one of the other formulations in new code.

NOTE: As described in Tip #132 (“Avoid Redundant Map Lookups”), don’t check if an item is in a container and then do another operation that implies a lookup (such as find, insert or remove).

Conclusion

Querying whether an item can be found in an associative container is a common operation, and a natural syntax for it is container.contains(value). Prefer that syntax when possible.

Tip of the Week #147: Use Exhaustive `switch` Statements Responsibly

2019-09-09T00:00:00-04:00

Originally posted as TotW #147 on April 25, 2018

By Jim Newsome

Updated 2020-04-06

Quicklink: abseil.io/tips/147

Introduction

Using the -Werror compiler flag, a switch statement over a value of an enum type without a default label will fail to compile if any enumerator of the enum doesn’t have a corresponding case. This is sometimes called an exhaustive or defaultless switch statement.

An exhaustive switch statement is an excellent construct for ensuring at compile time that every enumerator of a given enum is explicitly handled. However, we must ensure that we handle the fall-through case when the variable (legally!) has a non-enumerator value, and one of:

The owner of the enum guarantees no new enumerators will be added,
The owner of the enum is willing and able to fix our code when new enumerators are added (e.g. the enum definition is part of the same project),
The owner of the enum will not be blocked by breaking our build (e.g. their code is in a separate source control repository), and we’re willing to be forced to update our switch statements when updating to the latest version of the enum-owner’s code.

An Initial Attempt

Suppose we are writing a function that maps each enumerator of an enum to a std::string. We decide to use an exhaustive switch statement to ensure we didn’t forget to handle any of the enumerators:

std::string AnEnumToString(AnEnum an_enum) {
  switch (an_enum) {
    case AnEnum::kFoo:
      return "kFoo";
    case AnEnum::kBar:
      return "kBar";
    case AnEnum::kBaz:
      return "kBaz";
  }
}

Assuming that AnEnum indeed has only those three enumerators, this code will compile, and will seem to have the desired effect. However, there are two important issues that must be accounted for.

Enums with Non-Enumerator Values

In C++, enums are permitted to have values other than the explicit enumerators. All enums can legally take on at least all of the values representable by an integral type with just enough bits to represent every enumerator, and enums with a fixed underlying type (e.g. those declared with enum class) can take on any value representable by that type. This is sometimes intentionally leveraged to use an enum as a bitfield or to represent enumerators that didn’t exist when we compiled our code (as in proto 3).

So what happens in our code if an_enum isn’t one of the handled enumerator types?

In general when a switch statement doesn’t have a case matching the switch condition and doesn’t have a default case, execution falls through past the whole switch statement. This can lead to surprising behavior; in our example it leads to undefined behavior. After execution falls through the switch statement, it reaches the end of the function without returning a value, which is undefined behavior for a function with a non-void return type.

We can address this issue by explicitly handling the case where execution falls through the switch statement. This ensures we always get defined and predictable behavior at run time, while continuing to benefit from the compile-time check that all enumerators are explicitly handled.

In our example, we’ll log a warning and return a sentinel value. Another reasonable alternative, especially if we’re convinced that the function (currently) can’t receive a non-enumerator value, would be to immediately crash with a debuggable error message and stack trace.

std::string AnEnumToString(AnEnum an_enum) {
  switch (an_enum) {
    case AnEnum::kFoo:
      return "kFoo";
    case AnEnum::kBar:
      return "kBar";
    case AnEnum::kBaz:
      return "kBaz";
  }
  LOG(ERROR) << "Unexpected value for AnEnum: " << static_cast<int>(an_enum);
  return kUnknownAnEnumString;
}

We’ve now ensured that something reasonable happens for any possible value of an_enum, but there’s still potentially a problem.

What Happens When a New Enumerator Is Added?

Suppose someone later wants to add a new enumerator to AnEnum. Doing so causes AnEnumToString to no longer compile. Whether that’s a bug or a feature depends on who owns AnEnum and what guarantees they provide.

If AnEnum is part of the same project as AnEnumToString, then the engineer adding a new enumerator is likely to be blocked from submitting their change before fixing AnEnumToString due to compilation errors. They are also reasonably likely to be willing and able to do so. In this case our use of an exhaustive switch statement is a win: it successfully ensured that the switch statement is updated appropriately, and everyone is happy.

Similarly, if AnEnum is part of a different project in a different repository, then the breakage won’t surface until the engineers on our project try to update to a newer version of that code. If we expect that those engineers will be willing and able to fix the switch statement, then all is well.

However, if AnEnum is owned by a different project in the same repository the situation is a bit more precarious. A change to AnEnum might cause our code to break at head, and the engineer making the change might not be willing or able to fix it for us. Indeed, if there were many similar exhaustive switch statements over AnEnum, it’d be extremely challenging for them to fix all such usages.

For these reasons, it’s best to use exhaustive switch statements only on enum types that either we own, or whose owner has explicitly guaranteed that no new enumerators will be added.

In our example, let’s suppose that AnEnum is owned by a different project, but the documentation promises that no new enumerators will be added. Let’s add a comment so that future readers understand our reasoning.

std::string AnEnumToString(AnEnum an_enum) {
  switch (an_enum) {
    case AnEnum::kFoo:
      return "kFoo";
    case AnEnum::kBar:
      return "kBar";
    case AnEnum::kBaz:
      return "kBaz";
    // No default. The API of AnEnum guarantees no new enumerators will be
    // added.
  }
  LOG(ERROR) << "Unexpected value for AnEnum: " << static_cast<int>(an_enum);
  return kUnknownAnEnumString;
}

Conclusions

Exhaustive switch statements can be an excellent tool for ensuring that all enumerators are explicitly handled, provided that we:

Explicitly handle the case where the enum has a non-enumerator value, falling through the entire switch statement. In particular if the enclosing function has a return value, we must ensure that the function still either returns a value or crashes in a well-defined and debuggable way.
Ensure that one of:
- The owner of the enum type either guarantees no new enumerators will be added,
- The owner of the enum is willing and able to fix our code when new enumerators are added,
- If our code uses exhaustive switch statements and is broken due to an enumerator being added, the owner of the enum is not blocked by this breakage.

When making an enum type available to other projects, we should either:

Explicitly guarantee that no new enumerators will be added, so that users can take advantage of exhaustive switch statements.
Explicitly reserve the right to add new enumerators without notice, to discourage consumers from writing exhaustive switch statements. One idiomatic way of doing so is to add a sentinel enumerator clearly not meant to be used in API consumers’ exhaustive switch statements; e.g. kNotForUseWithExhaustiveSwitchStatements.

FAQ

Why does the compiler allow omitting a return statement after an exhaustive switch?

Omitting a final return can be safe if additional steps are taken to ensure that the enum variable can only be one of its enumerators. It’s often better in such cases to still defensively add a final return.
The enum I’m switching on already has exhaustive switch statements all over the place. Since the owners are already effectively prevented from adding new enumerators, won’t adding my own exhaustive switch statement be harmless?

It’s usually better to get an explicit policy from the owner before further increasing their maintenance burden.
What about protobuf enums?

For authoritative guidance, see the protobuf documentation.

Exhaustive switch statements on proto3 enum types are not recommended. The parser doesn’t guarantee that enum fields will have enumerator values. Additionally, it’s not possible to write an exhaustive switch statement over proto3 enum types without referencing special sentinel enumerators that should be considered internal implementation details of the protobuf tools.

Exhaustive switch statements on proto2 enum types that you own (or whose owners guarantee will never be moved to proto3 and will never have new enumerators added) are safe and recommended by the protobuf team. The protobuf parser guarantees that enum fields will be assigned a compile-time enumerator, though care should still be taken if the enum value isn’t guaranteed to have come from the parser (e.g. if it’s part of a proto object received as a function parameter).
What about scoped enumerations (enum class)?

Everything in this tip applies to all enumeration types in C++ at time of writing (i.e. through at least C++20).

References

B-tree Ordered Containers

2019-08-12T00:00:00-04:00

By Evan Brown, Engineer

The Abseil container library now includes B-tree containers that generally conform to the STL sorted container APIs:

absl::btree_set (meant to replace usage of std::set)
absl::btree_map (meant to replace usage of std::map)
absl::btree_multiset (meant to replace usage of std::multiset)
absl::btree_multimap (meant to replace usage of std::multimap)

You use a B-tree container just as you would an STL ordered container:

#include "absl/container/btree_map.h"

absl::btree_map<int, std::string> ducks =
    {{2, "dewey"}, {1, "huey"}, {3, "louie"},};

// Prints "huey, dewey, louie "
for (const auto& n : ducks) {
    std::cout << n.second << ", ";
}

B-trees have a different implementation from STL std::map containers, which require binary trees commonly implemented as red-black trees. While a red-black tree is limited to single-element nodes, with precisely two children, a B-tree may contain multiple values per node (M), with each node having (M+1) children. Having more values and children per node is more cache friendly because nodes are generally allocated separately so accessing additional nodes often results in cache misses.

Cache Friendliness

For search, insertion, and deletion, the number of nodes that need to be accessed in a sorted tree is proportional to the height of the tree. In balanced trees, this height is ~log_C(N), where C is the number of children per node and N is the number of values in the container; because b-trees have more children per node than binary trees, their heights are lower, and searching is faster.

For iteration, it is most cache efficient to store all values contiguously. B-trees store values contiguously in each node so it’s also generally more efficient to iterate through a B-tree than a binary tree.

Memory Overhead

B-trees also use significantly less memory per value in the tree because overhead is per node, and there are fewer nodes per value in B-trees. There is also an optimization in Abseil B-tree in which leaf nodes don’t store child pointers. Since the vast majority of nodes are leaf nodes (because of the higher branching factor due to more children per non-leaf node), this ends up saving a significant amount of memory.

API Difference from STL Sorted Containers

When values are inserted or removed from a B-tree, nodes can be split or merged and values can be moved within and between nodes (for the purpose of maintaining tree balance). This means that when values are inserted or removed from a B-tree container, pointers and iterators to other values in the B-tree can be invalidated. Abseil B-trees therefore do not provide pointer stability or iterator stability - this is in contrast to STL sorted containers that do provide these guarantees.

When to Use B-trees

B-trees are a good default choice for sorted containers, however, there are cases in which the STL alternatives may be more appropriate.

When value_type is large, fewer values can be stored per node so the benefits of B-tree are lessened.
When value_type is expensive to move, B-tree may become more expensive than STL alternatives because B-tree needs to move values within and between nodes to maintain balance, whereas binary trees can just move pointers instead. std::array<int32_t, 16> is an example of a value_type for which STL sorted containers currently outperform B-trees.
When pointer stability or iterator stability is required, B-trees aren’t a viable option (although usually code can be refactored to avoid these requirements).

For more information, consult the Abseil Container library documentation. Check out the B-tree Design Note for information on B-tree’s implementation.

Tip of the Week #90: Retired Flags

2019-05-10T00:00:00-04:00

Originally posted as TotW #90 on March 19, 2015

*by Titus Winters

One of the frustrating things about our (mis)use of command-line flags is the difficulty in removing a flag from binary and production servers safely (revisit https://abseil.io/tips/45 for some frustrating misuses). The trouble? A binary won’t start if you specify a flag that is no longer defined, and thus removal of flags may require coordination between C++ code and your job launching scripts and configurations.

In some cases, this coordination can be quite challenging (conditioning production code based on a binary version). If only there were a better way!

There is. A while back we added a new concept to the C++ command-line flags system called “retired flags” (ABSL_RETIRED_FLAG).

A retired flag creates no symbol in C++ (no more FLAGS_some_flag global variable to rely upon), does not appear in --help, but will be accepted when specified on the command line (although an ERROR will be logged). In this way, flags that are no longer used by the code can be “retired”, separating C++ changes from production configuration changes. Once the configuration changes are all clear, the retired flag can be removed once and for all.

Retired flags were designed to be used in many situations (including cases involving cross-repo non-atomic commit requirements). A step-by-step for a very simple flag retirement could look like this:

Remove uses of FLAGS_frobber from code.

If you’re following the advice of https://abseil.io/tips/45 and using flags primarily from main() this should be easy to do and to check for.

Change the definition of the flag to retire it. That is:

ABSL_FLAG(type, frobber, "default", "Which frobber to use?");

should become:

ABSL_RETIRED_FLAG(type, frobber, "default", "retired");

Wait for binary release(s). Once all the jobs in all of your serving cells are invoking the new binary, you can continue.
Remove the flag from your production configuration. Once searching the relevant production configs shows no hits for the frobber flag, you can continue.
Remove the retired flag.

We’ve had success with very complicated flag removals: the motivating example for this effort was the removal of flags defined in a legacy internal filesystem, which was more complicated because the flags were defined in a library (and thus used in numerous binaries). From what we’ve seen, even the most complicated flag removals can be enabled safely with this system. So, the next time you’re wondering how to remove a flag safely, consider retiring it first and taking it step-by-step.

Tip of the Week #45: Avoid Flags, Especially in Library Code

2019-05-10T00:00:00-04:00

Originally posted as TotW #45 on June 3, 2013

*by Titus Winters

“What I really want is the behavior of my code to be controlled by a global variable that cannot be statically predicted, whose usage is incompletely logged, and which can only be removed from my code with great difficulty.” – Nobody, Ever

The common use of flags in production code, especially within libraries, is a mistake. Don’t use flags unless it is truly necessary. There, we said it.

Flags are global variables, only worse: you can’t know the value of that variable by reading the code. A flag may not only be set at startup, but could potentially be changed later through arbitrary means. If you run a server in your binary, there is usually no guarantee that the value of your flag will remain unchanged from cycle to cycle, nor any notification if it were to change, nor any means for looking for such a change.

If your production environment logs the invocation of every binary directly, and stores those logs, great. Most environments don’t work that way. For library code, this uncertainty is especially nefarious: how can you tell when use of a particular feature is actually dead? The simple answer is: you can’t.

Flags also make it challenging to put the final stake in dead code. During migrations to new backends, you’d think that removing legacy code would simply be a matter of removing unnecessary build dependencies and issuing history’s most satisfying git rm. You’d be wrong. If your legacy binary has hundreds of flags defined and referenced by production code, simply removing the dead code would cause massive problems for your release engineers: almost no jobs would start up after such a change.

The worst part of all of this? An analysis that was done at Google during early 2012 found that the majority of C++ flags had never actually varied, as far as we can tell within the data retention limits described above.

Flags are appropriate in certain use cases, however: debugging without flag-enabled backtraces just wouldn’t be the same. Feature flags, when handled properly (and cleaned up afterward), are perfectly justified. Knobs that genuinely need to be tweaked by heroic SREs are a handy safety net. More broadly, flags used to pass name/value inputs to a binary, and used only in main(), are much more maintainable than positional parameters.

Even given those caveats, it’s time that we all take a good hard look at our usage of flags. The next time you’re tempted to add a flag to your library, spend a while looking for a better way. Pass configuration explicitly: this is almost always easier to reason about properly, and is certainly easier to maintain. Consider making numeric flags into compile-time constants. If you encounter new flags in code reviews, push back. Every flag introduction should be justified.

Tip of the Week #103: Flags Are Globals

2019-05-10T00:00:00-04:00

by Matt Armstrong

Define flags at global scope in a .cc file. Declare them at most once in a corresponding .h file.

Why Declare Things In Header Files?

Using header files is a reflex for most of us, so we may have forgotten why they are used:

Declaring something in a header file makes it easy to #include elsewhere. The entire program sees the same declaration.
Including this header in the .cc that defines the same entity ensures the definition matches declaration.
The header file serves as documentation for a package’s public API. Using anything but a package’s public API is poor form.
Including headers, instead of re-declaring entities, helps both tools and humans perform dependency analysis.

Abseil Flags Are As Vulnerable As Any Other Global

You can do this incorrectly with no link-time error. First, place the following in a .cc file:

// Defining --my_flag in a .cc file.
ABSL_FLAG(std::string, my_flag, "", "My flag is a string.");

And the following, erroneous, declaration of the flag in a different .cc file (perhaps a test):

// Declared in error: type should be std::string.
extern absl::Flag<int64> FLAGS_my_flag;

The program is ill-formed, and whatever happens is the result of undefined behavior. In my test program, this code compiled, linked, and crashed when the flag was accessed.

Recommendations

Design with command line flags as you would global variables.

Avoid flags if you can. See http://abseil.io/tips/45.
If you are using a flag to make a test easier to write, with no intent to ever set it in production, consider adding test-only APIs to your classes.
Consider treating flags as private static variables. If other packages need to access them, wrap them in functions.
Define your flags at global scope, not within a namespace, to get link errors when flag names conflict.
If a flag is accessed in multiple files, declare it in one .h file corresponding to its definition.
Use the ABSL_FLAG(type, ...) macro to define flags.

In Conclusion

Flags are global variables. Use them judiciously. Use and declare them as you would any other global variable.

Abseil Flags

2019-05-09T00:00:00-04:00

By Gennadiy Rozental, Abseil Engineer

Abseil is very happy to announce the release of the Abseil Flags library. Abseil’s flags library provides a standard, readable way to pass command-line values to a program.

#include <iostream>
#include <string>

#include "absl/flags/flag.h"
#include "absl/flags/parse.h"

ABSL_FLAG(std::string, name, "you", "Name of the person to greet");

int main(int argc, char** argv) {
  absl::ParseCommandLine(argc, argv);
  std::cout << "Hello " << absl::GetFlag(FLAGS_name) << "!" << std::endl;
  return 0;
}

$ greet
Hello you!
$ greet --name=Alice
Hello Alice!

Flag variables of the following types are supported out of the box:

bool
int32_t
int64_t
uint64_t
double
std::string
std::vector<std::string>
absl::Duration
absl::Time

For more information, consult the Abseil Flags library documentation.

Abseil Support for CMake

2019-04-02T00:00:00-04:00

By Jon Cohen, Abseil Engineer

CMake is a popular tool used to build multi-platform C++ projects. Abseil has had unofficial CMake support for some time, but support has never been as robust as that for Bazel. We are happy to announce that Abseil now fully supports the CMake build system.

Abseil supports CMake through the add_subdirectory command for full source inclusion, or by local installation into your project. Future Abseil LTS releases will be supported for installation in system-wide locations (such as /usr/local, CMake’s default install location).

We hope that this support will make it easier for users to adopt Abseil and for package managers to successfully and easily package Abseil.

For more information on getting Abseil working with CMake, consult our CMake Quickstart

Automated Upgrade Tooling for Abseil

2018-12-20T00:00:00-05:00

By Alex Strelnikov, Abseil Engineer

As we promised when we released our Compatibility Guidelines, we have developed a process for announcing and handling any API-breaking changes we may need to make within the Abseil code base. Our C++ Automated Upgrade guide outlines this process, which consists of clang-tidy tooling and associated communication regarding the change. A list of all such tools will be listed on our Upgrade Tools page.

At this time, we are also releasing our first such tool: a clang-tidy check for removing bug-prone implicit conversions in calls to several absl::Duration functions.

As outlined in our upgrade guide, we will always first serve notice of such a tool via this blog, and will also blog once we’ve officially removed the deprecated API.

CppCon 2018: Modern C++ Design

2018-11-29T00:00:00-05:00

Titus Winters and Modern C++

By Tom Manshreck, Abseil Tech Writer

CppCon 2018 was held in Bellevue, WA at the end of September.

C++ has changed a lot since the transformative introduction of C++11. It is now all too apparent that C++ API Design itself also needs to change as the lessons learned about, for example, type design become more understood.

Titus Winters reflects on the changes to C++ and how the introduction of new principles such as move-only types have affected API design in this two-part talk.

In the first part, Titus focuses on parameter passing and an API’s overload set in providing a powerful conceptual framework for API design. In the second part, we focus on the properties of well-designed types, and how to think about things like Regularity. We discuss how Regularity affects the design of non-owning reference types like string_view or span.

If you haven’t already, check out Titus’ original blog post on “Revisiting Regular Types” for more background information.

Civil Time in Abseil

2018-10-10T00:00:00-04:00

By Greg Miller, Bradley White, and Shaindel Schwartz

Almost every spring and fall there are news headlines about software that misbehaved during a daylight-saving transition. In much of the world, DST transitions occur multiple times per year, yet it is still a veritable minefield of latent bugs due to the complexities inherent in reasoning about civil-time discontinuities. To avoid these problems, a civil-time library must present the programmer with a correct — yet simplified — model that makes expressing the desired intent easy and writing bugs more obvious.

To that end, we are very pleased to introduce a new feature for the Abseil time library — civil time support. This update adds a set of constructs and functions that are used to represent and perform computations with civil times.

Abseil Time Constructs

When discussing time, we refer to two representations: absolute time, which represents a specific instant in time, and civil time, which represents the year, month, day, hour, minute and second (YYYY-MM-DD hh:mm:ss) of local, human-scale time. A “date,” for example, is a civil time - the legal, local name for the time that we are describing. A time zone defines the relationship between an absolute time and the civil time to which it corresponds:

The Abseil time library provides six new classes for representing civil times:

absl::CivilSecond
absl::CivilMinute
absl::CivilHour
absl::CivilDay
absl::CivilMonth
absl::CivilYear

These are all “value types” that enable easy and type-safe computations with civil times of varying alignment. They give you a clear vocabulary with which you can easily express your intent to the compiler and fellow humans.

Converting Between Absolute and Civil Times

The Abseil time library provides a set of functions to convert an absolute absl::Time and an absl::TimeZone to a civil time:

absl::ToCivilSecond()
absl::ToCivilMinute()
absl::ToCivilHour()
absl::ToCivilDay()
absl::ToCivilMonth()
absl::ToCivilYear()

absl::Time t = ...;
absl::TimeZone tz = ...;
absl::CivilDay cd = absl::ToCivilDay(t, tz);

The Abseil time library also provides the FromCivil() function to convert a civil time of any alignment and an absl::TimeZone to an absolute absl::Time:

absl::FromCivil()
absl::CivilSecond cs = ...;
absl::TimeZone tz = ...;
absl::Time t = absl::FromCivil(cs, tz);

More Information

For more complete information, see the Abseil time library documentation. Complete reference documentation is available within the Abseil time library header files.

Tip of the Week #153: Don't Use using-directives

2018-09-30T00:00:00-04:00

Originally posted as TotW #153 on July 17, 2018

By Roman Perepelitsa and Ashley Hedberg

Updated 2020-04-06

Quicklink: abseil.io/tips/153

I view using-directives as time-bombs, both for the parties that deal in them and the language system. – Ashley Hedberg with apologies to Warren Buffett

tl;dr

Using-directives (using namespace foo) are dangerous enough to be banned by the Google style guide. Don’t use them in code that will ever need to be upgraded.

If you wish to shorten a name, you may instead use a namespace alias (namespace baz = ::foo::bar::baz;) or a using-declaration (using ::foo::SomeName), both of which are permitted by the style guide in certain contexts (e.g. in *.cc files).

Using-directives at Function Scope

What do you think this code does?

namespace totw {
namespace example {
namespace {

TEST(MyTest, UsesUsingDirectives) {
  using namespace ::testing;
  Sequence seq;  // ::testing::Sequence
  WallTimer timer;  // ::WallTimer
  ...
}

}  // namespace
}  // namespace example
}  // namespace totw

The vast majority of C++ users think that the using-directive is injecting names into the scope where it’s declared. In the example above, that would be the scope of the function. In reality, the names are injected into the nearest common ancestor of the target namespace (::testing) and the usage namespace (::totw::example::anonymous) while the using directive is in scope. In our example, that’s the global namespace!

Thus, the code is roughly equivalent to the following:

using ::testing::Expectation;
using ::testing::Sequence;
using ::testing::UnorderedElementsAre;
...
// many, many more symbols are injected into the global namespace

namespace totw {
namespace example {
namespace {

TEST(MyTest, UsesUsingDirectives) {
  Sequence seq; // ::testing::Sequence
  WallTimer timer; // ::WallTimer
  ...
}

} // namespace
} // namespace example
} // namespace totw

This transformation is not exactly correct, as the names do not actually stay visible outside the scope of the using-directive. However, even a temporary injection into the global scope has some unfortunate consequences.

Let’s see what kind of changes can break this code:

If anyone defines ::totw::Sequence or ::totw::example::Sequence, seq will now refer to that entity instead of ::testing::Sequence.
If anyone defines ::Sequence, the definition of seq will fail to compile, as the reference to the name Sequence will be ambiguous. Sequence could mean ::testing::Sequence or ::Sequence, and the compiler doesn’t know which one you wanted.
If anyone defines ::testing::WallTimer, the definition of timer will fail to compile.

Thus, a single using-directive in a function scope has placed naming restrictions on symbols in ::testing, ::totw, ::totw::example, and the global namespace. Allowing this using-directive, even if only in function scope, creates ample opportunities for name clashes in the global and other namespaces.

If that example doesn’t look fragile enough, consider this:

namespace totw {
namespace example {
namespace {

TEST(MyTest, UsesUsingDirectives) {
  using namespace ::testing;
  EXPECT_THAT(..., proto::Partially(...)); // ::testing::proto::Partially
  ...
}

} // namespace
} // namespace example
} // namespace totw

This using-directive has introduced a namespace alias proto in the global namespace, roughly equivalent to the following:

namespace proto = ::testing::proto;

namespace totw {
namespace example {
namespace {

TEST(MyTest, UsesUsingDirectives) {
  EXPECT_THAT(..., proto::Partially(...)); // ::testing::proto::Partially
  ...
}

} // namespace
} // namespace example
} // namespace totw

The test will keep compiling until a header defining namespace ::proto, ::totw::proto, or ::totw::example::proto gets included transitively. At that point in time, proto::Partially becomes ambiguous, and the test stops compiling. This ties into the style guide’s rules on namespace naming: avoid nested namespaces, and don’t use common names for nested namespaces. (See Tip #130 and https://google.github.io/styleguide/cppguide.html#Namespace_Names for more on this topic.)

One might think that it’s safe to employ a using-directive for a closed namespace that has few symbols and guarantees that no more symbols will be ever added to it. (std::placeholders, which contains symbols _1 … _9, is an example of such a namespace.) However, even that isn’t safe: it precludes any other namespace from introducing symbols with the same names. In this sense, using-directives defeat the modularity provided by namespaces.

Unqualified using-directives

We’ve seen how one using-directive can go wrong. What happens if we have many of them, unqualified, in the same codebase?

namespace totw {
namespace example {
namespace {

using namespace rpc;
using namespace testing;

TEST(MyTest, UsesUsingDirectives) {
  Sequence seq;  // ::testing::Sequence
  WallTimer timer;  // ::WallTimer
  RPC rpc;  // ...is this ::rpc::RPC or ::RPC?
  ...
}

}  // namespace
}  // namespace example
}  // namespace totw

What could possibly go wrong here? A lot, as it turns out:

All the problems from our function-level example still exist, but two-fold: once for namespace ::testing, and once for namespace ::rpc.
If namespace ::rpc and namespace ::testing declare symbols with the same name, this code won’t compile if it does unqualified lookup on one of those names. This is important, because it demonstrates a terrifying scaling problem: since the full contents of each namespace is (generally speaking) injected into the global namespace, every new using-directive could add quadratic risk of name collisions and build failures.
If a sub-namespace such as ::rpc::testing is ever introduced, this code will stop compiling. (We have actually seen that namespace, so it is potentially just a matter of time until this snippet and that namespace are built together. Another reason to avoid deeply nested namespaces). The lack of namespace qualification is important here: this snippet may have compiled if the using-directives were fully-qualified and if there were no unqualified lookups on names common to both namespaces.
A newly-introduced symbol in ::totw::example, ::totw, ::testing, ::rpc, or the global namespace could collide with an existing symbol in any of those namespaces. That’s a big matrix of possibilities.

A brief aside: What namespace do you think RPC lives in? rpc would have been a perfectly reasonable guess, but it actually lives in the global namespace. Maintainability issues aside, the using-directives here make this code hard to read.

Why Do We Have This Feature, Then?

There are legitimate uses of using-directives within generic libraries, but they are so obscure and rare that they don’t deserve a mention here or in the style guide.

Parting Words

Using-directives are time-bombs: code that compiles today could easily stop compiling with the next language version or symbol addition. For external code that is short-lived and whose dependencies never change, this may be an acceptable risk. But beware: if you later decide that you want your short-lived project to continue working over time, those time-bombs may explode.

Tip of the Week #152: `AbslHashValue` and You

2018-09-28T00:00:00-04:00

Originally posted as TotW #152 on June 21, 2018

By Matt Kulukundis

Updated 2020-04-06

Quicklink: abseil.io/tips/152

I love Mozart, but I often make a terrible hash of it. – Simon Rattle

The absl::Hash framework [https://abseil.io/docs/cpp/guides/hash] is now the default hash implementation for the Swisstable family of hash tables (absl::{flat,node}_hash_{set,map}). All types hashable by this framework will automatically be useable as keys in Swisstables.

How Do I Use It?

Let’s say we have a simple Song struct (let’s agree that a song can be uniquely identified by these fields):

struct Song {
  std::string name;
  std::string artist;
  absl::Duration duration;
};

and we want to be able to store an absl::flat_hash_set<Song> or an absl::flat_hash_map<Song, CopyrightOwner>. All we have to do is add a simple friend function like:

struct Song {
  std::string name;
  std::string artist;
  absl::Duration duration;

  template <typename H>
  friend H AbslHashValue(H h, const Song& s) {
    return H::combine(std::move(h), s.name, s.artist, s.duration);
  }

  // Include your implementation of operator == and != here
};

and everything will work!

How Do I Test It?

We provide absl::VerifyTypeImplementsAbslHashCorrectly to verify that a type implements its overload correctly. This function has a few requirements:

The type must implement the == operator correctly.
The caller must provide instances of the type that include any interesting representations for their type. (For example, a type with a small size optimization should include equivalent instances that use the small size optimization and that do not.)

TEST(MyType, SupportsAbslHash) {
  EXPECT_TRUE(absl::VerifyTypeImplementsAbslHashCorrectly({
      MyType(),
      MyType(1, 2),
      MyType(2, 3),
      MyType(0, 0),
  }));
}

absl::VerifyTypeImplementsAbslHashCorrectly also supports testing heterogeneous lookup and custom equality operators.

Intrigued and want to know more? Read up at https://abseil.io/docs/cpp/guides/hash.

Tip of the Week #144: Heterogeneous Lookup in Associative Containers

2018-09-28T00:00:00-04:00

Originally posted as TotW #144 on March 23, 2018

By Samuel Benzaquen

Updated 2020-04-06

Quicklink: abseil.io/tips/144

Introduction

Associative containers associate an element with a key. Inserting into or looking up an element from the container requires an equivalent key. In general, containers require the keys to be of a specific type, which can lead to inefficiencies at call sites that need to convert between near-equivalent types (like std::string and absl::string_view).

To avoid this unnecessary work, some containers provide heterogeneous lookup. This feature allows callers to pass keys of any type (as long as the user-specified comparator functor supports them). See std::map::find for an example of this feature in an STL container.

Transparent Functors

A transparent functor is one that accepts more than one particular type. It must publicize this fact by providing an is_transparent inner type. The actual definition of this inner type is not relevant as it is used merely as a tag. It is common to use a using declaration of is_transparent set to void.

When a container detects a transparent functor their lookup functions will forward the user specified value intact instead of converting it to key_type first (through implicit or explicit conversion).

Implicitly supporting heterogeneous lookup can be dangerous, as the relationship between values might not be maintained after conversions. For example, 1.0 < 1.1, but static_cast<int>(1.0) == static_cast<int>(1.1). Thus, using a double to look up a value in a std::set<int> could lead to incorrect results. These potential bugs are the reason this feature is opt-in.

Using Heterogeneous Lookup For Performance

One common reason for using heterogeneous lookup is performance. We could construct the key_type, but doing so requires non-trivial work that we would rather avoid. For example:

std::map<std::string, int> m = ...;
absl::string_view some_key = ...;
// Construct a temporary `std::string` to do the query.
// The allocation + copy + deallocation might dominate the find() call.
auto it = m.find(std::string(some_key));

Instead, we can use a transparent comparator like so:

struct StringCmp {
  using is_transparent = void;
  bool operator()(absl::string_view a, absl::string_view b) const {
    return a < b;
  }
};

std::map<std::string, int, StringCmp> m = ...;
absl::string_view some_key = ...;
// The comparator `StringCmp` will accept any type that is implicitly
// convertible to `absl::string_view` and says so by declaring the
// `is_transparent` tag.
// We can pass `some_key` to `find()` without converting it first to
// `std::string`. In this case, that avoids the unnecessary memory allocation
// required to construct the `std::string` instance.
auto it = m.find(some_key);

What Else Is It Good For?

Cases exist where it is impossible or inconvenient to create a valid key_type object just to do a lookup. In those cases, we might want to use an alternative type that is much easier to produce, but contains the necessary information for the lookup. For example:

struct ThreadCmp {
  using is_transparent = void;
  // Regular overload.
  bool operator()(const std::thread& a, const std::thread& b) const {
    return a.get_id() < b.get_id();
  }
  // Transparent overloads
  bool operator()(const std::thread& a, std::thread::id b) const {
    return a.get_id() < b;
  }
  bool operator()(std::thread::id a, const std::thread& b) const {
    return a < b.get_id();
  }
  bool operator()(std::thread::id a, std::thread::id b) const {
    return a < b;
  }
};

std::set<std::thread, ThreadCmp> threads = ...;
// Can't construct an instance of `std::thread` with the same id, just to do the lookup.
// But we can look up by id instead.
std::thread::id id = ...;
auto it = threads.find(id);

STL Container Support and Alternatives

The standard ordered containers (std::{map,set,multimap,multiset}) support heterogeneous lookup. The standard unordered containers (std::unordered_{map,set,multimap,multiset}) do not support heterogeneous lookup until C++20.

The new family of Swiss Tables, however, support heterogeneous lookup for both string-like types (std::string, absl::string_view, etc.) and smart pointers (T*,std::shared_ptr, std::unique_ptr). They require both the hasher and the equality function to be tagged as transparent. All other key types require explicit opt-in from the user.

The B-Tree containers (absl::btree_{set,map,multiset,multimap}) also support heterogeneous lookup.

Protocol Buffers’ associative map’s implementation, google::protobuf::Map, supports heterogeneous lookup when the map is keyed with std::string using string-like keys (any type that is convertible to absl::string_view).

Tip of the Week #136: Unordered Containers

2018-09-28T00:00:00-04:00

Originally posted as TotW #136 on June 23, 2017

By Matt Kulukundis

Updated 2020-04-06

Quicklink: abseil.io/tips/136

“Sometimes, when the material is really good, you put expectations on yourself to make it the best possible show. You’re not just serving up the regular hash and doing your job and going home.” — Peter Dinklage

TL;DR: See https://abseil.io/docs/cpp/guides/container for official and up-to-date recommendations. This tip introduced the new types, but is not the canonical reference.

Introducing `absl::*_hash_map`

There is a new family of associative containers in town. They boast improvements to efficiency and provide early access to APIs from C++17. They also provide developers with direct control over their implementation and default hash functions, which is important for the long term evolution of a code base. New code should prefer these types over std::unordered_map . All of the maps and sets in this family have APIs that are nearly identical to std::unordered_map, so transitioning to them is easy.

For every absl::*_hash_map there is also an absl::*_hash_set; however, the diagrams will only depict the map case and we will often refer to just the map.

`absl::flat_hash_map` and `absl::flat_hash_set`

These should be your default choice. They store their value_type inside the main array. Because they move data when they rehash, elements don’t get pointer stability. If you need pointer stability or your values are large, consider using absl::node_hash_map instead, or absl::flat_hash_map<Key, std::unique_ptr<Value>>, possibly.

Warning: Because of pointer instability after rehash() code like map["a"] = map["b"] will access invalidated memory.

`absl::node_hash_map` and `absl::node_hash_set`

These allocate their value_type in nodes outside of the main array (like std::unordered_map). Because of the separate allocation, they provide pointer stability (the address of objects stored in the map does not change) for the stored data and empty slots only require 8 bytes. Additionally, they can store things that are neither moveable nor copyable.

We generally recommend that you use absl::flat_hash_map<K, std::unique_ptr<V>> instead of absl::node_hash_map<K, V>, and similarly for node_hash_set.

Swiss Tables and `absl::Hash`

2018-09-27T00:00:00-04:00

By Sam Benzaquen, Alkis Evlogimenos, Matt Kulukundis, and Roman Perepelitsa

We are extremely pleased to announce the availability of the new “Swiss Table” family of hashtables in Abseil and the absl::Hash hashing framework that allows easy extensibility for user defined types.

Last year at CppCon, We presented a talk on a new hashtable that we were rolling out across Google’s codebase. When asked about its release date, we may have been a touch optimistic. But hopefully it will have been worth the wait. As an added bonus, this release also comes with an entirely new framework for hashing your types. As with all things of this size, this is the work of a great many people.

Swiss Tables boast improvements to efficiency and provide C++11 codebases early access to APIs from C++17 and C++20.

These hash tables live within the Abseil container library.

`absl::flat_hash_map` and `absl::flat_hash_set`

The “flat” Swiss tables should be your default choice. They store their value_type inside the container’s main array to avoid memory indirections. Because they move data when they rehash, elements do not get pointer stability. If you require pointer stability or your values are large, consider using an absl::node_hash_map or absl::node_hash_set (or an absl::flat_hash_map<Key, std::unique_ptr<Value>>).

`absl::node_hash_map` and `absl::node_hash_set`

The “node” Swiss tables allocate their value_type in nodes outside of the main array (as in std::unordered_map). Because of the separate allocation, they provide pointer stability (the address of objects stored in the map does not change). As well, the stored data and empty slots only require 8 bytes. Additionally, they can store things that are neither moveable nor copyable.

We generally recommend that you use absl::flat_hash_map<K, std::unique_ptr<V>> instead of absl::node_hash_map<K, V>.

For more information about Swiss tables, see the Abseil container library documentation.

The `absl::Hash` hashing framework

The absl::Hash library consists of two parts:

absl::Hash<T>, a concrete hash functor object, which you can use out of the box
A generic hashing framework for specializing hashing behavior and making user-defined types hashable

This library is designed to be used as a replacement for std::hash and the various other hash functors. It provides several advantages over them:

It can hash objects of almost any standard type, including std::pair, std::tuple, and most standard containers.
It can be extended to support user-defined types. Our goal is that if it makes sense to hash an object of type Foo, then absl::Hash<Foo> will just work. These extensions are easy to write and efficient to execute.

Importantly, the underlying hash algorithm can be changed without modifying user code, which allows us to improve both it and types which utilize absl::Hash over time. For example, we might wish to change the hashing algorithm within a container to improve performance and to defend against some hash-flooding attacks.

The absl::Hash framework is the default hash implementation for the Swiss tables and does not need to be explicitly specified when working with that library.

For more information, see the Abseil hash library documentation.

Abseil Python Docs are Here!

2018-08-31T00:00:00-04:00

By Junjay Tan

Abseil Python now includes a documentation set on abseil.io!

Abseil Python provides various libraries for building Python applications. This code is collected from Google’s internal Python code base and has been extensively tested and used in production. They provide features for application startup, distributed command line flags, custom logging, and testing.

Until now, the only documentation for Abseil Python has been in the source code. This was comprehensive but did not always provide information for new users in an easy-to-read format.

We’re happy to announce an Abseil Python Quickstart Guide and module-specific programming guides similar to those for Abseil C++. These guides are patterned after those used internally at Google to onboard new Googlers (nooglers), and we hope they will be similarly helpful to you!

Clang-Tidy Checks for Abseil

2018-08-30T00:00:00-04:00

By Deanna Garcia and Hugo Gonzalez, Abseil Interns

Abseil wants to help developers avoid common mistakes unique to our collection of libraries. Therefore, we have developed a set of clang-tidy checks for Abseil, allowing you to catch these errors early on before they become bigger problems.

Clang-Tidy) is a useful tool to help developers write C++ code in a correct and efficient manner. Clang-Tidy now supports a set of Abseil checks designed to diagnose and fix typical programming errors specific to Abseil including:

Compatibility guideline violations
Style violations
Interface misuse
Bugs that can be deduced via static analysis

We hope that these checks will help projects that depend on Abseil have high standards of code quality.

Our checks can be found on llvm’s clang-tools-extra repository, in the clang-tidy/abseil directory. The following checks have been released:

Duration Division Check
Faster StrSplit Delimiter Check
No Internal Dependencies Check
No Namespace Check
Redundant StrCat Calls Check
StrCat-Append Check
String Find Starts with Check

For more information, consult Abseil’s Clang-Tidy Check Guide.

You can also write your own checks and share them with the Abseil community. For more information on clang-tidy, and if you are interested in learning more to get involved in writing your own Abseil clang-tidy checks, consult the Clang-Tidy documentation.

Coroutine Types

2018-07-13T00:00:00-04:00

By Titus Winters

The first hint of standard library design that takes advantage of coroutines (P1056) came through the Library Evolution Working Group (LEWG, responsible for design for the C++ standard library) during the Rapperswil meeting this summer. It was … surprising. As much as I want coroutines in the language, the design smell here was disconcerting. I want to point out what LEWG saw before we commit to this direction irrevocably - coroutine types as currently designed are baffling.

Background

Coroutines are an important new language feature for C++, but also a very old idea. In its current incarnation, the major force behind Coroutines in C++ is Gor Nishanov (whom I absolutely love working with). If you aren’t familiar with the history, or want a refresher on how Coroutines are a generalization of subroutines (function calls), try watching Gor’s talk from CppCon 2015: “C++ Coroutines, a negative overhead abstraction.”

In a function call/subroutine world, the caller invokes some function, the function runs, and then the function returns execution control to the caller. We’re all familiar with this. In an asynchronous programming model we often have a logically-related sequence of operations we want to execute (for instance, kicking off a sequence of async operations one by one) but cannot express as a single function, because we need to jump in and out of that flow of control. As a result, all necessary state gets bundled up and passed around as a “continuation,” to be invoked at some point in the future. This is generally known as continuation passing and you see it in languages/libraries/frameworks that are very functional-style, very asynchronous, or very callback-driven. It’s powerful, but not nearly as clear as direct style programming.

Coroutines are an attempt to provide a language-level ability to write in a direct style and execute in a continuation-passing style. For example, consider an example from P1056:

struct record {
  int id;
  std::string name;
  std::string description;
};
  
std::task<record> load_record(int id);
std::task<> save_record(record r);
  
std::task<void> modify_record() {
  record r = co_await load_record(123);
  r.description = "Look, ma, no blocking!";
  co_await save_record(std::move(r));
}

In this snippet we’re imagining a to-be-specified type std::task<T> that is in many ways a lighter-weight std::future<T>. It isn’t promising anything to do with concurrency or synchronization, and has no allocation or shared state. Functions that return std::task are coroutines - they can suspend (by calling co_await) and return execution to the caller - leaving any variables local to the coroutine’s stack intact.

A caller can invoke modify_record() as a coroutine, allowing the caller to be part of the continuation chain. This requires returning a coroutine-enabled type like std::task<> - but note that the types don’t have to line up.

std::task<void> f() {
  std::cout << "About to modify" << std::endl;
  co_await modify_record();
  std::cout << "Done modifying" << std::endl;
}  

In such an invocation, the two log statements will happen in order, but all sorts of computation may happen in the caller of f() between the two log outputs, because execution is yielded at the point of the co_await “call” to modify_record(). (This is expected: that is the whole point of coroutines.)

Note that because any function that uses the new coroutine keywords is now by definition a coroutine, there are some potential surprises. For example, if we were to invoke f() directly:

void invoke_f() {
  f();
}

This (maybe) compiles, but generates none of the logging output because the body of f() is not executed at all until something invokes co_await on the task returned by f() - which we just ignored. Many/most coroutine types are thus expected to be marked [[nodiscard]] - there is no point in calling a function that returns a type like std::task<> without operating on the task. (Remember: pay attention to those [[nodiscard]] warnings!)

The current specification provides no way for an ordinary (non-coroutine) function to usefully invoke a coroutine that returns task<T> nor to obtain the T that it wraps. In order to invoke f() like a function we would need some special coroutine-execution machinery - the cppcoro repository provides this in the form of a sync_wait() function. The specification of a sync_wait() has not yet been produced, but there will eventually be one overarching sync_wait() that works with anything satisfying the Awaitable concept.

There are a host of other interesting points and gotchas in coroutines, and also a competing proposal produced by my Google colleagues (P1063). All of that additional background is interesting and worth getting into, but is separate from the point of this post.

The `task<>` API

The main question here is “What does std::task<> look like?” Given that coroutines are inherently slicing apart something as fundamental as “what is a function call,” we’re likely going to see something interesting/exotic in the best case. Whether those exotic results are acceptable in trade for this very powerful feature is clearly a matter of taste, and what I want to bring to the attention of the community.

That said, here is the complete callable API for the proposed std::task:

template <typename T> class [[nodiscard]] task {
  public:
    task(task&& rhs) noexcept;
    ~task();
    auto operator co_await();  // exposition only
};   

Take a close look at that.

This is a type that wraps a T … but has no interfaces that mention that T. This is a type that can be move-constructed, and destroyed. Per the “exposition only” comment, it can be used via the new co_await keyword. That’s it. Strictly speaking, co_await doesn’t even return T, it returns a bunch of customization machinery that configures coroutine machinery so that you can get a T. The only way for a user to know that co_await yields a T is to see it in use, or read the comments. The user-facing API for coroutine types like this do not actually show you what you expect to see.

This would represent an unusual step for type design - producing types whose interface does not (and arguably cannot) describe its expected usage. IDEs are going to choke on this, autocompleters are useless, cppreference.com documentation will be novel at best.

The simplest current implementation of task<> (thanks Gor!) runs about 40 lines and is illustrative - most of the body of such a type is detailing its interaction with the coroutine machinery and definition of its related promise type.

It’s well worth spending a few minutes reading through P1056 and the above implementation link. It is certainly possible that coroutines require this complexity and subtlety - it is a fundamental extension to basic programming concepts. But we should be sure that this is well understood including its impact on the language, library, documentation, and the rest of the ecosystem. We should also be sure that we have found the right abstraction boundaries, not just something that works.

Coroutine Type Design

Since the earliest days of the Coroutines proposals it’s been clear that implementing coroutine types is not something that we expect most developers to do: this is specialist work, and libraries like Abseil, Boost, and the standard library will probably do the heavy lifting for most developers. Considering the results that we’ve been able to see with the coroutines Technical Specification and coroutine demos, that’s probably fine - this is powerful, and the resulting user code is pretty reasonable.

That said, if this is the style of specification and user-facing API for coroutine types, I have concerns. The user-facing APIs produced using this machinery are nonsensical and impossible to understand with normal patterns (like “reading the API”) - all of their details are literally hidden in the coroutine customization machinery. It’s technically correct and functional, of course, but something smells wrong.

On the other hand, some coroutine-backed designs seem basically OK. A generator demonstration uses iterators as the user-facing portion of the coroutine API, and the iterator certainly tells me that I get a T when I dereference generator<T>::begin(). Perhaps this is nothing more than “asynchrony is complicated” which should come as little surprise.

Looking at both the generator example and task<> example (both graciously provided by Gor), perhaps the real question is this: how confident are we (the committee and the community) that promise_type and the other coroutine customization points are the right design? Clearly they are a correct and functional design ; as always, I’m terribly impressed with the end result of the code that uses all of this. I’m less convinced that all of these apparent knobs are proveably the right set of basic operations and customization.

I’m not sure what the better answer is, and I have largely been uninvolved in my colleagues’ proposal (P1063) - I’m not sure that such a proposal would produce clearer types or customization points that I was more confident in. But even absent a better proposal, I want the community to look at this and pause. Is anyone else uncomfortable with designs like these? Are we sure we want to rush to include coroutines in the next C++ release, even with these design smells?

As is often the case, I urge the committee and community to take the time to be sure. If we proceed as-is, we’re likely stuck with these designs for a long time to come.

The Abseil str_format Library

2018-06-22T00:00:00-04:00

By Juemin Yang, Abseil Engineer

Abseil now includes a type-safe string formatting library: str_format. The str_format library is a typesafe replacement for the family of printf() string formatting routines within the <cstdio> standard library header. The str_format library provides most of the functionality of printf() type string formatting and a number of additional benefits:

Type safety, including native support for std::string and absl::string_view
Reliable behavior independent of standard libraries
Support for the POSIX positional extensions
Much faster (generally 2 to 3 times faster) than native printf functions
Streamable to a variety of existing sinks

For more information, consult Abseil’s StrFormat Guide If you are interested in the design of this library, check out our StrFormat Design Notes.

Long-Term Support (LTS) Branches

2018-06-18T00:00:00-04:00

By Tom Manshreck, Abseil Tech Writer

Abseil encourages developers to “live at head” but we understand that philosophy may not work for everyone. We are therefore providing snapshots of the Abseil codebase. These snapshots are available as “Long Term Support” (LTS) branches of Abseil, and we intend to provide a new snapshot every 6 months or so.

We pledge to support these LTS snapshots for at least 2 years. If critical bug fixes, such as security issues, require us to change Abseil, we will also change them within any supported LTS snapshot.

NOTE: we don’t want you to think of these snapshots as “versions.” They are simply a snapshot of the codebase at a specific point in time. If you cannot build from source or otherwise live at head, prefer to use the latest LTS branch of Abseil instead.

For more information, consult Abseil’s Release Management guide.

Revisiting Regular Types

2018-05-31T00:00:00-04:00

Good types are all alike; every poorly designed type is poorly defined in its own way. - Adapted with apologies to Leo Tolstoy

By Titus Winters

Abstract

With 20 years of experience, we know that Regular type design is a good pattern - we should model user-defined types based on the syntax and semantics of built-in types where possible. However, common formulations of Regular type semantics only apply to values, and for performance reasons we commonly pass by reference in C++. In order to use such a reference as if it were its underlying Regular type we need some structural knowledge of the program to guarantee that the type isn’t being concurrently accessed. Using similar structural knowledge, we can treat some non-Regular types as if they were Regular, including reference types which don’t own their data. Under such an analysis, string_view indeed behaves as if it were Regular when applied to common usage (as a parameter). However, span does not, and further it is (currently) impossible to have shallow copy, deep compare, and Regular const semantics in the same type in C++.

This analysis provides us some basis to evaluate non-owning reference parameters types (like string_view and span) in a practical fashion, without discarding Regular design.

Introduction

What are Regular types? In the space of type design, “Regular” is a term introduced by Alexander Stepanov. (The online/reduced form is Fundamentals of Generic Programming which I will cite in this essay. You’re encouraged to read the full book “Elements of Programming” for a more complete treatment.) By and large, the term Regular is meant to describe the syntax and semantics of built-in types in a fashion that allows user-defined types to behave sensibly.

“The C++ programming language allows the use of built-in type operator syntax for user-defined types. This allows us, as programmers, to make our user-defined types look like built-in types. Since we wish to extend semantics as well as syntax from built-in types to user types, we introduce the idea of a Regular type, which matches the built-in type semantics, thereby making our user-defined types behave like built-in types as well.”

A vastly-simplified summary, that has become popular in C++ design in later years, is “do as the ints do.” Generally speaking, for a snippet of generic code operating on some type T, if it works properly on ints and also works properly for your new type (similarly bug-free/comprehensible/etc.), you’ve designed a reasonable type.

Definitions of Regular

In fact, this evaluation of generic code is inherent in the (somewhat flexible) definitions for Regular; Stepanov defines certain properties for Regular, but different formulations may include a slightly different set of requirements. In particular, proposal P0898 aims to define Concepts for use in the C++ standard library; the requirements for Regular as presented in P0898 are somewhat reduced from Stepanov’s definition (ordering is not required). The crux of this difference seems to be that Stepanov’s definition stems from the requirements of a type with respect to the full set of C++98-era standard algorithms, while P0898 focuses primarily on the requirements to be used in a more modern library (specifically, the Ranges library as specified in Range v3 and/or the Ranges TS).

Stepanov's Regular Requirements	P0898 Concept Requirements for Regular
Default constructor	DefaultConstructible
Copy constructor	CopyConstructible
Destructor	Destructible
Movable?	Movable
	Swappable
Assignment	Assignable
Equality	EqualityComparable
Inequality	EqualityComparable
Total Ordering

Note: it’s generally assumed that Move construction would be included in Stepanov’s formulation, although move semantics were not present in C++ of that era. This is true even though int doesn’t have a move constructor: it is still move constructible (can be constructed from a temporary), and this is even the proper design for the type. Move+copy should be considered an overload set for optimization purposes. For more information, see TotW 148)

In either case, it’s important to bear a few (related) ideas in mind:

The various definitions of a Regular type are more alike than they are different.
After 20 years of experience, we’re confident that the definition of a Regular type is a useful abstraction, and approximates the right “default semantics” for user-defined types.
The definitions for a Regular type come from use in generic contexts. The essential aim is to define the syntax and semantics of a value type that mimics the behavior of a built-in type. The semantics of Regular allow us to reason about the use of the type within an algorithm, and hence define what the algorithm does.
The reasoning about code is much easier if the code consists of Regular types, instead of non-Regular ones, using the existing understanding of how built-in types work.

Both definitions focus heavily on semantics not just syntax. It is from these basic semantic properties of types that we find design invariants like the idea that copy, comparison, and const-ness are related. For instance, consider four of the most basic semantic requirements on Regular types from Stepanov’s early paper:

// comparison follows from copy
T a = b; assert(a==b);

// copy and assignment are the same
T a1; a1 = b; T a2 = b; assert(a1 == a2);

// copy/assignment is by value, not reference
T a = c; T b = c; a = d; assert(b==c);

// zap always mutates, unmutated values are untouched
T a = c; T b = c; zap(a); assert(b==c && a!=b);

The above are enough to define copy and assignment semantics: T has copy/assignment that operate by value, not by reference. However, all of this is assuming we have an ability to define equality. Stepanov points out that defining equality is a little squishy.

“Logicians might define equality via the following equivalence:

x == y ⇔ ∀ predicate P, P(x) == P(y)

That is, two values are equal if and only if no matter what predicate one applies to them, one gets the same result. This appeals to our intuition, but it turns out to have significant practical problems. One direction of the equivalence:

x == y ⇒ ∀ predicate P, P(x) == P(y)

is useful, provided that we understand the predicates P for which it holds. We shall return to this question later. The other direction, however:

∀ predicate P, P(x) == P(y) ⇒ x == y

is useless, even if P is restricted to well behaved predicates, for the simple reason that there are far too many predicates P to make this a useful basis for deciding equality.”

Programming languages inherently contain predicates that don’t exist in pure math, because the execution on computing hardware is a somewhat leaky abstraction. Consider: the question of “Are x and y aliases for the same memory location?” This is an easy predicate to write in C++, but nonsense for a traditional logician. In general, we focus on predicates that observe “the value” rather than the identity of the instance.

Stepanov goes on to describe the built-in notion of equality for most types: bitwise equality for ints and pointers. Floating point values are glossed over a bit via “although there are sometimes minor deviations like distinct positive and negative zero representations.” From this as a basis, we can begin to build up a definition of equality for aggregates, but immediately get into trouble because of heap allocations. “… objects which are naturally variable sized must be constructed in C++ out of multiple simple structs, connected by pointers. In such cases, we say that the object has remote parts. For such objects, the equality operator must compare the remote parts …”

This reasoning continues, including some discussion about the physical state vs. logical state of a type (for instance: the capacity of a vector does not figure into its equality comparison, nor does the use of SSO affect equality comparison for string).

Ultimately, Stepanov proposes the following definition, “although it still leaves room for judgement”:

“Definition: two objects are equal if their corresponding parts are equal (applied recursively), including remote parts (but not comparing their addresses), excluding inessential components, and excluding components which identify related objects.”

If we have a solid understanding of how to compare two instances of our type (by value, focusing on the logical state, not comparing by identity/memory location), and if our type implements all the syntactic/semantic requirements for Regular then we have implemented a Regular type. That should always be our starting point when designing a value type.

Modern Regular Types and const

With an agreed-upon understanding of Regular, we can start to examine the ways that Regular is used. We can reasonably assume that a Regular type can be fed through standard algorithms, since that’s a primary motivation for defining Regularity (or, conversely, the implied requirement for the algorithms, although the legacy algorithms are rarely specified in terms of semantic requirements).

So we can certainly express that this is safe and correct for Regular types, yes?

void DoSomething(const T& t);  // May be any valid C++

const T a = SomeT();
const T b = SomeT();
if (a == b) {
  DoSomething(a);
  assert(a == b);  // Note: we're assuming the semantics of ==
}

This seems straightforward: we have two const values. If they are equal, it doesn’t matter what operation we perform on one of them, they will remain equal. It also doesn’t matter if we modify any other global state. It does imply one additional requirement beyond being Regular: the normal semantics of const must be enforced - a const object must not change values. Consider the following type and body for DoSomething:

class Rotten {
 public:
  Rotten();
  Rotten(const Rotten& rhs) : val_(rhs.val_) {}
  Rotten& operator=(const Rotten& rhs) { val_ = rhs.val_; return *this; }

  bool operator==(const Rotten& rhs) const { return val_ == rhs.val_; }
  void Increment() const { val_++; }

 private:
  mutable int val_ = 0;
};

void DoSomething(const Rotten& r) { r.Increment(); }

This Rotten type meets the syntactic requirements for P0898 Regular: it is DefaultConstructible, CopyConstructible, Movable, Swappable, Assignable, EqualityComparable, and it has the semantics for all of those operations. Where it fails is in the General front-matter for standard library concepts. Consider, from P0898r2: “except where otherwise specified, an expression operand that is a non-constant lvalue or rvalue may be modified. Operands that are constant lvalues or rvalues must not be modified.”

This restriction (while being a little squishy about “modified”), plus the basic ideas of “equality” are enough to see that for at least the logical state of a type, const must mean const. Since Rotten goes out of its way to break that, the P0898 formulation of Regular rightly spots that bad type design and forbids it.

Since the original Stepanov paper never discusses const, but const is tied deeply to modern type design, I’ll primarily focus the remainder of the discussion on P0898 for simplicity and clarity.

Data Races and Thread Safety Properties

Let us digress a little to discuss something that is unrelated on the surface, but critical to the ensuing discussion: data races and the thread safety properties of types.

Very significant essays and presentations can (and should) be produced on the topics of data races in C++ and the thread safety properties of types. The following is at most a surface treatment. If you are unfamiliar with this domain, please find a good in-depth tutorial/refresher rather than relying on only this brief summary.

More or less: a data race occurs when at least one memory location is written by one thread and accessed (read or write) by another thread without synchronization between those operations. Any number of threads may read concurrently, but if any thread is writing, you must synchronize those operations. That synchronization can take many forms, ranging from the use of std::atomic up to a mutex or higher-level synchronization primitive. But no matter what you think you know about how execution works on your processor, on the C++ abstract machine there is no such thing as a safe data race. The C++ standard specifically calls this out: data races are undefined behavior. No correct program has undefined behavior. There is no wiggle room.

At a slightly higher level: how do we tell the difference between a read operation and a write operation for a type? For user-defined types we look at the API documentation, where the const qualifier nicely summarizes the read/write semantics of the API. For the vast majority of types (i.e. those that have the same thread-safety behavior that int does), concurrent (non-synchronized) calls to const methods are allowed, but if any concurrent call is made to a non-const method, there is the chance for a data race.

For instance: if you have an optional<int> shared among many threads, those threads may all ask has_value() or read from the contained int, so long as none of them overwrites the int. Such a store would take place via operator= (non-const) or assigning to the reference returned by the non-const overloads of value().

It has become common practice to classify types as either “thread-safe”, “thread-compatible”, or “thread-unsafe”, based on the conditions under which use of its API may result in a data race.

Thread-safe: No concurrent call to any API of this type causes a data race. This is useful for things like a Mutex. Generally speaking, thread-safe types are easiest to work with, but you pay for some of that usability in performance or API restrictions or both.
Thread-compatible: No concurrent call to any const operation on this type causes a data race. Any call to a non-const API means that instance must be used with external synchronization. C++ guarantees that standard library types are at least thread-compatible. This follows from the general pattern of Regular design, and “do as the ints do” as int is thread-compatible. In most cases, this is in-line with the philosophy of C++ - you do not pay for what you do not use. If you operate on an optional<int>, you can be sure that it isn’t grabbing a mutex. On the other hand, thread-compatible may have overhead in some cases: shared_ptr<> is unnecessarily expensive in cases where there is no sharing between threads, because of the use of atomics to synchronize the reference count.
Thread-unsafe: Even concurrent calls to const APIs on this type may cause data races - use of an instance of such a type requires external synchronization or knowledge of some form to be used safely. These are generally either used with a mutex or are used with knowledge like “I know that this instance is only accessed from this thread” or “I know that my whole program is only single threaded.” Types like this may be because of mutable members, or because of non-thread-safe data that is shared between instances.

It is worth mentioning that “const means const” is almost enough to ensure that a type is thread-compatible. It is possible to have members in your type that are not part of the logical state (for example: a reference count) but are mutable (either via the mutable keyword or through something like a const-pointer-to-non-const) and thus cause data races even when only const APIs are invoked. There’s a strong conceptual overlap between const-ness (conceptually, including both syntax and semantics) and thread-compatibility, and that overlap is actually equivalence in the case that there are no such mutable members.

Now considering thread-safety and const-ness, we can consider whether either the Stepanov or P0898 definitions of Regular say everything we want. If we’re following the model of int or the good standard library types, Regular types have a powerful property: if you have a const T, and pass that instance as const T& to some function that function can do nothing that can make your T change value (that isn’t inherently UB), become invalid, or otherwise become harder to use. In order to hold that property, your Regular type needs to be thread-compatible, or you need to constrain the function (promise to not share this instance among threads, for instance). Both approaches are useful.

Dependent Preconditions

One property of Regular that never seems entirely satisfying in less formal design conversations is that both int* and std::string are equivalently Regular, although any programmer will tell you that there is a lot more to worry about when working with int*: there are preconditions when using an int* that are harder to ensure than any precondition when using std::string.

For instance, std::string::operator[] has a precondition roughly of the form, index < size(). In fact, the only APIs on std::string that have preconditions of any form fall into two clear categories:

They are requirements that can be checked using other methods from the string API. For instance, the above precondition on operator[] can be checked by calling size().
They are requirements entirely focused on the data being passed in, not on string itself. For instance, the const char* constructor (non-NULL, valid allocation, nul-terminated), or the iterator-range construction/assignment operations (valid range).

For comparison, int* has at least one precondition of a fundamentally different flavor:

operator* - Requires that the pointer holds the address of a live int object.

This cannot be checked via any operation on int*, nor can it even be checked portably in any fashion given an arbitrary int*. Invoking this operation safely requires structural knowledge of the program. A type that has dependent preconditions has one or more such APIs; these are often (but not always) about properties of non-owned objects/external memory/etc.

APIs that have dependent preconditions are more complicated to use - they fundamentally require knowledge about the rest of the program in order to use safely. Types that have no such APIs are easier to use, and it is no surprise or coincidence that the majority of types that we consider “good” avoid this property. Or, put another way, types with dependent preconditions have a weaker form of the property we want from Regular: when passing a const T& to a function, that function may be able to invalidate the preconditions on some API of T through mechanisms that are outside of T. It isn’t enough to say that T is thread-compatible - we must know that nothing in the program is going to invalidate the dependent preconditions.

Let’s consider int* in the context of our Regular-code usage snippet:

using T = int*;
void DoSomething(const T& t);

const T a = SomeT();
const T b = SomeT();
if (a == b) {
  DoSomething(a);
  assert(a == b);
}

Is it really the case that DoSomething() can do anything we want and leave this snippet intact? For an int*, it definitely isn’t - certainly not to the same extent as it is for int. (There is perhaps some overlap between this and Lisa Lippincott’s recent work What is The Basic Interface.)

For instance, if we dereference the pointer, we may have introduced a data race: int* is only thread-compatible if never dereferenced, but we do not have knowledge that nothing else is modifying the underlying int. Or worse, if we dereference and write to the pointer, we make it a race if any thread is even reading the int.

void DoSomething(int* const t) { std::cout << *t << std::endl; }

P0898r2 gets at some of this in [concepts.lib.general.equality] p3: “Expressions required by this specification to be equality preserving are further required to be stable: two evaluations of such an expression with the same input objects must have equal outputs absent any explicit intervening modification of those input objects. [ Note: This requirement allows generic code to reason about the current values of objects based on knowledge of the prior values as observed via equality preserving expressions. It effectively forbids spontaneous changes to an object, changes to an object from another thread of execution, changes to an object as side effects of non-modifying expressions, and changes to an object as side effects of modifying a distinct object if those changes could be observable to a library function via an equality preserving expression that is required to be valid for that object. — end note ]”

In order to reason about a type that has dependent preconditions (like int*), to use it in standard algorithms, or to do much of anything with the type, we must have additional knowledge: why do we know that a given instance is being used in a race-free fashion?

The same is implicitly true for traditional Regular types. When handed a newly-constructed std::string, you know everything you need to know in order to operate on that object safely, even in the face of data races. If you don’t pass it to another thread, it isn’t shared. You are data race free purely by virtue of having the object and knowing there are no additional (mutable) references to it anywhere. If we instead only hand you a std::string& (as is the case for basically all usage of standard algorithms), we don’t actually have that knowledge. Without knowledge that it is unshared, you cannot compare it or copy it in a race free fashion.

This requirement is unstated, and pervasive. Everything in the Stepanov-era standard library implicitly assumes it. In the Ranges/Concepts era, we get text like P0898, rightly forbidding spontaneous changes to an object. But in the general case, the standard already says this, because of the language rules on data races, and the library rules that require most library types are effectively thread-compatible.

Put another way: what do you need to know when invoking a standard algorithm on some user-defined type? You need to know the syntax and semantics of its basic API - it needs to be Regular. But you also need to know that invocation of the algorithm on that instance is race free. A few types give you that by being thread-safe. The rest of the time, you have to know something about the structure of your program and how your instance interacts with the program in order to guarantee race-free.

Interestingly, it isn’t only operator* on a pointer that has dependent preconditions: it is implementation-defined behavior to invoke operator== on a pointer after the underlying object has been deleted. According to Richard Smith, it is “as-if we scribble over every pointer in the program that points to the object at the time of deletion.” So if we have

void DoSomething(int* const t) { delete t; }

we are already off in the realm of non-portable programs. And while implementation-defined is less bad than undefined, this does further demonstrate that even operator== for int* has preconditions that are impossible to check - we have to rely on structural knowledge of the program to use pointers correctly.

Which, luckily, clarifies most people’s intuition: pointers are more complicated than the other types we talk about as Regular. There’s room for expansion in our definition / usage / discussion of Regular. I believe that the missing piece for Regular is “thread compatible” in the general case (although there are other valid options), and existing design precedent in the library already follows that direction. Builtins and our vocabulary types are both Regular and thread-compatible.

Flavors of Race-Free, plus Regular

While Regular+thread-compatible is the most common (and most like int) combination that actually describes good/built-in-like types, other options may work in some situations. These options correspond roughly to the answers for “How do you know that operations on this instance do not cause data races?”

For any given instance of a type, you might know one of several possible things that allow you to operate on it race-free.

Thread-compatible and not shared with other threads for writing. If you’ve been handed a (non-racing) const T& you can operate on this in const fashion. If necessary, you can copy it to ensure there are no lurking references and perform any computation / mutation safely (but inefficiently). With minor knowledge (the instance isn’t shared), a T& can be used safely as if it were T.
It has dependent-preconditions, but for a particular instance + any dependent data, the program structure guarantees safe usage.
Single-threaded usage - There is only one thread in the program and thus all instances of the type are safe to use, or a given instance is known to not be shared among threads.

P0898r2’s [concepts.lib.general.equality] p3, cited above, would fix the standard library’s stance on Regular to be entirely the first option … at the cost of pointers not being considered Regular because of semantic requirements and implementation-defined behavior on comparison. (If deletion is “as-if all other pointers to this object are scribbled over” that certainly violates the restriction on the value of the object changing out from under us.)

All three of these options provide points in the type design space that are useful in conjunction with the existing definitions for Regular.

Thread-compatible + Regular is what we really want for user-defined types that mimic built-ins. This lets us reason about an instance in the expected fashion and use it efficiently in conjunction with generic algorithms. Types that have mutable data may have some overhead to support this.
Dependent-preconditions with knowledge that an instance + its dependent data are safe to use. This is the common usage for string_view when we use it as a non-owning parameter type: the underlying buffer will outlive the function call and is immutable for the duration of the call. Given that external knowledge of that underlying buffer, string_view behaves as if it were Regular. This makes sense, given that string_view was designed to be a drop-in replacement for const string&, and although references are not Regular types, std::string types are.
Single-threaded usage - This is easy to misuse, but can be an important area for optimization. Consider the discussions to provide a shared_ptr analogue that does not synchronize its reference count - if we know something about program structure, or can guarantee particular usage for an instance, we can design a more efficient type in this fashion. Given that knowledge, such a shared_ptr can still behave as if it were Regular.

Given a type and one of the above options to explain why it is safe to use a given instance, we can see that our nice property for Regular types still holds. Given a const T (and some program knowledge), we can perform any operation (that conforms to that program knowledge) on that const T without invalidating it or any of its API preconditions. Regular+thread-compatible types allow us this invariant with no constraints on the program or that operation. Lesser invariants require more knowledge - in return we tend to get lower-overhead, which is a very C++ style of tradeoff.

Evaluating `string_view`

If we know that the underlying buffer exists and is not being mutated, string_view behaves as if it were Regular: it is DefaultConstructible, CopyConstructible, Movable, Swappable, Assignable, EqualityComparable. If the underlying buffer is immutable for the life of our instance, the General Matter rules in P0898 don’t kick in: the value won’t magically change out from under us.

using T = string_view;
void DoSomething(const T& t);

const T a = SomeT();  // Assume SomeT() is providing a
                      // long-lived and stable buffer.
const T b = SomeT();

if (a == b) {
	
  // Won't modify the buffer, provided our assumption on SomeT() is correct.
  DoSomething(a);

  assert(a == b);
}

We can go back to Stepanov’s axioms about assignment and comparison.

string_view a = b; assert(a==b);
string_view a1; a1 = b;  string_view a2 = b; assert(a1 == a2);
string_view a = c; string_view b = c; a = d; assert(b==c);
string_view a = c; string_view b = c; zap(a); assert(b==c && a!=b);

So long as the values we are assigning to represent buffers that outlive the string_view and remain immutable, these axioms are held. (zap() has to actually change its parameter so that pre/post calls to zap are not equal - but remember that equality is about value not identity)

If we do not know that the underlying buffer exists and is not being mutated, we cannot evaluate any of these snippets. Even executing operator== runs the risk of undefined behavior either from use-after-free (if the buffer has disappeared) or data race (if the buffer is mutated).

Remember that this is a lot to ask: this knowledge of the underlying buffer is a lot more than the knowledge required when operating on something Regular like a string. I’ve probably seen hundreds of bugs stemming from people getting the object lifetimes wrong with string_view and its underlying buffer.

Without that external knowledge, it’s easy to see how we fail with all of Stepanov’s axioms (assign these string_views to temporaries). Similarly, without that knowledge, we fail at P0898r2’s [concepts.lib.general.equality] p3 - the value can change out from under us. We get (countless) examples like this:

string s1 = "hello";
string s2 = "hello";

string_view SomeT() {  // Give out references to different global strings
  static int count = 0;
  string_view ret = (count == 0 ? s1 : s2);
  count = (count + 1) % 2;
  return ret;
}

void DoSomething(const string_view sv) {  // modify an unrelated global
  s1[0] = 'a';
}

void f() {
  const string_view sv1 = SomeT();
  const string_view sv2 = SomeT();
  if (sv1 == sv2) {      // equal
    DoSomething(sv1);    // No program structure constraints, can modify a global.
    assert(sv1 == sv2);  // failure: "aello" != "hello"
  }
}

Side note: At some level I think this is a bogus example. Having participated for many years in mailing list discussions about the Google-internal type that inspired string_view, the way that people mishandle string_view has nothing to do with mutability of the underlying data, and everything to do with the underlying data being unowned and going out of scope. I estimate the prevalence of lifetime-requirement failure vs. underyling-mutability failure to be at least 50:1 - the risks of a type like this are almost entirely based around lifetime, not remote-mutation. It’s also much easier to construct an example for this snippet that fails because of insufficient buffer lifetime.

So no, string_view isn’t quite Regular. Given particular very constrained usage, it behaves as if it were Regular. That usage is very nearly required by the rest of the standard - you can’t even compare a string_view without undefined behavior unless you have knowledge about the program structure to guarantee that operation is safe. However, with that knowledge, string_view still behaves as if it were Regular.

Non-owning Reference Parameters

C++ is a language that is very concerned with 2 things: types, and efficiency. The type system for C++ is more complex than most other languages by a significant margin: this is the only mainstream language that I can imagine where it’s reasonable to envision a proposal for an infinite family of Null types (P0196).

At the same time, C++ focuses a great deal of effort on “do not pay for what you do not use”. We have a preponderance of non-Regular types in the core language: references are not Regular, but they are efficient. Passing by reference is basic - when invoking a function we don’t necessarily want to copy anything, we merely give a reference for the duration of the function call. An increasing amount of modern design is about providing flexibility when it comes to the specific types that are accepted - we overload on const char* and const string&, or we just accept string_view. Other user-defined types that have a contiguous buffer of characters can join in our overload set by providing a conversion to string_view.

This is, in some respects, one of the big areas for design work in C++ these days: building an increasing set of generic/type-erased types that accept a broad selection of related types and provide a uniform interface. Consider function, function_ref, span, and string_view, from recent standards discussion. In Google’s codebase I’m starting to see types like AnySpan which behave like span but further abstract T away from the underlying type (allowing non-null unique_ptr and T*, for instance).

From a usability perspective, we have to decide whether we are continuing down the path of building types that represent our duck-typing overload sets, or whether we are asking users to implement those overload sets inconsistently on an ad-hoc basis. I’m fairly sure that most of us will conclude that there is value in using these non-owning reference parameter types - if we can agree on how to design them.

When used as a parameter, we always know that the underlying data continues to exist and is as safe to access as making a copy of the data would have been. (That is, if the underlying data is a reference itself, and we don’t know if anything is mutating it, we’re already in trouble.)

Restated: the arguments against string_view and other reference types are that they are not Regular. And that’s true, they aren’t when evaluated in a vacuum. But if you have any structural knowledge about the program, they behave as if they are Regular in their most common usage - as parameters.

So since we’re probably going to keep building non-owning reference parameter types, can we stop arguing about them being bad because they aren’t Regular? They may be Regular enough for the use case they are designed for, and if a C++ programmer chooses to use them for more, they’ll still usually be Regular given knowledge of the lifetime of their referent. (Or the lifetimes won’t match up and we’ll be cursed for being a hard language, but that’s unavoidable.)

Taking into account APIs with dependent preconditions and implementation-defined behavior on deleted pointer comparison, string_view is no worse than int*, after all.

Evaluating `span`

When discussing reference types in the standard, the two common points of reference are string_view (already voted in with C++17) and span (heading to the working draft for C++20). Although there are numerous syntactic differences in the APIs of these two, semantically the major difference is in the mutability of the underlying data. A string_view is either a lightweight reference to a string that it will not mutate, or a non-owning pointer+length to a buffer it will not mutate (depending on whose conceptualization you follow), but in either case it does not mutate the underlying buffer. A const string_view is only interesting in that it cannot be trimmed nor reassigned.

On the other hand, span<T> allows non-const operations on the underlying T. If span were a container like vector we would know that a const span would imply only const access to the underlying T. In this sense, when comparing to string_view we have no precedent to draw upon.

Let’s apply the idea that we want span to be as close to Regular as possible. What would it take for us to operate on an instance as if it were Regular? Keep in mind that we are already assuming that the program structure forbids mutation of the buffers except via the span directly and thus DoSomething() cannot modify a1 or a2.

std::array<int, 3> a1 = {1, 2, 3};
std::array<int, 3> a2 = {1, 2, 3};
const span<int> s1 = a1;
const span<int> s2 = a2;

if (s1 == s2) {
  DoSomething(s1);
  assert(s1 == s2);
}

We know that can only use span as if it were Regular if we know that the underlying buffer isn’t being mutated by anything else. But that isn’t enough here: it’s easy to imagine a DoSomething that modifies a value through the const span and then breaks our assertion.

void DoSomething(const span<int>& s) {
  s[0] = 42; // Having a const span<T> doesn't make the underlying T const
}

Perhaps what we want is for span to only provide const access to the buffer when the span is const? We could make the const overload of span::operator[] provide const T&. Unfortunately, this isn’t enough for a copyable type.

void DoSomething(const span<int>& s) {
  span<int> copy = s; // Copying drops const but leaves the referent
  copy[0] = 42;  // Can modify through the copy
}

We have to treat the span and its dependent data as one, and that treatment must include const propagation. Unfortunately, we cannot get shallow copy (copy by pointer), const propagation (const reference implies const referent), and deep equality (compare by pointee) in the same type. You can, however, get close enough to const propagation by disallowing mutation of the underlying buffer.

For non-owning reference parameters like string_view and span, merely knowing the underlying buffer isn’t being mutated externally isn’t enough for it to be used as if it were Regular. We need to make one further choice:

The data cannot be mutated through the reference type, a la string_view
Equality is shallow: the logical state of the type is tied to the value of the pointer, rather than the value of what it points to
The reference type is designed in a clever fashion so that once you have a const Reference<T>, copies become at most Reference<const T>. (I suspect this is impossible in the language currently, and even if it isn’t it is likely to be awkward to work with, but I’m intrigued at the possibility).

The stated aims of span (to replace easily-mishandled instances of T*, len) cannot be met while having both Regular semantics and deep equality - the logical state of the type must be only the pointer+length, not the underlying unowned data. When given the choice, we should prefer to be more like Regular and change to shallow equality. It’s still arguable whether it will be Regular (dependent preconditions on operator[], and implementation-defined behavior for comparison on a deleted pointer), but it will be much closer.

With such a change to its comparison, and knowledge that the underlying buffer isn’t being modified, span will behave as if it were Regular. Given a const span and the required knowledge about its buffer, nothing we do to it can make its value change or invalidate any of its API preconditions.

Conclusion

If you have got the option, make your value types Regular and thread-compatible with no mutable or shared state. Do not take any of the above as justification to break that commandment lightly.

That said, Regular as everyone describes it does gloss over some things - we operate on Regular types by reference in standard algorithms constantly, and those operations aren’t safe without some form of structural knowledge of the program. Usually that knowledge is of the form “this instance isn’t shared to any other thread.” The best Regular types, those that model built-ins most closely, are thread-compatible and have no dependent preconditions. Types like int and string are easier to work with than int* or string_view.

When it comes to non-owning reference types like string_view or span, the required structural knowledge is more complex: it isn’t only that the instance isn’t shared, it’s that the instance plus its underlying data isn’t shared in a way that will cause data races. When operating on a type arbitrarily, that is hard to prove. But in the very common (and very relevant to C++ design priorities) case of building cheap non-owning reference parameters, we have that knowledge because of how parameter passing and function invocation work.

If knowledge that the underlying data isn’t shared is enough to make usage of an instance passed as a parameter behave like Regular, that is good. These types like string_view will have more sharp corners and take some getting used to, but in practice most of the problems come from underlying buffers going out of scope when used not as a parameter. Critically, such types must not allow mutation of their unowned underlying data.

For types where we want reference semantics but must allow for mutation of the underlying data, we must yield something. It is currently impossible to have a type that has shallow-copy, deep-compare, and Regular behavior when propagating const-ness: one or more of those properties must be given up. The expected usage of every type is different, so it is hard to provide one-size-fits-all guidance, but consider the following options:

shallow compare - Make the type compare based on what it points at by identity (pointer value) rather than dereferencing the underlying data.
SemiRegular - P0898 specifically defines the concept SemiRegular to denote types that are Regular except for a lack of operator==. These are still perfectly suitable for storing in many types of containers. Dropping operator== will leave no users confused at runtime when equality semantics do not match their expectation.

If equality isn’t the culprit for your type being irregular (even given knowledge of program structure), cut or rename the offending operations until it is a strict subset of Regular - it is better for your type to not reuse common syntax for operations that have non-Regular semantics.

As a satisfying side-note: classifying types based on the form of structural knowledge they need in order to operate on them safely provides us a way to separate types like int and int*. It has long been unsatisfying that pointers and values were both considered Regular in exactly the same fashion.

Accounting for this variation in “how much information do we need when working on an instance of this type safely”, we can look at reference types when their underlying data is extant and constant and determine whether the rest of their usage still looks Regular. Pleasantly, string_view works fine. We do find some weaknesses in the current span proposal, particularly when it comes to shallow vs. deep const.

Abseil No Longer Depends on CCTZ

2018-05-11T00:00:00-04:00

An Update On Our Dependencies

By Shaindel Schwartz, Abseil Engineer

As of the April 23 update, af78826 , Abseil no longer requires an external dependency on CCTZ. If you have included the CCTZ project solely to satisfy the Abseil dependency, you can now safely remove it from your project’s setup.

We are in the process of adding some updates to our Time library. As a transitional step in this set of planned updates, we have added an internal fork of the CCTZ types and dropped the dependency on the external CCTZ project. To avoid potential name conflicts with a project’s independent use of the CCTZ library, our internal copy lives in a unique, internal absl namespace. The internal copy is intended solely as an implementation detail underpinning our provided Time APIs. As with all internal types, users should not depend on or use them directly.

Note that if your project has its Abseil dependency pinned at a specific commit, you may still need the dependency on CCTZ. (For a commit earlier than af78826 , the dependency on CCTZ is still required.) We encourage you to live at head.

Tip of the Week #149: Object Lifetimes vs. `= delete`

2018-05-03T00:00:00-04:00

Originally posted as TotW #149 on May 3, 2018

By Titus Winters

Updated 2020-04-06

Quicklink: abseil.io/tips/149

Into the blue again after the money’s gone Once in a lifetime, water flowing underground –David Byrne

`=delete` for Lifetimes

Imagine you have an API that requires a reference to some long-lived object, but doesn’t take ownership of it.

class Request {
  ...

  // The provided Context must live as long as the current Request.
  void SetContext(const Context& context);
};

You think to yourself, “Hey, what happens if someone passes a temporary? That’s going to be a bug. But this is modern C++, I can prevent that!” So you rig together a change in the API, adding a deleted overload.

class Request {
  ...

  // The provided Context must live as long as the current Request.
  void SetContext(const Context& context);
  void SetContext(Context&& context) = delete;
};

Pleased with your work, you think “Hey, now the API says everything, the comment isn’t necessary.”

class Request {
  ...

  void SetContext(const Context& context);
  void SetContext(Context&& context) = delete;
};

Was this a good idea? Why or why not?

Don’t Design In a Vacuum

As presented, you might think that this is a good idea. However, as in many cases of API design it is tempting to look at the definition of the API, but more useful to look at how the API is used. So lets replay this scenario, taking into account usage.

A user attempting to use the original SetContext(), trying to get something simple to build and not knowing where to find the right Context object, just makes the suggested call.

request.SetContext(Context());

Without your =delete change, this builds, but fails at runtime (probably in a mysterious fashion). When looking at the SetContext API, the lifetime requirement is documented, and the code is changed to comply.

request.SetContext(request2.context());

A user attempting to use the “improved” SetContext() with your =delete change and no comment, on the other hand, first encounters the build break:

error: call to deleted member function 'SetContext'

  request.SetContext(Context());
  ~~~~~~~~^~~~~~~~~~

:4:8: note: candidate function has been explicitly deleted
  void SetContext(Context&& context) = delete;

The user then thinks “Well, I can’t pass a temporary”, but having no information about the actual requirement, what is the most likely fix?

Context context;
request.SetContext(context);

Now, the crux of the matter: How likely is it that the scope of the new automatic variable context is the right lifetime for this call? If your answer is anything less than 100%, the lifetime requirement comment is still necessary.

class Request {
  ...

  // The provided Context must live as long as the current Request.
  void SetContext(const Context& context);
  void SetContext(Context&& context) = delete;
};

Deleting a member of an overload set in this fashion is at best a half-measure. Yes, you avoid one class of bugs, but you also complicate the API. Relying on such a design is a sure-fire way to get a false sense of security: the C++ type system is simply not capable of encoding the necessary details about lifespan requirements for parameters.

Since the type system can’t actually get this right, we recommend you not complicate things with half-measures. Keep it simple - don’t try to rely on this pattern to disallow temporaries, it doesn’t work well enough to help.

`=delete` for “Optimization”

Let’s flip the situation around: perhaps it isn’t that you want to prevent temporaries, maybe you want to prevent copies.

future<bool> DnaScan(Config c, const std::string& sequence) = delete;
future<bool> DnaScan(Config c, std::string&& sequence);

How likely is it that a caller of your API will never need to keep their value? If you cannot be 100% sure that you know exactly how your API will be used, this is a recipe for annoying your users. Consider making copies and invoking such an API given normal (non-deleted) design:

Config c1 = GetConfig();
Config c2 = GetConfig();
std::string s = GetDna();

// Kick off scans for both configs.
auto scan1 = DnaScan(c1, s);
auto scan2 = DnaScan(c2, std::move(s));

Since we see that the second scan is the last use of s, we can just std::move into the value-consuming call. With the “cleverly optimized” version, the code gets more sloppy looking.

Config c1 = GetConfig();
Config c2 = GetConfig();
std::string s = GetDna();
std::string s2 = s;

// Kick off scans for both configs.
auto scan1 = DnaScan(c1, std::move(s));
auto scan2 = DnaScan(c2, std::move(s2));

APIs are provided as building blocks and abstractions - the ecosystem of APIs is a platform to be assembled together in new and surprising ways that are more than what the provider of any single API might predict. Believing that you know certainly that nobody should ever make a copy runs counter to that. Further, the problem of inefficiency and copying when moving would suffice is far broader than any single API, and likely better solved with some combination of profiling, training, code review, and static analysis.

In the rare case when you can know for sure that an API has to be used in a particular fashion: you probably should encode that in the types in question. Don’t operate on std::string as your representation of a DNA sequence; write a Dna class and make it move-only with an explicit (easy to scan for) way to do the expensive copy operation. Put another way: properties of types should be expressed in those types, not in the APIs that operate on them.

Ref-qualification

As a side note: it’s possible to apply the same reasoning for ref-qualifiers on destructive accessors. Consider a class like std::stringbuf - in C++20 it gained an accessor to consume the contained string, presented as an overload set with the existing accessor:

const std::string& str() const &;
std::string str() &&;

(See Tip #148 for more info on reference-qualified methods and overload sets). Looking at existing usage of std::stringbuf, nearly every use has a single stringbuf that is used to produce one string. Ignoring the legacy code, would it not be best to enforce that, and provide only the “efficient” ref-qualified destructive member?

Of course not, for similar reasons to the DnaScan example above: you cannot know certainly that nobody needs it, and it’s not unsafe to provide the const overload. Use ref-qualifiers only as an overload set for optimization, or when the ref-qualifiers are necessary to enforce semantic correctness.

Summary

It is tempting to try to use rvalue-references or reference qualifiers in conjunction with =delete to provide a more “user friendly” API, enforcing lifetimes or preventing optimization problems. In practice, those are usually bad temptations. Lifetime requirements are much more complicated than the C++ type system can express. API providers can rarely predict every future valid usage of their API. Avoiding these types of =delete tricks keeps things simple.

Tip of the Week #148: Overload Sets

2018-05-03T00:00:00-04:00

Originally posted as TotW #148 on May 3, 2018

By Titus Winters

Updated 2020-04-06

Quicklink: abseil.io/tips/148

One of the effects of living with electric information is that we live habitually in a state of information overload. There’s always more than you can cope with. –Marshall McLuhan

In my opinion, one of the most powerful and insightful sentences in the C++ style guide is this: “Use overloaded functions (including constructors) only if a reader looking at a call site can get a good idea of what is happening without having to first figure out exactly which overload is being called.”

On the surface, this is a pretty straightforward rule: overload only when it won’t cause confusion to a reader. However, the ramifications of this are actually fairly significant and touch on some interesting issues in modern API design. First let’s define the term “overload set”, and then let’s look at some examples.

What is an Overload Set?

Informally, an overload set is a set of functions with the same name that differ in the number, type and/or qualifiers of their parameters. (See Overload Resolution for all the gory details.) You may not overload on the return type from a function - the compiler must be able to tell which member of the overload set to call based on the invocation of the function, regardless of return type.

int Add(int a, int b);
int Add(int a, int b, int c);  // Number of parameters may vary

// Return type may vary, as long as the selected overload is uniquely
// identifiable from only its parameters (types and counts).
float Add(float a, float b);

// But if two return types have the same parameter signature, they can't form a
// proper overload set; the following won't compile with the above overloads.
int Add(float a, float b);    // BAD - can't overload on return type

String-ish Parameters

Thinking back on my earliest experiences with C++ at Google, I’m almost positive that the first overloads I encountered were of the form:

void Process(const std::string& s) { Process(s.c_str()); }
void Process(const char*);

The wonderful thing about overloads of this form is that they meet the letter and the spirit of the rule, in a very obvious fashion. There is no behavioral difference here: in both cases, we’re accepting some form of string-ish data, and the inline forwarding function makes it perfectly clear that the behavior of every member of the overload set is identical.

That turns out to be critical, and easy to overlook, since the Google C++ style guide doesn’t phrase it explicitly: if the documented behavior of the members of an overload set varies, then a user implicitly has to know which function is actually being called. The only way to ensure that they have a “good idea what is happening” without figuring which overload is being called is if the semantics of each entry in the overload set are identical.

So, the string-ish examples above work because they have identical semantics. Borrowing an example from the C++ Core Guidelines , we would not want to see something like:

// remove obstacle from garage exit lane
void open(Gate& g);

// open file
void open(const char* name, const char* mode ="r");

Hopefully, namespace differences suffice to disambiguate functions like these from actually forming an overload set. Fundamentally, this would be a bad design specifically because APIs should be understood and documented at the level of the overload set, not the individual functions that make it up.

StrCat

StrCat() is one of the most common Abseil examples for demonstrating that overload sets are often the right granularity for API design. Over the years, StrCat() has changed the number of parameters it accepts, and the form that it uses to express that parameter count. Years ago, StrCat() was a set of functions with varying arity. Now, it is conceptually a variadic template function … although for optimization reasons small-count arities are still provided as overloads. It has never actually been a single function - we just treat it conceptually as one entity.

This is a good use of overload sets - it would be annoying and redundant to encode the parameter count in the function name, and conceptually it doesn’t matter how many things are passed to StrCat() - it will always be the “convert to string and concatenate” tool.

Parameter Sinks

Another technique that the standard uses and that comes up in a lot of generic code is to overload on const T& and T&& when passing a value that will be stored: a value sink. Consider std::vector::push_back():

void push_back(const T&);
void push_back(T&&);

It’s worth considering the origin of this overload set: when the push_back() API first appeared, it contained push_back(const T&) which served as a cheap (and safe) parameter to pass. With C++11 the push_back(T&&) overload was added as an optimization for cases where the value is a temporary, or the caller has promised not to interfere with the parameter by writing out std::move(). Even though the moved-from object may be left in a different state, these still provide the same semantics for the user of the vector, so we consider them a well-designed overload set.

Put another way, the & and && qualifiers denote whether that overload is available for lvalue or rvalue expressions; if you have a var or var& argument, you will get the & overload, but if you have a temporary or have performed a std::move() on your expression, you will get the && overload. (See Tip #77 for more on move-semantics.)

Interestingly, these overloads are semantically the same as a single method – push_back(T) – but in some cases may be slightly more efficient. Such efficiency mostly matters when the body of the function is cheap compared to the cost of invoking the move constructor for T – possible for containers and generics, but unlikely in many other contexts. We generally recommend that if you need to sink a value (store in an object, mutate a parameter, etc) you just provide the single function accepting T (or const T&) for simplicity and maintainability. Only if you are writing very high-performance generic code is the difference likely to be relevant. See Tip #77 and Tip #117.

Overloaded Accessors

For methods on a class (especially a container or a wrapper), it is sometimes valuable to provide an overload set for accessors. Standard library types provide many great examples here - we’ll consider just vector::operator[] and optional::value().

In the case of vector::operator[], two overloads exist: one const and one non-const, which accordingly return a const or non-const reference, respectively. This matches our definition nicely; a user doesn’t need to know which thing is invoked. The semantics are the same, differing only in constness – if you have a non-const vector you get a non-const reference, and if you have a const vector you get a const reference. Put another way: the overload set is purely forwarding the const-ness of the vector, but the API is consistent – it just gives you the indicated element.

In C++17 we added std::optional<T>, a wrapper for at most one value of an underlying type. Just like vector::operator[], when accessing optional::value() both const and non-const overloads exist. However, optional goes one step further and provides value() overloads based on “value category” (roughly speaking, whether the object is a temporary). So the full pairwise combination of const and value category looks like:

T& value() &;
const T & value() const &;
T&& value() &&;
const T&& value() const &&;

The trailing & and && apply to the implicit *this parameter, just in the same way const qualifying a method does, and indicate acceptance of lvalue or rvalue arguments as noted in Parameter Sinks above. Importantly, however, you don’t actually need to understand move semantics to understand optional::value() in this case. If you ask for the value out of a temporary, you get the value as if it were a temporary itself. If you ask for the value out of a const ref, you get a const-ref of the value. And so on.

Copy vs. Move

The most important overload sets for types are often their set of constructors, especially the copy and move constructors. Copy and move, done right, form an overload set in all senses of the term: the reader should not need to know which of those overloads is chosen, because the semantics of the newly-constructed object should be the same in either case (assuming both constructors exist). The standard library is becoming more explicit about this: move is assumed to be an optimization of copy, and you should not depend on the particulars of how many moves or copies are made in any given operation.

Conclusion

Overload sets are a simple idea conceptually, but prone to abuse when not well understood - don’t produce overloads where anyone might need to know which function was chosen. But when used well, overload sets provide a powerful conceptual framework for API design. Understanding the subtleties of the style guide’s description of overload sets is well worth your time when thinking about API design.

Tip of the Week #117: Copy Elision and Pass-by-value

2018-05-03T00:00:00-04:00

Originally posted as TotW #117 on June 8, 2016

by Geoff Romer, (gromer@google.com)

“Everything is so far away, a copy of a copy of a copy. The insomnia distance of everything, you can’t touch anything and nothing can touch you.” — Chuck Palahniuk

Suppose you have a class like this:

class Widget {
 public:
  …

 private:
  string name_;
};

How would you write its constructor? For years, the answer has been to do it like this:

// First constructor version
explicit Widget(const std::string& name) : name_(name) {}

However, there is an alternative approach which is becoming more common:

// Second constructor version
explicit Widget(std::string name) : name_(std::move(name)) {}

(If you’re not familiar with std::move(), see TotW #77, or pretend I used std::swap instead; the same principles apply). What’s going on here? Isn’t it horribly expensive to pass std::string by copy? It turns out that no, sometimes passing by value (as we’ll see, it’s not really “by copy”) can be much more efficient than passing by reference.

To understand why, consider what happens when the callsite looks like this:

Widget widget(absl::StrCat(bar, baz));

With the first version of the Widget constructor, absl::StrCat() produces a temporary string containing the concatenated string values, which is passed by reference into Widget(), and then the string is copied into name_. With the second version of the Widget constructor, the temporary string is passed into Widget() by value, which you might think would cause the string to be copied, but this is where the magic happens. When the compiler sees a temporary being used to copy-construct an object, the compiler will¹ simply use the same storage for both the temporary and the new object, so that copying from the one to the other is literally free; this is called copy elision. Thanks to this optimization, the string is never copied, but only moved once, which is a cheap constant-time operation.

Consider what happens when the argument isn’t a temporary:

string local_str;
Widget widget(local_str);

In this case, both versions of the code make a copy of the string, but the second version also moves the string. That, again, is a cheap constant-time operation, whereas copying is a linear-time operation, so in many cases that will be a price well worth paying.

The key property of the name parameter that makes this technique work is that name has to be copied. Indeed, the essence of this technique is to try to make the copy operation happen on the function call boundary, where it can be elided, rather than in the interior of the function. This need not necessarily involve std::move(); for example, if the function needs to mutate the copy rather than store it, it may just be mutated in place.

When to Use Copy Elision

Passing parameters by value has several drawbacks which should be borne in mind. First of all, it makes the function body more complex, which creates a maintenance and readability burden. For example, in the code above, we’ve added a std::move() call, which creates a risk of accidentally accessing a moved-from value. In this function that risk is pretty minimal, but if the function were more complex, the risk would be higher.

Second, it can degrade performance, sometimes in surprising ways. It can sometimes be difficult to tell whether it’s a net performance win without profiling a specific workload.

As stated above, this technique applies only to parameters that need to be copied; it will be useless at best and harmful at worst when applied to parameters that don’t need to be copied, or only need to be copied conditionally.
This technique often involves some amount of extra work in the function body, such as the move assignment in the example above. If that extra work imposes too much overhead, the slowdown in the cases where the copy can’t be elided may not be worth the speedup in the cases where the copy can be elided. Note that this determination can depend on the particulars of your use case. For example, if the arguments to Widget() are almost always very short, or are almost never temporaries, this technique may be harmful on balance. As always when considering optimization tradeoffs, when in doubt, measure.
When the copy in question is a copy assignment (for example if we wanted to add a set_name() method to Widget), the pass-by-reference version can sometimes avoid memory allocation by reusing name_’s existing buffer, in cases where the pass-by-value version would allocate new memory. In addition, the fact that pass-by-value always replaces name_’s allocation can lead to worse allocation behavior down the line: if the name_ field tends to grow over time after it is set, that growth will require further new allocations in the pass-by-value case, whereas in the pass-by-reference case we only reallocate when the name_ field exceeds its historical maximum size.

Generally speaking, you should prefer simpler, safer, more readable code, and only go for something more complex if you have concrete evidence that the complex version performs better and that the difference matters. That principle certainly applies to this technique: passing by const reference is simpler and safer, so it’s still a good default choice. However, if you’re working in an area that’s known to be performance-sensitive, or your benchmarks show that you’re spending too much time copying function parameters, passing by value can be a very useful tool to have in your toolbox.

Strictly speaking the compiler is not required to perform copy elision, but it is such a powerful and vitally important optimization that it is extremely unlikely that you will ever encounter a compiler that doesn’t do it. ↩

Tip of the Week #24: Copies, Abbrv.

2018-04-20T00:00:00-04:00

Originally posted as TotW #24 on Nov 26, 2012

by Titus Winters, (titus@google.com) and Chandler Carruth (chandlerc@google.com)

“To copy others is necessary, but to copy oneself is pathetic.” - Pablo Picasso

Note: see also TotW #55 and TotW #77 for guidance on name counting and copies vs. moves.

One Name, No Copy; Two Names, Two Copies

When evaluating whether copies get made within any given scope (including cases triggering RVO), check how many names your data refers to.

You will have two copies of the data at any point where you have two live names for those copies. To a good first approximation, the compiler will (and often must) elide copies in all other cases.

Between the move semantics of STL containers (introduced automatically with the switch to C++11) and copy constructor elision by the compiler, we are rapidly converging on this rule providing not merely a lower bound on the number of copies, but a guarantee. If your benchmarks show that more copies are being made, it is likely a compiler bug; your compiler probably needs a fix.

So if your code is structured such that there are two names for the data at some point during the execution, you should expect a copy. If you avoid introducing a name which could possibly refer to the data, you’ll help ensure the compiler can remove the copy.

Examples

Let’s look at some examples of how this works in practice:

std::string build();

std::string foo(std::string arg) {
  return arg;  // no copying here, only one name for the data “arg”.
}

void bar() {
  std::string local = build();  // only 1 instance -- only 1 name

  // no copying, a reference won’t incur a copy
  std::string& local_ref = local;

  // one copy operation, there are now two named collections of data.
  std::string second = foo(local);
}

Most of the time, none of this matters. It is far more important to ensure that your code is readable and consistent, rather than worrying about copies and performance. As always: profile before you optimize. But, if you find yourself writing code from scratch – and can provide a clean and consistent API that returns its values – don’t discount code that seems like it would make copies: everything you learned about copies in C++ a decade ago is wrong.

Tip of the Week #120: Return Values are Untouchable

2018-04-20T00:00:00-04:00

Originally posted as TotW #120 on August 16, 2012

by Samuel Benzaquen, (sbenza@google.com)

Let’s suppose you have this code snippet, which seems to be working as expected:

absl::Status DoSomething() {
  absl::Status status;
  absl::Cleanup log_on_error = [&status] {
    if (!status.ok()) LOG(ERROR) << status;
  };
  status = DoA();
  if (!status.ok()) return status;
  status = DoB();
  if (!status.ok()) return status;
  status = DoC();
  if (!status.ok()) return status;
  return status;
}

A refactor changes the last line from return status; to return absl::OkStatus(); and suddenly the code stopped logging the errors.

What is going on?

Summary

Never access (read or write) the return variable after the return statement has run. Unless you take great care to do it correctly, the behavior is unspecified.

Return variables are implicitly accessed by destructors after they have been copied or moved ([stmt.return]), which is how this unexpected access occurs, but the copy/move may be elided, which is why behavior is unspecified.

This tip only applies when you are returning a non-reference local variable. Returning any other expression will not trigger this problem.

The Problem

There are 2 different optimizations on return statements that can modify the behavior of that code snippet: the named return value optimization (NRVO) and implicit moves.

The before code was working because NRVO is being performed and the return statement is not actually doing any work. The variable status is already constructed in the return address and the cleanup object is seeing this unique instance of the Status object after the return statement.

In the after code, NRVO is not being performed and the returned variable is being moved into the return value. The cleanup object is being run after the move operation is done and it is seeing a moved-from Status.

Note that the before code was not correct either, as it relies on NRVO for correctness. We encourage you to rely on NRVO for performance (See TotW #24), but not correctness. After all, NRVO is an optional optimization and compiler options or quality of implementation of the compiler can affect whether or not it happens.

Solution

Do not touch the return variable after the return statement. Be careful of destructors of local variables doing so implicitly.

The simplest solution is to separate the function in two. One that does all the processing, and one that calls the first one and does the post-processing (ie logging on error). Eg:

absl::Status DoSomething() {
  absl::Status status;
  status = DoA();
  if (!status.ok()) return status;
  status = DoB();
  if (!status.ok()) return status;
  status = DoC();
  if (!status.ok()) return status;
  return status;
}

absl::Status DoSomethingAndLog() {
  absl::Status status = DoSomething();
  if (!status.ok()) LOG(ERROR) << status;
  return status;
}

If you are only reading the value, you could also make sure to disable the optimizations instead. That would force a copy to be made all the time and the post-processing will not see a moved-from value. Eg:

absl::Status DoSomething() {
  absl::Status status_no_nrvo;
  // 'status' is a reference so NRVO and all associated optimizations
  // will be disabled.
  // The 'return status;' statements will always copy the object and Logger
  // will always see the correct value.
  absl::Status& status = status_no_nrvo;
  absl::Cleanup log_on_error = [&status] {
    if (!status.ok()) LOG(ERROR) << status;
  };
  status = DoA();
  if (!status.ok()) return status;
  status = DoB();
  if (!status.ok()) return status;
  status = DoC();
  if (!status.ok()) return status;
  return status;
}

Another Real Life Example

std::string EncodeVarInt(int i) {
  std::string out;
  proto2::io::StringOutputStream string_output(&out);
  proto2::io::CodedOutputStream coded_output(&string_output);
  coded_output.WriteVarint32(i);
  return out;
}

CodedOutputStream does some work on the destructor to trim unused trailing bytes. This function can leave garbage bytes on the string if NRVO does not happen.

Note that in this case you can’t force NRVO to happen and the trick to disable it won’t help. We must modify the return value before the return statement runs.

A good solution is to add a block and restrict the function to only return after the block is finished. Like this:

std::string EncodeVarInt(int i) {
  std::string out;
  {
    proto2::io::StringOutputStream string_output(&out);
    proto2::io::CodedOutputStream coded_output(&string_output);
    coded_output.WriteVarint32(i);
  }
  // At this point the streams are destroyed and they already flushed.
  // We can safely return 'out'.
  return out;
}

Conclusion

Don’t hold on to references to variables that are being returned.

You can’t control whether NRVO happens or not. Compiler versions and options can change this from underneath you. Do not depend on it for correctness.

You can’t control whether returning a local variable triggers an implicit move or not. The type you use might be updated in the future to support move operations. Moreover, future language standards will apply implicit moves on even more situations so you can’t assume that just because it is not happening now it won’t happen in the future.

Tip of the Week #11: Return Policy

2018-04-20T00:00:00-04:00

Originally posted as TotW #11 on August 16, 2012

by Paul S. R. Chisholm (p.s.r.chisholm@google.com)

Frodo: There’ll be none left for the return journey. Sam: I don’t think there will be a return journey, Mr. Frodo. – The Lord of the Rings: The Return of the King (novel by J.R.R. Tolkien, screenplay by Fran Walsh, Philippa Boyens, & Peter Jackson)

Note: this tip, though still relevant, preceded the introduction of move semantics in C++11. Please read this tip with the advice from TotW #77 also in mind.

Many older C++ codebases show patterns that are somewhat fearful of copying objects. Happily, we can “copy” without copying, thanks to something called “return value optimization” (RVO).

RVO is a long-standing feature of almost all C++ compilers. Consider the following C++98 code, which has a copy constructor and an assignment operator. These functions are so expensive, the developer had them print a message every time they’re used:

class SomeBigObject {
 public:
  SomeBigObject() { ... }
  SomeBigObject(const SomeBigObject& s) {
    printf("Expensive copy …\n", …);
    …
  }
  SomeBigObject& operator=(const SomeBigObject& s) {
    printf("Expensive assignment …\n", …);
    …
    return *this;
  }
  ~SomeBigObject() { ... }
  …
};

(Note that we’re intentionally avoiding discussion of move operations here. See TotW #77 for more information.)

Would you recoil in horror if this class had a factory method such as the following?

static SomeBigObject SomeBigObjectFactory(...) {
  SomeBigObject local;
  ...
  return local;
}

Looks inefficient, doesn’t it? What happens if we run the following?

SomeBigObject obj = SomeBigObject::SomeBigObjectFactory(...);

Simple answer: You probably expect there to be at least two objects created: the object returned from the called function, and the object in the calling function. Both are copies, so the program prints two messages about expensive operations. Real-world answer: No message is printed – because the copy constructor and assignment operator were never called!

How’d that happen? A lot of C++ programmers write “efficient code” that creates an object and passes that object’s address to a function, which uses that pointer or reference to operate on the original object. Well, under the circumstances described below, the compiler can transform such “an inefficient copy” into that “efficient code”!

When the compiler sees a variable in the calling function (that will be constructed from the return value), and a variable in the called function (that will be returned), it realizes it doesn’t need both variables. Under the covers, the compiler passes the address of the calling function’s variable to the called function.

To quote the C++98 standard, “Whenever a temporary class object is copied using a copy constructor … an implementation is permitted to treat the original and the copy as two different ways of referring to the same object and not perform a copy at all, even if the class copy constructor or destructor have side effects. For a function with a class return type, if the expression in the return statement is the name of a local object … an implementation is permitted to omit creating the temporary object to hold the function return value …” (Section 12.8 [class.copy], paragraph 15 of the C++98 standard. The C++11 standard has similar language in section 12.8, paragraph 31, but it’s more complicated.)

Worried that “permitted” isn’t a very strong promise? Fortunately, all modern C++ compilers perform RVO by default, even in debug builds, even for non-inlined functions.

How Can You Ensure the Compiler Performs RVO?

The called function should define a single variable for the return value:

SomeBigObject SomeBigObject::SomeBigObjectFactory(...) {
  SomeBigObject local;
  …
  return local;
}

The calling function should assign the returned value to a new variable:

// No message about expensive operations:
SomeBigObject obj = SomeBigObject::SomeBigObjectFactory(...);

That’s it!

The compiler can’t do RVO if the calling function reuses an existing variable to store the return value (though move semantics would apply for move-enabled types in this case):

// RVO won’t happen here; prints message "Expensive assignment ...":
obj = SomeBigObject::SomeBigObjectFactory(s2);

The compiler also can’t do RVO if the called function uses more than one variable for the return value:

// RVO won’t happen here:
static SomeBigObject NonRvoFactory(...) {
  SomeBigObject object1, object2;
  object1.DoSomethingWith(...);
  object2.DoSomethingWith(...);
  if (flag) {
    return object1;
  } else {
    return object2;
  }
}

But it’s okay if the called function uses one variable and returns it in multiple places:

// RVO will happen here:
SomeBigObject local;
if (...) {
  local.DoSomethingWith(...);
  return local;
} else {
  local.DoSomethingWith(...);
  return local;
}

That’s probably all you need to know about RVO.

One More Thing: Temporaries

RVO works with temporary objects, not just named variables. You can benefit from RVO when the called function returns a temporary object:

// RVO works here:
SomeBigObject SomeBigObject::ReturnsTempFactory(...) {
  return SomeBigObject::SomeBigObjectFactory(...);
}

You can also benefit from RVO when the calling function immediately uses the returned value (which is stored in a temporary object):

// No message about expensive operations:
EXPECT_EQ(SomeBigObject::SomeBigObjectFactory(...).Name(), s);

A final note: If your code needs to make copies, then make copies, whether or not the copies can be optimized away. Don’t trade correctness for efficiency.

Tip of the Week #143: C++11 Deleted Functions (`= delete`)

2018-03-15T00:00:00-04:00

Originally posted as TotW #143 on March 2, 2018

By Leonard Mosescu

Updated 2020-04-06

Quicklink: abseil.io/tips/143

Introduction

Interfaces, in a general sense, normally define the set of operations which can be invoked. Yet sometimes we may want to express the opposite: explicitly define a set of operations which should not be used. For example, disabling the copy constructor and copy assignment operator is a common way to restrict copy semantics for a particular type.

The language offers multiple options to effect such restrictions (and we’ll explore each one shortly):

Provide a dummy definition consisting solely of a runtime check.
Use accessibility controls (protected/private) to make the function inaccessible.
Declare the function, but intentionally omit the definition.
Since C++11: Explicitly define the function as “deleted”.

The pre-C++11 techniques range from runtime checks (#1) to compile-time (#2) or link time (#3) diagnostics. While battle-proven, these techniques are far from perfect: a runtime check is not ideal for the majority of situations where the constraint is static, and the link-time check delays the diagnostic to very late in the build process. Moreover, link time diagnostics are not guaranteed (missing a definition for an ODR-used function is an ODR violation) and the actual diagnostic messages are rarely developer-friendly.

A compile time check is better, but still flawed. It only works for member functions and is based on accessibility constraints, which are verbose, error-prone and susceptible to loopholes. Moreover, the errors that result from referencing such functions can be misleading, referring as they do to access restrictions rather than interface misuse.

The application of #2 and #3 to disable copying would look like this:

class MyType {
 private:
  MyType(const MyType&);  // Not defined anywhere.
  MyType& operator=(const MyType&);  // Not defined anywhere.
  // ...
};

Manually applying this for every class gets old really fast, so developers commonly package them in one of these ways:

The “mixin” approach (boost::noncopyable, non-copyable mixin)

class MyType : private NoCopySemantics {
  ...
};

The macros approach

class MyType {
 private:
  DISALLOW_COPY_AND_ASSIGN(MyType);
};

C++11 Deleted Definitions

C++11 addressed the need for a better solution through a new language feature: deleted definitions [dcl.fct.def.delete]. (See “deleted definitions” in the C++ standard draft.) Any function can be explicitly defined as deleted:

void foo() = delete;

The syntax is straightforward, resembling defaulted functions, although with a couple of notable differences:

Any function can be deleted, including non-member functions (in contrast to =default, which works only with special member functions).
Functions must be deleted on the first declaration only (unlike =default).

The key thing to keep in mind is that =delete is a function definition (it does not remove or hide the declaration). The deleted function is thus defined and participates in name lookup and overload resolution as any other function. It’s a special kind of “radioactive” definition which says “don’t touch!”.

Attempts to use a deleted function result in a compile time error with a clear diagnostic, which is one of the key benefits over the pre-C++11 techniques.

class MyType {
 public:
  // Disable default constructor.
  MyType() = delete;

  // Disable copy (and move) semantics.
  MyType(const MyType&) = delete;
  MyType& operator=(const MyType&) = delete;

  //...
};

// error: call to deleted constructor of 'MyType'
// note: 'MyType' has been explicitly marked deleted here
//   MyType() = delete;
MyType x;

void foo(const MyType& val) {
  // error: call to deleted constructor of 'MyType'
  // note: 'MyType' has been explicitly marked deleted here
  //   MyType(const MyType&) = delete;
  MyType copy = val;
}

Note: by explicitly defining the copy operations as deleted we also suppress the move operations (having user-declared copy operations inhibits the implicit declaration of the move operations). If the intention is to define a move-only type using the implicit move operations, =default can be used to “bring them back”, for example:

MyType(MyType&&) = default;
MyType& operator=(MyType&&) = default;

Other Uses

While the examples above are centered on copy semantics (which is likely the most common case), any function (member or not) can be deleted.

Since deleted functions participate in overload resolution they can help catch unintended uses. Let’s say we have the following overloaded print function:

void print(int value);
void print(absl::string_view str);

Calling print('x') will print the integer value of ‘x’, when the developer likely intended print("x"). We can catch this:

void print(int value);
void print(const char* str);
// Use string literals ":" instead of character literals ':'.
void print(char) = delete;

Note that =delete doesn’t affect just function calls. Attempting to take the address of a deleted function will also result in a compilation error:

void (*pfn1)(int) = &print;  // ok
void (*pfn2)(char) = &print; // error: attempt to use a deleted function

This example is extracted from a real world application: absl::StrCat(). Deleted functions are valuable any time a particular part of an interface must be restricted.

Defining destructors as deleted is stricter than making them private (although this is a big hammer and it may introduce more limitations than intended)

// A _very_ limited type:
//   1. Dynamic storage only.
//   2. Lives forever (can't be destructed).
//   3. Can't be a member or base class.
class ImmortalHeap {
 public:
  ~ImmortalHeap() = delete;
  //...
};

Yet another example, this time we want to only allow the allocation of non-array objects ([real world example][crashpad]):

// Don't allow new T[].
class NoHeapArraysPlease {
 public:
  void* operator new[](std::size_t) = delete;
  void operator delete[](void*) = delete;
};

auto p = new NoHeapArraysPlease;  // OK

// error: call to deleted function 'operator new[]'
// note: candidate function has been explicitly deleted
//   void* operator new[](std::size_t) = delete;
auto pa = new NoHeapArraysPlease[10];

Summary

=delete offers an explicit way to express parts of an interface which should not be referenced, also enabling better diagnostics than the pre-C++11 idioms. No piece of code, including compiler generated code, can reference a deleted function. For nuanced access control, the access specifiers or more elaborate techniques (for example, the passkey idiom as discussed in Tip #134) are more appropriate.

Important: Since the deleted definitions are part of the interface they should have the same access specifier as the other parts of the interface. Concretely, this means they should usually be public. In practice this also results in the best diagnostics (private and =delete doesn’t make much sense).

Credits: This tip includes key contributions and feedback from many people, special thanks to: Mark Mentovai, James Dennett, Bruce Dawson and Yitzhak Mandelbaum.

References

Tip of the Week #141: Beware Implicit Conversions to `bool`

2018-03-15T00:00:00-04:00

Originally posted as TotW #141 on January 19, 2018

By Samuel Freilich

Updated 2020-04-06

Quicklink: abseil.io/tips/141

Two Kinds of Null Pointer Checks

Checking a pointer for null before dereferencing is important to avoid crashes and bugs. This can be done in two ways:

if (foo) {
  DoSomething(*foo);
}

if (foo != nullptr) {
  DoSomething(*foo);
}

Both of these conditionals have the same semantics given that foo is a pointer, but the type-checking on the latter is a little tighter. Many types in C++ can be implicitly converted to bool, and additional caution is required when the pointed-to type can itself be converted to bool.

Consider the following code which could have two very different meanings:

bool* is_migrated = ...;

// Is this checking that `is_migrated` is not null, or was the actual
// intent to verify that `*is_migrated` is true?
if (is_migrated) {
  ...
}

This code is clearer:

// Looks like a null-pointer check for a bool*
if (is_migrated != nullptr) {
  ...
}

Both styles are acceptable in Google C++ code. So when the underlying type is not implicitly convertible to bool, follow the style of surrounding code. If the value in question is a “smart pointer” like std::unique_ptr, the semantics and tradeoffs are the same.

Optional Values and Scoped Assignments

What about optional (e.g. std::optional) values? They deserve more careful consideration.

For example:

std::optional<bool> b = MaybeBool();
if (b) { ... }  // What happens when the function returns std::optional(false)?

Putting the variable declaration in the conditional of the if statement limits the scope of the variable, but the value is implicitly¹ converted to a bool, so it may not be explicit which boolean property is tested.

The intent of the following code is clearer:

std::optional<bool> b = MaybeBool();
if (b.has_value()) { ... }

Note that, in fact, the code snippets above are equivalent: std::optional’s conversion to bool only looks at whether the optional is full, not at its contents. A reader may find it counterintuitive that optional(false) is true, but it’s immediately clear that optional(false) has a value. Again, it’s worth extra caution when the underlying type is implicitly convertible to bool.

One pattern for optional return values is to put a variable declaration in the conditional of the if statement. This limits the scope of the variable, but involves an implicit conversion to bool:

if (std::optional<Foo> foo = MaybeFoo()) {
  DoSomething(*foo);
}

Note: In C++17, if statements can contain an initializer, so the scope of the declaration can be limited while avoiding the implicit conversion:

if (std::optional<Foo> foo = MaybeFoo(); foo.has_value()) {
  DoSomething(*foo);
}

“Boolean-like” Enums

Let’s say you’ve taken the advice of Tip #94 and decided to use an enum in your function signature instead of a bool for better readability at the call sites. This kind of refactoring might introduce an implicit conversion in the function definition:

void ParseCommandLineFlags(
    const char* usage, int* argc, char*** argv,
    StripFlagsMode strip_flags_mode) {
  if (strip_flags_mode) {  // Wait, which value was true again?
    ...
  }
}

You can gain additional clarity by replacing the implicit conversion with an explicit comparison:

void ParseCommandLineFlags(
    const char* usage, int* argc, char*** argv,
    StripFlagsMode strip_flags_mode) {
  if (strip_flags_mode == kPreserveFlags) {
    ...
  }
}

Summary

In summary, be aware that implicit conversions to bool can be unclear, so consider writing more explicit code:

Compare pointer types to nullptr (especially if the pointed-at type is implicitly convertible to bool).
Test container emptiness with boolean functions like std::optional<T>::has_value() (especially if the contained type is implicitly convertible to bool). Use the optional initializer form for if to limit the scope of variables (Tip #165). Remember call-only interfaces though, and don’t take the address of value() or has_value(). The testing::Optional matcher can help in tests.
Compare enums to specific values.

More specifically, it’s a “contextual conversion”. ↩

What Should Go Into the C++ Standard Library

2018-02-27T00:00:00-05:00

By Titus Winters (titus@google.com), @TitusWinters

About the Author - Titus holds a PhD in Computer Science Education from UC Riverside. He created most of Google’s internal C++ coursework, and plays an active role in the C++ mentoring program that serves several thousand Google C++ engineers. He has been deeply involved in C++ library infrastructure since 2011, through the Google C++ Library Team and now Abseil. He chairs the Library Evolution Working Group (LEWG) - the sub-committee of the C++ Standard that focuses on the design of the C++ standard library.

A few interesting pieces on the direction and design for C++ have circulated recently. First off, I strongly recommend everyone read through Direction for ISO C++ ; it provides a lot of excellent detail and suggestions, as well as some much-needed guidance on how the language should evolve. In particular, I think it’s really important to note “No language can be everything for everybody,” the technical “pillars” upon which C++ rests, and the call to remind everyone to go read “The Design and Evolution of C++”. The Direction Group is very rightly showing concern about the breadth of things being proposed for the standard, as well as how to maintain velocity and stability as we continue to move forward at an ever more rapid pace.

Batteries Included?

An interesting piece by Guy Davidson also specifically focuses on what should be included in the C++ Standard Library, with emphasis on a proposed Graphics library. Guy (again, rightly) is concerned with what can and cannot be done with C++ easily and out of the box. Guy argues we should aim for “batteries included,” a much more all-inclusive approach than what is currently standardized. Especially when you compare C++ to other languages, there’s a pretty strong argument to be made for a more inclusive and even all-encompassing standard library - look at the richness of the standard libraries for Java or Python.

Interestingly, the Graphics library proposal is mentioned specifically in both of these pieces. In the Direction Group’s paper, Graphics is forwarded as a way to make C++ more accessible for beginners. In Guy’s piece, it’s an exemplar of the “batteries included” philosophy, drawing comparisons from other languages and trying to make C++ feel more fully-featured out of the box.

I don’t agree with either of those conclusions. But the reasoning behind this objection requires a little bit of a detour. Please bear with me.

Literate and Algorithmic Thinking

I’ve done studies on what undergraduate CS1 students know and think about CS when they show up, day 1. If you ask such students what it means to program on day 1, in a majority of cases they have basically no idea. The popular vision of programmers and hackers comes from movies like The Matrix, Hackers, and Swordfish, or the “It’s a UNIX system” scene from Jurassic Park. Programming is like magic and the act of programming in popular culture has more to do with portrayals of magic than opening up a text editor. A skilled programmer, like Harry Potter as much as Neo, can do anything - and will probably be surrounded by some impressive visual effects in the process.

Then, we introduce them to “Hello World”. Java gives me a great punching bag here: “public static void main string bracket-bracket args, system dot out dot println, quote Hello World”. Yes, you can draw comparison between the boilerplate necessary there with the incantations within magic, but if we are being honest we turn off CS1 students from that very first program if we aren’t careful. “Don’t worry about all of this stuff” is not a great way to inspire curiosity and a desire to learn. Boilerplate is the enemy for intro programmers. For a day 1 use-case, I love Python. You must explain “Print doesn’t mean to the printer, it just means to the screen. And quotes are just like in prose - saying exactly this thing in quotes.” For kick-starting some simple demonstrations without boilerplate, that’s pretty hard to beat. C++ falls somewhere between the two - if we stick to Hello World and iostreams, it’s not too bad. The well-travelled path is pretty narrow (explaining strings vs. char*s, namespaces, ADL, operators, etc), but it’s certainly not as bad as Java.

Once we get past the first day, we need to start thinking about what it means to become a programmer. By “programmer”, I don’t mean an expert in any given language - most anyone that can call themselves a programmer knows that the syntax and details in a specific language is a largely separable thing from the effort to learn to be an algorithmic / programmatic thinker. That transition into algorithmic thinking transcends any particular language. It’s a cognitive shift, thinking in abstractions and control flow, more than learning the syntax and edge cases for any particular language.

For comparison, there’s a book called Orality and Literacy by Walter Ong that goes into great detail on the differences between oral and literate people and cultures, pointing out the significant cultural and cognitive shifts that happen when you transition to a world that relies on the written word and the ability to communicate ideas without being face to face. I cannot prove it, but I firmly and fundamentally believe that a shift of equal magnitude happens when you transition from a literate mindset to an algorithmic one.

Assuming that the magnitude of the shift to an algorithmic mindset comparable to becoming literate, consider how hard is it to become literate. How much effort do we put into making that transition easy, with cardboard books and whole genres devoted to helping learners make that transition? Why would we think that shifting from literate to algorithmic is any less difficult? Bringing this back to C++: why would we assume that the right tool for a high-performance super-professional domain can also satisfy an “easy reader” in the process of becoming an algorithmic thinker? To me, and certainly to most people I know, C++ isn’t a kid-friendly Goodnight Moon - it’s more of a Finnegan’s Wake. Simply put: developing an algorithmic mindset is hard on its own, even in a language that is designed to hold your hand. C++ is not well-suited for being a starter language - come to us when you’ve gotten over those first hurdles.

Approaching it from another angle: according to P0939, C++ cannot afford to be all things to all people. Hopefully nobody disagrees when I say that C++ is a language that prioritizes efficiency and performance above pretty much everything else - including user-friendliness, ease of use, etc. Citing P0939 and P0684 - one of the few perspectives I expect everyone on the Committee to agree with is that “C++ leaves no room for another language between itself and the hardware”, and that you don’t pay for what you don’t use. Protections against programmer error can be costly, and are thus not a priority for C++. That’s all perfectly fine, so long as we keep that in mind and stay true to that philosophy. In many respects this is what makes C++ the important language it is: when efficiency counts, this is the language you go to. We will not make you pay for having your hand held.

If we know that we cannot be the one language for all tasks, and we know that we are focused on performance above all else - why are we thinking that a Graphics library to make it easy to visualize things for novices is a good fit?

As an educator, I don’t buy it.

What Makes C++ Special?

There’s a separate chain of reasoning that concerns me with Guy’s “Batteries Included” approach. Some of this was presented in Corentin’s reply to Guy’s article, but I’ve got a bit of a different approach and experience with it.

One of the other things that C++ has been good at, historically, is stability. In many respects I think we take that too far - I regularly leave standards meetings frustrated at our inability to fix mistakes, but I do also see the value that stability provides the community. At the committee level we pay pretty close attention to what will break user code (well, at least well-behaved user code, see P0921), up to and including support for pre-compiled code depending on an ever-changing standard library. That is, we provide not just API (source level) compatibility over time, we almost always provide ABI (binary level) compatibility. And that means that a huge number of possible changes are infeasible: if it involves how a type is represented in memory, it will not change for decades.

My friend Matt Kulukundis spoke at CPPCon about work that Google has been doing on hashing - a huge amount of performance and memory savings is available beyond what is provided by std::unordered_map. However, the standard can never realize those savings: even a change to std::hash is infeasible because we have to maintain binary compatibility with code compiled years ago with a sub-par approach to hashing. The implementation for std::hash as it stands leaves little room for important things like hash-flood mitigation. As a community, we’re paying in both resources and safety because of our addiction to binary compatibility.

Even ignoring binary compatibility, the standard is very reticent to make changes that might break existing code. As an example (one of many, I assure you), numerous proposals to resolve issues with std::function have circulated in the past few years. One of my favorite gripes is that it is the only type (AFAIK) in the standard library that is not thread-compatible: calls to const methods from multiple threads can still cause race conditions. This inconsistency is because the specification of the call operator is slightly bogus: even when called in a const context, std::function is perfectly willing to call non-const call operators, potentially mutating the contained callable. This defect was pointed out years ago, and it was pointed out that the only code that would be broken by fixing this would be code that is not thread safe and likely buggy. The committee was unwilling to break existing code - even at that early date.

Given this demonstration of prioritizing efficiency and stability, why would something as high-level and changeable as Graphics be a good fit for the standard? I often legitimately question whether hashed associative containers are a good fit for C++, given the potential for advancement that we’ve seen in the last few years. Anything complex enough that research tasks on how to optimize it are still ongoing should probably not be in the C++ standard. At least, not until we figure out how to deal with change a little more aggressively.

Dependency Management, not Standardization

The section of Guy’s “Batteries Included” piece that resonates with me is this: it isn’t that the standard has to provide these things, but that there needs to be some mechanism to readily distribute libraries and dependencies in the C++ community. It is far past time that we as a community find an answer that is somewhere between “this is chaos” and “this is in the standard”. This is particularly hard in C++, where the preponderance of build flags, preprocessor directives, build systems, compilers, standard library variants makes true portability challenging (to say the least). But I know it can be done. It will require library providers to be more clear about what compatibility they promise, and library consumers to understand what they are asking, but it can be done.

So, what should go in the standard library? Fundamentals. Things that can only be done with compiler support, or compile-time things that are so subtle that only the standard should be trusted to get it right. Vocabulary. Time, vector, string. Concurrency. Mutex, future, executors. The wide array of “sometimes useful data structure” and “random metaprogramming” could and should be shuffled off into a semi-standard repository of available but distinct utility libraries. Those may themselves have different design and stability philosophies, like Boost or Abseil - the principles that make the standard what it is are not necessarily universal even if they are good for the standard. Graphics is too much - many won’t want it, and those that think they do may not understand what they are asking for.

Graphics won’t make C++ a teaching language - learning to be an algorithmic thinker should be done in a language that has fewer sharp edges. C++ won’t make Graphics easy, and tying any sort of Graphics API to the legacy constraints of C++ won’t make for a great API in that space. But using the Graphics API as a high-value seed for a centralized repository of C++ dependencies … that would go a long way toward solving some very real problems.

Tip of the Week #93: using absl::Span

2018-02-22T00:00:00-05:00

Originally posted as TotW #93 on April 23, 2015

By Samuel Benzaquen

Updated 2023-05-08

Quicklink: abseil.io/tips/93

At Google we are accustomed to using string_view as function parameters and return types when we want to deal with unowned strings. It can make the API more flexible and it can improve performance by avoiding unneeded conversions to string. (Tip #1)

string_view has a more generic cousin called absl::Span (google3/third_party/absl/types/span.h). Note that though absl::Span is similar in utility to std::span, available in C++ 20, the two types are not exchangeable.

Span<const T> is to std::vector<T> what string_view is to string. It provides a read-only interface to the elements of the vector, but it can also be constructed from non-vector types (like arrays and initializer lists) without incurring the cost of copying the elements.

The const can be dropped, so where Span<const T> is a view into an array whose elements can’t be mutated, Span<T> allows non-const access to the elements. Unlike spans of const, however, these require explicit construction.

As Function Parameters

Some of the benefits of using Span as a function parameter are similar to those of using string_view.

The caller can pass a slice of the original vector, or pass a plain array. It is also compatible with other array-like containers, like absl::InlinedVector, absl::FixedArray, google::protobuf::RepeatedField, etc.

As with string_view, it is usually better to pass Span by value when used as a function parameter - this form is slightly faster, and produces smaller code.

Example:

void TakesVector(const std::vector<int>& ints);
void TakesSpan(absl::Span<const int> ints);

void PassOnlyFirst3Elements() {
  std::vector<int> ints = MakeInts();
  // We need to create a temporary vector, and incur an allocation and a copy.
  TakesVector(std::vector<int>(ints.begin(), ints.begin() + 3));
  // No copy or allocations are made when using Span.
  TakesSpan(absl::Span<const int>(ints.data(), 3));
}

void PassALiteral() {
  // This creates a temporary std::vector<int>.
  TakesVector({1, 2, 3});
  // Span does not need a temporary allocation and copy, so it is faster.
  TakesSpan({1, 2, 3});
}
void IHaveAnArray() {
  int values[10] = ...;
  // Once more, a temporary std::vector<int> is created.
  TakesVector(std::vector<int>(std::begin(values), std::end(values)));
  // Just pass the array. Span detects the size automatically.
  // No copy was made.
  TakesSpan(values);
}

Const Correctness for Vector of Pointers

A big problem with passing around std::vector<T*> is that you can’t make the pointees const without changing the type of the container.

Any function taking a const std::vector<T*>& will not be able to modify the vector, but it can modify the Ts. This also applies to accessors that return a const std::vector<T*>&. You can’t prevent the caller from modifying the Ts.

Common “solutions” include copying or casting the vector into the right type. These solutions are slow (for the copy) or undefined behavior (for the cast) and should be avoided. Instead, use Span.

Example: Function Parameter

Consider these Frob variants:

void FrobFastWeak(const std::vector<Foo*>& v);
void FrobSlowStrong(const std::vector<const Foo*>& v);
void FrobFastStrong(absl::Span<const Foo* const> v);

Starting with a const std::vector<Foo*>& v that needs frobbing, you have two imperfect options and one good one.

// fast and easy to type but not const-safe
FrobFastWeak(v);
// slow and noisy, but safe.
FrobSlowStrong(std::vector<const Foo*>(v.begin(), v.end()));
// fast, safe, and clear!
FrobFastStrong(v);

Example: Accessor

class MyClass {
 public:
  // This is supposed to be const.
  // Don’t modify my Foos, pretty please.
  const std::vector<Foo*>& shallow_foos() const { return foos_; }
  // Really deep const.
  absl::Span<const Foo* const> deep_foos() const { return foos_; }

 private:
  std::vector<Foo*> foos_;
};
void Caller(const MyClass* my_class) {
  // Accidental violation of MyClass::shallow_foos() contract.
  my_class->shallow_foos()[0]->SomeNonConstOp();
  // This one doesn't compile.
  // my_class->deep_foos()[0]->SomeNonConstOp();
}

Conclusion

When used appropriately, absl::Span can provide decoupling, const correctness and a performance benefit.

It is important to note that Span behaves much like string_view by being a reference to some externally owned data. All the same warnings apply. In particular, a Span must not outlive the data it refers to.

Tip of the Week #61: Default Member Initializers

2018-02-22T00:00:00-05:00

Originally posted as Totw #61 on Nov 12, 2013

by Michael Chastain (mec.desktop@gmail.com)

Updated October, 2016

Declaring Default Member Initialization

A default member initializer declares a default value for a member upon construction and looks like this:

class Client {
 private:
  int chunks_in_flight_ = 0;
};

This default initializer propagates into all constructors for that class, even constructors that C++ synthesizes. Initializing members in this way is useful for classes with lots of data members, especially for types such as bool, int, double, and raw pointers. Non-static data members of these fundamental types often slip through the cracks and end up uninitialized. Non-static data members of any type may have initializers, though.

Default member initializers are also useful for declarations of simple structs with no user-written constructor:

struct Options {
  bool use_loas = true;
  bool log_pii = false;
  int timeout_ms = 60 * 1000;
  std::array<int, 4> timeout_backoff_ms = { 10, 100, 1000, 10 * 1000 };
};

Member Initialization Overrides

If a class constructor initializes a data member that already has a default initializer, the initializer in the constructor supersedes the default:

class Frobber {
 public:
  Frobber() : ptr_(nullptr), length_(0) { }
  Frobber(const char* ptr, size_t length)
    : ptr_(ptr), length_(length) { }
  Frobber(const char* ptr) : ptr_(ptr) { }
 private:
  const char* ptr_;
  // length_ has a non-static class member initializer
  const size_t length_ = strlen(ptr_);
};

This code is equivalent to the older code:

class Frobber {
 public:
  Frobber() : ptr_(nullptr), length_(0) { }
  Frobber(const char* ptr, size_t length)
    : ptr_(ptr), length_(length) { }
  Frobber(const char* ptr)
    : ptr_(ptr), length_(strlen(ptr_)) { }
 private:
  const char* ptr_;
  const size_t length_;
};

Note that the first and second Frobber constructors have initializers for their non-static variables; these two constructors will not use the default initializer for length_. The third Frobber constructor, however, does not have an initializer for length_ so this constructor will use the default initializer for length_.

As always in C++, all non-static variables are initialized in the order of their declaration.

In the first 2 of the 3 Frobber constructors, the constructor provides an initializer for length_. The constructor initializer supersedes the default member initializer – the non-static class member initializer does not contribute to code generation for these constructors.

Note: Older documentation may refer to default member initializers as non-static data member initializers, abbreviated to NSDMIs.

Conclusion

Default member initializers won’t make your program any faster. They will help reduce bugs from omissions, especially when someone adds a new constructor or a new data member.

Be careful not to confuse a non-static class member initializer with a static class member initializer:

class Alpha {
 private:
  static int counter_ = 0;
};

This is an older feature. counter_ is static and this is a static declaration with an initializer. This is different from a non-static class member initializer, just as static member variables are different from non-static member variables.

Tip of the Week #134: `make_unique` and `private` Constructors.

2018-02-22T00:00:00-05:00

Originally posted as TotW #134 on May 10, 2017

By Yitzhak Mandelbaum, Google Engineer

Updated 2020-04-06

Quicklink: abseil.io/tips/134

So, you read Tip #126 and are ready to leave new behind. Everything’s going fine until you try to use std::make_unique to construct an object with a private constructor, and it fails to compile. Let’s take a look at a concrete example of this problem to understand what went wrong. Then, we can discuss some solutions.

Example: Manufacturing Widgets

You’re defining a class to represent widgets. Each widget has an identifier and those identifiers are subject to certain constraints. To ensure those constraints are always met, you declare the constructor of the Widget class private and provide users with a factory function, Make, for generating widgets with proper identifiers. (See Tip #42 for advice on why factory functions are preferable to initializer methods.)

class Widget {
 public:
  static std::unique_ptr<Widget> Make() {
    return std::make_unique<Widget>(GenerateId());
  }

 private:
  Widget(int id) : id_(id) {}
  static int GenerateId();

  int id_;
}

When you try to compile, you get an error like this:

error: calling a private constructor of class 'Widget'
    { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
                                 ^
note: in instantiation of function template specialization
'std::make_unique<Widget, int>' requested here
    return std::make_unique<Widget>(GenerateId());
                ^
note: declared private here
  Widget(int id) : id_(id) {}
  ^

While Make has access to the private constructor, std::make_unique does not! Note that this issue can also arise with friends. For example, a friend of Widget would have the same problem using std::make_unique to construct a Widget.

Recommendations

We recommend either of these alternatives:

Use new and absl::WrapUnique, but explain your choice. For example,

    // Using `new` to access a non-public constructor.
    return absl::WrapUnique(new Widget(...));

Consider whether the constructor may be safely exposed publicly. If so, make it public and document when direct construction is appropriate.

In many cases, marking a constructor private is over-engineering. In those cases, the best solution is to mark your constructors public and document their proper use. However, if your constructor needs to be private (say, to ensure class invariants), then use new and WrapUnique.

Why Can’t I Just Friend `std::make_unique` (or `absl::make_unique`)?

You might be tempted to friend std::make_unique (or absl::make_unique), which would give it access to your private constructors. This is a bad idea, for a few reasons.

First, while a full discussion of friending practices is beyond the scope of this tip, a good rule of thumb is “no long-distance friendships”. Otherwise, you’re creating a competing declaration of the friend, one not maintained by the owner. See also the style guide’s advice.

Second, notice that you are depending on an implementation detail of make_unique, namely that it directly calls new. If it is refactored so that it indirectly calls new – for example, in build modes using C++14 and later absl::make_unique is an alias for std::make_unique, and the friend declaration is useless.

Finally, by friending make_unique, you’ve allowed anyone to create your objects that way, so why not just declare your constructors public and avoid the problem altogether?

What About `std::shared_ptr`?

The situation is somewhat different in the case of std::shared_ptr. There is no absl::WrapShared and the analog – std::shared_ptr<T>(new T(...)) – involves two allocations, where std::make_shared can be done with one. If this difference is important, then consider the passkey idiom: have the constructor take a special token that only certain code can create. For example,

class Widget {
  class Token {
   private:
    explicit Token() = default;
    friend Widget;
  };

 public:
  static std::shared_ptr<Widget> Make() {
    return std::make_shared<Widget>(Token{}, GenerateId());
  }

  Widget(Token, int id) : id_(id) {}

 private:
  static int GenerateId();

  int id_;
};

Note that we are providing an explicit defaulted constructor. This is needed for C++17 and earlier, as otherwise Widget::Token would be an aggregate and could be aggregate-initialized by {}, effectively bypassing the private access.

For a full discussion of the passkey idiom, consult either of these articles:

Tip of the Week #88: Initialization: =, (), and {}

2018-02-15T00:00:00-05:00

Originally posted as TotW #88 on Jan 27, 2015

by Titus Winters (titus@google.com), on behalf of the Google C++ Style Arbiters

C++11 provided a new syntax referred to as “uniform initialization syntax” that was supposed to unify all of the various styles of initialization, avoid the Most Vexing Parse, and avoid narrowing conversions. This new mechanism means we now have yet another syntax for initialization, with its own tradeoffs.

C++11 Brace Initialization

Some uniform initialization syntax proponents would suggest that we use {}s and direct initialization (no use of the ‘=’, although in most cases both forms call the same constructor) for initialization of all types:

int x{2};
std::string foo{"Hello World"};
std::vector<int> v{1, 2, 3};

vs. (for instance):

int x = 2;
std::string foo = "Hello World";
std::vector<int> v = {1, 2, 3};

This approach has two shortcomings. First, “uniform” is a stretch: there are cases where ambiguity still exists (for the casual reader, not the compiler) in what is being called and how.

std::vector<std::string> strings{2}; // A vector of two empty strings.
std::vector<int> ints{2};            // A vector containing only the integer 2.

Second: this syntax is not exactly intuitive: no other common language uses something like it. The language can certainly introduce new and surprising syntax, and there are technical reasons why it’s necessary in some cases – especially in generic code. The important question is: how much should we change our habits and language understanding to take advantage of that change? Are the benefits worth the cost in changing our habits or our existing code? For uniform initialization syntax, we don’t believe in general that the benefits outweigh the drawbacks.

Best Practices for Initialization

Instead, we recommend the following guidelines for “How do I initialize a variable?”, both to follow in your own code and to cite in your code reviews:

Use assignment syntax when initializing directly with the intended literal value (for example: int, float, or std::string values), with smart pointers such as std::shared_ptr, std::unique_ptr, with containers (std::vector, std::map, etc), when performing struct initialization, or doing copy construction.

int x = 2;
std::string foo = "Hello World";
std::vector<int> v = {1, 2, 3};
std::unique_ptr<Matrix> matrix = NewMatrix(rows, cols);
MyStruct x = {true, 5.0};
MyProto copied_proto = original_proto;

instead of:

// Bad code
int x{2};
std::string foo{"Hello World"};
std::vector<int> v{1, 2, 3};
std::unique_ptr<Matrix> matrix{NewMatrix(rows, cols)};
MyStruct x{true, 5.0};
MyProto copied_proto{original_proto};

Use the traditional constructor syntax (with parentheses) when the initialization is performing some active logic, rather than simply composing values together.

Frobber frobber(size, &bazzer_to_duplicate);
std::vector<double> fifty_pies(50, 3.14);

vs.

// Bad code

// Could invoke an initializer list constructor, or a two-argument constructor.
Frobber frobber{size, &bazzer_to_duplicate};

// Makes a vector of two doubles.
std::vector<double> fifty_pies{50, 3.14};

Use {} initialization without the = only if the above options don’t compile:

class Foo {
 public:
  Foo(int a, int b, int c) : array_{a, b, c} {}

 private:
  int array_[5];
  // Requires {}s because the constructor is marked explicit
  // and the type is non-copyable.
  EventManager em{EventManager::Options()};
};

Never mix {}s and auto.
For example, don’t do this:
```
// Bad code
auto x{1};
auto y = {2}; // This is a std::initializer_list<int>!
```
(For the language lawyers: prefer copy-initialization over direct-initialization when available, and use parentheses over curly braces when resorting to direct-initialization.)

Perhaps the best overall description of the issue is Herb Sutter’s GotW post. Although he shows examples that include direct initialization of int with braces, his final advice is roughly compatible with what we present here with one caveat: where Herb says “where you prefer to see only the = sign”, we unambiguously prefer to see exactly that. In conjunction with more consistent use of explicit on multi-parameter constructors (see Tip #142), this provides a balance between readability, explicitness, and correctness.

Conclusion

The tradeoffs for uniform initialization syntax are not generally worth it: our compilers already warn against the Most Vexing Parse (you can use brace initialization or add parens to resolve the issue), and the safety from narrowing conversions isn’t worth the readability hit for brace-initialization (we’ll need a different solution for narrowing conversions, eventually). The Style Arbiters don’t think this issue is critical enough to make a formal rule on, especially because there are cases (notably in generic code) where brace initialization may be justified.

Tip of the Week #142: Multi-parameter Constructors and `explicit`

2018-02-15T00:00:00-05:00

Originally posted as TotW #142 on January 29, 2018

By James Dennett

Updated 2020-04-06

Quicklink: abseil.io/tips/142

“Explicit is better than implicit.” – PEP 20

TL;DR:

Most constructors should be explicit.

Introduction

Prior to C++11, the explicit keyword was meaningful only for constructors that could be called with a single argument, and our style guide required its use for such constructors so that they did not act as “converting constructors”. That requirement was not applied for multi-parameter constructors. Indeed the style guide used to discourage use of explicit for multi-parameter constructors as it had no meaning. That’s no longer the case.

In C++11, explicit became meaningful for copy-initialization from braced lists, such as when calling a function void f(std::pair<int, int>) with f({1, 2}) or when initializing a variable bad with std::vector<char> bad = {"hello", "world"};.

Wait a minute! In that last example the types don’t match. That can’t compile, can it? std::vector<std::string> good = {"hello", "world"}; would be reasonable, but a std::vector<char> can’t hold two std::strings. And yet it does compile (or at least it does with all current C++ compilers). What’s up with that? Let’s come back to that later, after we’ve talked more about explicit.

Constructors That Change Types, But Not Values

Constructors that aren’t marked as explicit can be invoked by the compiler to create a value without mentioning the type’s name. That’s great when we have the value we want already but the types just don’t quite match – we have a const char[] and we need a std::string, maybe, or we have two std::strings and we want a std::vector<std::string>, or we have an int and we want a BigNum. In short, it works well if the value is essentially the same before and after the conversion.

// The coordinates of a point in the Cartesian plane, w.r.t. some basis.
class Coordinate2D {
 public:
  Coordinate2D(double x, double y);
  // ...
};

// Computes the Euclidean norm of a given point `p`.
double EuclideanNorm(Coordinate2D p);

// Uses of the non-explicit constructor:
double norm = EuclideanNorm({3.0, 4.0});  // passing a function argument
Coordinate2D origin = {0.0, 0.0};         // initializing with `=`
Coordinate2D Translate(Coordinate2D p, Vector2D v) {
  return {p.x() + v.x(), p.y() + v.y()};  // returning a value from a function
}

By declaring the constructor Coordinate2D(double, double) without explicit we allow for passing {3.0, 4.0} to a function that takes a Coordinate2D parameter. Given that {3.0, 4.0} is a perfectly reasonable value for such an object, no confusion arises from this affordance.

Constructors That Do More

Implicitly calling a constructor isn’t such a good idea if its output is a different value than its input, or if it might have preconditions.

Consider a Request class with a constructor Request(Server*, Connection*). There’s no sense in which the value of the request object “is” the server and connection — it’s just that we can create a request that uses them. There could be many semantically different types that can be constructed from {server, connection}, such as a Response. Such a constructor should be explicit, so that we can’t pass {server, connection} to a function that accepts a Request (or Response) parameter. In such cases marking the constructor as explicit makes the code clearer for readers by requiring the target type to be named when it’s instantiated and helps to avoid bugs caused by unintended conversions.

// A line is defined by two distinct points.
class Line {
 public:
  // Constructs the line passing through the given points.
  // REQUIRES: p1 != p2
  explicit Line(Coordinate2D p1, Coordinate2D p2);

  // Determines whether this line contain a given point `p`.
  bool ContainsPoint(Coordinate2D p) const;
};

Line line({0, 0}, {42, 1729});

// Computes the gradient of `line`.  Returns an infinite value if `line` is
// vertical.
double Gradient(const Line& line);

By declaring the constructor Line(Coordinate2D, Coordinate2D) as explicit, we prevent code from passing unrelated points to Gradient without first deliberately turning them into a Line object. The “value” of a Line isn’t the two points, and not any two points define a Line.

As another example, the use of explicit for ownership transfer from a raw pointer in std::unique_ptr prevents a double-delete bug here:

std::vector<std::unique_ptr<int>> v;
int* p = new int(-1);
v.push_back(p);  // error: cannot convert int* to std::unique_ptr<int>
// ...
v.push_back(p);

To pass a std::unique_ptr the programmer must explicitly create one, which leaves a visual clue for readers that ownership is being transferred. The explicit constructor helps to document the constraint that the raw pointer must own its target.

Recommendations

Copy constructors and move constructors should never be explicit.
Make a constructor explicit unless its arguments “are” the value of the newly created object. (Note: the Google style guide currently requires all single-argument constructors to be explicit).
In particular, constructors for types where identity (address) is relevant to the value should be explicit.
Constructors that impose additional constraints on values (i.e., that have preconditions) should be explicit. Sometimes those are better implemented as factory functions (see Tip #42: Prefer Factory Functions to Initializer Methods).

Closing Remarks

This tip can be viewed as the flip-side of Tip #88: Initialization: =, (), and {}, which advises us to use copy-initialization syntax (with =) when initializing from “the intended literal value”. The advice of the present tip is to omit explicit only when tip 88 says to use copy initialization (which would otherwise be disallowed by the explicit keyword).

Finally, a word of warning: The C++ Standard Library doesn’t always get this right. Going back to our example (that mistakenly declared a std::vector<char> rather than a container of strings):

std::vector<char> bad = {"hello", "world"};

we find that std::vector has a templated “range” constructor taking a pair of iterators, which matches here and deduces the parameter types as const char*. If the recommendations in this tip were applied, that constructor would be explicit, because the value of a std::vector<char> is not two iterators (rather, it’s a sequence of characters). As it is, explicit was omitted, and this example code gives undefined behavior as the second iterator is not reachable from the first.

Tip of the Week #59: Joining Tuples

2018-01-25T00:00:00-05:00

Originally published as totw/59 on 2013-10-21

By Greg Miller (jgm@google.com)

Updated 2018-01-24

“Now join your hands, and with your hands your hearts.” –Henry VI, William Shakespeare

In March, 2013 we announced the new string joining API in Tip #36. The response to the new API was quite positive, and we worked to make the API even better. Topping the list of feature requests was the ability to join arbitrary lists of possibly heterogeneous data (I can only assume that Shakespeare was referring to joining a heterogeneous collection of hands and hearts). We didn’t go with the varargs or variadic template route, but we did add support for joining std::tuple objects, which addresses this need quite nicely. Simply create a std::tuple containing your heterogeneous data, and absl::StrJoin() will accept it just like any other container. Here are a few examples:

auto tup = std::make_tuple(123, "abc", 0.456);
std::string s = absl::StrJoin(tup, "-");
s = absl::StrJoin(std::make_tuple(123, "abc", 0.456), "-");

int a = 123;
std::string b = "abc";
double c = 0.456;

// Works, but copies all arguments.
s = absl::StrJoin(std::make_tuple(a, b, c), "-");
// No copies, but only works with lvalues.
s = absl::StrJoin(std::tie(a, b, c), "-");
// No copies, and works with lvalues and rvalues.
s = absl::StrJoin(std::forward_as_tuple(123, MakeFoo(), c), "-");

As with joining any container, the elements of the tuple are formatted using an absl::AlphaNumFormatter by default, but you can specify a custom join Formatter if your tuple contains elements that are not handled by the default formatter. To format a tuple with multiple custom element types, your custom Formatter object may contain multiple overloads of operator().

For example:

struct Foo {};
struct Bar {};

struct MyFormatter {
  void operator()(string* out, const Foo& f) const {
    out->append("Foo");
  }
  void operator()(string* out, const Bar& b) const {
    out->append("Bar");
  }
};

std::string s = absl::StrJoin(std::forward_as_tuple(Foo(), Bar()), "-",
                         MyFormatter());
EXPECT_EQ(s, "Foo-Bar");

The goal of the absl::StrJoin() API is to join any collection, range, list, or group of data using an intuitive and consistent syntax. We think joining std::tuple objects fits nicely with this goal and adds more flexibility to the API.

Tip of the Week #36: New Join API

2018-01-25T00:00:00-05:00

Originally published as totw/36 on 2013-03-21

By Greg Miller (jgm@google.com)

Updated 2018-01-24

“I got a good mind to join a club and beat you over the head with it.” – Groucho Marx

Many of you asked for a new joining API and we heard you. We now have one joining function to replace them all, and it is spelled absl::StrJoin(). You simply give it a collection of objects to be joined and a separator string, and it does the rest. It will work with collections of std::string, absl::string_view, int, double – any type that absl::StrCat() supports. If you need to join a type that will not StrCat(), you can also provide a custom Formatter for that type; we’ll see below how the use of a Formatter will let us nicely join a map.

Now for some quick examples:

std::vector<std::string> v = {"a", "b", "c"};
std::string s = absl::StrJoin(v, "-");
// s == "a-b-c"

std::vector<absl::string_view> v = {"a", "b", "c"};
std::string s = absl::StrJoin(v.begin(), v.end(), "-");
// s == "a-b-c"

std::vector<int> v = {1, 2, 3};
std::string s = absl::StrJoin(v, "-");
// s == "1-2-3"

const int a[] = {1, 2, 3};
std::string s = absl::StrJoin(a, "-");
// s == "1-2-3"

The following example passes a Formatter argument to format the pairs in a map, using a different separator. This makes the output nice and readable.

std::map<std::string, int> m = {{"a", 1}, {"b", 2}, {"c", 3}};
std::string s = absl::StrJoin(m, ";", absl::PairFormatter("="));
// s == "a=1;b=2;c=3"

You can also pass a C++ lambda expression as a Formatter.

std::vector<Foo> foos = GetFoos();

std::string s = absl::StrJoin(foos, ", ", [](std::string* out, const Foo& foo) {
  absl::StrAppend(out, foo.ToString());
});

Please refer to absl/strings/str_join.h for more details.

Tip of the Week #3: String Concatenation and operator+ vs. StrCat()

2018-01-25T00:00:00-05:00

Originally published as totw/3 on 2012-05-11

Updated 2017-09-18; revised 2018-01-22

Users are often surprised when a reviewer says, “Don’t use the string concatenation operator, it’s not that efficient.” How can it be that std::string::operator+ is inefficient? Isn’t it hard to get that wrong?

It turns out, such inefficiency isn’t clear cut. These two snippets have close to the same execution time, in practice:

std::string foo = LongString1();
std::string bar = LongString2();
std::string foobar = foo + bar;

std::string foo = LongString1();
std::string bar = LongString2();
std::string foobar = absl::StrCat(foo, bar);

However, the same is not true for these two snippets:

std::string foo = LongString1();
std::string bar = LongString2();
std::string baz = LongString3();
string foobar = foo + bar + baz;

std::string foo = LongString1();
std::string bar = LongString2();
std::string baz = LongString3();
std::string foobar = absl::StrCat(foo, bar, baz);

std::string temp = foo + bar;
std::string foobar = std::move(temp) + baz;

Specifically, note that the contents of foo and bar must be copied to a temporary location before they are placed within foobar. (For more on std::move, see Tip of the Week #77: Temporaries, moves, and copies.)

It is better instead to use absl::StrCat(), a nice helper function from absl/strings/str_cat.h that calculates the necessary string length, reserves that size, and concatenates all of the input data into the output - a well-optimized O(n). Similarly, for cases like:

foobar += foo + bar + baz;

use absl::StrAppend(), which performs similar optimizations:

absl::StrAppend(&foobar, foo, bar, baz);

As well, absl::StrCat() and absl::StrAppend() operate on types other than just string types: you can use absl::StrCat/absl::StrAppend to convert int32_t, uint32_t, int64_t, uint64_t, float, double, const char*, and string_view, like this:

std::string foo = absl::StrCat("The year is ", year);

Tip of the Week #10: Splitting Strings, not Hairs

2018-01-25T00:00:00-05:00

Originally published as totw/10 on 2012-08-16

By Greg Miller (jgm@google.com)

Updated 2018-01-24

I tend to have an odd split in my mind. –John Cleese

Splitting a string into substrings is a common task in any general-purpose programming language, and C++ is no exception. When the need arose at Google, many engineers found themselves wading through a morass of splitting functions in organically grown header files. You’d have hunted for the magical combination of input parameters, output parameters, and semantics that satisfies your need. After studying 50+ functions in a 600+ line header, you may have finally decided on something as tortuously named as SplitStringViewToDequeOfStringAllowEmpty().

To address this, the C++ Library Team implemented a new API for splitting strings, available for use in absl/strings/str_split.h

The new API replaced many of these splitting functions with a single absl::StrSplit() function. This function takes an input string to be split and a delimiter on which to split the string as arguments. absl::StrSplit() adapts the returned collection to the type specified by the caller. absl::StrSplit()’s implementation is efficient because absl::string_views are used internally; no copies are made unless the caller explicitly requests to store the results in a collection of string objects (which copy their data).

Enough talk, let’s see some examples:

// Splits on commas. Stores in vector of string_view (no copies).
std::vector<absl::string_view> v = absl::StrSplit("a,b,c", ',');

// Splits on commas. Stores in vector of string (data copied once).
std::vector<std::string> v = absl::StrSplit("a,b,c", ',');

// Splits on literal string "=>" (not either of "=" or ">")
std::vector<absl::string_view> v = absl::StrSplit("a=>b=>c", "=>");

// Splits on any of the given characters (',' or ';')
using absl::ByAnyChar;
std::vector<std::string> v = absl::StrSplit("a,b;c", ByAnyChar(",;"));

// Stores in various containers (also works w/ absl::string_view)
std::set<std::string> s = absl::StrSplit("a,b,c", ',');
std::multiset<std::string> s = absl::StrSplit("a,b,c", ',');
std::list<std::string> li = absl::StrSplit("a,b,c", ',');

// Equiv. to the mythical SplitStringViewToDequeOfStringAllowEmpty()
std::deque<std::string> d = absl::StrSplit("a,b,c", ',');

// Yields "a"->"1", "b"->"2", "c"->"3"
std::map<std::string, std::string> m = absl::StrSplit("a,1,b,2,c,3", ',');

For more information, take a look at absl/strings/str_split.h for details about how to use the Split API and absl/strings/str_split_test.cc for some more examples.

Thanks for reading. Now I really gotta split…

Tip of the Week #74: Delegating and Inheriting Constructors

2017-12-21T00:00:00-05:00

Originally posted as totw/74 on 2014-04-21

By Bradley White (bww@google.com)

“Delegating work works, provided the one delegating works, too.” – Robert Half

When a class has multiple constructors there is often a need to perform similar initialization in each variant. To avoid code duplication, many older classes resort to defining a private SharedInit() method that is called from the constructors. For example:

class C {
 public:
  C(int x, string s) { SharedInit(x, s); }
  explicit C(int x) { SharedInit(x, ""); }
  explicit C(string s) { SharedInit(0, s); }
  C() { SharedInit(0, ""); }
 private:
  void SharedInit(int x, string s) { … }
};

C++11 provides a new mechanism, delegating constructors, for dealing with such situations more clearly by allowing one constructor to be defined in terms of another. It is also an efficiency gain if the class has members that are expensive to default-initialize.

class C {
 public:
  C(int x, string s) { … }
  explicit C(int x) : C(x, "") {}
  explicit C(string s) : C(0, s) {}
  C() : C(0, "") {}
};

Note that if you delegate to another constructor you cannot also use a member initialization list – all initialization is done by the delegated constructor. Don’t go overboard though. If all the shared code does is set members, separate member-initialization lists or in-class initializers are probably clearer than using delegating constructors. Use good judgement.

Aside: an object is not considered complete until the delegating constructor returns; in practice, this only matters if the constructors can throw, as they leave the delegated-from objects in an incomplete state.

Another, less common form of constructor code duplication occurs when extending the behavior of a multi-constructor class through a wrapper. For example, consider a “veneer” subclass of C that just adds a new member function.

class D : public C {
 public:
  void NewMethod();
};

But what of the constructors for D? We’d like to simply re-use those from C rather than writing out all the forwarding boilerplate, and C++11 allows for this via the new inheriting constructors mechanism.

class D : public C {
 public:
  using C::C;  // inherit all constructors from C
  void NewMethod();
};

This new form of “using” for constructors matches its previous use for member functions.

Note, however, that constructors should really only be inherited when the derived class does not add new data members that need to be initialized explicitly. Indeed, the style guide cautions against inheriting constructors unless the new members (if any) have in-class initialization.

So, go ahead and use C++11’s delegating and inheriting constructors when they reduce duplication, eliminate forwarding boilerplate, or otherwise make your classes simpler and clearer.

Tip of the Week #42: Prefer Factory Functions to Initializer Methods

2017-12-21T00:00:00-05:00

Originally posted as totw/42 on 2013-05-10

By Geoffrey Romer (gromer@google.com)

Revised 2017-12-21

“The man who builds a factory builds a temple; the man who works there worships there, and to each is due, not scorn and blame, but reverence and praise.” – Calvin Coolidge

In environments where exceptions are disallowed (such as within Google), C++ constructors effectively must succeed, because they have no way to report failure to the caller. You can use an abort(), of course, but doing so crashes the whole program, which is often unacceptable in production code.

If your class’s initialization logic can’t avoid the possibility of failure, one common approach is to give the class an initializer method (also called an “init method”), which performs any initialization work that might fail, and signals that failure via its return value. The assumption is usually that the user will call this method immediately after construction, and if it fails the user will immediately destroy the object. However, these assumptions are not always documented, nor always obeyed. It’s all too easy for users to start calling other methods before initialization, or after initialization has failed. Sometimes the class actually encourages this behavior, e.g. by providing methods to configure the object before initializing it, or to read errors out of it after initialization fails.

This design commits you to maintaining a class with at least two distinct user-visible states, and often three: initialized, uninitialized, and initialization-failed. Making such a design work requires a lot of discipline: every method of the class has to specify what states it can be called in, and users have to comply with these rules. If this discipline lapses, client developers will tend to write whatever code happens to work, regardless of what you intended to support. When that starts to happen, maintainability nosedives, because your implementation has to support whatever combinations of pre-initialization method calls your clients have started to depend on. In effect, your implementation has become your interface. (See Hyrum’s Law.)

Fortunately, there’s a simple alternative that lacks these drawbacks: provide a factory function which creates and initializes instances of your class, and returns them by pointer or as absl::optional (see TotW #123), using null to indicate failure. Here’s a toy example using unique_ptr<>:

// foo.h
class Foo {
 public:
  // Factory method: creates and returns a Foo.
  // May return null on failure.
  static std::unique_ptr<Foo> Create();

  // Foo is not copyable.
  Foo(const Foo&) = delete;
  Foo& operator=(const Foo&) = delete;

 private:
  // Clients can't invoke the constructor directly.
  Foo();
};

// foo.c
std::unique_ptr<Foo> Foo::Create() {
  // Note that since Foo's constructor is private, we have to use new.
  return absl::WrapUnique(new Foo());
}

In many cases, this pattern gives you the best of both worlds: the factory function Foo::Create() exposes only fully-initialized objects like a constructor, but it can indicate failure like an initializer method. Another advantage of factory functions is that they can return instances of any subclass of the return type (though this is not possible if using absl::optional as the return type). This allows you to swap in a different implementation without updating user code, or even choose the implementation class dynamically, based on user input.

The primary drawback of this approach is that it returns a pointer to a heap-allocated object, so it’s not well-suited for “value-like” classes designed to work on the stack. However, such classes usually don’t require complex initialization in the first place. Factory functions also can’t be used when a derived class constructor needs to initialize its base, so initializer methods are sometimes necessary in the protected API of a base class. The public API can still use factory functions, though.

Tip of the Week #131: Special Member Functions and `= default`

2017-12-07T00:00:00-05:00

Originally posted as totw/131 on 2017-03-24

By James Dennett (jdennett@google.com)

Since the beginning of time, C++ has supported compiler-declared versions of some so-called special member functions: the default constructor, destructor, copy constructor and copy assignment operators. C++11 added move construction and move assignment to the list, and added syntax (=default and =delete) to give control over when those defaults are declared and defined.

What Does `=default` Do, and Why Would We Use It?

Writing =default is our way to tell the compiler “do what you would normally have done for this special member function”. Why might we want to do this rather than writing an implementation by hand or leaving the compiler to declare one for us?

We can change the access level (e.g., make a constructor protected instead of public), make a destructor virtual, or reinstate a function that would be suppressed (e.g., a default constructor for a class that has other user-declared constructors) and still get the compiler to generate the function for us.
Compiler-defined copy and move operations don’t need maintenance every time members are added or removed, if copying/moving the members is sufficient.
Compiler-provided special member functions can be trivial (when all of the operations they invoke are themselves trivial), which can make them faster and safer.
Types with defaulted constructors can be aggregates, and hence support aggregate initialization, whereas those with user-provided constructors cannot.
Explicitly declaring a defaulted member gives us a place to document the semantics of the resulting function.
In a class template, =default is a simple way to declare an operation conditionally, based on whether some underlying type provides it.

When we use =default on the initial declaration of a special member function, the compiler will check whether it can synthesize an inline definition for that function. If it can, it does. If it can’t, the function is actually declared as deleted, just as if we’d written =delete. That’s just what we’d need for transparently wrapping a class (say, if we’re defining a class template), but it can be surprising to readers.

If a function’s initial declaration uses =default, or if the compiler declares a special member function that is not user-declared, an appropriate noexcept specification is deduced, potentially allowing faster code.

How Does It Work?

Before C++11, if we needed a default constructor and already had other constructors, we’d have written one as:

class A {
 public:
  A() {}  // User-provided, non-trivial constructor makes A a non-aggregate.
};

From C++11 onwards we have more options.

class C {
 public:
  C() = default;  // misleading: C has a deleted default constructor
 private:
  const int i;  // const => must always be initialized.
};

class D {
 public:
  D() = default;  // unsurprising, but not explicit: D has a default constructor
 private:
  std::unique_ptr<int> p;  // std::unique_ptr has a default constructor
};

Clearly we shouldn’t write code like class C: in a non-template, use =default only if you intend the class to support the operation (and then test that it does). clang-tidy includes a check for this.

When =default is used after the first declaration of a special member function (i.e., outside of the class), it has a simpler meaning: it tells the compiler to define the function, and to give an error if it is unable to do so. When =default is used outside of the class, the defaulted function will not be trivial: triviality is determined by the first declaration (so that all clients agree on whether the operation is trivial or not).

If you don’t need your class to be an aggregate and you don’t need the constructor to be trivial then defaulting the constructor outside of the class definition, like examples E and F below, is often a good choice. Its meaning is clear to readers, and is checked by the compiler. For the special case of defaulting a default constructor or a destructor we could write {} instead of =default, but for other defaulted operations the compiler-generated implementation is less simple, and for consistency it’s good to write =default in all applicable cases.

class E {
 public:
  E();  // promises to have a default constructor, but...
 private:
  const int i;  // const => must always be initialized.
};
inline E::E() = default;  // compilation error here: would not initialize `i`

class F {
 public:
  F();  // promises to have a default constructor
 private:
  std::unique_ptr<int> p;  // std::unique_ptr has a default constructor
};
inline F::F() = default;  // works as expected

Recommendations

Prefer =default over writing an equivalent implementation by hand, even if that implementation is just {}. Optionally, omit =default from the initial declaration and provide a separate defaulted implementation.

Be cautious about defaulting move operations. Moved-from objects must still satisfy the invariants of their type, and the default implementations will usually not preserve relationships between fields.

Outside of templates, write =delete instead if =default wouldn’t provide an implementation.

Tip of the Week #123: `absl::optional` and `std::unique_ptr`

2017-12-07T00:00:00-05:00

Originally posted as totw/123 on 2016-09-06

By Alexey Sokolov (sokolov@google.com) and Etienne Dechamps (edechamps@google.com)

How to Store Values

This tip discusses several ways of storing values. Here we use class member variables as an example, but many of the points below also apply to local variables.

#include <memory>
#include "absl/types/optional.h"
#include ".../bar.h"

class Foo {
  ...
 private:
  Bar val_;
  absl::optional<Bar> opt_;
  std::unique_ptr<Bar> ptr_;
};

As a Bare Object

This is the simplest way. val_ is constructed and destroyed at the beginning of Foo’s constructor and at the end of Foo’s destructor, respectively. If Bar has a default constructor, it doesn’t even need to be initialized explicitly.

val_ is very safe to use, because its value can’t be null. This removes a class of potential bugs.

But bare objects are not very flexible:

The lifetime of val_ is fundamentally tied to the lifetime of its parent Foo object, which is sometimes not desirable. If Bar supports move or swap operations, the contents of val_ can be replaced using these operations, while any existing pointers or references to val_ continue pointing or referring to the same val_ object (as a container), not to the value stored in it.
Any arguments that need to be passed to Bar’s constructor need to be computed inside the initializer list of Foo’s constructor, which can be difficult if complicated expressions are involved.

As `absl::optional<Bar>`

This is a good middle ground between the simplicity of bare objects and the flexibility of std::unique_ptr. The object is stored inside Foo but, unlike bare objects, absl::optional can be empty. It can be populated at any time by assignment (opt_ = ...) or by constructing the object in place (opt_.emplace(...)).

Because the object is stored inline, the usual caveats about allocating large objects on the stack apply, just like for a bare object. Also be aware that an empty absl::optional uses as much memory as a populated one.

Compared to a bare object, absl::optional has a few downsides:

It’s less obvious for the reader where object construction and destruction occur.
There is a risk of accessing an object which does not exist.

As `std::unique_ptr<Bar>`

This is the most flexible way. The object is stored outside of Foo. Just like absl::optional, a std::unique_ptr can be empty. However, unlike absl::optional, it is possible to transfer ownership of the object to something else (through a move operation), to take ownership of the object from something else (at construction or through assignment), or to assume ownership of a raw pointer (at construction or through ptr_ = absl::WrapUnique(...), see TotW 126.

When std::unique_ptr is null, it doesn’t have the object allocated, and consumes only the size of a pointer¹.

Wrapping an object in a std::unique_ptr is necessary if the object may need to outlive the scope of the std::unique_ptr (ownership transfer).

This flexibility comes with some costs:

Increased cognitive load on the reader:
- It’s less obvious what’s stored inside (Bar, or something derived from Bar). However, it may also decrease the cognitive load, as the reader can focus only on the base interface held by the pointer.
- It’s even less obvious than with absl::optional where object construction and destruction occur, because ownership of the object can be transferred.
As with absl::optional, there is a risk of accessing an object which does not exist - the famous null pointer dereference.
The pointer introduces an additional level of indirection, which requires a heap allocation, and is not friendly to CPU caches; Whether this matters or not depends a lot on particular use cases.
std::unique_ptr<Bar> is not copyable even if Bar is. This also prevents Foo from being copyable.

Conclusion

As always, strive to avoid unnecessary complexity, and use the simplest thing that works. Prefer bare object, if it works for your case. Otherwise, try absl::optional. As a last resort, use std::unique_ptr.

	`Bar`	`absl::optional<Bar>`	`std::unique_ptr<Bar>`
Supports delayed construction		✓	✓
Always safe to access	✓
Can transfer ownership of `Bar`			✓
Can store subclasses of `Bar`			✓
Movable	If `Bar` is movable	If `Bar` is movable	✓
Copyable	If `Bar` is copyable	If `Bar` is copyable
Friendly to CPU caches	✓	✓
No heap allocation overhead	✓	✓
Memory usage	`sizeof(Bar)`	`sizeof(Bar) + sizeof(bool)`²	`sizeof(Bar)` when null, `sizeof(Bar) + sizeof(Bar)` otherwise
Object lifetime	Same as enclosing scope	Restricted to enclosing scope	Unrestricted
Call `f(Bar*)`	`f(&val_)`	`f(&opt_.value())` or `f(&*opt_)`	`f(ptr_.get())` or `f(&*ptr_)`
Remove value	N/A	`opt_.reset();` or `opt_ = absl::nullopt;`	`ptr_.reset();` or `ptr_ = nullptr;`

In case of a non-empty custom deleter there is also an additional space for that deleter. ↩
Also padding may be added. ↩

Tip of the Week #130: Namespace Naming

2017-11-30T00:00:00-05:00

Originally posted as totw/130 on 2017-02-17

By Titus Winters (titus@google.com)

The precision of naming takes away from the uniqueness of seeing — Pierre Bonnard

The earliest commit of the Google C++ Style Guide contains the guidance that many people are still using for namespace naming. Roughly, this can be summarized as “namespaces are derived from package paths.” Following on the heels of Java’s package naming requirements, this makes a lot of sense: we want to be able to uniquely identify symbols in C++ and we want there to be uniqueness and consistency in namespace choice.

Except in actuality, we don’t. We just didn’t realize for almost a decade.

Name Lookup

Let’s start with how name lookup works in C++ and how it’s different from Java.

namespace foo {
namespace bar {
void f() {
  Baz b;
}
}
}

In C++, lookup on an unqualified name (Baz) will search expanding scopes for a symbol of the same name: first in f() (the function), then in bar, then in foo, then in the global namespace.

In Java, there is no such thing as an unqualified symbol: either a symbol is a qualified name:

public void f() {
  com.google.foo.bar.Baz b = new com.google.foo.bar.Baz();
}

Or it is imported, either as a single package member or via wildcard:

import com.google.foo.bar.Baz;
import com.google.foo.bar.*;

In no case is Baz looked for outside of the package that is explicitly provided: wildcards don’t descend into child packages, nor is search extended into parent packages. As it turns out, this difference in how parent packages/namespaces are handled within Java and C++ is fundamental to why structural namespace naming (making the namespace structure match the package hierarchy) is a mistake within C++.

The Problem

The fundamental problem for building namespaces out of packages is that we rarely rely on fully-qualified lookup in C++, normally writing std::unique_ptr rather than ::std::unique_ptr. Coupled with lookup in enclosing namespaces, this means that for code in a deeply nested package (::division::section::team::subteam::project, for example) any symbol that is not fully qualified (std::unique_ptr) can in fact reference any of

::std::unique_ptr
::division::std::unique_ptr
::division::section::std::unique_ptr
::division::section::team::std::unique_ptr
::division::section::team::subteam::std::unique_ptr
::division::section::team::subteam::project::std::unique_ptr

And what’s worse: unqualified search starts at the bottom of that list and stops as soon as there is a namespace match. This means that your build can be broken if any of your transitive includes add a previously unused namespace that matches the leading namespace of any symbol you use out of an unqualified namespace. Strictly speaking, this doesn’t even have to be a build break: if someone adds something with a matching name and a syntactically-compatible API, the implementation of that API may be completely incompatible and cause widespread havoc at runtime. Obviously this isn’t too bad with std - nobody should ever be adding a nested namespace std - but what about more common namespaces? How about things like testing?

Names aren’t chosen to be unique. Since teams commonly create local utility packages to handle common tasks relating to the infrastructure they rely on, we wind up with local util and pipeline packages - and sub-namespaces. This is a recipe for unnecessary and unintended collisions.

For comparison, the problem in Java is far reduced: if you wildcard-import from two packages in Java and one adds a new symbol with the same name as the other package, your build can break. This is easily and completely solved by forbidding wildcard imports as is done in many Java styles.

Two Consistent Options, Three Approaches

There are two features that prevent this build-break-at-a-distance:

If no leaf namespace (search::foo::bar) matches any top-level namespace (::bar) or a sub-namespace of any parent of that leaf (search::bar), no name collisions will occur.
If there are no unqualified lookups, there will be no problems.

There are (at least) three ways to achieve this:

Always fully qualify everything outside of the current namespace. This is very verbose and sort of weird: nothing in C++ (including the standard library) is written with leading :: on every symbol.
Build some tooling to identify introduction of new namespaces and ensure that it doesn’t overlap with any other namespace in the same hierarchy. That is, do not add search::bar if there is a ::bar or a search::foo::bar.
Don’t nest deeply: a single top-level namespace per project gets the same result without long/complicated names, with less exposure to accidents, without causing surprise for new engineers, and without the need to build any tooling.

The current style guide suggests the last option, but allows for the old style (namespaces match package names) if necessary. This is largely because the Google didn’t want to cause too much anxiety or trigger anyone re-namespacing things. That said, if we had it to do over again in a fresh codebase we would unambiguously say this: one top-level namespace for public interfaces per project. Ensure uniqueness of namespaces via a common database. Thus we get (only) top-level namespaces like absl, and can have no ambiguity in lookup (barring collision between local symbols and those in the global namespace, but modern rules discourage the global namespace anyway).

Because there is so much code that existed before this change, and so much code following the old pattern even after this change, we find ourselves in a sort of half-way space, with some namespaces that often need to be fully qualified (::util), and some that are obviously unique and never need to be (std).

But It Keeps Things Organized!

I regularly hear people express that small/nested namespaces “keep things organized.” Putting things in their place feels right - why lump together something like StrCat() and make_unique() other than being in Abseil these have nothing to do with one another! Wouldn’t an absl::strings::utilities namespace help differentiate from absl::smart_ptrs?

In other languages this would probably be good - better organization with no downside. However, because of how lookup works (expanding into successive layers of containing namespace scopes) your fine-grained namespace is impacted by every symbol (and sub-namespace) added in every parent namespace. That is: while you don’t exactly “contain” the names from parent namespaces, name/namespace collisions matter nearly as much as if you do. Small/deeply-nested namespaces don’t shield you from this, they exacerbate it.

Best Practices

Practically speaking, the following is the best we can do given the realities of most codebases:

Have a database of some form for a codebase to identify the unique namespaces.
When introducing a new namespace, use that database and introduce it as a top-level.
If for some reason the above is impossible, never ever introduce a sub-namespace that matches a well-known top-level namespace. No sub-namespaces for absl, testing, util, etc. Try to give sub-namespaces unique names that are unlikely to collide with future top-levels.
When declaring namespace aliases and using-declarations, use fully qualified names, unless you are referring to a name inside the current namespace, as per TotW 119.
For code in util or other commonly-abused namespaces, try to avoid full qualification, but qualify if necessary.

The advice in TotW 119 also helps, for .cc files: our concern with fully-qualifying is not that it is bad, but that it is weird compared to C++ code in the rest of the world. Limited use in using-declarations strikes an acceptable balance. However, even complete adherence to this suggestion doesn’t fully mitigate the dangers from unqualified name lookup because we still have header files and don’t want to fully qualify every symbol in every header.

Tip of the Week #119: Using-declarations and namespace aliases

2017-11-30T00:00:00-05:00

Originally posted as totw/119 on 2016-07-14

By Thomas Köppe (tkoeppe@google.com)

This tip gives a simple, robust recipe for writing using-declarations and namespace aliases in .cc files that avoids subtle pitfalls.

Before we dive into the details, here is an example of the recipe in action:

namespace example {
namespace makers {
namespace {

using ::otherlib::BazBuilder;
using ::mylib::BarFactory;
namespace abc = ::applied::bitfiddling::concepts;

// Private helper code here.

}  // namespace

// Interface implementation code here.

}  // namespace makers
}  // namespace example

Remember that everything in this tip applies only to .cc files, since you should never put convenience aliases in header files. Such aliases are a convenience for the implementer (and the reader of an implementation), not an exported facility. (Names that are part of the exported API may of course be declared in headers.)

Summary

Never declare namespace aliases or convenience using-declarations at namespace scope in header files, only in `.cc` files.
Declare namespace aliases and using-declarations inside the innermost namespace, whether named or anonymous. (Do not add an anonymous namespace just for this purpose.)
When declaring namespace aliases and using-declarations, use fully qualified names (with leading `::`) unless you are referring to a name inside the current namespace.
For other uses of names, avoid fully qualifying when reasonable, see TotW 130.

(Remember that you can always have a local namespace alias or using-declaration in a block scope, which can be handy in header-only libraries.)

Background

C++ organizes names into namespaces. This crucial facility allows code bases to scale by keeping ownership of names local avoiding name collisions in other scopes. However, namespaces impose a certain cosmetic burden, since qualified names (foo::Bar) are often long and quickly become clutter. We often find it convenient to use unqualified names (Bar). Additionally, we may wish to introduce a namespace alias for a long but frequently used namespace: namespace eu = example::v1::util; We will collectively call using-declarations and namespace aliases just aliases in this tip.

The Problem

The purpose of a namespace is to help code authors avoid name collisions, both at the point of name lookup and at the point of linking. Aliases can potentially undermine the protection afforded by namespaces. The problem has two separate aspects: the scope of the alias, and the use of relative qualifiers.

Scope of the Alias

The scope at which you place an alias can have subtle effects on code maintainability. Consider the following two variants:

using ::foo::Quz;

namespace example {
namespace util {

using ::foo::Bar;

It appears that both using-declarations are effective at making the names Bar and Quz available for unqualified lookup inside our working namespace ::example::util. For Bar, everything is working as expected, provided there is no other declaration of Bar inside namespace ::example::util. But this is your namespace, so it is in your power to control this.

On the other hand, if a header is later included that declares a global name Quz, then the first using-declaration becomes ill-formed, as it attempts to redeclare the name Quz. And if another header declares ::example::Quz or ::example::util::Quz, then unqualified lookup will find that name rather than your alias.

This brittleness can be avoided if you do not add names to namespaces that you do not own (which includes the global namespace). By placing the aliases inside your own namespace, the unqualified lookup finds your alias first and never continues to search containing namespaces.

More generally, the closer a declaration is to the point of use, the smaller is the set of scopes that can break your code. At the worst end of our example is Quz, which can be broken by anyone; Bar can only be broken by other code in ::example::util, and a name that is declared and used inside an unnamed namespace cannot be broken by any other scope. See Unnamed Namespaces for an example.

Relative Qualifiers

A using-declaration of the form using foo::Bar seems innocuous, but it is in fact ambiguous. The problem is that it is safe to rely on the existence of names in a namespace, but it is not safe to rely on the absence of names. Consider this code:

namespace example {
namespace util {

using foo::Bar;

It is perhaps the author’s intention to use the name ::foo::Bar. However, this can break, because the code is relying on the existence of ::foo::Bar and also on the non-existence of namespaces ::example::foo and ::example::util::foo. This brittleness can be avoided by qualifying the used name fully: using ::foo::Bar.

The only time that a relative name is unambiguous and cannot possibly be broken by outside declarations is if it refers to a name that is already inside your current namespace:

namespace example {
namespace util {
namespace internal {

struct Params { /* ... */ };

}  // namespace internal

using internal::Params;  // OK, same as ::example::util::internal::Params

This follows the same logic that we discussed in the previous section.

What if a name lives in a sibling namespace, such as ::example::tools::Thing? You can say either tools::Thing or ::example::tools::Thing. The fully qualified name is always correct, but it may also be appropriate to use the relative name. Use your own judgment.

A cheap way to avoid a lot of these problems is not to use namespaces in your project that are the same as popular top-level namespaces (such as util); the Style Guide recommends this practice explicitly.

Demo

The following code shows examples of both failure modes.

helper.h:

namespace bar {
namespace foo {

// ...

}  // namespace foo
}  // namespace bar

some_feature.h:

extern int f;

Your code:

#include "helper.h"
#include "some_feature.h"

namespace foo {
void f();
}  // namespace foo

// Failure mode #1: Alias at a bad scope.
using foo::f;  // Error: redeclaration (because of "f" declared in some_feature.h)

namespace bar {

// Failure mode #2: Alias badly qualified.
using foo::f;  // Error: No "f" in namespace ::bar::foo (because that namespace was declared in helper.h)

// The recommended way, robust in the face of unrelated declarations:
using ::foo::f;  // OK

void UseCase() { f(); }

}  // namespace bar

Unnamed Namespaces

A using-declaration placed in an unnamed namespace can be accessed from the enclosing namespace and vice versa. If you already have an unnamed namespace at the top of the file, prefer putting all aliases there. From within that unnamed namespace, you gain an extra little bit of robustness against clashing with something declared in the enclosing namespace.

namespace example {
namespace util {

namespace {

// Put all using-declarations in here. Don't spread them over the file.
using ::foo::Bar;
using ::foo::Quz;

// In here, Bar and Quz refer inalienably to your aliases.

}  // namespace

// Can use both Bar and Quz here too. (But don't declare any entities called Bar or Quz yourself now.)

Non-aliased names

So far we have been talking about local aliases for distant names. But what if we want to use names directly and not create aliases at all? Should we say util::Status or ::util::Status?

There is no obvious answer. Unlike the alias declarations that we have been discussing until now, which appear at the top of the file far away from the actual code, the direct use of names affects the local readability of your code. While it is true that relatively qualified names may break in the future, there is a significant cost for using absolute qualifications. The visual clutter caused by the leading :: may well be distracting and not worth the added robustness. In this case, use your own judgment to decide which style you prefer. See TotW 130.

Acknowledgments

All credit is due to Roman Perepelitsa (romanp@google.com) who originally suggested this style in a mailing list discussion and who contributed numerous corrections and punchlines. However, all errors are mine.

CppCon 2017: How to Break an ABI

2017-11-30T00:00:00-05:00

Gennadiy Rozental’s Lightning Talk

By Tom Manshreck, Abseil Tech Writer

Breaking an ABI is seldom your first choice, but what do you do if you simply must break this contract with your developers?

Check out Gennadiy Rozental’s talk on how we implemented our own ::string class in Google, decided we eventually should move back to a std::string, and how we kept Google’s engineers happy during the transition.

Tip of the Week #126: `make_unique` is the new `new`

2017-11-23T00:00:00-05:00

Originally posted as totw/126 on 2016-12-12

By James Dennett (jdennett@google.com) based on a mailing list post by Titus Winters (titus@google.com)

As a codebase expands it is increasingly difficult to know the details of everything you depend on. Requiring deep knowledge doesn’t scale: we have to rely on interfaces and contracts to know that code is correct, both when writing and when reviewing. In many cases the type system can provide those contracts in a common fashion. Consistent use of type system contracts makes for easier authoring and reviewing of code by identifying places where there are potentially risky allocations or ownership transfers for objects allocated on the heap.

While in C++ we can reduce the need for dynamic memory allocation by using plain values, sometimes we need objects to outlive their scope. C++ code should prefer smart pointers (most commonly std::unique_ptr) instead of raw pointers when dynamically allocating objects. This provides a consistent story around allocation and ownership transfer, and leaves a clearer visual signal where there’s code that needs closer inspection for ownership issues. The side effect of matching how allocation works in the outside world post-C++14 and being exception safe is just icing.

Two key tools for this are absl::make_unique() (a C++11 implementation of C++14’s std::make_unique(), for leak-free dynamic allocation) and absl::WrapUnique() (for wrapping owned raw pointers into the corresponding std::unique_ptr types). They can be found in absl/memory/memory.h.

Why Avoid `new`?

Why should code prefer smart pointers and these allocation functions over raw pointers and new?

When possible, ownership is best expressed in the type system. This allows reviewers to verify correctness (absence of leaks and of double-deletes) almost entirely by local inspection. (In code that is exceptionally performance sensitive, this may be excused: while cheap, passing std::unique_ptr across function boundaries by value has non-zero overhead because of ABI constraints. That’s rarely important enough to justify avoiding it.)
Somewhat like the reasoning for preferring push_back() over emplace_back() (TotW 112), absl::make_unique() directly expresses the intent and can only do one thing (do the allocation with a public constructor, returning a std::unique_ptr of the specified type). There’s no type conversion or hidden behavior. absl::make_unique() does what it says on the tin.
The same could be achieved with std::unique_ptr<T> my_t(new T(args)); but that is redundant (repeating the type name T) and for some people there’s value in minimizing calls to new. More on this in #5.
If all allocations are handled via absl::make_unique() or factory calls, that leaves absl::WrapUnique() for the implementation of those factory calls, for code interacting with legacy methods that don’t rely on std::unique_ptr for ownership transfer, and for rare cases that need to dynamically allocate with aggregate initialization (absl::WrapUnique(new MyStruct{3.141, "pi"})). In code review it’s easy to spot the absl::WrapUnique calls and evaluate “does that expression look like an ownership transfer?” Usually it’s obvious (for example, it’s some factory function). When it’s not obvious, we need to check the function to be sure that it’s actually a raw-pointer ownership transfer.
If we instead rely mostly on the constructors of std::unique_ptr, we see calls like:
std::unique_ptr<T> foo(Blah());
std::unique_ptr<T> bar(new T());
It takes only a moment’s inspection to see that the latter is safe (no leak, no double-delete). The former? It depends: if Blah() is returning a std::unique_ptr, it’s fine, though in that case it would be more obviously safe if written as
std::unique_ptr<T> foo = Blah();
If Blah() is returning an ownership-transferred raw pointer, that’s also fine. If Blah() is returning just some random pointer (no transfer), then there’s a problem. Reliance on absl::make_unique() and absl::WrapUnique() (avoiding the constructors) provides an additional visual clue for the places where we have to worry (calls to absl::WrapUnique(), and only those).

How Should We Choose Which to Use?

By default, use absl::make_unique() (or std::make_shared() for the rare cases where shared ownership is appropriate) for dynamic allocation. For example, instead of: std::unique_ptr<T> bar(new T()); write auto bar = absl::make_unique<T>(); and instead of bar.reset(new T()); write bar = absl::make_unique<T>();
In a factory function that uses a non-public constructor, return a std::unique_ptr<T> and use absl::WrapUnique(new T(...)) in the implementation.
When dynamically allocating an object that requires brace initialization (typically a struct, an array, or a container), use absl::WrapUnique(new T{...}).
When calling a legacy API that accepts ownership via a T*, either allocate the object in advance with absl::make_unique and call ptr.release() in the call, or use new directly in the function argument.
When calling a legacy API that returns ownership via a T*, immediately construct a smart pointer with WrapUnique (unless you’re immediately passing the pointer to another legacy API that accepts ownership via a T*).

Summary

Prefer absl::make_unique() over absl::WrapUnique(), and prefer absl::WrapUnique() over raw new.

Tip of the Week #109: Meaningful `const` in Function Declarations

2017-11-23T00:00:00-05:00

Originally posted as totw/109 on 2016-01-14

By Greg Miller (jgm@google.com)

This document will explain when const is meaningful in function declarations, and when it is meaningless and best omitted. But first, let us briefly explain what is meant by the terms declaration and definition.

Consider the following code:

void F(int);                     // 1: declaration of F(int)
void F(const int);               // 2: re-declaration of F(int)
void F(int) { /* ... */ }        // 3: definition of F(int)
void F(const int) { /* ... */ }  // 4: error: re-definition of F(int)

The first two lines are function declarations. A function declaration tells the compiler the function’s signature and return type. In the above example, the function’s signature is F(int). The constness of the function’s parameter type is ignored, so both declarations are equivalent (See “Overloadable declarations”.)

Lines 3 and 4 from the above code are both function definitions. A function definition is also a declaration, but a definition also contains the function’s body. Therefore, line 3 is a definition for a function with the signature F(int). Similarly, line 4 is also a definition for the same function, F(int), which will result in an error at link time. Multiple declarations are allowed, but only a single definition is permitted.

Even though the definitions on lines 3 and 4 declare and define the same function, there is a difference within their function bodies due to the way they are declared. From the definition on line 3, the type of the function-parameter variable within the function will be int (i.e., non-const). On the other hand, the definition on line 4 will produce a function-parameter variable within the function whose type is const int.

Meaningful `const` in Function Declarations

Not all const qualifications in function declarations are ignored. To quote from “Overloadable declarations” ([over.load]) in the C++ standard (emphasis added):

const type-specifiers buried within a parameter type specification are significant and can be used to distinguish overloaded function declarations

The following are examples where const is significant and not ignored:

void F(const int* x);                  // 1
void F(const int& x);                  // 2
void F(std::unique_ptr<const int> x);  // 3
void F(int* x);                        // 4

In the above examples, the x parameter itself is never declared const. Each of the above functions accepts a parameter named x of a different type, thus forming a valid overload set. Line 1 declares a function that accepts a “pointer to an int that is const”. Line 2 declares a function that accepts a “reference to an int that is const”. And line 3 declares a function that accepts a “unique_ptr to an int that is const”. All of these uses of const are important and not ignored because they are part of the parameter type specification and are not top-level const qualifications that affect the parameter x itself.

Line 4 is interesting because it does not include the const keyword at all, and may at first appear to be equivalent to the declaration on line 1 given the reasons cited at the beginning of this document. The reason that this is not true and that line 4 is a valid and distinct declaration is that only top-level, or outermost, const qualifications of the parameter type specification are ignored.

To complete this example, let us look at a few more examples where a const is meaningless and ignored.

void F(const int x);          // 1: declares F(int)
void F(int* const x);         // 2: declares F(int*)
void F(const int* const x);   // 3: declares F(const int*)

Rules of Thumb

Though few of us will ever truly master all the delightful obscurities of C++, it is important that we do our best to understand the rules of the game. This will help us write code that is understood by other C++ programmers who are following the same rules and playing the same game. For this reason, it is important that we understand when const qualification is meaningful in a function declaration and when it is ignored.

Although there is no official guidance from the Google C++ style guide, and there is no single generally accepted opinion, the following is one reasonable set of guidelines:

Never use top-level const on function parameters in declarations that are not definitions (and be careful not to copy/paste a meaningless const). It is meaningless and ignored by the compiler, it is visual noise, and it could mislead readers.
Do use top-level const on function parameters in definitions at your (or your team’s) discretion. You might follow the same rationale as you would for when to declare a function-local variable const.

Tip of the Week #99: Nonmember Interface Etiquette

2017-11-16T00:00:00-05:00

Originally posted as totw/99 on 2015-06-24

Revised 2017-10-10

The interface of a C++ class is not constrained to its members or to its definition. When evaluating an API, we must consider definitions beyond the class body that can be as much a part of its interface as its public members.

These external interface points include template specializations like hashers or traits, nonmember operator overloads (e.g. logging, relationals), and other canonical nonmember functions designed for use with argument-dependent lookup (ADL), most notably swap().

A few of these are illustrated below for some sample class space::Key:

namespace space {
class Key { ... };

bool operator==(const Key& a, const Key& b);
bool operator<(const Key& a, const Key& b);
void swap(Key& a, Key& b);

// standard streaming
std::ostream& operator<<(std::ostream& os, const Key& x);

// gTest printing
void PrintTo(const Key& x, std::ostream* os);

// new-style flag extension:
bool ParseFlag(const string& text, Key* dst, string* err);
string UnparseFlag(const Key& v);

}  // namespace space

HASH_NAMESPACE_BEGIN
template <>
struct hash<space::Key> {
  size_t operator()(const space::Key& x) const;
};
HASH_NAMESPACE_END

There are some important risks associated with making such extensions incorrectly, so this article will try to present some guidance.

The Proper Namespace

Interface points that are functions are usually designed to be found by argument-dependent lookup (ADL, see TotW 49). Operators and some operator-like functions (notably swap()) are designed to be found by ADL. This protocol only works reliably when the function is defined in a namespace associated with the type being customized. The associated namespaces include those of its base classes and class template parameters. A common mistake is to place these functions in the global namespace. To illustrate the problem, consider the following code in which good(x) and bad(x) functions are called with identical syntax:

namespace library {
struct Letter {};

void good(Letter);
}  // namespace library

// bad is improperly placed in global namespace
void bad(library::Letter);

namespace client {
void good();
void bad();

void test(const library::Letter& x) {
  good(x);  // ok: 'library::good' is found by ADL.
  bad(x);  // oops: '::bad' is hidden by 'client::bad'.
}

}  // namespace client

Note the difference between library::good() and ::bad(). The test() function is relying on the absence of any function called bad() in namespaces enclosing the call site. The appearance of client::bad() hides ::bad() from the test caller. Meanwhile, the good() function is found regardless of what else exists in the test() function’s enclosing scope. The C++ name lookup sequence will only yield a global if a name search from closer lexical scopes to the call site fails to find any names.

This is all very subtle, which is the point, really. It’s all much simpler if we default to defining functions alongside the data on which they operate.

A Quick Note on In-Class Friend Definitions

There’s a way to add non-member functions to a class from within the class definition. Friend functions can be defined directly inside a class.

namespace library {
class Key {
 public:
  explicit Key(string s) : s_(std::move(s)) {}
  friend bool operator<(const Key& a, const Key& b) { return a.s_ < b.s_; }
  friend bool operator==(const Key& a, const Key& b) { return a.s_ == b.s_; }
  friend void swap(Key& a, Key& b) {
    swap(a.s_, b.s_);
  }

 private:
  std::string s_;
};
}  // namespace library

These friend functions have a special property of ONLY being visible through ADL. They’re a little strange in that they are defined in the enclosing namespace, but don’t appear in name lookup there. These in-class friend definitions must have inlined bodies for this stealth property to kick in. See “Friend Definitions” for more detail.

Such functions will not hide global functions from fragile call sites in their namespace or appear in diagnostic messages for unrelated calls to functions of the same name. They get out of the way, essentially. They’re also very convenient to define, easy to discover, and they have access to class internals. Probably the biggest drawback is that they will not be found in cases where the enclosing class is implicitly convertible from an argument type.

Note that access (i.e. public, private, and protected) has no effect on friend functions, but it might be polite to place these in the public section anyway, just so they’ll be more visible during perusal of the public API points.

The Proper Source Location

To avoid violations of the one-definition rule (ODR), any customizations on a type’s interface should appear where they can’t accidentally be defined multiple times. This usually means they should be packaged with the type in the same header. It’s not appropriate to add this sort of nonmember customization in a *_test.cc file, or in a “utilities” header off to the side where it can be overlooked. Force the compiler to see the customization and you will be more likely to catch violations.

A function overload (including operator overloads) intended as a nonmember extension should be declared in a header that defines one of its arguments.

The same goes for template specializations. Template specializations can be packaged with the primary template definition, or packaged with the type on which it’s being specialized. For partial specializations or with multiple parameters it’s a judgment call. It’s usually pretty clear in practice which site is better. The important thing is that the specialization should not be hidden in client code: it should be as visible as the template and the user-defined types involved.

When to Customize

Don’t customize the behavior of a class in a test. This is dangerous and unfortunately very common. Test source files are not immune to these dangers and should follow the same rules as production code. We find a lot of inappropriate operators in *_test.cc files that are written with the intent to “get EXPECT_EQ to compile” or some other pragmatic concern. Unfortunately they are still ODR risks (if not violations) and can make library maintenance difficult. These rogue definitions might even stand in the way of library maintenance as adding these operators upstream would break the very tests that needed them and defined their own. For testing there are alternatives available with a little more effort.

Note that ADL works from the original declaration point for a type. Typedefs, type aliases, alias templates, and using declarations do not create types and have no impact on ADL. This can make it a little tricky to locate the proper placement for customizations, but this cannot be helped, so just do it.

Don’t augment the interface to types generated by protobufs. This is another common pitfall. You may own the .proto file, but that doesn’t mean you own the C++ API it generates, and your augmentations could block improvements to the generated C++ API. This is an ODR risk because you can’t ensure that your augmentations are seen whenever the generated header is included.

When defining T, it may be tempting to define behavior for templates like std::vector<T> or std::pair<T,T>. Though your customizations may take precedence and may do what you expect, you may be conflicting with other expected customizations defined on the broader class template.

It’s possible to define some customizations for raw pointers. It may be tempting in some cases to supply these customizations for T* along with T. This is not advised. It’s dangerous because the customization may conflict with the expected ordinary behavior of pointers (e.g. the way they’re logged, swapped, or compared). It’s best to leave pointers alone.

What to Do When You’re Stuck

Following these guidelines can be challenging. Much of the inappropriate overloading and specialization seen in C++ code is motivated by a small set of root causes. Below is a partial listing with successful workarounds. If you run into a library API that can’t be worked with, please send a note to the owners to see about adding the appropriate customization hooks. Common APIs should be usable without breaking these interface packaging guidelines.

Testing a Type with `EXPECT_EQ`, etc.

temptation: EXPECT_EQ requires operator==, and operator<< or GoogleTest’s PrintTo.

workaround: Write lightweight gmock matchers with MATCHER_P instead of relying exclusively on EXPECT_EQ etc.

workaround: Create local (this is essential) wrapper types you DO truly own and provide customizations on those, possibly using trivial inheritance as a shortcut.

Using `T` as a Container Key

temptation: Container default functor types may rely on operator<, operator==, and hash<T>.

workaround: Use more custom comparators or custom hashers. Use more typedefs for your associative container types to hide these details from client code.

Logging Containers of `T`

temptation: Defining operator<< overloads for standard containers.

workaround: Don’t try to log the container directly.

Take-Aways

A type’s behavior is not completely defined by its class definition. Non-member definitions and specializations also contribute. You may need to keep reading past that closing brace to really understand how a class works.

Be aware of when and where it’s safe to add these customizations. Adding inappropriate definitions may get your code to work for now, but you could be adding fragility and maintenance blockers for other engineers down the line to deal with.

Tip of the Week #65: Putting Things in their Place

2017-11-16T00:00:00-05:00

Originally posted as totw/65 on 2013-12-12

By Hyrum Wright (hyrum@hyrumwright.org)

“Let me ’splain. No, there is too much. Let me sum up.” –Inigo Montoya

C++11 added a new way to insert elements into standard containers: the emplace() family of methods. These methods create an object directly within a container, instead of creating a temporary object and then copying or moving that object into the container. Avoiding these copies is more efficient for almost all objects, and makes it easier to store move-only objects (such as std::unique_ptr) in standard containers.

The Old Way and the New Way

Let’s look at a simple example using vectors to contrast the two styles. The first example uses pre-C++11 code:

class Foo {
 public:
  Foo(int x, int y);
  …
};

void addFoo() {
  std::vector<Foo> v1;
  v1.push_back(Foo(1, 2));
}

Using the older push_back() method, two Foo objects are constructed: the temporary argument and the object in the vector that is move-constructed from the temporary.

We can instead use C++11’s emplace_back() and only one object will be constructed directly within the memory of the vector. Since the “emplace” family of functions forward their arguments to the underlying object’s constructor, we can provide the constructor arguments directly, obviating the need to create a temporary Foo:

void addBetterFoo() {
  std::vector<Foo> v2;
  v2.emplace_back(1, 2);
}

Using Emplace Methods for Move-Only Operations

So far, we’ve looked at cases where emplace methods improve performance, but they also make previously impossible code feasible, such as storing move-only types like std::unique_ptr within containers. Consider this snippet:

std::vector<std::unique_ptr<Foo>> v1;

How would you insert values into this vector? One way would be to use push_back() and construct the value directly within its argument:

v1.push_back(std::unique_ptr<Foo>(new Foo(1, 2)));

This syntax works, but can be a bit unwieldy. Unfortunately, the traditional way of getting around this confusion is fraught with complexity:

Foo *f2 = new Foo(1, 2);
v1.push_back(std::unique_ptr<Foo>(f2));

This code compiles, but it leaves ownership of the raw pointer unclear until the insertion. What’s worse, the vector now owns the object, but f2 still remains valid, and could accidentally be deleted later on. To an uninformed reader, this ownership pattern can be confusing, particularly if construction and insertion are not sequential events as above.

Other solutions won’t even compile, because unique_ptr isn’t copyable:

std::unique_ptr<Foo> f(new Foo(1, 2));
v1.push_back(f);             // Does not compile!
v1.push_back(new Foo(1, 2)); // Does not compile!

Using emplace methods can make it more intuitive to insert the object while it’s being created. In other cases, if you need to move the unique_ptr into the vector, you can:

std::unique_ptr<Foo> f(new Foo(1, 2));
v1.emplace_back(new Foo(1, 2));
v1.push_back(std::move(f));

By combining emplace with a standard iterator, you can also insert the object at an arbitrary location in the vector:

v1.emplace(v1.begin(), new Foo(1, 2));

That said, in practical terms we wouldn’t want to see these ways to construct a unique_ptr - Use std::make_unique (from C++14) or absl::make_unique (if you’re still on C++11).

Conclusion

We’ve used vector as an example in this Tip, but emplace methods are also available for maps, lists and other STL containers. When combined with unique_ptr, emplace allows for good encapsulation and makes the ownership semantics of heap-allocated objects clear in ways that weren’t possible before. Hopefully this has given you a feel for the power of the new emplace family of container methods, and a desire to use them where appropriate in your own code.

Tip of the Week #112: emplace vs. push_back

2017-11-16T00:00:00-05:00

Originally posted as totw/112 on 2016-02-25

By Geoff Romer (gromer@google.com)

Revised 2017-08-30

“The less we use our power, the greater it will be.” — Thomas Jefferson

As you may already know (and if not, TotW 65), C++11 introduced a powerful new way to insert items into containers: emplace methods. These let you construct an object in-place inside the container, using any of the object’s constructors. That includes the move and copy constructors, so it turns out any time you could use a push or insert method, you can use an emplace method instead, with no other changes:

std::vector<string> my_vec;
my_vec.push_back("foo");     // This is OK, so...
my_vec.emplace_back("foo");  // This is also OK, and has the same result

std::set<string> my_set;
my_set.insert("foo");        // Same here: any insert call can be
my_set.emplace("foo");       // rewritten as an emplace call.

This raises an obvious question: which one should you use? Should we perhaps just discard push_back() and insert(), and use emplace methods all the time?

Let me answer that question by asking another: what do these two lines of code do?

vec1.push_back(1<<20);
vec2.emplace_back(1<<20);

The first line is quite straightforward: it adds the number 1048576 to the end of the vector. The second, however, is not so clear. Without knowing the type of the vector, we don’t know what constructor it’s invoking, so we can’t really say what that line is doing. For example, if vec2 is a std::vector<int>, that line merely adds 1048576 to the end, as with the first line, but if vec2 is a std::vector<std::vector<int>>, that second line constructs a vector of over a million elements, allocating several megabytes of memory in the process.

Consequently, if you have a choice between push_back() and emplace_back() with the same arguments, your code will tend to be more readable if you choose push_back(), because push_back() expresses your intent more specifically. Choosing push_back() is also safer: suppose you have a std::vector<std::vector<int>> and you want to append a number to the end of the first vector, but you accidentally forget the subscript. If you write my_vec.push_back(2<<20), you’ll get a compile error and you’ll quickly spot the problem. On the other hand, if you write my_vec.emplace_back(2<<20), the code will compile, and you won’t notice any problems until run-time.

Now, it’s true that when implicit conversions are involved, emplace_back() can be somewhat faster than push_back(). For example, in the code that we began with, my_vec.push_back("foo") constructs a temporary string from the string literal, and then moves that string into the container, whereas my_vec.emplace_back("foo") just constructs the string directly in the container, avoiding the extra move. For more expensive types, this may be a reason to use emplace_back() instead of push_back(), despite the readability and safety costs, but then again it may not. Very often the performance difference just won’t matter. As always, the rule of thumb is that you should avoid “optimizations” that make the code less safe or less clear, unless the performance benefit is big enough to show up in your application benchmarks.

So in general, if both push_back() and emplace_back() would work with the same arguments, you should prefer push_back(), and likewise for insert() vs. emplace().

Tip of the Week #49: Argument-Dependent Lookup

2017-11-09T00:00:00-05:00

Originally posted as totw/49 on 2013-07-14

“…whatever disappearing trail of its legalistic argle-bargle one chooses to follow…” –Antonin Scalia, U.S. v Windsor dissenting opinion

Overview

A function call expression such as func(a,b,c), in which the function is named without the :: scope operator, is called unqualified. When C++ code refers to a function by an unqualified name, the compiler performs a search for a matching function declaration. What is surprising to some people (and different from other languages) is that in addition to the caller’s lexical scope, the set of search scopes is augmented by namespaces associated with the function argument types. This additional lookup is called Argument-Dependent Lookup (ADL). It’s definitely happening in your code, so you’ll be much better off with a basic understanding of how it works.

Name Lookup Basics

A function call must be mapped to a single function definition by the compiler. This matching is done in two independent serial processing stages. First, name lookup applies some scope searching rules to produce a set of overloads matching the name of the function. Then overload resolution takes those overloads produced by that name lookup and tries to choose a best match for the arguments given at the call site. Keep this distinction in mind. Name lookup comes first, and it doesn’t try to make any determination as to whether a function is a good match or not. It doesn’t even consider the argument count. It just searches scopes for a function name. Overload resolution is a complex topic in its own right, but it’s not our focus right now. Just know that it’s a separate processing stage that gets its inputs from name lookup.

When an unqualified function call is encountered, several independent search sequences can occur for that function name, each attempting to match the name to a set of overloads. The most obvious search is the outward search starting from the lexical scope of the call site:

namespace b {
void func();
namespace internal {
void test() { func(); } // ok: finds b::func().
} // b::internal
} // b

This name lookup has nothing to do with ADL yet (func() has no arguments). It is simply a search outward from the site of the function call, proceeding outward from local function scope (if applicable), to class scope, enclosing class scope, and base classes (if applicable), then to namespace scope, and out into enclosing namespaces, and finally the global :: namespace.

As name lookup progresses through a sequence of increasingly widening scopes, the process stops as soon as any function with the target name is found, whether or not that function’s arguments are compatible with the arguments supplied by the call site. When a scope is encountered containing at least one function declaration with the target name, the overloads in that scope become the result of that name lookup.

This is illustrated in the example below:

namespace b {
void func(const string&);  // b::func
namespace internal {
void func(int);  // b::internal::func
namespace deep {
void test() {
  string s("hello");
  func(s);  // error: finds only b::internal::func(int).
}
}  // b::internal::deep
}  // b::internal
}  // b

It’s tempting but incorrect to think a func(s) expression will overlook the obviously bad match of b::internal::func(int), and continue looking to the next outward enclosing scope to find b::func(const string&). However, name lookup doesn’t consider argument types. It finds something called func and stops in b::internal, leaving the evaluation of “obviously bad” to the overload resolution phase. The b::func(const string&) function is never even seen by overload resolution.

An important implication of the scoped search order is that overloads in a scope appearing earlier in the search order will hide overloads from later scopes.

Argument-Dependent Lookup (ADL)

If a function call passes arguments, a few more parallel name lookups are launched. These extra lookups consider each associated namespace of each of the function call’s arguments. Unlike lexical scope name lookup, these argument- dependent lookups do not proceed to enclosing scopes.

Results of lexical scope name lookup and all ADLs are merged together, to form the final set of function overloads.

The Simple Case

Consider the following code:

namespace aspace {
struct A {};
void func(const A&);  // found by ADL name lookup on 'a'.
}  // namespace aspace

namespace bspace {
void func(int);  // found by lexical scope name lookup
void test() {
  aspace::A a;
  func(a);  // aspace::func(const aspace::A&)
}
}  // namespace bspace

Two name lookups are launched to resolve the call to func(a). The lexical scope name lookup starts in the local function scope of bspace::test(). It finds no func there and proceeds to the scope of namespace bspace, in which it finds func(int) and stops. The other name lookup, which is due to ADL, starts in the namespace associated with the argument a. In this case, that’s only namespace aspace. That lookup finds aspace::func(const aspace::A&) and stops. Overload resolution therefore receives two candidates. These are ‘bspace::func(int)’ from lexical name lookup, and ‘aspace::func(const aspace::A&)’ from the single ADL lookup. In overload resolution, the func(a) call resolves to aspace::func(const aspace::A&). The bspace::func(int) overload is not a good match for the argument type and so it is rejected by overload resolution.

The lexical name search and each of the additional ADL-triggered name searches can be considered to occur in parallel, with each returning a set of candidate function overloads. The results of all such searches are thrown in a bag and they compete via overload resolution to determine the best match. If there’s a tie for best match, the compiler issues an ambiguity error; “There can be only one.” If no overload is a good match, that’s an error too. So more precisely, “there must be exactly one”, which doesn’t sound as cool in a movie trailer.

Type-Associated Namespaces

The previous example was the simple case, but a more sophisticated type can have many namespaces associated with it. The set of namespaces associated with a type includes any namespace of any type that appears as a part of the argument type’s full name, including its template parameter types. It also includes the namespaces of direct and indirect base classes. For example, a single argument that expands to the type a::A<b::B, c::internal::C*> will produce searches beginning in the a, b and c::internal namespaces (and any other namespaces associated with the constituent types a::A, b::B, or c::internal::C), each looking for the called function name. The following example shows a few of these effects:

namespace aspace {
struct A {};
template <typename T> struct AGeneric {};
void func(const A&);
template <typename T> void find_me(const T&);
}  // namespace aspace

namespace bspace {
typedef aspace::A AliasForA;
struct B : aspace::A {};
template <typename T> struct BGeneric {};
void test() {
  // ok: base class namespace searched.
  func(B());
  // ok: template parameter namespace searched.
  find_me(BGeneric<aspace::A>());
  // ok: template namespace searched.
  find_me(aspace::AGeneric<int>());
}
}  // namespace bspace

Tips

With the fundamental name lookup mechanism fresh in your mind, consider the following tips which may help you when you are working with real C++ code.

Type Aliases

Sometimes determining the set of namespaces associated with a type will take a bit of detective work. typedef and using declarations can introduce aliases for a type. In those cases the aliases are fully resolved and expanded to their source types before the list of namespaces to search are chosen. This is one way in which typedef and using declarations can be a bit misleading, because they can lead you to make incorrect predictions about which namespaces will be searched by ADL. This is demonstrated below:

namespace cspace {
// ok: note that this searches aspace, not bspace.
void test() {
  func(bspace::AliasForA());
}
}  // namespace cspace

Caveat Iterator

Be careful with iterators. You don’t really know with what namespaces they are associated, so don’t rely on ADL for resolving function calls involving iterators. They might just be pointers to the elements, or they might be in some namespace private to the implementation that has nothing to do with the container’s namespace.

namespace d {
int test() {
  std::vector<int> vec(a);
  // maybe this compiles, maybe not!
  return count(vec.begin(), vec.end(), 0);
}
}  // namespace d

The above code has a dependency on whether std::vector<int>::iterator is int* (which is possible) or some type in a namespace that has a count overload (like std::count()). It’s possible that this will work on some platforms and not others, or that it will work in debug builds with instrumented iterators, but not in optimized builds. It’s better to just qualify the function name. If you want to call std::count(), spell it that way.

Overloaded Operators

An operator (e.g. + or <<) can be thought of as a kind of function name, e.g. operator+(a,b) or operator<<(a,b), and are also unqualified. One of the most important uses of ADL is the search for operator<< used during logging. Usually we see something like std::cout << obj; for some obj, let’s say of type O::Obj. This statement is like an unqualified function call of the form operator<<(std::ostream&, const O::Obj&), which will find overloads in the std namespace from the std::ostream parameter, the O namespace from the O::Obj parameter, and of course any overloads picked up from the lexical scope search from the call site.

It’s important to place such operators in the same namespace as the user-defined type they’re meant to operate upon: in this case within namespace O. If the operator<< is placed in an outer namespace like :: (the global namespace), that operator will work for a while until someone quite innocently places an unrelated operator<< in namespace ‘O’ for some other type. It takes a bit of discipline but saves a lot of confusion later to follow the simple rule of defining all operators and other associated nonmember functions next to the type’s definition in the same namespace.

Fundamental Types

Note that the fundamental types (e.g. int, double, etc) are not associated with the global namespace. They are associated with no namespace. They do not contribute any namespaces to ADL. Pointer and array types are associated with their pointee or element types.

Refactoring Gotchas

Refactorings that change the types of arguments to an unqualified function call can affect which, if any, overloads are considered. Just moving a type into a namespace and leaving behind a typedef in the old namespace for compatibility doesn’t help, and really just makes the problem harder to diagnose. Be careful when moving types to new namespaces.

Similarly, moving a function to a new namespace and leaving behind a using declaration might mean that unqualified calls won’t find it anymore. Sadly, they might still compile by finding a different function you didn’t intend them to find. Be careful when moving functions to new namespaces.

Final Thought

Relatively few programmers understand the exact rules and corner cases involved with function lookups. The language spec contains 13 pages of rules about what exactly goes into a name search, including special cases, details about friend functions, and enclosing class scopes to keep your head spinning for years. Despite all this complexity, if you keep the basic idea of parallel name searches in mind, you’ll be on solid footing for understanding how your function calls and operators are resolving. You will now be able to see how seemingly remote declarations end up being chosen when you invoke functions or operators. And you’ll be a little better able to diagnose puzzling build errors like ambiguities or name-hiding effects when they happen.

Tip of the Week #135: Test the Contract, not the Implementation

2017-11-09T00:00:00-05:00

Originally posted as TotW #135 on June 5, 2017

By James Dennett

Updated 2020-04-06

Quicklink: abseil.io/tips/135

“If you have one true friend you have more than your share.” — Thomas Fuller

C++ has a somewhat elaborate access control mechanism using public, protected, private, and friend. Test code has its own rules of etiquette for using those facilities, and GoogleTest augments them with its own FRIEND_TEST macro. Use of FRIEND_TEST should be a last resort, not a preferred choice.

Test the Contract

We write tests to find bugs in the implementation of a component’s contract, or to give us sufficient confidence that there are no such bugs. When using test-driven development (TDD), we also write tests to help us to design that contract. Tests that depend on unspecified aspects of a component are brittle, and prone to reporting failures even when the production code is working correctly.

Prefer to test via the public interface of a component. More generally, tests should verify the contract of a component and, just as with any other client, should not make assumptions beyond what’s guaranteed.

Techniques for Providing Access from Tests

A number of techniques can be used to allow test code the access needed to do its job. Some are enumerated here, roughly arranged from best to worst.

Augmenting the Public API for Tests

Sometimes it’s hard to get sufficient coverage when testing via a minimal interface. If your component is implementing a very narrow interface specified by a base class (e.g., one with only a ProcessItem virtual function) and it’s impractical to gain sufficient confidence from a test that uses only that interface, consider creating a new, testable component containing the implementation details. Then the class containing the virtual function can be so simple that it needs only minimal testing. BUILD visibility can be used to restrict use of your implementation class (if needed and if your build system supports it).

If a test depends on just one or two private functions, consider making those functions part of the public interface. This isn’t so bad: you’ll need them to have a clearly documented interface anyway, and other clients (not just the test) might find them useful. If, after consideration, you decide that a function really is only for tests, then it should be documented as such, and maybe named with a ForTesting suffix.

Using Peers to Avoid Exposing Implementation

If the test still needs to have access to private implementation details, create a test peer (sometimes called a test spouse). A test peer is a friend class of the class under test, often defined in the _test.cc file (though some prefer to define it in the same file as the class that befriends it), and used to provide controlled access to the class under test to the test code. The test peer won’t be able to live in an anonymous namespace as its exact name needs to match the friend declaration, but the rest of the test code can be in an anonymous namespace as usual. The name of a test peer class typically ends in Peer.

(Avoid) Using `FRIEND_TEST`

While common in older code, FRIEND_TEST should not be used in new code. It introduces reverse coupling, making the production header file depend on details of the associated unit test. It forces the tests to move out of the anonymous namespace. Every FRIEND_TEST grants a test function unrestricted access to the class under test; in long test functions it can be hard to see where the test modifies the state of the class under test. It requires use of an unusual header file provided by GoogleTest for inclusion in production code, whereas almost all of GoogleTest is intended only for use in tests. And finally it scales badly, requiring new FRIEND_TEST uses to be added to the production header when new tests are added. In practice this often results in header files having lengthy blocks of FRIEND_TEST lines.

(Don’t) Make the Entire Test Fixture a `friend`

Making the entire test fixture a friend of the class under test (with friend class MyClassTest;) is strongly discouraged. Compared to the options above, it allows the entire test fixture (but not the tests themselves, which are sub-classes of the fixture) unrestricted and unannotated access to every member of the class under test, meaning that readers of the test code have no visual clue of when the test is breaking encapsulation. It also forces the test fixture to be outside of the anonymous namespace. Compared to a friend fixture, a test peer makes the code much more self-documenting for readers, and costs just a little extra work for code authors.

Summary of Recommendations

Prefer to test the client interface of a component, and keep tests independent of private implementation details.
Factor out a testable, possibly test-only subcomponent if the client interface isn’t sufficient to thoroughly exercise the unit under test.
Sometimes it’s reasonable to add to the public interface in order to make a component testable.
If necessary, access private members from tests using a test peer rather than using FRIEND_TEST.
Do not befriend an entire test fixture. Use one of the more targeted approaches described above.

Tip of the Week #107: Reference Lifetime Extension

2017-11-02T00:00:00-04:00

Originally posted as totw/107 on 2015-12-10

By Titus Winters (titus@google.com)

There have been some reports of confusion about references and lifetimes after TotW 101, so in this Tip we’ll dive more into the question, “When does reference lifetime extension apply?”

string Foo::GetName();

const string& name = obj.GetName();  // Is this safe/legal?

In short, lifetimes of temporaries (referents) will be extended if and only if

A local const T& (or T&&, although Google style generally ignores that) is initialized to the result of an expression (usually a function call) returning a temporary T or the T subobject of a temporary (e.g. a struct containing T).

The standard-ese may be a little tricky to unpack, so lets discuss some of the edge cases to clarify:

This doesn’t work when assigning to T&, it must be const T&. (It’s a compilation error.)
This doesn’t work if there is a (non-polymorphic) type conversion at play. For instance, assigning to const absl::string_view& from string does not extend the lifetime of the string. (You also don’t want const absl::string_view& in the first place, but that’s a separate issue).
This doesn’t work when you’re getting the subobject indirectly: the compiler doesn’t look through function calls (getters or the like). The subobject form only works when you’re directly assigning from a public member variable subobject of the temporary. (So it doesn’t come up often because we don’t have many expressions that return temporary structs.)
The case where type conversion is allowed is when assigning to a T& from a temporary of U when T is a parent class of U. Please don’t do that: it’s even more confusing for readers than the other cases.

If the lifetime of the temporary is extended, it will last until the reference goes out of scope. If the lifetime of the temporary isn’t extended in the above fashion, the T being referred to is destroyed at the end of the statement (when we get to the next ;).

As per TotW 101 you probably shouldn’t rely on lifetime extension in the explicit case of reference-initialization: it’s not gaining you much/any performance, and it is subtle, fragile, and prone to cause extra work for your reviewers and future maintainers.

There are subtle cases where lifetime extension is happening and is necessary and beneficial (like ranged-for over a temporary container), but again, the extension is only for the result of the temporary expression, not any sub-expressions. For instance, these work:

std::vector<int> GetInts();
for (int i : GetInts()) { }  // lifetime extension on the vector is important

// Return string_views of size 1 for each char in this string.
std::vector<absl::string_view> Explode(const string& s);

// Lifetime extension kicks in on the vector, but *not* on the temporary string!
for (absl::string_view s : Explode(StrCat("oo", "ps"))) { }  // WRONG

This does not work:

MyProto GetProto();

// Lifetime extension *doesn't work* here: sub_protos (a repeated field)
// is destroyed by MyProto going out of scope, and the lifetime extension rules
// don't kick in here to magically lifetime extend the MyProto returned by
// GetProto().  The sub-object lifetime extension only works for simple
// is-a-member-of relationships: the compiler doesn't see that sub_protos()
// itself returning a reference to an sub-object of the outer temporary.
for (const SubProto& p : GetProto().sub_protos()) { }  // WRONG

Tip of the Week #101: Return Values, References, and Lifetimes

2017-11-02T00:00:00-04:00

Originally posted as totw/101 on 2015-07-29

By Titus Winters (titus@google.com)

Consider the following code snippet:

const string& name = obj.GetName();
std::unique_ptr<Consumer> consumer(new Consumer(name));

In particular, I’d like to draw your attention to the &. Is it appropriate? What should we check? What can go wrong? I find a decent number of C++ programmers that aren’t entirely clear on references, but do generally know that they “avoid making copies.” As with most issues in C++, it’s more complicated than that.

Case by Case: What is Being Returned and How is it Being Stored?

There are two (maybe three) important questions here:

What type is returned (in this example, by GetName())?
What type are we storing into/initializing (in this example, what is the type of name)?
If we are returning a reference, does the object being returned by reference have any lifetime limitations?

We’ll continue with string as our example type, but the same arguments generalize for most non-trivial value types.

Returning string, initializing string: This is usually RVO and is guaranteed to be, at worst, a move for modern types (See TotW 77).
Returning string& or const string&, initializing string: This is a copy (there must be some long-living object which we are returning a reference to, so once we initialize a new string there are two names for that data, hence a copy. See TotW 77). Sometimes this is valuable, like if you need your string to outlast the lifetime guarantees provided by the function.
Returning string, initializing string&: This won’t compile, as you cannot bind a reference to a temporary.
Returning const string&, initializing string&: This won’t compile, as you’ve dropped the const inappropriately.
Returning const string&, initializing const string&: This is no cost (you’re effectively returning just a pointer). However, you have inherited any existing lifetime restrictions: how long is that reference good for? Most accessor methods that return a reference are returning a member - at most, the reference can be valid for the life of the containing object.
Returning string&, initializing string&: This is the same as #5, but with the additional caveat: the returned reference is non-const, so any modifications to your reference will be reflected in the source.
Returning string&, initializing const string&: The same as #5.
Returning string, initializing const string&: You’d think, given #3, that this wouldn’t work. However, the language has special-case support for this: if you initialize a const T& with a temporary T, that T (in this case string) is not destroyed until the reference goes out of scope (in the common case of automatic or static variables).

Scenario #8 is what allows most reflexive use of references (that is “Oh, I don’t want to copy so I’ll just assign into a reference” without necessarily thinking about what is being returned). However, because of #1, it’s also not really doing anything for you: there probably wouldn’t have been a copy in the first place. Further, now readers of your code have to contend with your local variable being of type const string& instead of string, and thus worry about whether the underlying string has gone out of scope or changed.

Put another way: when code reviewing the original snippet, I have to worry about:

Is GetName() returning by value or by reference?
Does the constructor for Consumer take string, const string& or string_view?
Does the constructor have any lifetime requirements placed on that parameter? (If it isn’t just string.)

However, if you just declare name as string in the first place, it’s generally no less efficient (because of RVO and move semantics), and at least as likely to be safe with respect to object lifetimes.

Additionally, if there is an object lifetime issue, it’s often simpler to spot when storing as string: rather than looking at the interplay between the lifetime promise of GetName()’s returned reference and the lifetime requirements of SetName(), having your own string means only having to look at the local code and SetName().

All of which is to say: avoiding copies is fine, so long as you’re not making things more complicated. Making code more complicated when there wasn’t a copy in the first place is not a good tradeoff.

CppCon 2017: 'A Type, by Any Other Name'

2017-10-31T00:00:00-04:00

Jon Cohen: “A Type, by Any Other Name”

By Tom Manshreck, Abseil Tech Writer

It’s Halloween, so find out what spooky things (like ADL) lurk when doing a major renaming/refactoring on a codebase as large as Google’s. Oooh, scary, kids!

Tip of the Week #94: Callsite Readability and bool Parameters

2017-10-26T00:00:00-04:00

Originally posted as totw/94 on 2015-04-27

By Geoff Romer (gromer@google.com)

Revised 2017-10-25

“Of the many forms of false culture, a premature converse with abstractions is perhaps the most likely to prove fatal to the growth of … vigour of intellect.” — George Boole.

Suppose you come across code like this:

int main(int argc, char* argv[]) {
  ParseCommandLineFlags(&argc, &argv, false);
}

Can you tell what this code does, and in particular what the last argument means? Now suppose you’ve come across this function before, and you know that the last argument has to do with whether command-line flags are left in argv after the call. Can you tell which is true and which is false?

Of course you don’t know, because this is hypothetical, but even in real code, we have better things to do with our brains than memorize the meaning of every function parameter, and better things to do with our time than go look up the documentation for every function call we encounter. We have to be able to make pretty good guesses about what function calls mean, just from looking at the callsite.

Well-chosen function names are the key to making function calls readable, but they’re often not enough. We usually need the arguments themselves to give us some clues about what they mean. For example, you might not know what to make of absl::string_view s(x, y); if you’d never seen string_view before, but absl::string_view s(my_str.data(), my_str.size()); and absl::string_view s("foo"); are much clearer. The trouble with bool parameters is that the argument at the callsite is very often a literal true or false, and that gives the reader no contextual cues about the meaning of the parameter, as we saw in the ParseCommandLineFlags() example. This problem is compounded if there’s more than one bool parameter, because now you have the additional problem of figuring out which parameter is which.

So how can we fix code like that example? One (bad) possibility is to do something like this:

int main(int argc, char* argv[]) {
  ParseCommandLineFlags(&argc, &argv, false /* preserve flags */);
}

The drawbacks of this approach are evident: it’s not clear if that comment is describing the meaning of the parameter or the effect of the argument. In other words, is it saying that we’re preserving the flags, or is it saying that it’s false that we’re preserving the flags? Even if the comment manages to make that clear, there’s still a risk that the comment will get out of sync with the code.

A better approach is to specify the name of the parameter in the comment:

int main(int argc, char* argv[]) {
  ParseCommandLineFlags(&argc, &argv, /*remove_flags=*/false);
}

This is much clearer, and much less likely to fall out of sync with the code. Clang-tidy will even check that the comment has the correct parameter name. A less ambiguous but longer variant is to use an explanatory variable:

int main(int argc, char* argv[]) {
  const bool remove_flags = false;
  ParseCommandLineFlags(&argc, &argv, remove_flags);
}

However, explanatory variable names are not checked by the compiler, and so they may be erroneous. This is a particular problem when you have multiple bool parameters, which might be transposed by the caller.

All these approaches also rely on programmers to consistently remember to add these comments or variables, and to do so correctly (although clang-tidy will check the correctness of parameter-name comments).

In many cases, the best solution is to avoid using bool parameters in the first place, and use an enum instead. For example, ParseCommandLineFlags() could be declared like this:

enum ShouldRemoveFlags { kDontRemoveFlags, kRemoveFlags };

void ParseCommandLineFlags(int* argc, char*** argv, ShouldRemoveFlags remove_flags);

so that the call could look like this:

int main(int argc, char* argv[]) {
  ParseCommandLineFlags(&argc, &argv, kDontRemoveFlags);
}

You can also use an enum class, as described in TotW 86, but in that case you’d want to use a slightly different naming convention, e.g.:

enum class ShouldRemoveFlags { kNo, kYes };
…
int main(int argc, char* argv[]) {
  ParseCommandLineFlags(&argc, &argv, ShouldRemoveFlags::kNo);
}

Obviously, this approach has to be implemented when the function is defined; you can’t really opt into it at the callsite (you can fake it, but for little benefit). So when you’re defining a function, particularly if it’s going to be widely used, the onus is on you to think carefully about how the callsites will look, and in particular to be very skeptical of bool parameters.

Tip of the Week #86: Enumerating with Class

2017-10-26T00:00:00-04:00

Originally posted as totw/86 on 2015-01-05

By Bradley White (bww@google.com)

“Show class, … and display character.” - Bear Bryant.

An enumeration, or simply an enum, is a type that can hold one of a specified set of integers. Some values of this set can be given names, and are called the enumerators.

Unscoped Enumerations

This concept will be familiar to C++ programmers, but prior to C++11 enumerations had two significant shortcomings: the enumeration names were:

in the same scope as the enum type, and
implicitly convertible to values of some integer type.

So, with C++98 …

enum CursorDirection { kLeft, kRight, kUp, kDown };
CursorDirection d = kLeft; // OK: enumerator in scope
int i = kRight;            // OK: enumerator converts to int

but, …

// error: redeclarations of kLeft and kRight
enum PoliticalOrientation { kLeft, kCenter, kRight };

C++11 modified the behavior of unscoped enums in one way: the enumerators are now local to the enum, but continue to be exported into the enum’s scope for backwards compatibility.

So, with C++11 …

CursorDirection d = CursorDirection::kLeft;  // OK in C++11
int i = CursorDirection::kRight;             // OK: still converts to int

but the declaration of PoliticalOrientation would still elicit errors.

Scoped Enumerations

The implicit conversion to integer has been observed to be a common source of bugs, while the namespace pollution caused by having the enumerators in the same scope as the enum causes problems in large, multi-library projects. To address both these concerns, C++11 introduced a new concept: the scoped enum.

In a scoped enum, introduced by the keywords enum class, the enumerators are:

only local to the enum (they are not exported into the enum’s scope), and
not implicitly convertible to integer types.

So, (note the additional class keyword) …

enum class CursorDirection { kLeft, kRight, kUp, kDown };
CursorDirection d = kLeft;                    // error: kLeft not in this scope
CursorDirection d2 = CursorDirection::kLeft;  // OK
int i = CursorDirection::kRight;              // error: no conversion

and, …

// OK: kLeft and kRight are local to each scoped enum
enum class PoliticalOrientation { kLeft, kCenter, kRight };

These simple changes eliminate the problems with plain enumerations, so enum class should be preferred in all new code.

Using a scoped enum does mean that you’ll have to explicitly cast to an integer type should you still want such a conversion (e.g., when logging an enumeration value, or when using bitwise operations on flag-like enumerators). Hashing with std::hash will continue to work though (e.g., std::unordered_map<CursorDirection, int>).

Underlying Enumeration Types

C++11 also introduced the ability to specify the underlying type for both varieties of enumeration. Previously the underlying integer type of an enum was determined by examining the sign and magnitude of the enumerators, but now we can be explicit. For example, …

// Use "int" as the underlying type for CursorDirection
enum class CursorDirection : int { kLeft, kRight, kUp, kDown };

Because this enumerator range is small, and if we wished to avoid wasting space when storing CursorDirection values, we could specify char instead.

// Use "char" as the underlying type for CursorDirection
enum class CursorDirection : char { kLeft, kRight, kUp, kDown };

The compiler will issue an error if an enumerator value exceeds the range of the underlying type.

Conclusion

Prefer using enum class in new code. You’ll reduce namespace pollution, and you may avoid bugs in implicit conversions.

enum class Parting { kSoLong, kFarewell, kAufWiedersehen, kAdieu };

Tip of the Week #64: Raw String Literals

2017-10-26T00:00:00-04:00

Originally published as totw/64 on 2013-12-09

By Titus Winters (titus@google.com)

Updated 2017-10-23

Quicklink: abseil.io/tips/64

"(?:\"(?:\\\\\"|[^\"])*\"|'(?:\\\\'|[^'])*')"; — A cat walking over the keyboard . . . or maybe what the fox says . . . no, actually just a highly- escaped regexp found in real C++ code.

Odds are you’ve had trouble getting your regular expression understood properly in C++ due to escaping issues. Similarly, you’ve probably been annoyed with preserving quotes and newlines when embedding a text version of Protobuf or JSON data into your unittests. When you have to use significant escaping (or worse, multi-layer escaping), code clarity drops precipitously.

Luckily, there’s a new C++11 feature that removes this need for escaping: raw string literals.

The Raw String Literal Format

A raw string literal has the following special syntax:

R"tag(whatever you want to say)tag"

tag is a sequence of up to 16 characters (and an empty tag is both OK and common). The characters after ‘“tag(‘ and before the first following occurrence of ‘)tag”’ are used literally as the contents of the string literal. ‘tag’ may contain any character but parentheses, backslash, and whitespace.

Examine the difference:

const char concert_17_raw[] =
    "id: 17\n"
    "artist: \"Beyonce\"\n"
    "date: \"Wed Oct 10 12:39:54 EDT 2012\"\n"
    "price_usd: 200\n";

versus:

const char concert_17_raw[] = R"(
    id: 17
    artist: "Beyonce"
    date: "Wed Oct 10 12:39:54 EDT 2012"
    price_usd: 200)";

Special Cases

Note that indentation rules, combined with the fact that raw string literals may contain newlines, leave you with an awkward choice on how to indent the first line of your raw string block. Because text protobufs ignore whitespace, this problem can be avoided by throwing in a leading newline (ignored by the parser) in that case, but not all uses of raw strings are so forgiving.

A non-empty tag is useful when the sequence )" happens to appear in your string and therefore can’t act as the closing delimiter:

std::string my_string = R"foo(This contains quoted parens "()")foo";

Conclusion

Raw string literals are certainly not an everyday tool for most of us, but there are times when good use of this new language feature will increase readability. So the next time you’re scratching your head trying to figure out if you need two \\s or four, try raw string literals instead. Your readers will thank you for it, even if regular expressions are still hard:

R"regexp((?:"(?:\\"|[^"])*"|'(?:\\'|[^'])*'))regexp";

CppCon 2017: The design of absl::StrCat

2017-10-23T00:00:00-04:00

Jorg Brown’s Lightning Talk

By Tom Manshreck, Abseil Tech Writer

Check out Jorg Brown’s Lightning Talk on the design of absl::StrCat() for background on what problem it was designed to solve and why it was designed the way it was.

Tip of the Week #77: Temporaries, Moves, and Copies

2017-10-20T00:00:00-04:00

Originally published as totw/77 on 2014-07-09

By Titus Winters (titus@google.com)

Updated 2017-10-20

Quicklink: abseil.io/tips/77

In the ongoing attempt to figure out how to explain to non-language-lawyers how C++11 changed things, we present yet another entry in the series “When are copies made?” This is part of a general attempt to simplify the subtle rules that have surrounded copies in C++ and replace it with a simpler set of rules.

Can You Count to 2?

You can? Awesome. Remember that the “name rule” means that each unique name you can assign to a certain resource affects how many copies of the object are in circulation. (See TotW 55 on Name Counting for a refresher.)

Name Counting, in Brief

If you’re worrying about a copy being created, presumably you’re worried about some line of code in particular. So, look at that point. How many names exist for the data you think is being copied? There are only 3 cases to consider:

Two Names: It’s a Copy

This one is easy: if you’re giving a second name to the same data, it’s a copy.

std::vector<int> foo;
FillAVectorOfIntsByOutputParameterSoNobodyThinksAboutCopies(&foo);
std::vector<int> bar = foo;     // Yep, this is a copy.

std::map<int, string> my_map;
string forty_two = "42";
my_map[5] = forty_two;          // Also a copy: my_map[5] counts as a name.

One Name: It’s a Move

This one is a little surprising: C++11 recognizes that if you can’t refer to a name anymore, you also don’t care about that data anymore. The language had to be careful not to break cases where you were relying on the destructor (say, absl::MutexLock), so return is the easy case to identify.

std::vector<int> GetSomeInts() {
  std::vector<int> ret = {1, 2, 3, 4};
  return ret;
}

// Just a move: either "ret" or "foo" has the data, but never both at once.
std::vector<int> foo = GetSomeInts();

The other way to tell the compiler that you’re done with a name (the “name eraser” from TotW 55) is calling std::move().

std::vector<int> foo = GetSomeInts();
// Not a copy, move allows the compiler to treat foo as a
// temporary, so this is invoking the move constructor for
// std::vector<int>.
// Note that it isn’t the call to std::move that does the moving,
// it’s the constructor. The call to std::move just allows foo to
// be treated as a temporary (rather than as an object with a name).
std::vector<int> bar = std::move(foo);

Zero Names: It’s a Temporary

Temporaries are also special: if you want to avoid copies, avoid providing names to variables.

void OperatesOnVector(const std::vector<int>& v);

// No copies: the values in the vector returned by GetSomeInts()
// will be moved (O(1)) into the temporary constructed between these
// calls and passed by reference into OperatesOnVector().
OperatesOnVector(GetSomeInts());

Beware: Zombies

The above (other than std::move() itself) is hopefully pretty intuitive, it’s just that we all built up weird notions of copies in the years pre-dating C++11. For a language without garbage collection, this type of accounting gives us an excellent mix of performance and clarity. However, it’s not without dangers, and the big one is this: what is left in a value after it has been moved from?

T bar = std::move(foo);
CHECK(foo.empty()); // Is this valid? Maybe, but don’t count on it.

This is one of the major difficulties: what can we say about these leftover values? For most standard library types, such a value is left in a “valid but unspecified state.” Non-standard types usually hold to the same rule. The safe approach is to stay away from these objects: you are allowed to re-assign to them, or let them go out of scope, but don’t make any other assumptions about their state.

Clang-tidy provides some some static-checking to catch use-after move with the bugprone-use-after-move check. However, static-analysis won’t ever be able to catch all of these - be on the lookout. Call these out in code review, and avoid them in your own code. Stay away from the zombies.

Wait, `std::move` Doesn’t Move?

Yeah, one other thing to watch for is that a call to std::move() isn’t actually a move itself, it’s just a cast to an rvalue-reference. It’s only the use of that reference by a move constructor or move assignment that does the work.

std::vector<int> foo = GetSomeInts();
std::move(foo); // Does nothing.
// Invokes std::vector<int>’s move-constructor.
std::vector<int> bar = std::move(foo);

This should almost never happen, and you probably shouldn’t waste a lot of mental storage on it. I really only mention it if the connection between std::move() and a move constructor was confusing you.

Aaaagh! It’s All Complicated! Why!?!

First: it’s really not so bad. Since we have move operations in the majority of our value types (including protobufs), we can do away with all of the discussions of “Is this a copy? Is this efficient?” and just rely on name counting: two names, a copy. Fewer than that: no copy.

Ignoring the issue of copies, value semantics are clearer and simpler to reason about. Consider these two operations:

void Foo(std::vector<string>* paths) {
  ExpandGlob(GenerateGlob(), paths);
}

std::vector<string> Bar() {
  std::vector<string> paths;
  ExpandGlob(GenerateGlob(), &paths);
  return paths;
}

Are these the same? What about if there is existing data in *paths? How can you tell? Value semantics are easier for a reader to reason about than input/output parameters, where you need to think about (and document) what happens to existing data, and potentially whether there is an pointer ownership transfer.

Because of the simpler guarantees about lifetime and usage when dealing with values (instead of pointers), it is easier for the compiler’s optimizers to operate on code in that style. Well-managed value semantics also minimizes hits on the allocator (which is cheap but not free). Once we understand how move semantics help rid us of copies, the compiler’s optimizers can better reason about object types, lifetimes, virtual dispatch, and a host of other issues that help generate more efficient machine code.

Since most utility code is now move-aware, we should stop worrying about copies and pointer semantics, and focus on writing simple easy-to-follow code. Please make sure you understand the new rules: not all legacy interfaces you encounter may be updated to return by value (instead of by output parameter), so there will always be a mix of styles. It’s important that you understand when one is more appropriate than the other.

Tip of the Week #55: Name Counting and unique_ptr

2017-10-20T00:00:00-04:00

Originally published as totw/55 on 2013-09-12

by Titus Winters (titus@google.com)

Updated 2017-10-20

Quicklink: abseil.io/tips/55

“Though we may know Him by a thousand names, He is one and the same to us all.” - Mahatma Gandhi

Colloquially, a “name” for a value is any value-typed variable (not a pointer, nor a reference), in any scope, that holds a particular data value. (For the spec-lawyers, if we say “name” we’re essentially talking about lvalues.) Because of std::unique_ptr’s specific behavioral requirements, we need to make sure that any value held in a std::unique_ptr only has one name.

It’s important to note that the C++ language committee picked a very apt name for std::unique_ptr. Any non-null pointer value stored in a std::unique_ptr must occur in only one std::unique_ptr at any time; the standard library is designed to enforce this. Many common problems compiling code that uses std::unique_ptr can be resolved by learning to recognize how to count the names for a std::unique_ptr: one is OK, but multiple names for the same pointer value are not.

Let’s count some names. At each line number, count the number of names alive at that point (whether in scope or not) that refer to a std::unique_ptr containing the same pointer. If you find any line with more than one name for the same pointer value, that’s an error!

std::unique_ptr<Foo> NewFoo() {
  return std::unique_ptr<Foo>(new Foo(1));
}

void AcceptFoo(std::unique_ptr<Foo> f) { f->PrintDebugString(); }

void Simple() {
  AcceptFoo(NewFoo());
}

void DoesNotBuild() {
  std::unique_ptr<Foo> g = NewFoo();
  AcceptFoo(g); // DOES NOT COMPILE!
}

void SmarterThanTheCompilerButNot() {
  Foo* j = new Foo(2);
  // Compiles, BUT VIOLATES THE RULE and will double-delete at runtime.
  std::unique_ptr<Foo> k(j);
  std::unique_ptr<Foo> l(j);
}

In Simple(), the unique pointer allocated with NewFoo() only ever has one name by which you could refer it: the name “f” inside AcceptFoo().

Contrast this with DoesNotBuild(): the unique pointer allocated with NewFoo() has two names which refer to it: DoesNotBuild()’s “g” and AcceptFoo()’s “f”.

This is the classic uniqueness violation: at any given point in the execution, any value held by a std::unique_ptr (or more generally, any move-only type) can only be referred to by a single distinct name. Anything that looks like a copy introducing an additional name is forbidden and won’t compile:

scratch.cc: error: call to deleted constructor of std::unique_ptr<Foo>'
  AcceptFoo(g);

Even if the compiler doesn’t catch you, the runtime behavior of std::unique_ptr will. Any time where you “outsmart” the compiler (see SmarterThanTheCompilerButNot()) and introduce multiple std::unique_ptr names, it may compile (for now) but you’ll get a run-time memory problem.

Now the question becomes: how do we remove a name? C++11 provides a solution for that as well, in the form of std::move().

 void EraseTheName() {
   std::unique_ptr<Foo> h = NewFoo();
   AcceptFoo(std::move(h)); // Fixes DoesNotBuild with std::move
}

The call to std::move() is effectively a name-eraser: conceptually you can stop counting “h” as a name for the pointer value. This now passes the distinct-names rule: on the unique pointer allocated with NewFoo() has a single name (“h”), and within the call to AcceptFoo() there is again only a single name (“f”). By using std::move() we promise that we will not read from “h” again until we assign a new value to it.

Name counting is a handy trick in modern C++ for those that aren’t expert in the subtleties of lvalues, rvalues, etc: it can help you recognize the possibility of unnecessary copies, and it will help you use std::unique_ptr properly. After counting, if you discover a point where there are too many names, use std::move to erase the no-longer-necessary name.

Tip of the Week #122: Test Fixtures, Clarity, and Dataflow

2017-10-20T00:00:00-04:00

Originally published as totw/122 on 2016-08-30

By Titus Winters (titus@google.com)

Updated 2017-10-20

Quicklink: abseil.io/tips/122

Be obscure clearly. — E.B. White

How does test code differ from production code? For one thing, tests are untested: when you write messy spaghetti code that is spread over several files and has hundreds of lines of SetUp how can anyone be sure that the test is really testing what it needs to? Too often your code reviewers will have to assume that the setup makes sense and are at best spot-checking the logic for each individual test case. In those cases, your test is likely to fail if something changes, but it’s rarely clear whether that something is the right something.

On the other hand, if you keep each test simple and as straightforward as possible, it’s easier to see that it is correct by inspection, understand the logic, and review for higher quality test logic. Let’s look at a few simple ways to achieve that.

Dataflow in Fixtures

Consider the following example:

class FrobberTest : public ::testing::Test {
 protected:
  void ConfigureExampleA() {
    example_ = "Example A";
    frobber_.Init(example_);
    expected_ = "Result A";
  }

  void ConfigureExampleB() {
    example_ = "Example B";
    frobber_.Init(example_);
    expected_ = "Result B";
  }

  Frobber frobber_;
  string example_;
  string expected_;
};

TEST_F(FrobberTest, CalculatesA) {
  ConfigureExampleA();
  string result = frobber_.Calculate();
  EXPECT_EQ(result, expected_);
}

TEST_F(FrobberTest, CalculatesB) {
  ConfigureExampleB();
  string result = frobber_.Calculate();
  EXPECT_EQ(result, expected_);
}

In this fairly simple example, our tests span 30 lines of code. It’s very easy to imagine less simple examples that are 10x that: certainly more than will fit on any single screen. A reader or code reviewer that wants to validate that the code is correct has to scan around as follows:

“OK, this is a FrobberTest, where’s that defined … oh, this file. Great.”
“ConfigureExampleA … that’s a FrobberTest method. It’s operating on some member variables. What types are those? How are they initialized? OK, Frobber and two strings. Is there a SetUp? OK, default constructed.”
“Back to the test: OK, we calculate a result and compare it against expected_ … what did we store there again?”

Compare to the equivalent code written in a simpler style:

TEST(FrobberTest, CalculatesA) {
  Frobber frobber;
  frobber.Init("Example A");
  EXPECT_EQ(frobber.Calculate(), "Result A");
}

TEST(FrobberTest, CalculatesB) {
  Frobber frobber;
  frobber.Init("Example B");
  EXPECT_EQ(frobber.Calculate(), "Result B");
}

With this style, even in a world where we have hundreds of tests, we can tell with only local information exactly what is going on.

Prefer Free Functions

In the previous example, all of the variable initialization was nice and terse. In real tests, that isn’t always true. However, the same ideas about dataflow and avoiding fixtures may apply. Consider this protobuf example:

class BobberTest : public ::testing::Test {
 protected:
  void SetUp() override {
    bobber1_ = PARSE_TEXT_PROTO(R"(
        id: 17
        artist: "Beyonce"
        when: "2012-10-10 12:39:54 -04:00"
        price_usd: 200)");
    bobber2_ = PARSE_TEXT_PROTO(R"(
        id: 21
        artist: "The Shouting Matches"
        when: "2016-08-24 20:30:21 -04:00"
        price_usd: 60)");
  }

  BobberProto bobber1_;
  BobberProto bobber2_;
};

TEST_F(BobberTest, UsesProtos) {
  Bobber bobber({bobber1_, bobber2_});
  SomeCall();
  EXPECT_THAT(bobber.MostRecent(), EqualsProto(bobber2_));
}

Again, the centralized refactoring leads to a lot of indirection: declarations and initialization are separate, and potentially far away from actual use. Further, because of SomeCall() in the middle, and the fact we’re using a fixture and fixture member variables, there’s no way to be sure that bobber1_ and bobber2_ weren’t modified between initialization and the EXPECT_THAT validation, without checking the details of SomeCall(). More scrolling around is likely necessary.

Consider instead:

BobberProto RecentCheapConcert() {
  return PARSE_TEXT_PROTO(R"(
      id: 21
      artist: "The Shouting Matches"
      when: "2016-08-24 20:30:21 -04:00"
      price_usd: 60)");
}
BobberProto PastExpensiveConcert() {
  return PARSE_TEXT_PROTO(R"(
      id: 17
      artist: "Beyonce"
      when: "2012-10-10 12:39:54 -04:00"
      price_usd: 200)");
}

TEST(BobberTest, UsesProtos) {
  Bobber bobber({PastExpensiveConcert(), RecentCheapConcert()});
  SomeCall();
  EXPECT_THAT(bobber.MostRecent(), EqualsProto(RecentCheapConcert()));
}

Moving the initialization into free functions makes it clear that there is no hidden dataflow. Well chosen names for the helpers mean that you can likely review the test for correctness without even scrolling up to see the details of the helper.

Five Easy Steps

You can generally improve test clarity by following these steps:

Avoid fixtures where reasonable. Sometimes it’s not.
If you are using fixtures, try to avoid fixture member variables. It is far too easy to start operating on those in ways akin to globals: data flow is hard to track since absolutely any code path in the fixture may modify the member.
If you’ve got variables that need complex initialization that would make each test hard to read, consider a helper function (not part of the fixture) that documents that initialization and returns the object directly.
If you must have fixtures that contain member variables, try to avoid methods that operate on those members directly: pass them in as parameters whenever possible to make the dataflow clear.
Try to write tests before writing headers: if you start with a usage that is pleasant to test, your API is usually better, and your tests are almost always clearer.

CppCon 2017: C++ as a 'Live at Head' Language

2017-10-04T00:00:00-04:00

Titus Winters’ Plenary Keynote

By Tom Manshreck, Abseil Tech Writer

If you didn’t get a chance to check out Titus Winters’ plenary keynote in person, check it out below.

Find out why “Living at Head” is a good thing!

Welcome to Abseil!

2017-09-26T00:00:00-04:00

A New Common Libraries Project

By Titus Winters, Abseil Lead

Today we are open sourcing Abseil, a collection of libraries drawn from the most fundamental pieces of Google’s internal codebase. These libraries are the nuts-and-bolts that underpin almost everything that Google runs. Bits and pieces of these APIs are embedded in most of our open source projects, and now we have brought them together into one comprehensive project. Abseil encompasses the most basic building blocks of Google’s codebase: code that is production tested and will be fully maintained for years to come.

Our C++ code repository is available at https://github.com/abseil/abseil-cpp.

By adopting these new Apache-licensed libraries, you can reap the benefit of years (over a decade in many cases) of our design and optimization work in this space. Our past experience is baked in.

Just as interesting, we’ve also prepared for the future: several types in Abseil’s C++ libraries are “pre-adoption” versions of C++17 types like string_view and optional - implemented in C++11 to the greatest extent possible. We look forward to moving more and more of our code to match the current standard, and using these new vocabulary types helps us make that transition. Importantly, in C++17 mode these types are merely aliases to the standard, ensuring that you only ever have one type for optional or string_view in a project at a time. Put another way: Abseil is focused on the engineering task of providing APIs that remain stable over time.

Consisting of the foundational C++ and Python code at Google, Abseil includes libraries that will grow to underpin other Google-backed open source projects like gRPC, Protobuf and TensorFlow. We love those projects, and we love the users of those projects - we want to ensure smooth usage for these things over time. In the next few months we’ll introduce new distribution methods to incorporate these projects as a collection into your project.

Continuing with the “over time” theme, Abseil aims for compatibility with major compilers, platforms and standard libraries for approximately 5 years. Our 5-year target also applies to language version: we assume everyone builds with C++11 at this point. (In 2019 we’ll start talking about requiring C++14 as our base language version.) This 5-year horizon is part of our balance between “support everything” and “provide modern implementations and APIs.”

Highlights of the initial release include:

Zero configuration: most platforms (OS, compiler, architecture) should just work.
Pre-adoption for C++17 types: string_view, optional, any. We’ll follow up with variant soon.
Our primary synchronization type, absl::Mutex, has an elegant interface and has been extensively optimized.
Efficient support for handling time: absl::Time and absl::Duration are conceptually similar to std::chrono types, but are concrete (not class templates) and have defined behavior in all cases. Additionally, our clock-sampling API absl::Now() is more heavily optimized than most standard library calls for std::chrono::system_clock::now().
String handling routines: among internal users, we’ve been told that releasing absl::StrCat(), absl::StrJoin(), and absl::StrSplit() would itself be a big improvement for the open source C++ world.

The project has support for C++ and some Python. Over time we’ll tie those two projects together more closely with shared logging and command-line flag infrastructure. To start contributing, please see our contribution guidelines and fork us on GitHub. Check out our documentation and community page for information on how to contact us, ask questions or contribute to Abseil.

Tip of the Week #1: `string_view`

2017-09-26T00:00:00-04:00

Originally posted as TotW #1 on April 20, 2012

By Michael Chastain

Updated 2020-08-18

Quicklink: abseil.io/tips/1

What’s a `string_view`, and Why Should You Care?

When creating a function to take a (constant) string as an argument, you have three common alternatives: two that you already know, and one of which you might not be aware:

// C Convention
void TakesCharStar(const char* s);

// Old Standard C++ convention
void TakesString(const std::string& s);

// string_view C++ conventions
void TakesStringView(absl::string_view s);    // Abseil
void TakesStringView(std::string_view s);     // C++17

The first two cases work best when a caller has the string in the format already provided, but what happens when a conversion is needed (either from const char* to std::string or std::string to const char*)?

Callers needing to convert a std::string to a const char* need to use the (efficient but inconvenient) c_str() function:

void AlreadyHasString(const std::string& s) {
  TakesCharStar(s.c_str());               // explicit conversion
}

Callers needing to convert a const char* to a std::string don’t need to do anything additional (the good news) but will invoke the creation of a (convenient but inefficient) temporary string, copying the contents of that string (the bad news):

void AlreadyHasCharStar(const char* s) {
  TakesString(s); // compiler will make a copy
}

What to Do?

Google’s preferred option for accepting such string parameters is through a string_view. This is a “pre-adopted” type from C++17 - for now, use absl::string_view even if std::string_view is available.

An instance of the string_view class can be thought of as a “view” into an existing character buffer. Specifically, a string_view consists of only a pointer and a length, identifying a section of character data that is not owned by the string_view and cannot be modified by the view. Consequently, making a copy of a string_view is a shallow operation: no string data is copied.

string_view has implicit conversion constructors from both const char* and const std::string&, and since string_view doesn’t copy, there is no O(n) memory penalty for making a hidden copy. In the case where a const std::string& is passed, the constructor runs in O(1) time. In the case where a const char* is passed, the constructor invokes a strlen() automatically (or you can use the two-parameter string_view constructor).

void AlreadyHasString(const std::string& s) {
  TakesStringView(s); // no explicit conversion; convenient!
}

void AlreadyHasCharStar(const char* s) {
  TakesStringView(s); // no copy; efficient!
}

Because the string_view does not own its data, any strings pointed to by the string_view (just like a const char*) must have a known lifespan, and must outlast the string_view itself.

This means that using string_view for storage is often questionable: you need some proof that the underlying data will outlive the string_view. For example, the following struct probably doesn’t make sense if it might be stored beyond the result of a function call that accepts it as a parameter:

struct TestScore {
  absl::string_view username;  // Probably should be a `std::string`
  double score;
};

If your API only needs to reference the string data during a single call, and doesn’t need to modify the data, accepting a string_view is sufficient.

If you need to use the data later or modify the data, you can explicitly convert to a C++ string object using std::string(my_string_view). Another option, discussed in Tip #117, is to pass a std::string by value and use std::move in callers when applicable.

Adding string_view into an existing codebase is not always the right answer: changing parameters to pass by string_view can be inefficient if those are then passed to a function requiring a std::string or a NUL-terminated const char*. It is best to adopt string_view starting at the utility code and working upward, or with complete consistency when starting a new project.

A Few Additional Notes

Unlike other string types, you should pass string_view by value just like you would pass an int or a double because string_view is a small value.
Marking a string_view as const only affects whether the string_view object itself can be modified, and not whether it can be used to modify the underlying chars – it never can. This is exactly analogous to how a const char* can never be used to modify the chars, even if the pointer itself can be modified.
For function parameters, don’t qualify string_view with const in function declarations (see Tip #109). You may use const to qualify string_view in function definitions at your (or your team’s) discretion, e.g., to be consistent with the surrounding code (Tip #109). For other local variables, using const is neither encouraged nor discouraged (https://google.github.io/styleguide/cppguide.html#Use_of_const).
string_view is not necessarily NUL-terminated. Thus, it’s not safe to write:
```
    printf("%s\n", sv.data()); // DON’T DO THIS
```
Prefer this instead (see Tip #124):
```
    absl::PrintF("%s\n", sv);
```
You can log a string_view just like you would a string or a const char* :
```
 LOG(INFO) << "Took '" << sv << "'";
```
You can convert an existing routine that accepts const std::string& or NUL-terminated const char* to string_view safely in most cases. The only danger we have encountered in performing this operation is if the address of the function has been taken, this will result in a build break as the resulting function-pointer type will be different.
string_view has a constexpr constructor and a trivial destructor; keep this in mind when using in static and global variables (see the style guide) or constants (see Tip #140).
A string_view is an effective reference and may not be the best choice for member variables (also see Tip #180).

Abseil Blog & Tips

Performance Tip of the Week #97: Virtuous ecosystem cycles

Case studies

SwissMap

Size class-driven allocation

Sized deallocation

Size class improvements

Turbo troubles

Lessons

Don’t assume the status quo is fixed

Focus on problems and outcomes, not exact solutions

Create and use leverage

Understand what is seen and unseen

Closing words

Performance Tip of the Week #99: Illuminating the processor core with llvm-mca

Preliminaries: Varint optimization

Analyzing the code

Throughput versus latency

Understanding dependency chains

Optimizing CRC32C

Limitations

Closing words

Further reading

Performance Tip of the Week #98: Measurement has an ROI

Getting the big picture right

Significant figures

Precision and accuracy

Increasing statistical power through aggregation

Balancing complexity and simplicity

Conclusion

Performance Tip of the Week #26: Fixing things with hashtable profiling

Starting the analysis

Finding stuck bits

Case study: InternalSubscriber::MessageReceived

Case study: ProcessWatchDone

Finding tables with many collisions

Case study: PortMapperImpl::UpdateActiveMap

Hashtable statistics

Detecting bad hashtables in unit tests

Closing words

Performance Tip of the Week #95: Spooky action at a distance

Normalization techniques

Choosing the right partitioning scheme

Cache locality and memory bandwidth

Better hugepage coverage lifted all boats

Distributed systems

Closing words

Performance Tip of the Week #94: Decision making in a data-imperfect world

Focusing on outcomes

Changing decisions

Failing fast

Updating our priors

Data censorship

Profiler limitations

Partial populations

Closing words

Performance Tip of the Week #93: Robots never sleep

Prevent problems via technical means

Strategies for taming subtlety

Compile-time hardening

Runtime hardening

Static analysis

Ratchet-and-pawl

Costs of avoided guarantees

Validating policy choices

Closing words

Performance Tip of the Week #90: How to estimate

Why estimate?

How big is it?

Exploring profiles

Leveraging other data sources

How much can we change?

Past performance

Speed of light

Analytical methods

Refactoring

Partial prototyping

Partial deployment and counterfactuals

Things to look out for

Calibrate

Case study: `InternalSubscriber::MessageReceived`

Case study: `ProcessWatchDone`

Case study: `PortMapperImpl::UpdateActiveMap`

Tip of the Week #232: When to Use `auto` for Variable Declarations

`std::make_unique` and Other Google-wide Factory Functions

Otherwise: Avoid Using `auto`

`std::clamp`

`std::midpoint`

`std::lerp`