Memory customization is not always a top priority when a design team plans a new system-on-chip (SoC) project. But often it should be.

This may not be an obvious statement. Granted, SRAM claims a lot of area on most SoCs. The speed and power consumption of SRAM arrays can affect the overall chip performance and energy efficiency.

But today’s memory compilers are flexible tools that support a variety of cell designs. At Faraday, for example, the 14FFC compiler offers eight variants, tuned to diverse needs, ranging from high-density to high-performance to ultra-low-power. So why do you consider custom memory?

One answer to the above question is the need for an unusual word or bit length. Relatively simple customization can produce the exact SRAM configuration required for a specific instance, not just the compiler’s closest approximation.

Similarly, there are times during floorplanning—or, more concerningly, during timing closure—when giving an SRAM instance an unusual aspect ratio can ease a difficult situation. This may be a more complex customization, requiring changes to array layout and routing, multiplexers, drivers, and cell designs.

Recently, we designed a multi-Mbit SRAM array with an aspect ratio of nearly 1:19. This memory architecture is ideally suited for seamless integration into frame-buffer applications specifically designed for display processing. The memory configuration, characterized by its unique aspect ratio, is carefully engineered to accommodate wide I/O widths and specialized non-2ⁿ-column multiplexing requirements.

Figure 1 Special aspect ratio memory in this case is x = 1775 um, y = 95 um; giving an SRAM instance an unusual aspect ratio can ease a difficult situation. Source: Faraday Technology

Another situation involves yield and reliability. Compilers typically only generate a specific number of redundant columns of bit cells. In the event of a bit failure, the array can disconnect the offending cell’s column and replace it with a redundant column if one is available. This technique is effective if failures only occur in one or a few columns.

But for various reasons, some designs require more protection: redundant columns and redundant rows. The additional cells, routing, and logic to implement this expanded redundancy can be achieved by customizing the array.

Figure 2 This memory offers redundant rows and columns for additional rows and columns. Source: Faraday Technology

An automotive case study

Another example of memory customization comes from a recent SoC design we participated in. The project was for a mission-critical automotive SoC. Our customer specified an Automotive Grade 1 (AG1) operating ambient temperature range of -40 to +125 °C.

Within that range, the customer required an extended operating life, as is customary for automotive electronics. And the chip would require ISO 26262 functional safety certification, which would require enhanced failure analysis and documentation during design.

This project illustrates the level of detail sometimes needed in memory customization. But it also shows the extent of additional support—analysis, documentation, design assistance, and test services—that a custom memory design can entail.

We determined that existing tools could produce an array that would operate reliably over the AG1 temperature range in the short term. But to achieve the required operating life, we had to address aging issues in the circuitry.

First, there was the issue of high-current signals on the array’s word lines and bit lines. The customer was rightly concerned that, over the operating life and at elevated temperatures, the high currents could cause sufficient electromigration to trigger chip failure. So, we redesigned the line drivers and the array, preserving array performance, signal integrity, and line-direction management while reducing the risk of electromigration.

Bias temperature instability (BTI) was another threat to chip life: time and elevated temperature cause a gradual but significant drift in MOSFET threshold voltages. Unfortunately, NMOS and PMOS devices age differently under BTI. So very gradually, the timing of rising and falling signal edges can diverge. Eventually, this can lead to circuit failure at points where the relative arrival times of two signals, one positive-going and one negative-going, are critical. Accordingly, we altered the memory design.

We further inspected the remaining control logic for the risk of developing race conditions over time and adjusted timing margins to account for eventual threshold-voltage drift. The result was a significant improvement in SRAM’s expected operating life.

Functional safety

Certification under ISO 26262 was another requirement. This comprehensive standard delves deep into the design process to ensure that chip failure modes are identified, traced to their root causes, and addressed. This process extends to IP used in the design and to the original circuitry. So, the documentation required for ISO 26262 certification was deliverable for the custom memory team.

Two primary documents are required: a Design Failure Mode and Effects Analysis (DFMEA) and a safety manual. The former, as its name suggests, is an exhaustive list of the ways the IP could cause an error, the possible causes of those failure modes, and the remedial actions taken. The safety manual, in contrast, is an instruction manual for the chip and system designers who will integrate the IP into the overall design.

One entry in the DFMEA might include a failure in which a bit cell flips, corrupting data in the SRAM. Under this heading, list potential causes of a flipped bit, including design-rule violations in the cell array, radiation upset, and aging. For each reason, there would be a list of controls to prevent it or detect it, an assessment of the remaining failure risk, and recommendations for further action.

The safety manual tells IP integrators and system developers how to use the IP without violating the conditions for which it was designed. Directions might include, for instance, input signal and supply voltage ranges, noise limits, substrate noise and temperature limits, output loading specifications, and maximum duty cycle limits.

Why custom memory design?

As these examples illustrate, custom memory design can adapt an array exactly to functional, timing, or layout requirements of a particular SoC. It can also produce arrays for demanding performance, environmental, or reliability requirements.

But seeing a customer through to a finished SoC requires far more than just providing the design files for a custom SRAM array. The design partner should be ready to assist the SoC design team with integration, provide verification and test support, and thoroughly document the characteristics and requirements of the new SRAM design.

In addition, the partner should be able to work intimately with the SoC foundry to ensure yield, and with the test vendor to ensure adequate test coverage for the new array. In many cases, it’s an advantage for the partner to have in-house testing capability.

Roger Chen is deputy division manager for memory IP development at Faraday Technology.

Related Content

• Accurate memory models for all
• Geopolitics Is Rewriting Memory Sourcing
• The Memory Wall Is Real, Here Is the Door
• Developing a Design Methodology for Embedded Memories
• Beyond Bandwidth: The Industry is Striving for Custom Memory

The post Would custom memory ease your SoC design? appeared first on EDN.

Connectivity is all well and good…well, sort of, as it invariably comes with a price, literally and/or figuratively. Simple’s sometimes best, all things considered, and ambient-light power’s also nice.

When you want to monitor and adjust the internal humidity (and temperature, while you’re at it) of your residence or other facility, a “smart” connected hygrometer such as the one I tore down last month is convenient, since you can check both the measurements-of-the-moment and longer-term legacy trends from anywhere (even when you’re away) using your mobile device. A “smart” hygrometer can even alert you when those measurements stray beyond predefined boundary conditions. And if it includes a built-in display, you can keep your smartphone stowed away and still see the data.

All that connectivity and integrated intelligence comes with a bill-of-materials cost adder, however. And there’s always also the latent (or not) potential for hackers to gain access to that same data stream. While you might not care if someone halfway around the world (or down the street, for that matter) knows your home’s humidity and temperature, you’ll undoubtedly care a lot more if that same “smart” hygrometer ends up being a penetration “vector” for a broader attack, revealing your location and Wi-Fi network login details, for example, along with providing strangers with access to more privacy-violating LAN devices such as security cameras.

Acceptable = respectable

As such, a non-connected sensor is a credible (and sometimes the preferable) alternative. At the beginning of April, I saw a two-pack of BaldrTherm 2.2” solar-powered digital thermometer and hygrometers marked down to $9.99 at Amazon and, curious to try out (and tear down) such a device myself, pressed “purchase”.

I’ve subsequently seen the same two-pack listed there for as low as $8.99, exemplifying a broader BaldrTherm promotion that I’m guessing is motivated by a product line transition combo of redesign and migration to larger, more visible data-rich, 3.2” display devices:

with in-progress awkward consequences:

And to be clear, the company offers plenty of “connected” product variants, too. But today we’ll dive inside a fully standalone-operation offering, complete with a solar cell power option that’s more broadly photon-source agnostic (albeit presumably still visible light spectrum-centric).

Since I know how much you all love conceptual teardown “stock” images, I’ll start with one of ‘em:

And now for our actual patient, as usual beginning with some outer box shots, also as-usual accompanied by a 0.75″ (19.1 mm) diameter U.S. penny for size comparison purposes:

Flip open either of the latter two flaps:

and inside you’ll find two slips o’literature (the “user manual”, such as it scantly is, can be accessed in PDF form here):

and two sleeve-swathed examples of today’s teardown victim:

Diminutive in size and price

Here’s the now-“naked” device from various perspectives. Note the transparent piece of plastic (which BaldrTherm refers to as an “insulation sheet”) sticking out one side, which keeps the battery inside from prematurely draining while sitting on store shelves pre-purchase, until removed by the buyer-now-owner (and whose very presence was initially confusing to me, as I’d assumed the energy storage cell in the interior was solar-rechargeable; keep reading).

In spite of the battery still being disconnected, and after a brief delay after initial exposure to my home office’s overhead lighting:

the display came on and the device started working:

I was initially surprised by this unexpected functional transition, until I pondered and realized the underlying reason why, which the user manual also spells out:

Time to get inside. You may have already noticed in one of the earlier overview shots the two coin edge-inviting slots (one of them doing double-duty for the “insulating sheet”) on one side.

Had I thought to grab the penny I had handy, they might have sufficed. As it was, the flexible tip of the “spunger” I was trying to use made it ineffective, so much so that I peeled off the backside sticker to see if I could find any screw heads underneath it. Nope:

Switching to a flat-head screwdriver eventually accomplished my objective, however:

Hot and (not) heavy

Here’s where things started getting interesting and, in retrospect, amusing. I happened to notice that, presumably during the initial disassembly process, the spring terminal at the anode (“negative”) end of the AAA battery inside had become dislodged.

Normally, such batteries’ cases have a thin plastic outer insulating layer that prevents short-circuits with the cathode directly below it:

Not in this case (bad pun intended), however, or maybe it got scratched during disassembly, too. Because when I grabbed the sides of the battery to remove it, my fingertips got scorched. I quickly grabbed the aforementioned flat-head screwdriver and flipped the battery out of the chassis that way instead.

While I waited for it to cool, I carefully rolled it around and learned that it was a non-rechargeable conventional alkaline cell, instead.

In retrospect, including not only a rechargeable battery but also the necessary recharging circuitry in the design would have ballooned the bill-of-materials cost, and I later noticed that the documentation made it clear that the battery was not to be replaced, apparently if for no other reason than to preclude owner burns and other potential mishaps.

If so, though, then why the tempting coin-shaped slots on one side? Inquiring minds want to know. Surprisingly, the cell still held a meaningful modicum of charge; I’d apparently been sufficiently speedy in noticing and rectifying the short-circuit circumstances:

And the device still worked, both with the battery removed:

and with it temporarily reinstalled once safe to touch again.

Internal details

Onward. The solar cell is tenuously held in place with a single piece of tape on one side and the case sides on the other.

The PCB to which it’s attached is conversely more firmly ensconced by two screws.

You know what comes next:

We have a liftoff:

Now for the other, more circuitry-meaningful front side:

Flipping the LCD over reveals its elastomeric connector on one end, which normally presses up against electrical contacts on the PCB itself:

This is one rugged little device; pressing the two halves back together with my fingers and exposing the solar cell to light reignites the display and broader sensing-and-reporting capabilities (albeit with the measured temperature presumably inflated by my body proximity).

Here’s a closeup of the PCB frontside:

showing the elastomer-mating contacts at bottom, a piece of insulating tape at upper left and normally between the LCD backside and a 220-µF capacitor first glimpsed in the assembly rear-view images I shared earlier:

and at upper right, and left-to-right, the humidity and temperature sensors. Underneath the identification-blocking black epoxy blob in the center is presumably the SoC.

Capacitor and missing-battery buffers

In closing, after putting everything back together, the device still worked, after a brief wakeup delay and initially for only a short and cyclical timeframe.

After which, functionality eventually stabilized as long as sufficient light remained available.

Specifically, I’m guessing, commensurate with the fact that there’s still no battery (re)installed. What’s the relationship here? It has to do, I think, with the core purpose of that previously noted capacitor. Remember my “backup batteries and supercaps” piece from last month? This is effectively the supercapacitor, intended to smooth out transient ambient illumination variability-induced impermanence in the solar cell’s output.

I’m guessing that the capacitor is taking a few system-reboot cycles to get to full stored charge capacity, particularly given that there’s (abnormally, versus the normal configuration) no battery installed to alternatively supply the system with the necessary electrons. Agree or disagree, readers? As always, please let me know your thoughts on this and/or anything else that caught your fancy in the comments!

—Brian Dipert is the associate editor, as well as a contributing editor, at EDN.

Related Content

• Humidifiers and such: How much “smart” is too much?
• Smart hygrometers: Still largely useful even without integrated visual monitors
• The Tapo Hub: TP-Link joins the low-bandwidth, long-range RF club
• TP-Link’s Tapo H100: Smart sensing unencumbered
• IoT device vulnerabilities are on the rise
• Backup batteries and supercaps: Geriatric hardware traps

The post Not smart, but solar: Analyzing another thermo-plus-hygrometer appeared first on EDN.

Fifteen miles above you, a small styrofoam box is shrieking into the void. Its voice is binary—relentlessly transmitting temperature, pressure, and wind speed from the freezing stratosphere. In two hours, it will be gone, torn apart by the very atmosphere it was sent to measure.

This is the radiosonde’s hidden existence: the most successful yet expendable Internet of Things (IoT) device ever launched.

From balloons to big data

Radiosondes are the unsung workhorses of atmospheric science. First launched in the 1930s, these lightweight sensor packages ride weather balloons into the upper atmosphere, relaying streams of temperature, pressure, and humidity data that form the backbone of modern weather forecasting.

Every day, hundreds are released worldwide, their short lives fueling the long-range models that guide aviation, agriculture, and disaster preparedness. Though each unit is designed to perish after a single flight, the collective impact of radiosondes is enduring—an invisible infrastructure that keeps our understanding of the sky precise and predictive.

Vehicle vs. instrument: Understanding the weather balloon system

While people often use the terms interchangeably, a weather balloon and a radiosonde are distinct components of a single flight system. The weather balloon is an expendable transport vehicle; a large latex sphere filled with hydrogen or helium designed to provide the lift necessary to reach the stratosphere.

In contrast, the radiosonde is the scientific payload; a small, battery-operated instrument package tethered below the balloon. While the balloon’s only job is to climb until it bursts, the radiosonde performs the actual work of measuring temperature, humidity, and pressure and then transmitting that data via radio waves to meteorologists on the ground in real-time.

Figure 1 A sonde balloon and a radiosonde facilitate upper-air observations for numerical weather prediction models. Source: Azista Aerospace

The science of atmospheric sounding: How radiosondes work

A radiosonde primarily tracks pressure, temperature, and humidity using sensitive electronic sensors. While these provide the “ingredients” of the air, the device also tracks wind speed and direction by monitoring its own movement via GPS; as the balloon drifts, its change in position reveals exactly how the wind is blowing at different altitudes.

Together, these measurements allow meteorologists to build a complete vertical profile of the atmosphere—from the ground all the way up to the stratosphere. Furthermore, these variables are used to calculate geopotential height, which determines the precise altitude of pressure levels used to map global weather patterns.

Figure 2 The balloon-borne DFM-17 radiosonde provides atmospheric data for meteorological sounding. Source: graw

In essence, a radiosonde is a portable weather station integrated with a radio transmitter. Suspended from a rubber or latex balloon, the device ascends deep into the stratosphere to capture high-altitude data, transmitting real-time measurements of temperature, pressure, and humidity to a receiving station. The maximum altitude is determined by the diameter and thickness of the balloon.

By tracking the unit’s trajectory via GPS, meteorologists also map the strength and direction of winds aloft, creating a comprehensive vertical profile of the atmosphere. The flight concludes when the balloon reaches its expansion limit and bursts, triggering a small parachute to slow the radiosonde’s descent. While many units land in inaccessible areas, others are recovered by the public and returned for refurbishment, closing the loop on a single atmospheric mission.

Radiosonde system: Vertical layers from balloon to ground station

A radiosonde system is organized in vertical layers, beginning with the sounding balloon, also known as the sonde balloon, which ascends into the upper atmosphere carrying the payload. Suspended beneath is the radiosonde unit, integrating a glass bead thermistor for precise temperature measurement, a capacitive humidity sensor to monitor moisture levels, and a GPS receiver to provide accurate position, altitude, and wind data.

These measurements are transmitted through the radiosonde transmitter to a ground-based receiver and processing system, where the data is decoded and analyzed. This layered architecture—from balloon to ground station—creates a continuous vertical profile of atmospheric conditions, enabling reliable weather forecasting, climate monitoring, and deeper research into atmospheric dynamics.

Beyond the core radiosonde unit, several design enhancements improve measurement accuracy and reliability. The capacitive humidity sensor is equipped with a miniature heater element to prevent condensation and ensure stable reading in saturated conditions. The glass bead thermistor used for air temperature measurement is often treated with hydrophobic coating, reducing the impact of water droplets and improving response time in cloud environments.

Many radiosondes also include an optional barometric pressure sensor, adding direct pressure measurements to complement GPS-derived altitude data. These refinements—heater stabilization, protective coatings, and auxiliary pressure sensing—extend the robustness of the radiosonde system, ensuring dependable atmospheric profiles even in challenging weather regimes.

Figure 3 The Vaisala Radiosonde RS41-SGP features a specialized chassis that integrates a high-precision pressure sensor into its compact design, ensuring robust and accurate atmospheric profiling even in GNSS-challenged environments. Source: Vaisala

Radio subsystem: Transmission and data handling

The radio subsystem of a radiosonde is engineered for efficient, narrow-band communication between the airborne unit and the ground station.

Modern designs support programmable frequencies and channel selection, allowing flexible operation across different meteorological networks. Transmission parameters include controlled bandwidth allocation, adjustable transmitter power, and defined coverage ranges to ensure reliable signal reception over long ascents. Data is typically modulated using Gaussian frequency-shift keying (GFSK), balancing spectral efficiency with robustness against noise.

The downlink stream carries structured data bits at a specified sampling rate, enabling continuous atmospheric profiling. For pre-launch verification, many systems integrate near field communication (NFC) capability, allowing quick ground checks of sensor calibration and transmitter health. Together, these radio features—programmable channels, efficient modulation, and diagnostic NFC—form the backbone of dependable data delivery from balloon to ground station.

Here is a side note regarding AFSK vs. GFSK. Earlier radiosonde systems often relied on audio frequency-shift keying (AFSK), a simple scheme that encodes data by alternating between two audio tones. While easy to implement, AFSK suffers from poor spectral efficiency and limited robustness in noisy RF environments.

So, modern designs have largely transitioned to GFSK, which applies Gaussian filtering to smooth frequency shifts. This reduces bandwidth usage, minimizes adjacent-channel interference, and improves reliability when multiple sondes are launched simultaneously. In practice, GFSK delivers cleaner signals and higher data integrity, making it the preferred modulation method for today’s radiosonde telemetry.

Figure 4 Modern pocket-sized radiosondes, such as the Windsond S2, capture real-time weather profiles for immediate analysis. Source: Sparv Embedded

Telemetry and ground receiver

While the airborne unit handles transmission, the ground receiver ensures accurate acquisition, synchronization, and validation of the telemetry stream. Selective filtering and error-detection routines safeguard data integrity even under weak-signal conditions, while multi-channel capability allows simultaneous monitoring of several sondes during coordinated launches. Once captured, the telemetry is processed through digital signal blocks that reconstruct temperature, humidity, pressure, and positional data into usable atmospheric profiles.

Modern systems further enhance reliability with multi-GNSS technology, leveraging multiple satellite constellations to improve positional accuracy and wind profiling. Coupled with real-time visualization interfaces, operators can track balloon ascent, sensor health, and data quality throughout the flight. By combining robust acquisition with intelligent decoding, the receiver transforms radiosonde measurements into actionable meteorological information for forecasting systems.

External payloads and research extensions

Beyond standard meteorological instrumentation, radiosondes can be adapted to carry external payloads for specialized research. A common example is the ozone sonde, which measures ozone concentration profiles using electrochemical sensors to support atmospheric chemistry studies.

Other payloads may include aerosol samplers, radiation detectors, or custom research modules, depending on mission objectives. These add-on packages are typically integrated beneath the radiosonde unit, sharing the balloon lift and telemetry link while operating within defined weight and power budgets.

By accommodating external payloads, radiosonde platforms extend their role from routine weather monitoring to flexible airborne laboratories, enabling targeted investigations into atmospheric composition, pollution transport, and climate dynamics.

High-altitude scavenger hunt

Every day, thousands of radiosondes drift back to Earth, largely unnoticed by the world below. However, with a modest receiver and a bit of technical curiosity, these silent travelers become the centerpiece of a high-tech scavenger hunt.

Radiosonde hunting, also known as radiosonde tracking, is a unique hobby that bridges the gap between radio engineering, software-defined radio (SDR), and outdoor exploration. By leveraging specialized hardware and open-source software, enthusiasts can intercept live telemetry, decode atmospheric data in real time, and pinpoint a sonde’s landing site for recovery.

Radiosondes as tools and inspiration

Radiosondes have proven indispensable across a wide application range—from core meteorology and climate science to agricultural forecasting, where vertical profiles of humidity, temperature, and wind inform crop management and irrigation planning. Their adaptability extends further through external payloads such as ozone sondes, and even specialized launch techniques like double-balloon configurations, which extend flight duration and altitude coverage for advanced research missions.

Yet radiosondes are more than just instruments of record; they are also objects of curiosity and experimentation. Around the world, enthusiasts collect spent sondes, hack their electronics, and repurpose them for creative experiments, turning routine weather balloons into platforms for learning and innovation. This dual identity—precision tool for science and playground for exploration—underscores why radiosondes continue to inspire both professionals and hobbyists alike.

Well, whether you are a researcher, a student, or a curious tinkerer, radiosondes invite you to explore the atmosphere, experiment with technology, and contribute to the collective understanding of our dynamic skies.

T. K. Hareendran is a self-taught electronics enthusiast with a strong passion for innovative circuit design and hands-on technology. He develops both experimental and practical electronic projects, documenting and sharing his work to support fellow tinkerers and learners. Beyond the workbench, he dedicates time to technical writing and hardware evaluations to contribute meaningfully to the maker community.

The post Radiosondes: Disposable guardians of the sky appeared first on EDN.

A new technology addresses a key performance barrier limiting the use of GaN-on-silicon semiconductors in mainstream RF applications. According to Scott Bibaud, president and CEO of Atomera, this will change the economics of GaN in RF by unlocking breakthrough RF performance on low-cost silicon substrates.

Gallium nitride (GaN) devices for high-performance RF applications are typically built on silicon carbide (SiC) substrates; while they offer robust performance, they are also costly and difficult to scale. On the other hand, silicon substrates offer a lower-cost, more scalable foundation with the potential to support larger wafer sizes and greater compatibility with standard silicon manufacturing.

However, GaN-on-silicon underperforms in RF applications due to parasitic channel losses that reduce efficiency, especially at high frequencies. Enter Atomera’s Mears Silicon Technology (MST), which claims to reduce these losses while offering robust linearity and lower-cost GaN solutions for 5G and other high-frequency RF devices.

MST—a quantum-engineered thin-film technology—introduces a thin, oxygen-modified layer near the surface of the silicon wafer to create a more favorable platform for GaN growth, making silicon a more viable foundation for high-performance RF devices. This controlled layer modifies the silicon lattice structure and helps block the diffusion of electrical dopants. That, in turn, improves crystal quality at the GaN-silicon interface.

MST can improve various wafer-level reliability measures in nitrided oxide planar devices. Source: Atomera

Incize, which provides characterization and modeling services for RF semiconductors, has performed RF characterization of the first MST-enabled samples. The Belgian company reports a substantial reduction in parasitic interface charge and a significant reduction in RF losses.

“Beyond the small-signal improvements, the large-signal results are particularly compelling,” said Mostafa Emam, founder and CEO of Incize. “Then there is a linearity benefit that extends into the high-power regime, approaching performance levels typically associated with advanced RF SOI technologies.”

In Atomera’s own testing, MST enabled more than a 10x reduction in parasitic channel charge, reducing a key mechanism of RF power loss and supporting improved high-frequency GaN device performance. The test data also shows that MST enables devices to handle significant power while maintaining signal quality—linearity—under stress.

Robert Mears, founder and CTO of Atomera, is quick to add that linearity is a top concern for RF designers. “The new data shows MST GaN-on-silicon achieving both the ultra-low RF losses and linearity metrics of advanced trap-rich RF SOI,” he said. “At the benchmark input power of 30 mW, the linearity is exceptional, 1000x better than the GaN-on-silicon reference wafer.”

Atomera, a semiconductor materials and technology licensing company, is based in Los Gatos, California.

Related Content

• GaN on silicon or SiC?
• A Guide to GaN-on-Silicon
• A brief history of gallium nitride (GaN) semiconductors
• Why RF Technologies Should Consider GaN Over Silicon
• GaN-on-Si Technology Makes Headway in RF Applications

The post The RF-ready GaN-on-silicon with lower parasitic losses appeared first on EDN.

Editor’s note: This is a multi-part series on how to design a digital-controlled PFC. Previous entries: 

• How to design a digital-controlled PFC, Part 1
• How to design a digital-controlled PFC, Part 2
• How to design a digital-controlled PFC, Part 3

High efficiency is a mandatory requirement in some applications, especially in data centers. The recently announced 80 Plus Ruby certification sets the highest efficiency standard for data center power-supply units (PSUs), as shown in Table 1. The new efficiency requirement is not only higher than 80 Plus Titanium at each load condition, but also requires 90% efficiency at a 5% load, which has never been specified before.

Table 1 “Ruby” is the most recent and most stringent of the 80 Plus certification levels

With totem-pole bridgeless power factor correction (PFC) offering the best efficiency among all PFC topologies, digital control can further push the efficiency capabilities of this topology to new levels. In the fourth and final installment of this series, I will first introduce several digital methods to improve efficiency and then discuss some special PFC requirements including re-rush current control, electrical metering (e-metering) and PFC with a baby boost converter.

Dynamic dead time to achieve ZVS for synchronous switch

Theoretically, the PFC synchronous switch can operate with zero voltage switching (ZVS), but there must be a proper dead time between when the boost switch turns off and the synchronous switch turns on. As illustrated in Figure 1, assuming a positive cycle, when boost switch Q2 turns off, the inductor current (I_L) starts to charge the output capacitance (C_OSS) of Q2 and discharge the output capacitance C_OSS of Q1, and the switch-node voltage rises.

If Q1 turns on before the switch-node voltage rises to the output voltage (V_OUT), this is hard switching, and the switching losses are high. If Q1 turns on too late after the switch-node voltage rises to V_OUT, the current will conduct in the third quadrant of Q1 with diode-like behavior. Since the gallium nitride field-effect transistor used for Q1 has a higher V_SD drop compared to a silicon metal-oxide semiconductor field-effect transistor body diode, this induces a higher third-quadrant conduction loss.

Figure 1 This equivalent circuit describes a PFC synchronous switch during dead time. (Source: Texas Instruments)

Ideally, Q1 should turn on at the exact moment when the switch-node voltage rises to V_OUT. Given the I_L, V_OUT and C_OSS of Q1 and Q2, the following equation calculates the time to charge the switch node from 0 to V_OUT:

You can use firmware to dynamically adjust the dead time calculated from the equation to maintain ZVS for the synchronous switch.

CCM_TCM multimode control

A totem-pole bridgeless PFC can operate in either continuous conduction mode (CCM) or triangular current mode (TCM); each has its advantages and disadvantages. Table 2 provides a high-level comparison between the two modes.

Table 2 Continuous conduction mode (CCM) and triangular current mode (TCM) options both have pros and cons for totem-pole power factor correction (PFC) operation purposes.

Ideally, the totem-pole bridgeless PFC could operate with multimode, as shown in Figure 2. At heavy loads or at the peak of an AC half cycle, the desired PFC input current is high and the PFC operates in CCM mode. When the load reduces or around the AC zero-crossing area where the desired PFC input current is low, the PFC switches to TCM mode and operates with ZVS.

Compared to pure CCM mode, this multimode operation has better efficiency at light loads because of ZVS. Compared to pure TCM mode, because the inductor current ripple is much lower, there is no need to use multiphase interleaved operation; therefore, this multimode operation significantly reduces the size and system costs. By combining the advantages of both CCM and TCM, this multimode operation can meet both high-efficiency and high-power-density requirements.

Figure 2 CCM_TCM multimode operation can meet both high-efficiency and high-power-density requirements. (Source: Texas Instruments)

Reference 1 provides details about this control method and its implementation. Figure 3 compares the efficiency (tested on the same board) between this CCM_TCM multimode control method and traditional CCM control, with efficiency improving as much as 2%.

(a)	(b)

Figure 3 CCM_TCM multimode control delivers efficiency improvements versus traditional CCM control in both low line (a) and high line (b) environments. (Source: Texas Instruments)

Special burst mode – AC cycle skipping

Burst mode is widely used to improve efficiency at light loads. Unlike traditional pulse-width modulation (PWM) pulse-skipping burst mode, where you skip PWM pulses randomly, here I would like to introduce a special burst mode: AC cycle skipping, which is you skip one or more AC cycles in light loads.

In other words, you would turn the PFC off for one or more AC cycles and turn the PFC back on for the next AC cycle. The turnon and turnoff instance occurs at the AC zero crossing such that the whole AC cycle is skipped. Since PFC turnon and turnoff at inductor current equal zero, there is less stress and electromagnetic interference.

The number of AC cycles to skip is reverse-proportional to the load; the lighter the load, the more AC cycles skipped. Figure 4 shows the skipping of one and two AC cycles, respectively. Channel 1 is the AC voltage, and channel 4 is the AC current.

(a)	(b)

Figure 4 Shown here is AC cycle skipping at a light loads: one cycle (a) and two cycles (b). (Source: Texas Instruments)

Once the PFC turns off, the switching losses, driving losses and reverse-recovery losses all drop to zero, and the power losses are just the PFC standby power.

When turning off the PFC to skip AC cycles, both the current loop and voltage loop need to be frozen; otherwise, the integrators in those loops will build up to generate a big PWM pulse when the PFC turns back on, causing a large current spike.

Determining whether the PFC enters a light load requires the load information. Normally there is no current sensor at the PFC output; therefore, it’s not possible to directly measure the output load. However, because the PFC voltage-loop output is proportional to the load, you can use the voltage-loop output as a rough indicator to determine whether the PFC is operating with a light load.

If you must precisely skip an appropriate number of AC cycles to maintain V_OUT ripple within a specified range, you will need accurate load information, which you can obtain through an integrated e-meter function that I will discuss after the next section.

A big concern with AC cycle skipping is the V_OUT drop during a load transient. Assuming that a load step-up occurs when the PFC is off, V_OUT may drop too much.

To address this issue, you can compare V_OUT to a predefined threshold through a comparator. Once V_OUT is below this threshold, the PFC will immediately exit burst mode, disable AC cycle skipping, and return to normal operation. The PFC will handle the transient response as if there is no such special burst mode.

AC cycle skipping can also help reduce total harmonic distortion (THD) at light loads. Reference 2 compares THD with and without this method.

Re-rush current limit

The AC input voltage could suddenly drop out when PFC is operating normally. Since the load is still applied, the PFC V_OUT could drop to a lower value. Then, when the AC voltage returns, if the AC input voltage is higher than V_OUT, there will be an inrush current. This current is called the re-rush current.

Previously, the re-rush current was unspecified and there was no special control action for this event, it solely relied on the power-stage components’ ability to handle re-rush current. Test results show that re-rush current can jump more than 10 times higher than the PFC-rated maximum input current. Such a high re-rush current can either damage the power supply or reduce its lifetime.

The recently released Modular Hardware System– Common Redundant Power Supply (M-CRPS) specification requires limiting re-rush current when the input voltage resumes after an input brownout or blackout event on the power supply used in a data center. As shown in Figure 5, the root-mean-square (RMS) value of re-rush current should not exceed 5 times the maximum PSU rating over one-half cycle of input frequency, or 3.5 times the maximum PSU rating over one cycle of input frequency. In addition, the input current of the PSU should settle to a value less than or equal to two times the maximum PSU rating of the PSU within two cycles of the input frequency after applying the AC input.

Figure 5 The Modular Hardware System– Common Redundant Power Supply (M-CRPS) specification documents limits on both re-rush current and timing. (Source: Texas Instruments)

Reference 3 provides a firmware-based solution to handle this re-rush current so that when the AC voltage comes back from dropout, both the re-rush current (when V_IN > V_OUT) and the non-re-rush current (when V_IN < V_OUT) are well controlled – not exceeding the M-CRPS limit specification, but high enough to rapidly boost V_OUT.

E-metering

Power supplies in data centers are required to measure the input power in real time and report the measurement to the host; this is called e-metering. The M-CRPS specification requires an input power measurement error within ±1% when the load is >125W, within ±1.25W when the load is between 50W and 125W, and within ±5W when the load is <50W. To achieve such high measurement accuracy, the e-meter function is traditionally implemented through a dedicated metering device, as shown in Figure 6a.

(a)	(b)

Figure 6 These circuit diagrams show a traditional e-meter and PFC control (a), as well as combining an e-meter with PFC control (b). (Source: Texas Instruments)

A current shunt placed on the PFC input side senses the input current, with a voltage divider (not shown in Figure 6a) across the AC line and AC neutral senses the input voltage. A dedicated metering device receives this current and voltage information and calculates the input power and input RMS current information, sending the results to the host.

With a digital controller, since analog-to-digital converters (ADCs) of the microcontroller (MCU) are measuring both the input voltage and input current, it becomes possible to integrate the e-meter function into PFC control code. Figure 6b shows this e-meter configuration.

A current shunt senses the input current and an isolated delta-sigma modulator (the AMC1306 from Texas Instruments) measures the voltage drop across the current shunt. The delta-sigma modulator output is sent to the PFC controller MCU. The current information will be used for both e-metering and PFC current-loop control. A voltage divider senses the input voltage, which is then measured by the MCU’s ADC directly, just as in traditional PFC control. Reference 4 has more details about e-meter implementation and calculation.

Integrating e-meter functionality into PFC control code eliminates the need for a dedicated metering device, not only reducing system costs, but also simplifying printed circuit board layout and expediting the design process.

PFC with a baby boost converter

In server applications, a bulk capacitor (C_BULK in Figure 7) is required to hold PSU output in regulation for more than 10mS after AC dropout. To accomplish this, a 3kW server PSU would need a total capacitance of over 1.3mF, which would consume at least 30% of the overall space. To improve power density, you must reduce the bulk capacitance.

Adding a baby boost converter between PFC and DC/DC, as shown in Figure 7 and described in Reference 5, can achieve high power density. The baby boost converter is a compact boost converter that only operates during AC dropout events.

Figure 7 A PFC with a baby boost converter can achieve high power density. (Source: Texas Instruments)

Figure 8 is a flow chart of baby boost converter operation. During normal operation, the baby boost converter is off and bypassed by a BYPASS FET Q4. When AC line dropout occurs and V_BULK drops to a certain level, Q4 turns off, and the baby boost converter turns on to allow V_BB to maintain its nominal value. If AC power returns, V_BULK will rise; once V_BULK rises to a certain level, MCU turns off the baby boost converter, turns on BYPASS FET Q4, and the PFC resumes normal operation.

Figure 8 This flow chart outlines the various stages of baby boost converter operation.

Conclusion

I hope that the information imparted in this series enables you to design your own digital-controlled PFC and meet ever-more-strict specifications. You will find that digital control is so flexible that is possible to implement advanced control algorithms that would be difficult to implement with analog control. A digital-controlled power supply also offers impressive performance.

References

1. Sun, Bosheng. “A novel CCM-TCM multimode control method for totem-pole bridgeless PFC.” Texas Instruments Analog Design Journal article, literature No. SLYT877, 1Q 2026.
2. Sun, Bosheng. “AC cycle skipping improves PFC light-load efficiency.” Texas Instruments Analog Design Journal article, literature No. SLYT585, 3Q 2014.
3. Sun, Bosheng. “How to limit PFC re-rush current.” Texas Instruments Analog Design Journal article, literature No. SLYT865, 1Q 2025.
4. Sun, Bosheng. “A low-cost and high-accuracy e-meter solution.” EDN, Aug. 26, 2024.
5. Yu, Sheng-Yang, Benjamin Genereaux, and LiehChung Yin. “Improve power density with a baby boost converter in a PFC circuit.” Texas Instruments Analog Design Journal article, literature No. SLYT830, 2Q 2022.

Related Content

• How to design a digital-controlled PFC, Part 1
• How to design a digital-controlled PFC, Part 2
• How to design a digital-controlled PFC, Part 3
• A low-cost and high-accuracy e-meter solution

The post How to design a digital-controlled PFC, Part 4 appeared first on EDN.

Any discussion of modern AI system performance must include MLCommons and its MLPerf benchmark suite, which has become the industry’s de facto standard for measuring machine learning performance. Since its debut in 2018, MLPerf has provided a neutral, peer-reviewed framework for comparing hardware and software platforms across a broad range of AI workloads.

The original MLPerf benchmarks reflected the dominant AI workloads of the late 2010s. Early inference tests focused on models such as image classification with ResNet-50, natural language processing with Bidirectional Encoder Representations from Transformers (BERT), object detection with RetinaNet, and recommendation with Deep Learning Recommendation Model (DLRM).

These workloads were important and representative at the time, but they shared one characteristic: they were highly parallel and relatively easy to map onto GPU architectures.

For several years, benchmark results reinforced a simple narrative. Each new generation of accelerators delivered higher throughput, lower latency, and better energy efficiency. Because the workloads aligned well with GPU strengths, the benchmark curves rose steadily and predictably.

The generative AI shockwave: Rewriting the rules of MLPerf

Autoregressive LLMs introduced a fundamentally different inference pattern. Prompt processing remained highly parallel, but token generation became sequential and memory bound. Suddenly, raw TeraFLOPS no longer told the whole story.

MLPerf began incorporating this new reality in stages. Inference v4.0 introduced the first LLM benchmark based on Meta platform Llama 2 70B. This benchmark measured token throughput and provided the industry with its first standardized method for comparing LLM inference systems.

MLPerf Inference v5.0 released in 2025 significantly expanded the generative AI focus. It added Llama 3.1 405B Instruct, a 405-billion parameter model with a 128,000-token context window. The benchmark also introduced an interactive variant of Llama 2 70B that imposed strict limits on Time to First Token (TTFT) and Time Per Output Token (TPOT), two metrics that directly capture user experience in conversational applications.

These additions were pivotal because they exposed the core weakness of GPU-based inference systems. When unconstrained by latency, GPUs could buffer requests, create large batches, and deliver excellent throughput. Under interactive latency limits, batching opportunities shrank, hardware utilization dropped, and throughput fell sharply.

In other words, MLPerf began measuring not just how fast a system could run under ideal conditions, but also how responsive it remained under realistic conditions.

Inference disaggregation: Optimization of resources

This evolution reached another milestone in MLPerf Inference v5.1 and the emerging v6.x era. The benchmark suite broadened its focus to include increasingly sophisticated workloads, including reasoning models such as DeepSeek-R1 and more demanding long-context applications. At the same time, submissions began showcasing system-level optimizations such as inference disaggregation, where prompt processing and decoding are assigned to different accelerator pools.

Disaggregation has become one of the most consequential developments in modern inference benchmarking.

Historically, MLPerf treated each benchmark run as a single system under test, leaving vendors free to optimize their hardware and software stacks as they saw fit. As long as submissions complied with accuracy and latency requirements, any architectural technique was fair game.

This openness allowed participants to introduce increasingly sophisticated serving strategies. One of the most effective has been the separation of prefill and generation across distinct groups of accelerators. The prefill cluster handles the compute-intensive prompt processing stage, while the generation cluster focuses exclusively on token decoding.

In controlled benchmark scenarios, where prompt lengths and output lengths are known in advance, disaggregation can produce dramatic gains. By eliminating interference between the two phases, systems reduce preemption and improve latency-sensitive throughput.

Yet this raises an important question. Does the benchmark still measure accelerator capability, or is it increasingly measuring system orchestration? The answer is both.

Modern AI performance depends on the interaction between processor, memory hierarchy, interconnect fabric, runtime software, and serving algorithms. MLPerf has evolved accordingly. It now rewards system-level innovation rather than isolated chip performance.

That shift is entirely appropriate, but it also means benchmark results must be interpreted carefully.

A disaggregated configuration optimized for long document summarization may perform brilliantly in MLPerf while delivering more modest benefits in production environments where workloads vary continuously. Real-world deployments must cope with unpredictable prompt lengths, bursty traffic, and rapidly changing ratios of prefill to generation demand.

Consequently, MLPerf increasingly measures a system’s ability to align resources with a known workload profile. This is a valuable metric, but it’s not synonymous with universal real-world performance.

Illustrative comparison: MLPerf 5.x versus MLPerf 6.x

Table below illustrates how benchmark methodology evolved as MLPerf shifted from throughput-oriented LLM tests to more latency-sensitive and system-aware workloads. The numbers are representative rather than exact, but they reflect the broad trends seen in published results and vendor disclosures.

Publicly discussed MLPerf inference results based on Llama 3.1 405B LLM run on a leading-edge GPU-based processor in three scenarios (off-line, server mode, and interactive mode) highlight MLPerf’s evolution. Source: Author

From chip benchmark to system benchmark

The history of MLPerf mirrors the evolution of AI itself.

The early benchmark suites focused on relatively static workloads that aligned naturally with the strengths of GPU architectures. Tasks such as image recognition, recommendation systems, and conventional deep learning inference relied heavily on dense matrix operations and large-scale parallelism, allowing GPUs to demonstrate exceptional throughput and scalability. In that era, benchmark leadership was closely associated with raw compute capability, memory bandwidth, and increasingly larger accelerator configurations.

The rise of generative AI fundamentally changed that equation.

As autoregressive LLMs became the dominant workload, MLPerf evolved accordingly, introducing larger models, longer context windows, interactive server scenarios, and increasingly strict latency constraints. These additions exposed a critical reality: while GPUs remain extraordinarily efficient during the highly parallel prefill phase, they are far less efficient during token generation, where inference becomes sequential, memory-bound, and heavily dependent on latency-sensitive execution.

This shift transformed the meaning of benchmark performance.

Modern MLPerf results no longer measure the capabilities of an isolated accelerator alone. Instead, they measure the effectiveness of an entire inference architecture.

Disaggregation, scheduling policies, key-value (KV) cache management, streaming pipelines, runtime orchestration, and workload balancing have become just as important as the underlying silicon itself. In many cases, the benchmark winner is no longer the system with the most compute power, but the one that most effectively adapts a fundamentally sequential workload to hardware originally designed for massively parallel graphics and HPC computation.

As a result, benchmark interpretation has become significantly more nuanced. The headline numbers increasingly reflect how intelligently the system orchestrates resources across racks of accelerators, separates prefill from generation, minimizes preemption, and maintains throughput under realistic latency constraints. MLPerf has evolved from a pure hardware benchmark into a broader measure of system architecture and software orchestration.

At the same time, this evolution reveals something even more profound. The latest MLPerf 6.x requirements implicitly highlight the growing limitations of conventional GPU architectures for real-time LLM inference. The industry has reached a point where increasingly sophisticated scheduling mechanisms and disaggregated serving infrastructures are being used to compensate for a deeper architectural mismatch between autoregressive inference and massively parallel processors.

In many respects, the benchmark itself is beginning to suggest the next major transition in AI infrastructure design.

Rather than continuing to optimize architectures originally developed for graphics rendering and parallel numerical computing, the future may require entirely new inference-centric architectures built specifically for the unique characteristics of the LLM generation. Such architectures would need to deliver high utilization and low latency even with very small batch sizes—potentially down to a single user request—while minimizing data movement, reducing memory bottlenecks, and supporting continuous token generation without relying on increasingly complex orchestration layers to hide inefficiencies.

In that sense, MLPerf has become more than a benchmark suite. It is now a window into the architectural tensions shaping the future of AI computing, revealing both the extraordinary adaptability of modern accelerator systems and the growing need for a fundamentally new class of inference hardware designed from the ground up for the realities of autoregressive AI.

Lauro Rizzatti is a business development executive with Vsora, a technology company offering semiconductor solutions that redefine design performance. He is a noted chip design verification consultant and industry expert on hardware emulation.

Editor’s Note

This is Part 2 of the mini-series that examines how LLM inference forced changes to MLPerf benchmarking. In Part 1, contributor Lauro Rizzattti analyzes LLM inference across its two processing phases—prefill versus generation—and highlights how this workflow exposes structural inefficiencies in GPU-based accelerators.

Related Content

• Strategies to Dominate the AI Accelerator Market
• A closer look at LLM’s hyper growth and AI parameter explosion
• The role of AI processor architecture in power consumption efficiency
• AI GPU computing delivers data-center performance on the factory floor
• The truth about AI inference costs: Why cost-per-token isn’t what it seems

The post MLPerf and the rise of latency-aware LLM benchmarking appeared first on EDN.

Clock speeds get faster. Per-cycle (and per-clock edge) address and data dollops get larger. And protocols get more efficient. But here we’re talking about external, not internal, buses.

Back in 2023, I devoted two blog posts’ worth of content to comparing various memory card technologies, products and speed bin options, initially in March (identifying a fake card in the process) and more in-depth in July. Since then, I’ve come across numerous examples of both evolutionary and revolutionary successors to the devices discussed in that two-part series, not to mention those covered in even more distant-past writeups (themed, for example, around the cameras, digital audio recorders and other devices that leverage such storage).

I’ve had this follow-up piece in my to-do list for a while now, and I’ve finally decided to actualize my longstanding aspiration before the dust pile accumulating on this specific list entry gets any deeper. Not every technology to be discussed in the paragraphs to follow will likely achieve high-volume market success, mind you, with any sooner-or-later failures not necessarily the result of implementation shortcomings, either. Note, for example, that today’s (and past) industry supply constraints encourage manufacturers to “double down” on maximizing the output and profitability of existing approaches, versus devoting scarce capacity to dubious bets.

That said, win or lose there’s usually an interesting story behind each approach. Without further ado…and with the upfront qualifier that I’ll be intentionally delaying any discussion of USB-interface memory devices until later, since their connector locations compel them to be fully external to the system, either sticking straight out of it or cable-tethered to it…and that for related reasons, I won’t be covering eMMC and other fully internal formats, either…and lastly, that I’ll be skipping over legacy formats that were proprietary and/or otherwise non-impactful…

Historical precedents

A short writeup, “History Repeating” at Virginia Tech’s website, begins as follows:

Variations on the repeating-history theme appear alongside debates about attribution. Irish statesman Edmund Burke is often misquoted as having said, “Those who don’t know history are destined to repeat it.” Spanish philosopher George Santayana is credited with the aphorism, “Those who cannot remember the past are condemned to repeat it,” while British statesman Winston Churchill wrote, “Those that fail to learn from history are doomed to repeat it.”

Long-time readers may recall that I’ve referenced variants of this same quote theme in several past writeups, consistently with a negative connotation involving the downsides of ignorance to the past. That said, excessive dependence on history lessons can also be problematic, resulting in evolutionary, overly constraining baby-steps that suppress alternative more revolutionary strides, which may lead to failure but may also dramatically leap beyond traditional approaches.

I’ll leave you to decide for yourselves what to conclude from this first case study, admittedly too personal to likely allow me to be completely arms-length about it! Embedded within the tuple (card identifier) data structures reported by Intel’s Series 2 flash memory cards were the initials of the small team of developers, myself among them, who designed their ASIC (30 years ago…yikes!). I subsequently led the technical marketing launch of the 28F008SA 8 Mbit flash memories inside those same cards, followed by the definition, development and introduction of 16 and 32 Mbit component successors and cards based on them, all in the early-to-mid-1990s.

Products such as these, representing the industry’s first removable and high capacity (for the era, at least) memory cards, added these tuple structures and other enhancements in order to deliver full Personal Computer Memory Card International Association (PCMCIA, later known as PC Card) compatibility, in contrast to Series 1 precursors which were more elementary multi-component arrays along with address decode and chip select logic. Intel’s and others’ similar products were specifically referred to as linear flash memory PC Cards, both to differentiate them from other PCMCIA card types—modems, ISDN and SCSI, for example, and living on (at least to a degree) with CableCARDs—and from alternative ATA-interface flash memory cards.

The key difference between the two memory card types centered on where the flash media management intelligence was located: in the card itself for ATA flash PC Cards, thereby presenting a standardized hardware and software interface to the system regardless of what (and whose) media was inside, versus in the system, implemented as software and/or dedicated hardware, for the linear flash PC card approach. Proponents of the latter scheme touted its claimed reduced media bill-of-materials cost, not to mention the potential ability to direct-execute code out of it (acting as a big parallel-interface chip), but it was inherently relevant for only NOR (vs NAND) memory suppliers, along with being a “heavier lift” for system developers. For these and other reasons, the ATA approach eventually won out in the marketplace.

Miniaturization

That said, Intel and several of its NOR flash memory partner/competitors had also taken a stab at miniaturizing the linear flash PC Card with the creatively named (ha!) Miniature Card format:

Other flash memory suppliers countered with the ultimately much more popular CompactFlash card, now maintained by the aptly named CompactFlash Association (CFA), whose hardware interface was similarly PCMCIA-derived albeit instead (as with the ATA flash PC Card precursor) focused on the IDE/ATA (and later, UDMA) command set:

Amid this “where is the media management intelligence best located” debate, two other notable contending approaches of the same timeframe also bear mentioning. The first, SmartMedia, was championed by Toshiba (as well as, later, by its primary competitor, Samsung):

SmartMedia was essentially a single (although a few variants embedded multiple) NAND flash memory die embedded within a thin plastic membrane, plus a multi-contact metallic interface that wirebond-direct-connected to the die with no intervening media controller intelligence.

Conceptually sounds like linear flash PC Cards and their derivatives, doesn’t it? Yes…and no. For one thing, SmartMedia was much smaller than either Miniature Card or Compact Flash. For another, it was based on NAND flash memory, which was more HDD-like in its core attributes  (notably erase block size and speed) than NOR, simplifying system-side media management development. And then there was the fact that Toshiba wasn’t just a semiconductor supplier; its various systems divisions were potential SmartMedia implementers, and the company also did a good job of cultivating business from other Japanese and broader Asian systems manufacturers.

Finally, near the end of the last century (in 1997, to be exact), Sandisk and systems partners Siemens and Nokia unveiled the MultiMediaCard (MMC), which ultimately came in multiple dimension options, as well as in both standard and clock-boosted performance variants:

MMC is best known today in its aforementioned non-removable eMMC form, which itself is being slowly supplanted by the embedded variant of the MIPI- and SCSI-based Universal Flash Storage (UFS) (an organization whose own removable-version standard ironically has conversely been underwhelmingly adopted by the industry). Today’s generational successor to MMC is the Secure Digital (SD) card, originally referred to as SecureMMC:

which built on the MMC foundation with “enhancements including a digital rights management (DRM) feature, a more durable physical casing, and a mechanical write-protect switch.” The SD standard’s successive iterations have expanded the available clock speed, protocol and electrical contact count options in a backwards-compatible fashion to keep pace with flash memory performance gains, such as in this high-end V90 card from OWC:

The microSD Card derivative tackled substantive dimensional decreases with notable success; here’s one alongside the SmartMedia card I showed you earlier:

One interesting newer SD (and microSD) card specification variation that I became aware of recently when shopping for storage media for a couple of new Raspberry Pi cards is the Application Performance Class. Quoting from Kingston Technology documentation:

A new classification has been presented with the introduction of Android’s Adopted Storage Device feature. The App Performance Class assures minimum random and sequential performance speeds to meet both run and store execution time requirements under given conditions. It does this simultaneously while providing storage for pictures, videos, music, files and other important data. Basically, they’re ideal for use in smartphones and mobile gaming devices that run applications at random read and write speeds while also being used for storage.

 There are two ratings for the App Performance Class which are known as A1 and A2. A1 has a minimum random read of 1500 IOPS and a minimum random write of 500 IOPS while A2 has a minimum random read of 4000 IOPS and a minimum random write of 2000 IOPS. Both A1 and A2 have a minimum sustained write speed of 10MB/s. The App Performance Class is something to consider [editor: for example] when planning on installing Android apps on a microSD card.

And, by the way, unlike the SmartMedia competitor of the day, both MMC and successor SD Cards notably also embed (despite their smaller sizes) media management intelligence that simplifies and standardizes the system implementation. Moore’s Law strikes again, eh?

Hang tight; I’ll be right back

Believe it or not, I originally envisioned this being, and wrote it as, a single unified blog post. However, as thought of more (and more…and more…) things to include, the wordcount grew (and grew…and grew…), transforming it into something resembling a small book (I exaggerate, but you get my drift). Having passed through 1,500 words at the beginning of this paragraph, I’m instead going to pause for now, intending (God willing) to share the other half of this now-two-part series with you next week. Until then, please share in the comments your thoughts on what I’ve covered so far!

—Brian Dipert is the associate editor, as well as a contributing editor, at EDN.

Related Content

• Memory cards: Specifications and (more) deceptions
• SD card speeds: question your assumptions
• AI boom and the politics of HBM memory chips

The post Memory card interfaces keep pace with the internal bus evolution race: Part 1 appeared first on EDN.

Recent frontier LLM inference benchmarks have highlighted a recurring pattern. GPU-based systems deliver outstanding throughput when latency is not a concern, but their performance drops sharply once real-time response requirements are imposed.

This behavior is sometimes attributed to software inefficiencies or suboptimal system tuning. In reality, the root cause lies much deeper. It reflects a fundamental mismatch between how GPUs are architected and how autoregressive inference works.

LLM inference: Prefill versus generation

To understand this limitation, it is useful to examine the two distinct phases of LLM inference: prefill and generation.

During the prefill phase, the model processes the entire input prompt in one pass. The prompt is tokenized, embedded, and propagated through every layer of the transformer network. At each layer, the model computes the attention relationships among all tokens and builds the key-value (KV) cache, which stores the intermediate data needed for subsequent token generation.

This stage maps extremely well onto GPU hardware. GPUs were designed to execute thousands of identical operations in parallel. In the prefill phase, the model performs massive matrix multiplications over large tensors, exactly the type of workload for which GPUs excel. When all tokens are available upfront, the calculations can be distributed across tens of thousands of cores, resulting in very high arithmetic utilization.

The generation phase is fundamentally different.

Once the KV cache has been created, the model begins producing output tokens one at a time. Each token depends on all tokens that came before it. This sequential dependency means that, regardless of how much hardware is available, the model cannot generate the next token until the current one has been completed.

For every generated token, the model must read the parameters for every layer, consult the KV cache, compute the next token probabilities, and then repeat the autoregressive process. The amount of computation per token is relatively modest, but the amount of data movement remains substantial.

Two faces of GPU architecture: Why modern GPUs struggle with real-time latency constraints

This is where the GPU architecture begins to work against the workload.

GPUs achieve peak efficiency when they execute large, highly parallel workloads with regular memory access patterns. Token generation offers neither. The workload is small, inherently sequential, and dominated by repeated memory accesses rather than dense arithmetic. Many of the GPU’s compute units remain idle while the device waits for data to arrive from high-bandwidth memory.

In other words, generation is not compute-bound; it’s memory-bound.

The distinction is crucial. In a compute-bound workload, adding more arithmetic units improves performance. In a memory-bound workload, performance is limited by how quickly data can be moved to the processors. Once memory bandwidth becomes the bottleneck, additional compute resources provide diminishing returns.

This explains why GPUs can appear extraordinarily efficient when throughput is measured without latency constraints. In that scenario, inference servers are free to buffer requests and combine them into large batches. Batching allows the system to process many token streams simultaneously, effectively transforming numerous small sequential tasks into a larger parallel workload that better matches the GPU’s strengths.

The role of batch sizes in GPU’s utilization

At first glance, batching in AI inference may appear straightforward. Unlike image inference where every sample in a batch completes simultaneously, LLM inference involves many conversations progressing independently and asynchronously. Some requests finish quickly, others may continue for hundreds or even thousands of decoding iterations, and new requests may arrive continuously while older conversations are still active.

The workload therefore becomes highly dynamic and irregular. Specifically, the generation of each request ends only when the model produces a special “end-of-sequence” token indicating that the response is complete.

This characteristic fundamentally changes the nature of inference scheduling.

This is where continuous batching becomes essential. Continuous batching is the runtime orchestration algorithm responsible for managing the simultaneous execution of multiple conversations across the same accelerator resources. Instead of treating inference as a sequence of isolated batches, the scheduler continuously inserts, removes, pauses, and resumes requests as tokens are generated.

The objective is to maximize hardware utilization while minimizing user-visible latency. As batch sizes increase, hardware utilization rises and throughput improves dramatically. However, batching comes at the cost of response time.

When users expect low latency, the system cannot afford to delay requests while waiting to accumulate a large batch. Each request must be processed almost immediately. As batch sizes shrink, the GPU loses the parallelism needed to keep its compute resources busy. Utilization falls, and throughput drops accordingly.

This is the central architectural limitation of GPUs in LLM inference.

The issue becomes even more pronounced when the same accelerator must handle both prefill and generation. Prefill is a large, compute-intensive task, while generation consists of many smaller, latency-sensitive operations. When new prompts arrive, the system may need to interrupt ongoing token generation to perform prompt processing. These context switches, often referred to as preemption, increase latency and reduce efficiency further.

Inference disaggregation: A clever shortcut to mitigate GPU’s inefficiencies

To mitigate this problem, system designers have begun disaggregating inference. Instead of assigning both phases to the same accelerator pool, they dedicate one group of GPUs to prefill and another to generation. The prefill GPUs build the KV cache and transfer it to the generation GPUs, which decode tokens independently.

This separation eliminates interference between the two phases and allows each group of GPUs to operate more efficiently. Prompt processing can proceed continuously without disrupting active token generation, and generation can continue without interruption.

In controlled benchmark environments, where prompt lengths, output lengths, and request patterns are known in advance, this approach can deliver substantial improvements.

Yet the underlying limitation of GPU architectures remains.

Inference disaggregation: Does it scale in real-world applications?

The generation phase is still sequential and memory bound. No amount of software optimization can eliminate the need to read model weights and cached data for each token. The disaggregated approach simply reduces scheduling inefficiencies and isolates the phases so that GPU resources are used more effectively.

Whether this strategy can scale efficiently in real-world applications depends on workload predictability.

The real-world AI services process a highly variable mix of requests. Some consist of long prompts and short responses. Others involve short prompts and long outputs. Demand can shift rapidly over time, changing the ideal ratio between prefill and generation resources.

Adapting to these changes requires dynamically reallocating accelerators. That process is not instantaneous. Devices must be initialized, model parameters loaded, and serving infrastructure synchronized. If traffic patterns are highly volatile, the overhead of reconfiguration can offset much of the benefit.

The broader lesson is that GPU performance in LLM inference is governed by more than raw TeraFLOPS.

The prefill phase showcases the strengths of GPUs, leveraging dense matrix operations and massive parallelism. The generation phase exposes their weaknesses, forcing highly parallel processors to execute a fundamentally sequential, memory-dominated workload.

As a result, the impressive throughput numbers often reported in unconstrained benchmarks can be misleading. They reflect idealized conditions in which batching hides architectural inefficiencies. Once latency constraints are introduced, those inefficiencies become visible.

The challenge for the industry is not simply to build larger GPUs, but to develop architectures and system designs better aligned with the realities of autoregressive inference.

Until then, the most significant limitation in real-time LLM serving will remain the same: generation is a sequential, memory-bound process running on hardware originally optimized for massively parallel computation.

Lauro Rizzatti is a business development executive with Vsora, a technology company offering semiconductor solutions that redefine design performance. He is a noted chip design verification consultant and industry expert on hardware emulation.

Editor’s Note

In a two-part series, contributor Lauro Rizzattti examines how LLM inference forced changes to MLPerf benchmarking. He will illustrate the evolution of the MLPerf benchmark and detail how generative AI forced a radical shift in AI hardware evaluation in the upcoming Part 2.

Related Content

• Strategies to Dominate the AI Accelerator Market
• A closer look at LLM’s hyper growth and AI parameter explosion
• The role of AI processor architecture in power consumption efficiency
• AI GPU computing delivers data-center performance on the factory floor
• The truth about AI inference costs: Why cost-per-token isn’t what it seems

The post The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking appeared first on EDN.

Abundant use of the AI acronym is increasingly evident at various industry events. Strip away the hype layer and look deeper, however, and interesting trends still emerge into view.

This is my third straight year covering Microsoft’s developer-focused conference, following up on the 2024 and 2025 show editions. And interestingly (at least to me), the event timing, both in an absolute sense and relative to other notable industry trade shows, has shifted each year.

• 2024’s Build took place on May 21-23, the week after Google’s I/O developer event (May 14-16) and several weeks before Computex (June 4-7).
• Last year, all three conferences took place on the same week.
• And this year, the Google I/O and Microsoft Build cadence returned to separate-weeks spacing, two weeks apart this time. Conversely, Build and Computex were still in the same-week slot.

Why the upfront focus on this seeming nuance? Well, for one thing, Computex conversely is a consumer-tailored show. That’s why, for example, Microsoft and Nvidia co-announced one new computer (information on which I’ll share shortly) at Computex, while introducing another with a different form factor but the exact same processing subsystem at Build.

Plus, in emphasizing a point that is likely already obvious to at least some of you, any chronological spacing between two companies’ events enables the latter to fine-tune its announcements and messaging to react to the former…and the more spacing the better from a reaction-robustness standpoint.

Speaking of announcements, let’s get to them, shall we? Microsoft CEO Satya Nadella and his various lieutenants, along with a couple of special guests, covered a lot of ground in the 2.5-hour kickoff keynote, the video of which I’ve embedded below. I’ll hit what I thought were the highlights in the following paragraphs.

AI inference-accelerating hardware

About those computers I just mentioned…stop me if you’ve heard this before. Microsoft and a partner roll out new Windows-on-Arm computer platforms, both mobile and mini-desktop in shape, and intended for both consumers and developers. Two years ago, that partner was Qualcomm, the SoCs were the Snapdragon X Elite and Plus, and the consumer mobile systems were the Surface Laptop and Pro (also accompanied by other OEMs, in a nod to Microsoft’s broader Windows-on-Arm aspirations). The developer mini-desktop was the Snapdragon Dev Kit for Windows, which never made it to production: Qualcomm “indefinitely paused” it only a few months later.

This outcome was more than a bit of a surprise to me, albeit not a complete surprise, as I’d been hearing for some time of both chronic hardware and software issues with the platform. That said, I already owned (and still use) its two Qualcomm application processor-based, developer-tailored predecessors, the Qualcomm-branded ECS LIVA Mini Box QC710:

and Microsoft’s “Project Volterra” (officially: Windows Dev Kit 2023) system:

So, the Snapdragon Dev Kit for Windows was unsurprisingly on my wish list, too.

Hopefully, Nvidia will have better luck, although the situation still feels somewhat embryonic. Let’s discuss consumer mobile system(s) first: launched at Computex and coming “this fall” at an as-yet-unannounced price, is the Microsoft Surface Laptop Ultra, based on Nvidia’s RTX Spark SoC.

While you might not immediately recognize the processor from its new marketing moniker, you’ve heard about it (from me, to be precise) before. It was previously known as the N1 and N1X, as well as the GB10, and it’s the outcome of a co-development project with MediaTek, who contributed the up-to-20-core CPU constellation and reportedly also took lead on full-chip integration, including the NVLink interconnect to the up-to-6,144 core GPU cluster.

The SoC’s development has been lengthy and troubled, if longstanding and widespread rumors are to be believed, and industry analyst skepticism remains existent. It first appeared in a Linux-based system, the DGX Spark (rebranded from its initial name, Project DIGITS), last October.

And now, Nvidia has determined that the RTX Spark is finally ready for Windows-based laptops (and not just from Microsoft itself, just as was the case two years before with Qualcomm). But not now. “This fall”. At a price to be announced later, but likely stratospheric if due only to the industry constraints-driven currently pricey “up to 128 GB of unified memory”. And what about the developer mini-desktop system, the Surface RTX Spark Dev Box, unveiled at Build?

There’s…umm…a waitlist. Microsoft CEO Nadella invited the Build attendees to join him on it. None of which inspires much in the way of confidence. Maybe one or both systems will be available for sale in time to end up on this November’s edition of my yearly “Holiday shopping guide for engineers”, but at this point, I’d be (pleasantly, mind you) surprised.

If you’re once again feeling déjà vu, by the way, it’s because Microsoft and Nvidia have been here before. The initial attempt at bringing a Windows-on-Arm system to market, the Surface with Windows RT, was based on an Nvidia Tegra SoC. I personally owned one and ended up tearing it apart after it eventually died. The hardware was first-rate for the time, although a dearth of native software in conjunction with woeful x86 code emulation support doomed it.

That was 2012. Jump forward again to the other, earlier-mentioned déjà vu moment, when Qualcomm’s announced partnership with Microsoft in 2024, and I feel compelled to point out that by no means is it seemingly deceased (or even on life support, for that matter). I recently acquired a gently used Microsoft Surface Pro 11 based on Qualcomm’s Snapdragon X Plus to replace my long-in-the tooth Surface Pro X.

The SP11 has 16 GBytes of RAM and a 1 TByte SSD and runs solely on its integrated battery all day with ease, even when emulating x86. Microsoft systems based on second-generation Snapdragon X2 Elite (and presumably also Plus) SoCs are seemingly coming soon. And on a similar note, Microsoft’s still churning out branded systems based on x86 CPUs, too, with most recent updates less than a month ago.

Agentic-centric O/Ss

One particularly memorable quote from Nadella in the keynote was the following:

“There’s a real platform shift. We’re moving from building operating systems, devices for apps, to agents.”

Indicative of this forecasted shift is Project Solara, explained in part by means of a conversation between Nadella and Qualcomm President and CEO Cristiano Amon.

At Build, there was also an Android-derived proof-of-concept demonstration showing agent-based interactions with (and between) a smart speaker with a screen, mobile devices, and intelligent ID cards. Google also spoke a great deal about agentic AI at its I/O developer conference two weeks ago; instead of repeating myself again, I’ll refer you to my coverage of that event for the background info if you need it.

Speaking of agents, Microsoft also announced Execution Containers, which keep agents from accessing unintended, critical regions of other agents and applications, the underlying operating system and system hardware. And for when you want to communicate with them, OpenClaw founder Peter Steinberger showed up on stage for introducing Scout, an OpenClaw AI Assistant gateway.

If you’re thinking it sounds at least something like Gemini Spark, which Google announced two weeks back, you’re not off-base. Remember my comments at the beginning of this piece about competing-event timing and ordering and effects on later-event messaging?

Homegrown models

Last but not least, let’s touch on an event topic that prompted the “AI Independence” title of this piece. In late April, OpenAI and Microsoft “redefined” their business relationship, which among other changes fundamentally freed both companies from the various exclusivity arrangements that had previously defined (and arguably dominated) it. While a “divorce” might be overstating the result (but then again, maybe not), a “softer” term such as “conscious uncoupling” wouldn’t be far off.

One tangible outcome of this redefinition was clearly evident this week, as Mustafa Suleyman, head of Microsoft AI, unveiled seven new homegrown AI models with capabilities spanning image, voice, and transcription functions, and claimed performance matching if not exceeding that of Google, OpenAI and other competitors’ models, both open- and closed-source. I was particularly interested in Suleyman’s declaration regarding MAI-Thinking-1, the flagship reasoning model, that:

“We trained it from the ground up on clean data, without distillation from third-party models.”

And with that, I’ll wrap up for today. As always, I welcome your thoughts in the comments on the topics I’ve covered here, as well as any others that might have caught your eye—Microsoft’s ongoing research work on quantum computing, for example, including the development of Majorana 2, the sequel to last year’s premier quantum computing chip from the company.

Next Monday, Tim Cook (along with, presumably, his CEO successor John Ternus) will hit the stage to kick off Apple’s yearly Worldwide Developers Conference (WWDC), completing the yearly big-tech-company developer conference triumvirate. I’ll see you back here then, if not before!

—Brian Dipert is the associate editor, as well as a contributing editor, at EDN.

Related Content

• Google I/O 2026: Agentic AI gets serious
• Microsoft Build 2025: Arm (and AI, of course) thrive
• Microsoft’s Build 2024: Silicon and associated systems come to the fore
• A holiday shopping guide for engineers: 2025 edition

The post Build 2026: Accumulating evidence of Microsoft’s AI independence appeared first on EDN.

Altera has partnered with Mercury Systems and VadaTech to expand its Agilex 9 FPGA ecosystem with COTS VPX boards for mission-critical defense platforms. These solutions integrate Agilex 9 medium-band Direct RF FPGAs into VPX architectures, including SOSA-aligned OpenVPX, to help defense customers accelerate time-to-market, reduce SWaP, and enable flexible software-defined RF capabilities.

The Agilex 9 FPGAs combine RF data converters, FPGA fabric, and high-speed transceivers into a unified, programmable architecture, enabling real-time processing of large volumes of RF data at the edge. This integration supports distributed, multi-domain operations that require rapid decision-making and adaptation to changing mission requirements. The devices deliver the bandwidth, performance, and I/O needed for demanding embedded applications such as adaptive radar, cognitive electronic warfare, and secure, software-defined communications.

Mercury Systems’ DRF5660 boards and VadaTech’s VPX540 boards with Agilex 9 Direct RF AGRM027 FPGAs are available for order today.

Agilex 9 Direct RF series

Altera

The post Agilex 9 FPGAs power COTS VPX boards appeared first on EDN.