The idea of an option is relatively simple. Basically an option gives you the right to buy (a call option) or sell (a put option) some good or security at a fixed price by some date in the future. A call is a bet that the price will go up. If you have an option to buy 100 shares of stock X at $100 per share and the stock hits $110 before the option expires, you score a profit of $1,000 (10 X $100) less the initial cost of the option. A put works the same but in reverse; you profit if the price of the stock falls below the option’s “striking price.”

While the concept is straightforward, the pricing of options, and decisions on when to buy or exercise them is anything but. Quantitative analysts, or quants, use statistical models to consider large numbers of possibilities over time and to find the likeliest outcome.

One of the oldest methods for assessing prices over time is the Monte Carlo simulation, a technique that requires generating large numbers of random data points following some distribution. According to TradingMarkets.com, the big French bank BNP Paribas ran up against capacity constraints running its Monte Carlo pricing models on a cluster of CPUs and moved to a CUDA-based GPU system to get higher performance.

Probably the best known approach to financial modeling is the Black-Scholes equation, for which two of its developers, Myron Scholes and Robert Merton, won the 1997 Nobel Prize in economics (the third, Fischer Black, had died in 1995.) Black-Scholes is a differential equation that assumes the security prices are a random walk, that is, that day-to-day price fluctuations are random.

OnEye, and Australian firm specializing in high-performance financial computing, found running a CUDA-based Black-Scholes model on an NVIDIA GPU produced results almost 700 times faster than running the same analysis on a 2.21 GHz AMD Athlon 64 X2 CPU alone. The GPU system was able to price over 4 billion options per second.

Another way to analyze options is to look at prices over time. In the binomial model, a price goes up or down by a fixed amount at each time interval with each direction of movement having its own probability. The result is a tree-like structure in which each level represents a time period and each leaf a possible price. Marayam Ganesan, Roger D. chamberlain. And Jeremy Buhler at Washington University in St. Louis ran a CUDA binomial model on an NVIDIA GPU. “We theorize an optimal speedup of 15× over a comparable parallel implementation for a problem size of 1000 time steps,” they wrote. “In general the expected speed-up is proportional to the square root of the problem size.”

The trinomial model is a little more complicated in that it lets a price go up, down, or stay the same at each time interval. Two French researchers, Gregoire Jauvion and Tuan Nguyen built a trinomial option pricing model for a system with an Intel quad-core 3.2 GHz Core 2 Duo CPU and an NVIDIA GPU. The CUDA code on the GPU ran nearly 32 times faster for a model pricing 64 options over 1,024 time periods.

One theme that runs through all the research reports of the use of CUDA for options pricing models is that getting the maximum performance gain requires careful planning to take advantage of the strengths and minimize the weaknesses of GPU processing. But beyond that, it doesn’t seem to have posed any particular challenges. As John nelson, who writes the Path Dependent blog on programming, complex systems, and trading wrote:

“I started learning CUDA yesterday; I wrote my first simple CUDA program today. The library does have a non-negligible learning curve, but it is not steep. It largely is a matter of learning the most efficient ways to work with CUDA (e.g. shared, local, or constant memory). Happily, this is an incremental process; You can learn to write bad yet working CUDA applications while slowly learning to write them better; And, as a bonus, even your bad code is likely to run laps around your CPU (for finance apps anyway.)"

This post is an entry in The World Isn’t Flat, It’s Parallel series running on nTersect, focused on the GPU’s importance and the future of parallel processing. Today, GPUs can operate faster and more cost-efficiently than CPUs in a range of increasingly important sectors, such as medicine, national security, natural resources and emergency services. For more information on GPUs and their applications, keep your eyes on The World Isn’t Flat, It’s Parallel.

Anjul Patney and Duane Merrill are two of this year’s NVIDIA graduate fellows. Patney. Patney is a third-year doctoral student at the University of California at Davis, working under John Owens, a specialist in general purpose programming on GPUs. “I have a strong interest in understanding the evolution of programmable graphics pipelines, a possibility recently enabled by the flexible CUDA, OpenCL and Direct Compute programming models..” Patney says. “I am interested in identifying both software and hardware principles that will define these pipelines in the future. In this pursuit I often run into some very intriguing problems in graphics and parallel computing. It's a lot of fun.”

His area of interest is graphics rendering. As he wrote in his research statement in applying for the fellowship: ”My research goal is to explore infrastructures that allow complex, dynamic, yet fully customizable data structures and algorithms to help build new rendering schemes. I wish to deploy them in an open-source library, to serve as an abstraction over which flexible graphics applications can efficiently interact with the underlying hardware."

Patney was first exposed to parallel processing techniques as an undergraduate at the Indian Institute of Technology, Delhi. “Early training in parallel and multicore programming is very important for interested undergrads,” he says.

Merrill is a doctoral student at the University of Virginia. Merrill is working on applying the techniques of GPU processing to some of the basic building blocks of software, things such as sorting, the efficient processing of lists, and graph problems that he describes as “the nuts and bolts of data-intensive computer science, yet are the hardest to make scream on the GPU.”

One of the challenges of GPU programming is that since it is relatively new—the first version of CUDA was only released three years ago—programmers have not had time to create the big libraries of pre-tested, ready-to-run code that can make software development must faster and easier—a situation Merrill hopes to help remedy. “We simply don't have many collections of reusable software components at our disposal, particularly at [the level] where the brunt of cooperative performance gains are derived,” he says. “This is the hardest software to construct, make fast, and get correct,” he says. ”This is why the prospect of writing extremely high performance, high quality code for GPUs is such a challenge.”

Merrill did his undergraduate work at Virginia, then spent several years in industry before returning to graduate school. In his undergraduate days, GPU programming “wasn’t on the radar screen,” but that has changed. “The experience is much different these days,” he says. “Many of the undergraduate computer science students in our department opt to take a parallel-computing elective that has units on SPMD [single-program, multiple data] computing. This course is usually instructed by my advisor, Andrew Grimshaw, and I teach the GPGPU unit, in which we use the CUDA framework exclusively.”

In addition to a $25,000 stipendfor the 2010-11 academic year, the fellowship also provides engineering and technical support to 9 graduate students at 9 different universities. “The relationship NVIDIA fosters with university researchers through its fellowship program provides a conduit for ideas and technology to flow between academia and industry,” says NVIDIA researcher and former NVIDIA fellow Jared Hoberock. “In addition to financial sponsorship, the fellowship affords student researchers a unique opportunity to cultivate a dialogue with the finest engineering minds in the industry .”

NVIDIA will be accepting applications for the 2011-12 graduate fellowships beginning in November. Details on the program and on application procedures are on the NVIDIA website.

This post is an entry in The World Isn’t Flat, It’s Parallel series running on nTersect, focused on the GPU’s importance and the future of parallel processing. Today, GPUs can operate faster and more cost-efficiently than CPUs in a range of increasingly important sectors, such as medicine, national security, natural resources and emergency services. For more information on GPUs and their applications, keep your eyes on The World Isn’t Flat, It’s Parallel.

We’ve written before about how GPUs are transforming the field of medical imaging. In the case of Paris-based startup UsefulProgress, GPUs are enabling a kind of real-life X-ray vision. From a single scan, UsefulProgress technology can produce a high-definition 3D digital anatomy that reveals the underlying layers of bones, vessels, tissues and muscles in a body. 

The imagery that the UsefulProgress technology produces is stunning. You don’t have to be a medical student to appreciate a detailed 3D fly-through of the human brain or skeletal system (see the UsefulProgress website for examples). Surgeons can use UsefulProgress’s images as a pre-operative dry run, while doctors can employ them for non-invasive diagnostics. Students at the University of Paris Descartes medical school use the technology to study the human body and learn surgical techniques.

At the Emerging Companies Summit, the head of UsefulProgress, Sylvain Ordureau, met with NVIDIA Vice President of Business Development Jeff Herbst to talk about the technology. The proprietary software-hardware solution works with medical imaging technology such as CT, MRI and X-ray. The UsefulProgress solution takes the hundreds of image “slices” produced from such scans and stitches them together into a 3D volume. NVIDIA GPUs and CUDA are used for the image processing as well as the image display. The high resolution (8000x8000 pixels) of the images and large file sizes make this the sort of computational problem ideally suited to parallel processing.

Doctors aren’t the only ones who want a way to peer inside the human body without performing surgery. Archaeologists who work with human mummies have the same need – in many cases their subjects are too fragile to withstand an autopsy. For instance, after scanning an enigmatic mummy at Paris’s Musée de l’Homme, the UsefulProgress technology can create the 3D images within seconds, allowing archaeologists to peer inside the skull cavity of the thousand-year-old subject, discovering clues to its origin and history.

Although medical imaging is its primary use case, UsefulProgress is finding other applications for its 3D volume rendering, including materials scanning, pharmaceutical research and even helping gemologists get a glimpse inside precious stones like diamonds before the stones are cut.

OLAP is a technique for taking a deep dive into a subset of what may be a very large database. Say, for example, you wanted to analyze sales by product, store location, and time of day. Data for those three variables are compacted into a “cube,” so called because each of the variables can be regarded as a dimension or axis and the entire data space can be visualized as a cube. In the real world, analyses typically involve more variables and the cube becomes an impossible to visualize hypercube of many dimensions.

The data compression is important because efficient processing requires that the data being analyzed be held completely in memory. Mattias Krämer, vice president for technology at OLAP software maker Jedox AG, says that multidimensional OLAP techniques can allow 20 GB of data to be compacted into a 2 GB cube, small enough to be stored in memory even on a relatively modest system.

Jedox, based in Freiburg, Germany, makes a set of tools called Palo Suite that, among other things, lets analysts run OLAP using familiar tools such as Microsoft Excel. The newest version of Palo, developed in cooperation with the University of Freiburg and the University of Western Australia, uses NVIDIA’s CUDA C to boost OLAP performance through GPU processing.

Credit: Jedox AG

The company explains the advantage of the GPU approach by analogy. Say you had to deliver newspapers to a large number of homes. You could use a truck and find the most efficient route to visit the houses one after another. Or you could use a fleet of bicycles, each delivering a paper to one house. The bicycles (GPU processor cores) are slower and less powerful than the truck (the CPU), but the combination of sheer numbers and the elimination of the need to find an optimal route makes this method more efficient.

Interestingly, Palo GPU is part of the component of the suite that runs on a server. Servers are typically equipped with the most minimal graphics systems. They often lack displays altogether since they are frequently administered remotely, and even when a local display is present, it is rarely used for anything more demanding than server administration or scanning logs.

But Palo GPU makes a case for equipping a server with one or more high-end GPUs exclusively to get the benefit of GPU parallel processing. In fact, a number of server makers just announced integrated CPU-GPU servers and blade systems using NVIDIA Tesla 20-series (Fermi) GPUs.

“Depending on the structure of the OLAP cube, the company has been seeing performance boosts of 40X to 100X, compared to CPU processing. As performance [has been] the key factor in business intelligence applications for years now, these are really good numbers,” Krämer says.

“The tremendous computing power of today’s GPUs is achieved by using an array of processor cores that outnumbers current CPU cores by almost two orders of magnitude,” Jedox says in a soon-to-be-published white paper. “The massive parallelism offered by a GPU has been used to solve many problems with speedups ranging from tens to hundreds compared to a single processor. The popularity of general-purpose computing on graphics processing units (GPGPU) has gained further momentum with the releases of programming interfaces such as NVIDIA’s CUDA C and OpenCL or ATI Stream SDK which allow programmers to develop algorithms for GPUs using common languages such as C with only minimal extensions. The CUDA framework in particular has led to a dramatic increase in applications implemented for GPUs. Apart from graphics applications, GPUs are nowadays utilized in many other areas of computing, such as physics simulations, protein folding, cryptanalysis, and many more.”

While calling CUDA “a very good development tool for our purposes,” there are some improvements Krämer would like to see. One is better support for the C++ programming language, the object-oriented extension of C. Another, more basic, change would be to make it easier for CUDA applications to run as “services” under Windows, an approach that makes more reliable and easier to administer. NVIDIA recently released a new Tesla Compute Cluster driver that addresses this issue for Tesla products.

This post is an entry in The World Isn’t Flat, It’s Parallel series running on nTersect, focused on the GPU’s importance and the future of parallel processing. Today, GPUs can operate faster and more cost-efficiently than CPUs in a range of increasingly important sectors, such as medicine, national security, natural resources and emergency services. For more information on GPUs and their applications, keep your eyes on The World Isn’t Flat, It’s Parallel.

Increasingly, that means the graphics systems of the computers at the heart of imaging technology end up doing double duty. High-performance graphics processors drive the super high-resolution displays that these devices demand. And the enormous processing power of those GPUs is used for the computational heavy lifting needed to create the images.

Computerized tomography (CT) is the oldest and still one of the most useful of the computerized imaging techniques. A CT scanner takes a series of X-ray pictures by rotating around the body. These pictures are then assembled to create an image of a cross-section of the body. A series of these slices can be used to construct a 3D image of a section of the body.

National Institutes of Health

This sounds simple, but it takes a tremendous amount of math to get from individual exposures to the reconstructed image. Specifically, it involves the computation of a large number of Fourier transforms whose effect is to use data about neighboring points to increase the information about each point in the image.

If you have been following this series, you know that this is the sort of computation that profits tremendously from parallel processing, since the computation for each point can be carried out independently of any others. The effect can be dramatic. Klaus Mueller’s RapidCT project at the State University of New York-Stony Brook found that using GPU processing could reduce the time needed for a CT scan reconstruction from 135 seconds to less than seven seconds. And that was on an NVIDIA GeForce 8800 GTX GPU, a fairly modest unit by current high-performance standards.

UC San Diego

X-rays, including CT scans, depend on the fact that certain tissues, such as bones, absorb far more radiation than others. Magnetic resonance imaging (MRI) depends of the different response of tissue types to strong magnetic fields and is more effective than CT in resolving soft tissue. Like CT, MRI depends of mathematics to reconstruct the data into an image. MRI math is a lot more complicated, but again, it can be set up for efficient parallel processing.

The Georgia Tech Center for Advanced Brain Imaging (CABI) uses Jacket software from Accelereyes, which supports GPU processing in MathWorks MATLAB. CABI uses functional MRI (fMRI) scans to observe brain activity as it happens.

The key challenge in analyzing the scans, is “segmentation,” the process of dividing the brain image into the tissues, such as blood vessels, grey matter, white matter and cerebro-spinal fluid. CABI uses Accelereyes Statistical Parametric Mapping for bspline interpolation, a mathematical technique that accurately fits smooth curves to data points.

“The main factor that makes b-spline interpolation work nicely with parallelization is that it can be performed independently over individual data points,” says Ani Dasgupta, a Geogia Tech graduate student at CABI. “So as opposed to a CPU performing interpolation one data point at a time, a GPU would take up a chunk of points and interpolate them in parallel, since the interpolation of one point does not depend on the outcome of others.”

Parallel processing on the GPU has resulted in a 3.6X speedup of segmentation time, compared with CPU-only processing on an Intel quad core Nahalem-class processor. More recent tests point to a speedup of up to 15X.

Dasgupta also says that GPU-based parallel processing has gotten much easier: “Another important factor in favor of this is the ease of programming with GPUs after the advent of CUDA from nVIDIA. Traditionally, solving non-graphics problems on the GPU used to be very difficult, involving treating non-graphics data like vertices or pixel data points and using complicated graphics APIs to process this information. But with CUDA, a person who knows the C language can get around with writing a program for the GPU. So things are now better and easier for scientists from other fields, should they wish to do such work, than it was before.”

Spencer Technologies

Ultrasound imaging uses high-frequency sound waves to create pictures of internal tissues. But again, it takes a lot of calculation to convert what is basically a pattern of echoes into a usable image. Spencer Technologies, another Accelereyes customer, uses Doppler ultrasound, which, like Doppler radar, uses frequency shifts to detect motion, to analyze blood flow in the brain or motion of the brain relative to the skull. Using an NVIDIA GeForce GTX 280 and Accelereyes Jacket software, Spencer was able to calcuate brain displacement 12 times faster than with CPU processing alone. Spencer think is will soon be able to get a 16X improvement.

The bottom line on all these projects is that GPU-based parallel computing is producing better medical images faster. And as the tools get easier to use, the benefits should spread.

This post is an entry in The World Isn’t Flat, It’s Parallel series running on nTersect, focused on the GPU’s importance and the future of parallel processing. Today, GPUs can operate faster and more cost-efficiently than CPUs in a range of increasingly important sectors, such as medicine, national security, natural resources and emergency services. For more information on GPUs and their applications, keep your eyes on The World Isn’t Flat, It’s Parallel.

Adobe Photoshop is an example of an image enhancement tool that many computer users are familiar with (and one that uses GPU processing when available to speed its often intense computations.) Photoshop, however, is primarily intended for photographers and other creative artists, while researchers generally need specialized, and often automated tools to extract the information they need.

Satellite view of NVIDIA’s Santa Clara, CA, campus in Google Earth

Intelligence work remains one of the important uses of image enhancement. The unmanned Predator unmanned aircrafts over Afghanistan and Iraq are flown by remote control from Nellis Air Force Base in Nevada, and depend on real time processing of images from on board cameras, satellites, and other sources to identify and verify targets. The capabilities of the U.S. National Reconnaissance Office’s Keyhole image intelligence satellites is classified, but it is generally believed that they are capable, with enhancement, of resolving features as small as a few centimeters (the U.S. government generally limits civilian satellite imagery to a resolution of one-half meter.)

Scientific research is also a huge user of image processing and climate change is an area of particular concern. Tracking the shrinking of polar ice caps and the extent of sea ice from space is relatively easy, but researchers are often interested in much more subtle alterations, such as the changes in species of trees and other plants that occurs with variation in temperature and rainfall. This requires much better resolution than spotting an iceberg. The mathematical techniques used to process these remote sensing images are computationally intensive and can benefit greatly from GPU computing techniques.

A number of image enhancement techniques can make the information hidden in satellite photos usable. The laws of physics limit how small an object can be resolved depending on the size of the lens or mirror and the distance to the target. But image processing can do better by bringing out features that the optics alone leave fuzzy. One way to improve on that is through edge detection, a mathematical technique that can be used to sharpen out-of-focus images by figuring out where sharp edges should be.

False color imaging is another technique that can be used to discover hidden data in images. Sometimes it is used to highlight features, such as differing types of vegetation, that would be hard to spot in a true color image, where all green plants, for example, look pretty much alike. A common use of false color is to capture data that would otherwise be invisible because it is captured outside the spectrum of visible light, usually in the infrared or ultraviolet.

Mort Canty is a physicist at the Jülich Research Center in southwestern Germany, where he does research in remote sensing and image processing, including geometric and radiometric corrections and map projections of satellite data. He has become a convert to GPU processing to get the maximum computational bang for the buck. (For the technically inclined, he mostly uses the IDL image processing language and ENVI software from ITT Visual Information Systems and it the author of Image Analysis, Classification, and Change Detection in Remote Sensing: With Algorithms for ENVI/IDL. He’s also been experimenting with Tech-X’s GPUlib as another way to run his computations on a NVIDIA GPU using CUDA.

ENVI used to process false color image of nonlinear clustering (Mort Canty)

“Recently I have been working with nonlinear image transformations, which are quite computationally intensive,” he says. “GPULib is giving me a tenfold speedup on an old NVIDIA GeForce graphics card (which I hope to replace soon with a Fermi GPU).” He gives the main credit to CUDA’s extremely fast execution of matrix operations: “For someone with limited time or programming abilities, a high-level package like GPULib can open a door to parallel processing which might otherwise have remained closed.”

With satellite data now available either free (from Google and Microsoft) or in specialized forms at low cost, image enhancement software proliferating, and GPUs providing massive processing power, the use of image enhancement to extract information from remote sensing data is likely to explode.

This post is an entry in The World Isn’t Flat, It’s Parallel series running on nTersect, focused on the GPU’s importance and the future of parallel processing. Today, GPUs can operate faster and more cost-efficiently than CPUs in a range of increasingly important sectors, such as medicine, national security, natural resources and emergency services. For more information on GPUs and their applications, keep your eyes on The World Isn’t Flat, It’s Parallel.

The shape of a molecule is determined by the interactions of the electrons in its constituent atoms, ultimately at the level of quantum physics. In the simple case of water, the physics causes the two hydrogen atoms to bond to oxygen at a 105° angle. Proteins, however, can contain thousands of atoms arranged in a helix that twists and turns into complex shapes.

As in so many other fields, the immense power of computers to complete in seconds. computations that would take human beings lifetimes has revolutionized chemistry, giving rise to the collaboration of chemistry, physics, mathematics and computer science known as computational chemistry. Pharmaceutical research, for example, used to be a hit-or-miss process testing thousands of chemicals for pharmacological effects, with far more misses than hits. Now researchers are more likely to figure out what sort of molecule they need, set out to design it, and then figure out a way to synthesize it.

For large molecules--and proteins and other biologically active molecules that are often very large--this becomes a daunting computational task. Fortunately, it is one that lends itself well to the efficiencies of parallel computing. Most of this work used to be carried out on supercomputers or custom-designed clusters of workstations and servers. More recently, the work is moving to massively multi-core graphics processing units, such as NVIDIA’s Tesla.

The results of this can be very impressive. TeraChem, from PetaChem LLC is a quantum chemistry software package optimized for GPUs. Running analyses of several molecules on a workstation with four Tesla GPUs, TeraChem performed 8 to 50 times faster than the widely used General Atomic and Molecular Structure System (GAMESS) software running on a cluster of 256 quad core CPUs. A quad Tesla workstation is hardly your garden variety desktop—the 240-core Tesla C1060s go for about $1,300 apiece—but the setup outperformed far more expensive and complex hardware.

Harvard chemist Alán Aspuru-Guzic is a convert to GPU computing. His quantum chemistry research group analyses molecultes using electron correlation. This approach requires solutions to Schrödinger equations, differential equations that describe changes in the state of the system over time. An exact solution to a Schrödinger equation requires knowing all possible quantum states at the same time, a complicated version of the famous mind experiment of Schrödinger’s cat, which may be alive, dead, or both inside a sealed box. That’s a problem that can only be solved on a quantum computer, a device that unfortunately exists only in computer science labs, and there only in a primitive and not very usable state. Eventually, quantum computers will be available to computational chemists. But, says Aspuru-Guzik, “it will take a decade, maybe two decades. It’s hard to predict.”

Lacking quantum computers, researchers have to settle for close approximations to the exact solutions, but even these require tremendous computational effort. “In the meantime we have the GPU,” says Aspuru-Guzik. “The GPU is a very attractive alternative because it is cheap. It’s the future of computing.”

Aspuru-Guzik’s toolkit includes Q-Chem 3.1, a commercial quantum chemistry program, and CUBLAS, a high-powered linear algebra system based on CUDA, NVIDIA’s technology for general computing on GPUs. They found that using GPUs to assist in the multiplication of large matrices, another job ideally suited to parallel processing, sped the task by a factor of 13 over the use of the CPU alone. Most of us, of course, will never try to figure out the interactions of electrons in a molecule, nor will we have any use for a quantum computer. But the result of this work is important to all of us because it means better understanding of basic chemical processes and ultimately such things as faster development of new drugs. You can even get involved through a Stanford University project called Folding@home. that uses idle time on thousands of computers to compute protein folding. And if your computer has a modern GPU, you’ll become part of the parallel revolution.

A tool like NVIDIA’s CUDA parallel computing model makes the GPU cores, up to 240 of them on the latest NVIDIA Tesla GPUs, available to programs. But to take maximum advantage of it, you have to be a skilled C or C++ programmer. The problem is that many of the people who would benefit most from high-performance computing are not software developers by profession. They write customized code out of necessity, but their primary work is in chemistry, geology, astronomy, physics or biology.

Tech-X Corp., a Boulder, CO, software and consulting company specializing in high-performance scientific computing, is working to change that. Its GPUlib is a tool that brings GPU-based computing into the high-level tools used by researchers, including ITT Visual Information Solutions’ IDL, Mathworks’ MATLAB, and that trusty old laboratory standby, Fortran.

“Parallel computing used to be a very elite field,” says Peter Messmer, vice president for space applications at Tech-X. “Few applications are designed to take advantage of it. GPU processing makes it much more mainstream.” Until GPU processing came along, the cheapest way to get very high performance in the lab was by building a cluster of relatively inexpensive PCs, but this took skills that researchers who weren’t computer scientists or electrical engineers often lacked. “The GPU makes it much more mainstream,” says Messmer.

GPU cores are best at vector processing, math in which large arrays of data are manipulated simultaneously, since that is what is needed for the GPU’s primary task of rendering graphics. That made Tech-X’s choice of working with IDL and MATLAB a natural, since these tools are already optimized for manipulating vector data. Typical uses include image processing for astronomical and remote sensing data and medical imaging.

Messmer says a major challenge is just getting researchers to try GPU computing. “People have heard a lot about GPU computing, but they are skeptical,” he says. “They remember the field-programmable gate array hype from a few years ago, where it turned out to be too complicated for people to do anything. GPUlib helps because it maps to how people already think about problems.”

GPUlib is free for academic use and $495 for commercial use. It is available for Windows, Mac, and Linux.

This post is an entry in The World Isn’t Flat, It’s Parallel series running on nTersect, focused on the GPU’s importance and the future of parallel processing. Today, GPUs can operate faster and more cost-efficiently than CPUs in a range of increasingly important sectors, such as medicine, national security, natural resources and emergency services. For more information on GPUs and their applications, keep your eyes on The World Isn’t Flat, It’s Parallel.

Some time around the middle of the last decade, the race to ever-faster computing hit the wall. Until then, designers had delivered soaring performance through three well-understood technologies: shrinking the already microscopic transistors, cramming more of them into each processor, and running them at higher speeds

The problem was that faster processor performance translated into higher power consumption and more heat, and even if you could find a way to get rid of the excess heat before the chips fried, continuation of the trend posed unacceptable economic and environmental costs.

An alternative route to faster computing had been around for some time. Instead of driving the processors harder, use more of them.  Mainframe computers and servers had long used multiple processors to handle heavy loads, but advances in chip technology made it possible to combine multiple processors on a single chip, an approach that is both more efficient and much cheaper. Today, high-performance computing is a story of dividing computational workloads over multiple processor cores. In the case of personal computers, this means both a handful of cores in the CPU and dozens, sometimes hundreds of cores in the graphics processing unit (GPU).

But multiprocessor hardware brings with it a significant software challenge. From the beginning of modern computing in the 1940s, programs had been designed to work sequentially. Funding, mostly by the Defense Advanced Projects Research Agency, produced some successes in systems with large numbers of processors designed to solve computations problems by breaking them into many pieces that could be run simultaneously, but these massively parallel systems never achieved commercial viability.

One reason is that most common computing problems, and the algorithms used to solve them, are not well suited to this sort of breakup. And sequential thinking seems to be wired into our brains. Neuroscientist Jill Bolte Taylor says the right hemisphere of the brain, which processes sensory signals, does parallel processing but the left hemisphere, which is responsible for analytic thinking, “functions like a serial processor.” For better or worse, programming is a left-brain activity.

The biggest mathematical impediment to parallel approaches is that many processes are recursive: each step depends on the result of previous steps. Consider the simple problem of finding the greatest common divisor of two integers. The standard method of doing this, the Euclidean algorithm, has been known for over 2,000 years and uses repeated subtraction.

For example, if you want to find the greatest common divisor of 2,987 and 1,751, start by subtracting 1,751 from 2,987. Repeatedly subtract the difference (switching the order if needed to prevent negative numbers) until the result is 0. In this case, the two numbers have a largest common factor of 103. It’s a beautiful and efficient process, but it is inherently sequential because each subtraction depends on the previous result.

The great exception to the dominance of serial thinking is graphics. A very simple, common, and typical need in graphics is the need to rotate an image. If you remember some trigonometry, you may recall a simple formula to rotate point counterclockwise through an angle & Theta:

The importance of this is that each point can be processed independently of every other point. If you had as many processors as points, the entire transformation could be computed in a single, massively parallel operation. And the same is true of many more complex graphics tasks.

The parallel-friendly nature of graphics work led to the early incorporation of a multi-processor architecture into graphics processing units (GPU). NVIDIA’s top-of-the-line Tesla GPUs currently feature 240 processor cores. While these cores are not as flexible as CPU processors, they excel at certain tasks, such as the vector operations that lie at the heart of many intense computational problems.

Software for effective utilization of large numbers of cores, both CPU and GPU, remains a challenge but things are getting better. NVIDIA helped lead the way with the CUDA parallel programming model which enabled general-purpose computation on NVIDA GPUs and with extensions to the C programming language that made the processors accessible.   Developers can thus program NVIDIA’s CUDA GPUs using languages such as C, C++, and Fortran via the CUDA toolkit and PGI’s CUDA Fortran compiler respectively and also using multiple driver-level APIs such as OpenCL and DirectCompute.

One of the biggest challenges facing software developers is that to get more performance for existing applications and to develop new more compute-intensive applications, they have no choice but to consider parallelizing their applications, whether they choose multi-core CPUs or many core GPUs.   Based on the last few years of development, the CUDA parallel programming model has established itself as an “easier” way to do parallel programming (it still isn’t easy, but CUDA does make certain things easier).  Also, GPUs can offer a tremendous performance advantage over CPUs and so combining these two elements offers developers a way to develop more innovative applications.

Breast cancer detection today relies on 2D imaging systems and invasive procedures like biopsy. The former is often imprecise due to various factors such as operator expertise, the latter is painful and often unnecessary. But because doctors don’t want to be wrong on something like cancer, they’ll act aggressively on anything that looks suspicious – even though 80% of biopsies come back negative. That’s a lot of unnecessary fear and distress for patients.

TechniScan’s solution uses 3D ultrasound to create a detailed picture that may help doctors in the process of finding and treating breast cancer. (This video gives a detailed look at how the system works.) Basically, the machine’s scanner rotates all the way around a patient’s breast, capturing a scan every 2 degrees, and then composites a detailed 3D image. Each image is around 8 to 9 million voxels (the 3D equivalent of a pixel) and requires more than 120 million Fast Fourier Transform (FFT) calculations to build.

Now comes the hard part – tackling a computational load of this size in a real-world hospital environment. For 3D ultrasound to make sense for hospitals, the process needed to be both cost effective and time effective.

TechniScan realized that – great as 3D ultrasound promised to be – it wouldn’t get broad traction unless calculations could completed within 30 minutes. That’s the magic number that allows a patient to get her results during her appointment and keeps the device running every hour.

Throw CPUs at this problem, and they could work – but not in the footprint, not at the cost and definitely not at the same speed as GPUs. TechniScan’s system uses two Tesla C1060 GPUs to process images in less than 30 minutes. It would take more than an hour for a quad-core CPU cluster to do the same job. For TechniScan – and for patients everywhere – that’s not a viable result.

TechniScan Medical Systems is in the process of applying for 510(k) clearance of the WBU System. The system is not cleared for sale at this time.