This morning, I woke up to find the following comment in the MATLAB® Newsgroup:

Over two years ago, MathWorks® started to build a clone of Jacket, which you now know as the GPU computing support in the Parallel Computing Toolbox (TM).  At the time, there were many naysayers suggesting that Jacket would somehow be eclipsed by the clone.  Made sense, right?

Wrong!  Here we are 2 years later and the clone is still a poor imitation. There are several technical reasons for this, but if you are serious about getting great performance from your GPU, Jacket is the better option.  Look at all the real customers that are getting big benefit. Here are some other recent benchmarks from the Walking Randomly Blog that show Jacket on a laptop is faster than PCT (TM) on a Tesla:

If it were easy to imitate Jacket, then MathWorks® would have siphoned away all the Jacket users.  The truth is that it is not easy to build great GPU software, and the Jacket user base continues to explode.  Jacket is not only better than PCT (TM) but is also getting better at a faster rate.  Here’s to another 2, 5, and 10+ years of great speeds for all of you Jacket programmers!

To Juliette and others out there, if you really want PCT (TM) to get better, you might consider asking MathWorks® to spending less time cloning and more time working with others who are adding value to the MATLAB® ecosystem.

Today was Day 1 of the Keeneland Workshop.  Many great talks were given, across a broad range of GPU computing topics.

With last week’s ArrayFire Webinar fresh in mind, it was interesting to see similar conclusions drawn in a presentation by Kyle Spafford of Oak Ridge National Laboratory.  Kyle independently ran a number of benchmarks over a period of time which show how quickly OpenCL has matured and where it yet has room for improvement.  The slide below comes from Kyle’s presentation.  For numbers >1, CUDA is faster.  For numbers <1, OpenCL is faster.  Performance in most cases is close to equivalent.

Just as we showed in the ArrayFire webinar, OpenCL performance is quite comparable with CUDA performance.  The Achilles heel for OpenCL right now seems to be in the FFT and a few other cases related to texture memory optimizations.

Many other great talks were given at the workshop. ArrayFire and Jacket were also covered in the library talks.

If you are looking at solving HPC type problems with GPUs, you should follow the activities of the Keeneland group.  These guys are leaders in GPU computing for HPC.

In case you missed it, we recently held an ArrayFire Webinar, focused on exploring the tradeoffs of OpenCL vs CUDA.

This webinar is part of an ongoing series of webinars held each month to present new GPU software topics as well as programming techniques with Jacket and ArrayFire.

For those of you who missed it, we provide a recap here. Lots of questions were fielded by our team, so it’s a must-watch. We hope to see you at the next one!

Recap

Download the slides.  Here is a transcript of the content portion of the webinar:

AccelerEyes is pleased to present today’s ArrayFire webinar looking at OpenCL and CUDA Trade-offs and Comparisons. Everyday, we interact with many programmers in various stages of GPU development projects. In making GPU project decisions, there is a lot of information to absorb from a variety of sources. The intent of this webinar is to condense this information into the key points that matter, to help you digest GPU computing software decisions.

Over the last 5 years, as we’ve collected information from our GPU computing customers, we’ve determined that there are 5 core GPU software features that programmers seek in a technology. These include:

• Performance:  This is the core motivation for GPU computing and relates to quality. “How good will my code end up?” is the question here. “How fast can I push it?”
• Scalability:  This is related to labor costs and quality of results. “If I develop my code on a workstation, can I launch it on a cluster without major headaches?” is the question here.
• Portability:  This is related to flexibility and costs, both labor costs and fixed hardware costs. It can also lead to a question of quality, and freedom to move code to better or newer hardware that emerges is important.
• Community:  This is a broad term meant to encompass other terms like support, logevity, commitment, etc. It is important that the technology platform has a lot of users and momentum. It is important that investments made today persist and pay dividends down the road.
• Programmability:  A sense of how much labor and effort are required to get good performance. This includes a sense of the software platforms maturity, including robustness to bugs and availability of functionality.

We’ve arranged this webinar to address these five GPU Software Features for the two major GPU computing platforms, CUDA and OpenCL. For each of these features, we will share some of our thoughts on how CUDA and OpenCL compare. Then we will conclude the discussion of each feature with some comments on how the feature relates to ArrayFire (those slides have an ArrayFire logo in the upper right hand corner of the slide).

The first feature is Performance. Both CUDA and OpenCL are fast, and on GPU devices they are much faster than the CPU for data-parallel codes, with 10X speedups commonly seen on data-parallel problems.

Both CUDA and OpenCL can fully utilize the hardware. They are both entirely sufficient to extract all the performance available in whatever hardware device. But we italicized the word “can” here, because the devil is really in the details. Performance depends upon a slew of variables, including hardware type, algorithm type, and code quality. It is nearly impossible to guess how much speedup you can extract from a piece of code. In our experience, nearly all science, engineering, and financial codes can get great benefit out of GPU hardware, but the big question is how difficult is it to transform your algorithm to realize those benefits.  We’ll discuss programmability later on.

We will present ArrayFire performance results in CUDA and OpenCL at the end of the webinar.

The second feature we’ll discuss is Scalability. Scalability can mean many things, so we’ve broken this discussion down into 3 kinds of scaling.

From Laptops to Single GPU machines, both CUDA and OpenCL codes scale without any code change. This is a very common use case. We see nearly half of GPU computing users leverage a laptop at some stage of GPU development and later move the code to a different “performance” hardware setup, like a workstation or cluster. Both CUDA and OpenCL make life easy for this use case.

From a Single GPU Machine to a Multi-GPU Machine, both CUDA and OpenCL require user managed code for low-level synchronization of communication between the multiple GPUs. This is a headache, but with patience, manageable by both CUDA and OpenCL.

From a Multi-GPU machine to a Cluster, neither CUDA nor OpenCL really offer much assistance. Rather, programmers tend to write their own MPI code that handles all the cluster communication and then use CUDA and OpenCL directly in each node.

With respect to Scability, there are some other interesting developments of note. The first is that there is some new technology in CUDA called GPUDirect that is aimed at reducing memory transfer overheads when communicating between multiple GPUs. It has optimizations to reduce overhead by allowing Peer-to-peer memory transfers between GPUs on the same PCI express bus. It also has optimization to reduce the overhead of moving data from GPU memory to a network interface card. This is certainly and interesting development, but it is too new for us to say if it is actually something people will use.

The second interesting development is in Mobile GPU Computing. OpenCL has quickly become the most pervasive way to do GPU computing on mobile devices, including smartphones and tablets. Companies like ARM, Imagination Technologies, Freescale, Qualcomm, Samsung, and others are all enabling their mobile GPUs to run OpenCL codes. There are more mobile devices sold each year than there are PCs, so this is a huge community that is beginning to put its support behind OpenCL. At AccelerEyes, we have done several GPU consulting projects on mobile GPUs and are believers that there is big benefit to accelerating apps, especially computer vision and video processing apps, directly on the phone or tablet.

In scaling from Laptops to Single GPU machines, ArrayFire’s just-in-time compiler automatically makes optimizations for the GPU type, without any code change. In this sense, both the CUDA and OpenCL versions of ArrayFire enjoy scalability here.

From a single GPU machine to a multi-GPU machine, ArrayFire has a big advantage. The ArrayFire deviceset() function makes mutli-GPU computing super simple. No need to mess with synchronization issues. ArrayFire automatically manages memory and queues up all GPUs in your system with a full workflow, ensuring good resource utilization.

From multi-GPU machines to clusters, ArrayFire is the same as CUDA and OpenCL. There is not really any added benefit and users write and manage their own MPI code.

The third feature is Portability. This is perhaps the most reconizable difference between CUDA and OpenCL. CUDA only runs on NVIDIA GPUs, while OpenCL is the open industry standard and runs on AMD, Intel, NVIDIA, and other hardware devices.

With respect to CUDA, there was a recent announcement at NVIDIA’s GPU Technology Conference in Asia that said CUDA would become more open, and the press carried it as saying that CUDA would become open source. This is definitely a step that GPU programmers are happy to see. But it remains to be seen what this actually means. There are two comments that we’ll make on this announcement:

1. From what we can tell, parts of the CUDA compiler will be open sourced to a limited number of groups. These groups will likely try to build compiler adaptations that enable CUDA code to run on other devices, if you use their compilers. From the announcement, it appears that the CUDA libraries, like CUBLAS and CUFFT, will not be open sourced. This is a critical disctinction, because as we’ll show later on, libraries are key.
2. Creating a compiler that can automatically generate tuned code for various hardware devices with very different architectures is extremely difficult. These kinds of projects continue to remain popular in hardcore academic research, but have yet to mature their way into actually widespread utility. Ocelot is an example research project in this category.

Also, with respect to portability, CUDA does not provide CPU fallback. Currently, developers using CUDA typically put if-statements in their code that distinguish between the presense or absense of a GPU device at runtime. In contract, with OpenCL, CPU fallback is supported and makes code maintenance much easier.

ArrayFire is fully portable. The same ArrayFire code runs on CUDA or OpenCL. The only difference is the version of the ArrayFire library that you link against in your code.

The main caveat here is that today, the OpenCL version of ArrayFire only supports a subset of the functionality available in the CUDA version. This is due to the fact that our CUDA code base has been around much longer. It is also due to the fact that there is less of an OpenCL software ecosystem, as we’ll discuss next.

The fourth feature is Community. This is the feature that encompasses support, longevity, commitment, etc. As those things are hard to measure, we put together a proxy. It is interesting to look at the number of Forum topics on NVIDIA’s CUDA forms at nearly 27 thousand and AMD’s OpenCL forums at 4 thousand. Also, on a neutral 3rd party site stackoverflow (btw if you don’t you stackoverflow, you should) has tags for CUDA and OpenCL, with the number of CUDA tags being over 3X the number of OpenCL tags. As you would expect, there are many more people doing CUDA programming today due to the great investmenet NVIDIA has put into building the ecosystem for GPU computing.

With respect to AccelerEyes, we have over 14 hundred GPU topics on our forums, which is the largest community of GPU programmers supported by any software company. The next largest is the veteran HPC company, PGI, which has 485 topics on their GPU forums.

Community also has to do with ecosystem and other tools. We will cover a discussion of those as they relate to libraries in a moment.

The fifth and final feature is Programmability. Both CUDA and OpenCL are low-level. It is time consuming to do GPU kernel development in either of those platforms. The bulk of that time is often spent in redesigning algorithms to exploit data-parallelism.

This is why the entire GPU computing market has lately shifted a major focus towards programmability.

To understand the landscape, let’s look at this simple two-by-two, where we have Faster vs Slower technologies on the y-axis, and time-consuming vs easy-to-use technologies on the x-axis. As a baseline, you can consider SSE or AVX instructions on the CPU as something that is time consuming to write and won’t end up giving you the data-parallel performance that you can expect out of a GPU.

Writing GPU kernels in CUDA or OpenCL leads to much faster code, but is likewise very time-consuming to develop.

In the opposite corner, compiler directives have recent become popular. The claim of these is that you can sprinkle a few pragmas into your code and that the compiler will figure out how to get the code to run well on the GPU. While in some simple cases you might get a little benefit, there is simply no compiler today that is capable of automatically generating good, fast code for GPUs from standard serial CPU code. Compilers simply can’t figure out how to morph serial algorithms into data-parallel algorithms.

Which is why libraries are so key to GPU computing. In a library, you get access to a set of functions that have already been hand-optimized and tuned to exploit data-parallelism. Libraries include within them the benefits of speed that come from writing kernels. But they are also written with ease-of-use in mind and merely require a similar level of intrusion as is required by the compiler directive. This is why ArrayFire has been and continues to be so successful.

Libraries really make all the difference in GPU computing. To compare and contract CUDA versus OpenCL, it is important to look at the comparison of libraries available. Raw math libraries available in CUDA include CUBLAS, CUFFT, CULA, and Magma. These are pretty much complete providing the majority of all routines necessary for dense matrix operations.

CUDA also has CUSPARSE which is a good start for sparse linear algebra routines, but still needs to mature.

CUDA libraries only run on NVIDIA GPUs. NVIDIA does not provide libraries for OpenCL.

Raw math libraries in AMD’s OpenCL have matured a lot recently. With clAmdBlas and clAmdFft, you get most of the important Blas routines, radix 2, 3, and 5 FFT routines (which covers the most common cases). There is no LAPACK function support and there are no sparse data support libraries.

But a very important point is that AMD’s libraries run not only on AMD devices, but on all OpenCL-compliant devices, including NVIDIA GPUs.

Due to these developments at AMD, AccelerEyes is proud to have recently supported OpenCL in both our Jacket product (which applies to MATLAB code) and our ArrayFire product (which applies to C/C++, Fortran, and Python).

Our OpenCL support is new and not nearly as mature as our support of CUDA. But our initial OpenCL support is better than our initial CUDA support was when we first launched our CUDA products. And we expect OpenCL to continue to mature rapidly in the near future.

This concludes the slide portion of the presentation. In what follows, we will spend some time showing benchmarks of OpenCL versus CUDA in ArrayFire code, with a particular focus on the raw math libraries we just discussed.

Checkout the video below for the benchmarks and Q/A session.  Enjoy!

Video

The question above comes from María (@turbonegra).  She follows us @accelereyes.  Many of you are wondering when ArrayFire support for new CUDA version 4.1 will be released.  The answer: work is currently under way.

CUDA 4.1 includes a new Fermi compiler, and many people in the GPU ecosystem have reported slowdowns from upgrading to the new CUDA version. So we’ve delayed releasing ArrayFire and Jacket support for CUDA 4.1 because we want to verify performance and reliability across all our unit tests, performance regressions, and customer code samples.  Our tests sweep across various driver versions and everything from mobile GeForce cards through server-grade Tesla and Fermi chips.

We are still working through the testing and verification at the moment. While NVIDIA’s general claim of 10% performance benefit are a bit fluffy, things do look good overall, and we believe CUDA 4.1 is ready for prime time. Various FFT, BLAS, and LAPACK routines show solid performance enhancements. In the few places where we have observed a performance degradation (e.g. linear texturing), we are working closely with NVIDIA to ensure these are addressed in coming driver and toolkit releases.

We’ll post here and on twitter when CUDA 4.1 support ready in ArrayFire and Jacket. Stay tuned!

PS:  Thanks to María for prompting this post

In case you missed it, we recently held a webinar on how to accelerate common medical imaging applications using an easy, powerful programming library with Jacket for MATLAB®.

This webinar was part of an ongoing series of webinars that will help you learn more about the many applications of Jacket and ArrayFire, while interacting with AccelerEyes GPU computing experts.  Gallagher Pryor, CTO of AccelerEyes, used the Bayesian Image Segmentation algorithm as a simple use-case to show how easy it is to convert CPU code to GPU code with Jacket (only 4 lines of CPU code needed to be changed!).

For those of you who missed it, we uploaded the webinar on Youtube. We hope to see you at the next one!

We recently reported that Jacket could be used over Windows Remote Desktop connections as long as you had an NVIDIA Tesla device in TCC mode. With the latest NVIDIA driver updates, Tesla and Quadro devices can be put into TCC mode, making it possible to use Jacket over Remote Desktop with both Tesla and Quadro devices.

We have tested this out with the NVIDIA Quadro 4000 as well as Quadro 6000 GPUs. The system had a Tesla C2050 connected to the display, and the Quadro in TCC mode. Here’s the ginfo output:

>> ginfo
Jacket v2.0 (build 80c7ba4) by AccelerEyes (64-bit Windows)
License Type: Designated Computer ([JACKET_ROOT]\jacket\engine\jlicense.dat)
Addons: MGL4, JMC, SDK, DLA, SLA
CUDA toolkit 4.0, driver 285.62
GPU1 Quadro 4000, 2048 MB, Compute 2.0 (single,double)
Memory Usage: 1977 MB free (2048 MB total)

Jacket over Remote Desktop is documented extensively on the AccelerEyes Wiki. Please check that page for more information.

AccelerEyes invites you to participate in series of webinars designed to help you learn more about Jacket for MATLAB® and ArrayFire for C/C++/Fortran/Python, a comprehensive library of GPU-accelerated functions.

GPU Programming for Medical Image Segmentation: January 18, 2012 at 3:00 p.m. EST

There’s a huge volume of data generated using acquisition modalities like computer tomography (CT), magnetic resonance imaging (MRI), positron emission tomography or nuclear medicine. A common need is to manipulate and transmit this data using compression techniques in as little time as possible. During this webinar we will show Jacket’s superior speed and handling volumes from subscripting to convolutions.  Come and learn how to accelerate common medical imaging applications using an easy, powerful programming library with Jacket for MATLAB®.

OpenCL and CUDA Trade-Offs and Comparison: February 15, 2012 at 3:00 p.m. EST

The OpenCL standard continues to mature and is now (or soon will be) supported by a variety of GPUs and manycore processors. At AccelerEyes, we remain at the forefront of OpenCL development. ArrayFire OpenCL is a fast software library for GPU computing with a simple API.  In this informative webinar, our team of GPU experts will discuss OpenCL and CUDA trade-offs and comparisons.  In addition, you’ll get to see ArrayFire OpenCL in action with real code.

GPU Programming for Financial Computing:  March 15, 2012 at 3:00 p.m. EST

Quantitative analysts are discovering the benefits of leveraging GPUs in tackling complex financial computing models. Using Jacket’s computational horsepower, analysts can employ a variety of functions to achieve speedups in trade signal generation, complex derivative pricing, evaluating risk scenarios, and more. In this webinar, we’ll discuss the latest developments in GPU programming for financial computing.

Each webinar will be conducted by AccelerEyes’ team of GPU computing experts and will include live demos of Jacket and ArrayFire.   We hope you will join us as we discuss exciting developments in GPU computing software!

One of the biggest areas where GPUs are providing benefit is with scientific computing. With libraries like Sage and SciPy providing a huge collection of functions and algorithms for free, Python has become one of the favorite tools for developers around the world. Even though these libraries have C/C++ back-ends, performance on large problems quickly becomes an issue and can kill productivity.

On the heals of our free release of Arrayfire C/C++, we’re excited to release ArrayFire Python. All of this is FREE for most users (see below for clarification)!

The structure of ArrayFire/Python is loosely based on NumPy in that it uses a single array object that can contain multiple data types. You can convert NumPy arrays to ArrayFire arrays and vice versa. If you already have your application using NumPy arrays, this is a quick way to jump in and tweak critical sections.

import numpy as np
import arrayfire as af
a = np.random.rand(5,5)
b = af.array(a)
c = b.host() # c is the same as a

Alternatively, you can generate data on the device:

r = af.randu(5, 5)
o = af.ones(5,5)
z = af.zeros(5,5)

Once you have the data you need, you can utilize hundreds of functions to convert your code entirely onto the GPU. You’ll find much of the API follows directly from NumPy itself.

Screenshot taken while running raindrop example

Jumping right in, here is an example showing the Monte-Carlo calculation of pi

from arrayfire import *
def pi(samples=20000000):
    x = randu(samples, 1)
    y = randu(samples, 1)
    return 4 * sum(mul(x, x) + mul(y, y) < 1) / samples

You can visit our website to download the latest version of ArrayFire. You can find the Python wrapper in arrayfire/python directory. Installation instructions are in README, and a few examples are included that show off both compute and visualizations.

Our Forums are the best place to get the latest info and help.

AccelerEyes provides this software for free in the hope that some of you might be interested in hiring us to port your code to the GPU.  If that is interesting, let us know!

* ArrayFire is free for use on a single GPU.  To run ArrayFire on larger hardware systems, contact sales@accelereyes.com.

New Multi-GPU functionality , added support for OpenCL devices, and much more…

AccelerEyes announces the release of Jacket version 2.0, adding GPU computing capabilities for use with MATLAB®.  Version 2.0 delivers even more speed through a host of new improvements, maximizing GPU device performance and utilization.

Notable new features include a multi-GPU interface and support for OpenCL devices. With Jacket v2.0, your M-code is now portable across all major GPU devices, including AMD/ATI, Intel, and NVIDIA chips.

Jacket is the premier GPU software plugin for MATLAB®, better than alternative solutions.  It is relied upon by thousands of organizations for rapid prototyping and problem solving across a range of government, manufacturing, energy, media, biomedical, financial, and scientific research applications.

Multi-GPU Details:

• Control over all GPUs in your program through simple, fast GPU selection functions.  Jacket automatically handles communication between the GPU devices, without the need to launch bulky parallel computing workers
• GINFO, GSELECT, GSYNC all extended to handle multiple devices

OpenCL Details:

• Supports single precision, floating point, real, and complex types
• Supports array math, FFTs, element-wise operations, and more
• Selection of any OpenCL compliant device listed in GINFO
• Currently available as a FREE beta feature, download now

Other Notable Improvements:

• New Base Jacket Functions, such as MEDIAN and PROD
• Additional Image Processing Library functions
• Additional Statistics Library functions
• Additional Signal Processing Library functions
• Support for binary to decimal conversion with BI2DE, DE2BI
• New Demos (included in every download):
  • Defense Optical Flow Tracking example, Music Visualizer example, and new Jacket CPU v. GPU demo
• Financial example of Black-Scholes with GCOMPILE is 35X faster than CPU
  

Visit our company website and see the v2.0 release notes for the full list of enhancements.

Pricing and Availability

Jacket v2.0 is available for download on the AccelerEyes website.  Pricing for a Jacket base license with support for a single GPU is $999.00 USD for commercial and $350.00 USD for academic customers.  AccelerEyes provides 12 months of software maintenance and updates with each software license.  Volume packages and development bundles are also now available at special price points.

Try our Professional Services

AccelerEyes provides professional GPU consulting services.  Our team of engineers guarantees great results from GPU computing.  Equipped with Jacket and years of experience, our experts deliver results in fewer hours than any other consulting firms.  Set up a free GPU consultation today.

Lenovo and AccelerEyes have a joint solution for optimizing M code on Lenovo workstations.  The combined HPC solution combines high Intel Xeon CPU performance for daily productivity with unprecedented NVIDIA graphics (GPU) performance for parallel computing with Jacket. Jacket’s comprehensive benchmark suite, when run on Lenovo ThinkStation systems, shows tremendous amounts of speedups for a wide variety of computationally-intensive applications.