blog

Filtering GfxBench2D Results

Tue, 04 Feb 2014 18:09:24 +0000

The number of GfxBench2D results has been steadily increasing over time, to the point that sifting through the results is getting a bit unwieldy. It has always been the intention to add ways to filter and analyse the data, but lack of time has prevented these from materialising.

Well, I'll let you in on a little secret: it's possible to filter the GfxBench2D results by motherboard, CPU, and graphics card. Why is it secret? Because the user interface (UI) isn't done. The actual functionality has been there for a while.

Since the UI still is nowhere near done, and the number of results keep rising, here are some quick links to filtered data for the AmigaOS results:

Filter by motherboard:

A1200 Blizzard PPC results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/BlizzardPPC-%7C-A1200/
A4000 CyberStorm PPC results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/CyberStormPPC-%7C-A4000/
AmigaOne XE/SE results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/AmigaOne/
AmigaOne-X1000 results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/AmigaOne+X1000/
Pegasos II results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/Pegasos+II/
Sam440 EP/flex results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/Sam440EP/
Sam460ex results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/Sam460ex/

Filter by CPU:

IBM PowerPC 750 CXe results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/CPU/IBM+PowerPC+750+CXe/
IBM PowerPC 750 FX results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/CPU/IBM+PowerPC+750+FX/
IBM PowerPC 750 GX results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/CPU/IBM+PowerPC+750+GX/
Motorola MPC 7441/7451 Vger results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/CPU/Motorola+MPC+7441-%7C-7451+Vger/
Motorola MPC 7445/7455 Apollo results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/CPU/Motorola+MPC+7445-%7C-7455+Apollo/
Motorola MPC 7447/7457 Apollo results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/CPU/Motorola+MPC+7447-%7C-7457+Apollo/

Filter by graphics card:

Elbox Radeon 9250 PCI results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/GraphicsCard/25/
XFX Radeon HD 4670 results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/GraphicsCard/105/
Sapphire Radeon HD 6570 1GB Ultimate results: hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/GraphicsCard/248/

Graphics cards are filtered by ID because the names aren't a reliable reference. This does mean that this filter won't be of much use until the UI is available

It is also possible to combine filters. For example, the following link shows results for Pegasos II machines with a Motorola MPC 7447/7457 Apollo CPU:

hdrlab.org.nz/benchmark/gfxbench2d/OS/AmigaOS/Motherboard/Pegasos+II/CPU/Motorola+MPC+7447-%7C-7457+Apollo/

As said earlier, adding the filtering UI is still planned, as are more methods of analysing the results. In the meantime, the links above should help narrow down the results.

P.S. If you want to stay up-to-date with what's happening here on HDRLab, follow me on Google+.

GfxBench2D for AmigaOS Version 2.8

Tue, 19 Jun 2012 02:09:19 +0000

I hadn't expected to release another update so soon, but GfxBench2D version 2.6 had one puzzling problem: the altivec based copy routines actually caused a significant drop in performance on G4 AmigaOne-XEs (A1-XEs). What made this puzzling, was that the altivec copy routines improved performance on G4 Pegasos II machines. Both of these machines use MPC74xx CPUs, so what could be causing this strange behaviour? This was a bit of a mystery.

Today, that mystery is mystery no more.Section 7.3 of the MPC7450 Reference Manual states:

AltiVec loads and stores—The 60x bus protocol does not support a 16-byte bus transaction. Therefore, cache-inhibited AltiVec loads, stores, and write-through stores take an alignment exception. Note that AltiVec loads and stores that are not quad word–aligned also take an alignment exception in MPX bus mode. If either of these conditions is encountered, a re-write of the alignment exception routines in software is required.

As it turns out, the A1-XE uses the 60x protocol, whereas the Pegasos II uses MPX bus mode. Hence, the altivec copy routines worked perfectly on the Pegasos II, but triggered an huge stream of alignment exceptions. AmigaOS' exception handler ensured that the data being copied still got delivered, but the overhead resulted in much poorer performance. Thanks goes to Jörg Strohmayer, for figuring out what was going wrong.

Version 2.8 works around this problem by disabling the altivec routines for all old AmigaOnes (i.e., the A1-XE/SE and the Micro-AmigaOne).

GfxBench2D Client Updates for AmigaOS and Windows

Sat, 16 Jun 2012 02:48:25 +0000

Both the AmigaOS and Windows versions of GfxBench2D clients have been updated. There was a minor issue in which a graphics card whose name contains particular characters would fail to save. The reason is that files cannot use certain characters, and so attempting to save a file with say, a slash '/' character (which is used to separate directories in a path) would not work. This issue has now been fixed.

The AmigaOS version also updates the RAM-to-VRAM and VRAM-to-RAM tests to to take advantage of the altivec unit in processors that have one. Altivec (a.k.a. VMX or Velocity Engine) is a vector processing unit which can perform operations on multiple numbers (i.e., vectors) simultaneously. The x86 equivalent are the SSE extensions, of which there are multiple versions. It just so happens that the altivec unit can also be used to speed up non-DMA copies.

How Altivec Speeds up Transfers to/from VRAM

Why would using altivec help? Well, computers tend to be more efficient at transferring data in large blocks instead of individual bytes, or even individual 32-bit words (i.e., 4 bytes). In the case of PCI-Express, there is a huge overhead for transferring data in tiny blocks. According to PLX Technology, efficiency drops to 60-70% when transferring blocks of 64 bytes, and drops off sharply below that (source). This puts CPU-based transfers at a significant disadvantage, unless the PCI-Express controller has means of merging multiple writes/reads into larger blocks.

Altivec vectors are 128-bits long, or 16 bytes. This main PowerPC arithmetic unit works on numbers up to 64 bits (8 bytes). So, when altivec reads or writes data to/from VRAM, it requests data in blocks twice the size that non-altivec reads/writes do. The larger block size results in more efficient data transfer.

On the AmigaOne-X1000, this difference results in approximately double-the transfer rate in either direction. To see the difference, compare this result from version 2.5, with this result from 2.6. Version 2.6's Copy to/from VRAM results are roughly twice that of 2.5, and four times the speed that WritePixelArray()/ReadPixelArray() currently manage.

GfxBench2D Version	RAM to VRAM (MiB/s)	VRAM to RAM (MiB/s)
2.6	446.85	24.18
2.5	223.28	12.28

it is still a fraction of the speed that is possible with DMA (support for which is still to-do in the RadeonHD graphics driver), but it is a big improvement over the non-altivec copy routines.

EDIT (2012/06/18): It appears that the new improved code is significantly slower on AmigaOne-XEs with 74xx processors. On other machines though, the new code does improve performance of the RAM To VRAM test. I have decided that I will not add special code that is specific to one particular motherboard and CPU combination. Developers may find these results of use, as they try to optimise their code to work well on as many machines as possible.

EDIT 2 (2012/06/19): After discovering the cause of the problem, I decided to release an update that works around the issue after all.

Developers Please use WritePixelArray()/ReadPixelArray()

This might come as a shock given the results above, but I strongly urge developers to use WritePixelArray()/ReadPixelArray() for transferring data between RAM and VRAM. There are two reasons for this. Firstly, the same altivec enabled algorithm that GfxBench2D 2.6 uses will make its way into a future version of AmigaOS 4.x (hint: I used GfxBench2D as a test-bed for developing copy routines). More importantly, WritePixelArray()/ReadPixelArray() use DMA on the Sam460ex, and this will eventually be expanded to other platforms too. DMA is the only way to achieve high efficiency transfers with PCI Express, which you will not be able to access directly from your applications. There is no way that a CPU-based copy routine can come anywhere close to the performance of DMA. If you insist on using your own routines, then at least include a speed comparison test into your software, and use WritePixelArray()/ReadPixelArray() when it is faster.

Old versions are Banned

Since the results of one of the tests has changed in the AmigaOS version, results are no longer comparable between version 2.6 and previous versions. Hence, results from all previous versions of GfxBench2D can no longer be uploaded to the server. Results from old versions are still available, but have been marked with an exclamation mark and a warning.

GfxBench2D: Botched Release

Wed, 25 Jan 2012 14:34:09 +0000

Given the complexity of the release process, I guess that it was inevitable that I would forget something with one of the releases. GfxBench2D versions 2.3/2.4 for AmigaOS and 2.7 for Windows were unable to upload results due to me forgetting to update the login details. More importantly, I forgot to test that functionality before releasing it, thus failing to notice the mistake. This has now been fixed with version 2.5 for AmigaOS, and 2.8 for Windows. My sincerest apologies for the inconvenience.

So, please download the latest version.

GfxBench2D Gets an Icon

Thu, 12 Jan 2012 18:00:52 +0000

Both the AmigaOS and Windows versions of GfxBench2D have been updated to add an icon. This is something that I always wanted to add, but lacked the time (and I'm not a graphic artist). Anyway, thanks to Thomas Blatt - who created the icon - this is now a reality.

GfxBench2D for Windows 2.6

Mon, 10 Oct 2011 02:09:18 +0000

The Windows version of GfxBench2D has received another update. This version includes a workaround for an issue with buggy graphics drivers. On a small number of computers it would hang at the WritePixelArray test. This was due to a driver bug affecting the CopySubresource() function. It would allow the source-buffer to be locked for writing before copying of data from that buffer to the graphics card had completed, potentially enabling the old data to be overwritten before it had been used. Fortunately, there was a different method to wait until the operation was done, which waits for the GPU to be idle. This same method is used prior to each test to ensure the most accurate results.

The updated version can be found at the usual place.

Minor GfxBench2D Update

Mon, 19 Sep 2011 02:11:31 +0000

A minor update to GfxBench2D has been released for both AmigaOS (now at version 2.2) and Windows (now at version 2.5). This is a purely cosmetic update, and does not affect the benchmarking process in any way.

This update adds a user-agent string to the upload client, and adds the word "recommended" to the upload requester. The user-agent string is something that the upload client passes to the server to identify itself. Previously this was left blank. While this makes no difference operationally, it is something that should be filled in. The change to the upload requester's text is there to (hopefully) encourage people to upload their results. I have noticed a number of downloads that did not result in new results being posted. While uploading is 100% optional, the more people who upload, the more data there is for everyone to compare performance with.

GfxBench2D for Windows Updated to Version 2.4

Sat, 03 Sep 2011 19:41:54 +0000

GfxBench2D for Windows is now at version 2.4. It is available at the usual place, i.e., the download page. This version fixes a few bugs, the most important of which is a bug that prevented certain multi-core machines from running the tests. If you find any bugs, please contact me via email, or post to the GfxBench2D forum.

GfxBench2D updates for AmigaOS and Windows

Mon, 22 Aug 2011 03:06:58 +0000

Both the AmigaOS and Windows versions of GfxBench2D have been updated, and are available for download. Some bugs were found in the Windows version over the last few days, resulting in version 2.2. At this stage the Windows version is still considered to be beta. The AmigaOS version is now at 2.1. It incorporates some bug-fixes and changes that were primarily the result of developing the Windows version.

Update 2011/08/23: The Windows version is now at 2.3 after another minor bug was found.

GfxBench2D for Windows

Thu, 18 Aug 2011 03:25:01 +0000

I am please to announce that the Windows version of GfxBench2D is now available. Windows XP or higher is required. The website code has also been updated in order to accommodate multiple Operating Systems (OSes).

If any developers are interested in helping to create a version of GfxBench2D for other OSes, please contact me.

GfxBench2D the First Release

Sat, 18 Jun 2011 20:26:03 +0000

Welcome to yet another development log. This post marks the first release of the GfxBench2D benchmark tool, and the corresponding web application (web-app). The benchmark tool tests the performance of graphics cards for a range of different operations, and (optionally) uploads the results to this website for display. While it is still a work-in-progress (isn't most software?), it has reached a point at which it is useful.

The original idea for this project was a result of my Radeon HD driver for AmigaOS 4.x project. Graphics card driver writers are particularly interested in how fast the graphics card can perform operations. So, I started writing a 2D graphics benchmarking tool so that I could measure performance, identify any areas that need work, and monitor progress as the driver is developed further. This helped me track down the PCI-to-PCI bridge initialisation problem that was seriously hampering performance on Sam440 machines (you will find details about this here, on the last paragraph in the section called "The Long Road to Here").

While the early version of the benchmarking tool was helpful, collating results and generating graphs manually was rather tedious and time consuming. I wanted to get results from more graphics cards, and others were asking for benchmark results too. So, I thought about how I could automate the process and enable other people to send results quickly, and the GfxBench2D web-app was born. A web-API was developed to upload results, and an upload client was added to the benchmark tool. At this point I also added the tests that I still wanted it to perform.

As mentioned earlier, this is still a work-in-progress. In particular, there are more features that I would like to add to the website for displaying and comparing results. It is also planned to port the benchmarking tool to other operating systems too. In the meantime, I invite you to check out the results that have already been submitted, and/or download the benchmarking tool to try out on your own machines (AmigaOS 4.x only at the time of writing).