SieveAhl: A Simple CPU Benchmark Set for 68K Macs

SieveAhl is an implementation of the FPU-independent Sieve of Eratosthenes and a modified version of the FPU-dependent Ahl's Simple Benchmark for the 68K classic Macintosh line. It makes a small and (possibly) useful method for comparing performance between 68K Macintoshes (and potentially Power Macs), and runs on practically any Mac extant.

Probable unsupported minimum: Mac 512K or Ke (or original Mac 128K with upgraded RAM), System 1.1
Tested minimum: Mac Plus, System 1.1 (yes, really)
Recommended minimum: Mac Plus, System 6.0.8

Download from here: SieveAhl.sit (StuffIt archive, 46K)

Operation is fairly obvious. Run SieveAhl and select the desired benchmark (Sieve is considerably slower, Ahl is very fast). Choose the desired number of runs (500 is the default to which the reference standard was calibrated). Using more runs generates more accurate timings, which due to the timing method chosen, are only accurate to the jiffy (1/60th of a second).

In MultiFinder and Systems 7 through 9, the system appears to hang while the benchmark runs because SieveAhl will not yield the CPU to other tasks. This is intentional and normal, to ensure that the CPU is given a proper workout with no interfering activity. (Nevertheless, it wouldn't hurt to disable non-essential extensions and control panels beforehand to eliminate them as a source of interference.) Be sure to choose a number of iterations that will finish during this epoch, however. If you decide you want your system back, it is safe to force-quit SieveAhl.

It is strongly recommended that virtual memory be disabled, as it may cause an unpredictable performance hit.

The reference standard is "Mr. T" -- our token relatively vanilla Mac Plus (hi, Sellam!) with 2.5MB RAM and System 6.0.8, and the stock 8MHz 68000 CPU.

For how the benchmarks were implemented, there is source and discussion available.

Performance

Here are some results from my not-so-copious collection of 68K Macs, and a few "emulator" results for comparison. The Sieve benchmark seems less affected by non-CPU factors (OS, FPU, cache, etc.). One issue might be differences in the SANE library used for floating point math between system versions, not to mention the fact that SANE uses the hardware FPU when present. Cache also has a profound affect on Ahl score, probably for the same reason (accelerates the SANE code). Interestingly, the Daystar PowerCentral control panel seems to accelerate the IIci's FPU score even with no Daystar hardware present!

The IIsi+FPU, cacheless IIci and SE/30 were chosen because they are basically very similar architectures (68030 + 68882 FPU), differing significantly only in system speeds. Comparing the relative Sieve (and Ahl with all other factors remaining constant) percentages corresponds very nicely with proportional differences in the respective system's clock rates (the 25MHz cacheless IIci is 20% faster than the 20MHz IIsi+FPU, for example; the 20MHz IIsi+FPU is 25% faster than the 16MHz SE/30 -- do the math and see). This is a most gratifying demonstration of SieveAhl's validity.

System					Sieve% (runs)	Ahl% (runs)

Mac Plus, OS 6.0.8 (reference)		100% (500)	100% (500)

SE/30, OS 6.0.8				666% (500)	695% (2000)
SE/30, OS 7.1				667% (500)	726% (2000)
IIsi, OS 7.1				838% (500)	567% (2000)
IIsi + FPU, 8*24*GC, OS 7.1		839% (500)	948% (2000)
IIci, no L1, 8*24*GC, OS 7.1		1047% (500)	1175% (2000)
 + Daystar PowerCentral control panel			2012% (2000)
IIci, Apple 32K L1, 8*24*GC, OS 7.1	1051% (500)	1423% (2000)
 + Daystar PowerCentral control panel			2506% (2000)
IIci, Daystar '030 50MHz+FPU+32K L1, 8*24*GC, OS 7.1, PowerCentral
   control panel			2121% (500)	4122% (10,000)

vMac 0.1.9 on host Power Macintosh 7300+G3/500/1MB, Plus ROM
 + System 1.1				444% (500)	264% (500)
 + System 3.2				444% (500)	267% (500)
 + System 7.0.1				443% (500)	267% (500)
Fusion PC 3.0 on host AMD 5x86/133, DOS 6.22, IIci ROM, Fast Math ON,
   VESA refreshed video @15fps, OS 7.1	561% (500)	3299% (5000)
Built-in ROM 68LC040 emulator on host Power Macintosh 7300+G3/500/1MB
 + OS 9.1				39548% (50,000)	90802% (500,000)

Special Considerations on PPC and OS X

SieveAhl will run on PowerPC-based Macs and should run on OS X, but the choice of reference platform makes it obvious this is a 68K benchmark. Because the Power Macintoshes do not natively run 68K code, the extra overhead of the emulator built-in to the Power Mac ROM must be factored in. Thus, you're really testing the emulator *and* the CPU.

This becomes an issue in comparing Macs with different architectures, such as NuBus Power Mac vs. PCI Power Mac; Old World vs. New World; or even software with third-party 68K emulation acceleration such as Connectix Speed Doubler. In all of these cases SieveAhl's performance is affected by the efficiency of the emulator code as well as the system architecture, and values generated on one architecture may not be comparable to another or directly comparable to the reference. (It is known that since the majority of the PCI Power Macintosh line has the same ROM checksum, values obtained are comparable in this line because the 68K emulation code must also be identical. For example, the 7300 and 8600 both have a ROM checksum of $960E4BE9. The CopyROM utility will tell you the size and checksum of your system ROM.)

Furthermore, since OS X is preemptive, it defeats SieveAhl's attempt to hog the system to get as much CPU time as it can. Results may be skewed or inflated on OS X for this reason.

Cameron Kaiser