[HECnet] Benchmarks - WHETSTONE.C

[Clem Cole]

Mark - that is the theory, but in practice it is not going to tell you much that you can really take to the bank.    Why?   Because Whetstone is a poor benchmark -- the results vary too much depending on compiler, language implementation, architecture etc -- its too easy to game.    please do some goggling -- around 1985 it fell out of favor because it as so meaningless.

John Mashey (then late of PWB and at the time ??CTO?? of MIPS) wrote a great piece which he cribbed the name from  Benjamin Disraeli  called:   "Lies, Damned Lies and Benchmarks" - IIRC whetstone was one of the benchmarks he  rips apart in that paper.   John and others really set out to create a set of benchmarks that could work regardless of compiler and architecture that told you something reasonable about the system.    That group would form SPEC.


So, if you really want to do this type of analysis, I think for the effort you suggesting, it be worth the extra time to dig up a copy of specint and run that not whetstone.    What is cool is that all the manufacturers published their spec numbers so you can see what you should expect and what you get so you'll have real base line and be able to make some more direct  inferences.

Clem

On Tue, Jan 15, 2013 at 1:44 PM, Mark Benson <md.benson at gmail.com> wrote:

On 15 Jan 2013, at 01:41, sampsa at mac.com wrote:

> Somebody was asking for a better benchmark than VUPS earlier, I found WHETSTONE.C on GORVAX which calculates MIPS. Not sure how much better it is than VUPS.COM, but here are some results:
>
> GORVAX (SIMH VAX, on a Core i5)
>          Loops: 1000, Iterations: 1, Duration: 5 sec.
>          C Converted Double Precision Whetstones: 20.0 MIPS
>
> CHIMPY (DS10)
>          Loops: 10000, Iterations: 1, Duration: 2 sec.
>          C Converted Double Precision Whetstones: 500.0 MIPS
>
> RHESUS (rx2600)
>          Loops: 100000, Iterations: 1, Duration: 4 sec.
>          C Converted Double Precision Whetstones: 2500.0 MIPS
>
> I've attached the C source to this message, and will put binaries for all three platforms as well as the source in RHESUS::[.MEDIALIB.WHETSTONE]

The cool thing about this is is compiles on other platforms, so we can do cool stuff like:

- Compare the available MIPS inside an emulator to the available MIPS on the host OS, giving a MIPS:MIPS ratio for emulator code
- Compare directly the power of older CPUs with newer ones and also derive ap erformance MIPS/Watt figure for various systems.
- Find out how incredibly CPUs have advanced in 50 years!

--

Mark Benson

http://DECtec.info
Twitter: @DECtecInfo
HECnet: STAR69::MARK

Online Resource & Mailing List for DEC Enthusiasts.