Assignment 3: Benchmark analysis

Due: 5:00pm, Thu 7 Feb. Value: 40 pts. Submit to Moodle.

Altogether there were nine solutions to Assignment 1. The following table summarizes the last four numbers reported in the assignment: the ratio of grendel or ozark to a lab computer on each of the two benchmark programs. The graph at right plots the relative performance of grendel and ozark according to each solution, measured as the geometric mean of the two ratios for each computer.

Primality Gaussian grendel ozark grendel ozark Solution A 1.70 2.59 2.17 3.15 Solution B 0.33 1.00 3.72 2.37 Solution C 4.34 7.69 2.89 2.18 Solution D 2.62 2.19 2.40 1.60 Solution E 0.00 0.00 5.92 4.80 Solution F 3.40 6.25 2.34 1.77 Solution G 0.00 0.00 1.00 1.75 Solution H 2.33 2.67 3.28 4.97 Solution I 0.54 0.85 2.33 1.59

Surprisingly, there is quite a bit of variety. That's a good indicator that most of these have some major problems!

I've randomly assigned three of these solutions for you to review, as shown below. (If you work with another student on this assignment, use the following algorithm: Let X be the person whose last name comes earlier in the alphabet, and let Y be the other person. You should review the three solutions assigned to X. But if Y's solution to Assignment 1 is included on X's list, then you should replace Y's solution with any one of the solutions assigned to Y — but do not choose X's solution to Assignment 1, and of course do not choose a solution already appearing on X's list. If that proves impossible, please contact me.)

Britton:	A, B, C
Bucuti:	C, A, G
Clary:	I, C, E
John:	H, E, D
Kimenyi:	D, I, G
Laborde:	F, A, B
Inema:	F, E, H
McNew:	F, I, G
Yociss:	B, D, H

You should submit an ASCII text file containing four parts, in the following format.

(your name(s) here)

Solution X:

(your analysis of X here)

Solution Y:

(your analysis of Y here)

Solution Z:

(Your analysis of Z here)

Conclusion:

(Which of the three you were assigned is soundest? Which is least sound?)

Each analysis should be addressed to the person writing the solution — that is, you should be courteous even as you are being critical. In fact, I hope to share your critique with the student in question. Your critique should address all technical shortcomings you see in the work, with adequate explanation. (You need not address writing quality or coding style — except inasmuch as you believe the report misses some important details that would not allow you to reproduce the results given.)

Please remember to review the code in each solution: In some instances, the benchmarks are less meaningful because of significant bugs.

As a very rough length guideline, I expect that a typical analysis of a single solution would be at least 100 words. However, that is just a rough minimum — some may be much longer. If you have a critique that is particularly short, you might highlight some notable features that make that solution particularly strong. The final section of your report — the conclusion — need not include more than a single sentence.

Your solution will be evaluated on the quality of your writing, the correctness of your analysis (are your points correct?), and the completeness of your analysis (do you include each shortcoming that should be included?).