Since the release on eXtreme Gammon, there have been several independent studies on the strength of the program. We'd like to thank all the people who did all that hard and very long work, particularly Michael Depreli.

• Michael Depreli Study
• Mike Corbett Study

Michael Depreli Study: published on BGonline.org (finished on April 25^th 2010)

The study is comparing the different top programs (at multiple strength level). Using 500 money games any difference of opinion is analyzed very deeply using a rollout. The mistake each program/level makes are accumulated.
This is a long process (more than 4500 moves or cube decisions needed to the rolled). The project got completed after 6 month of intense analysis. Rollouts were made using GnuBG (Rollout parameters GnuBG 2-ply world class 1296 trials or 2.33 JSD (98% conf) if sooner).
All number are normalized equity.

We'd like to commend Michael for his extraordinary dedication and all his hard work to get that project completed.

The process does not take into account search interval. Each candidate was analyzed in the level requested regardless of the search interval used.

BOT	Ply	Checker Play	Missed Double	Wrong Double	Wrong Take	Wrong Pass	Total
eXtreme Gammon	XGR+	13.397	1.088	0.658	0.970	0.241	16.354
eXtreme Gammon	XGR	22.269	1.661	0.783	2.173	0.264	27.150
eXtreme Gammon	5-ply	17.169	1.507	0.789	2.859	0.450	22.774
GnuBG	4-ply	21.599	2.663	0.644	4.061	0.127	29.094
eXtreme Gammon	4-ply	22.967	0.426	1.647	0.818	0.555	26.413
GnuBG	3-ply	29.313	0.903	10.276	0.775	5.880	47.147
eXtreme Gammon	3-ply	27.814	1.831	1.528	3.996	0.520	35.689
GnuBG	2-ply	33.247	2.763	1.670	4.261	0.476	42.417
Snowie 4	3-ply	37.424	1.922	1.139	3.651	0.867	45.003
BGBlitz	3-ply	41.286	1.692	10.864	4.168	2.159	60.169

Legend:

• Ply: Search depth as defined for the program (GnuBG 2-ply is equivalent to other bot 3-ply)
• Total: total equity lost

Here is a chart that shows the relative strength based on this study (in Elo compared to XG 3-ply).
The speed test were performed by GameSite 2000 ltd and is not from an independent source.
Speed was evaluated using a core i7 computer analyzing of a money session and a match.

Speed test were made using the using a search interval where the last ply looks up to 4 moves within 0.080 equity (eXtreme Gammon: Huge for 3-ply, GnuBG 3-ply and 4-ply: Large)

Note about BGBlitz: as it cannot analyze a full match. Its speed was determined using Rollout speed.

Click to see a bigger picture.

Mike Corbett Study:

Phil Simborg ran a test at Mike Corbett's request on 10 positions of his book.

On 9 positions eXtreme Gammon did better than Snowie. GnuBG did better on 6 (all move in 3-ply (2-ply for GnuBG).

As the positions picked were the ones Snowie gets wrong this test does not reflect the difference between eXtreme Gammon and Snowie. It does, however show the difference between eXtreme Gammon and GnuBG.

Page Number	eXtreme did better than Snowie	Gnu did better than Snowie	Error avoided by eXtreme Gammon
1			0.059
2			0.076
15			0.024
23			0.001
61			0.012
68			None
70			0.054
83			0.051
87			0.166
133			0.023