Here are a few games to compare the accuracy of the analysis of eXtreme Gammon, GnuBG, and Snowie.
To see a side by side comparison of the program
features follow this link
The following protocol has been used to compare the different programs:
- Analyze the game in eXtreme Gammon Deep
setting (3-ply)
- Analyze the game in Snowie Precise (3-ply)
- Analyze the game in GnuBG World-Class
(equivalent 3-ply)
- Additionally, Analyze the game in eXtreme Gammon XG roller Setting
- Manually mark all the moves where any of the
above disagree
- Rollout in eXtreme Gammon using the
predefined setting "3-ply Rollout with Variance
reduction" (1440 game in 3-ply)
- For each computer: add all the errors and
equity lost
Two matches have been analyzed, more will follow
(it's quite a time consuming process).
Note that the games were randomly selected. They were not hand picked because eXtreme did quite well
The first game is between eXtreme and Snowie. The
second is a self play by GnuBG (world-class).
Here are the results
Game 1:
 
| |
eXtreme Gammon |
XG Roller |
Snowie |
GnuBG |
| |
 |
| Equity lost |
0.431 |
0.378 |
1.060 |
0.625 |
| Error made |
6 |
6 |
19 |
13 |
| Blunder made |
1 |
none |
4 |
none |
| Estimated Elo1 |
2222 |
2224 |
2196 |
2214 |
| Snowie Rating2 |
0.5 |
0.4 |
1.2 |
0.7 |
| Rank |
2nd |
1st |
4th |
3rd |
Game
2:
| |
eXtreme Gammon |
XG Roller |
Snowie |
GnuBG |
| |
|
| Equity lost |
0.935 |
0.784 |
1.839 |
1.160 |
| Error made |
16 |
13 |
27 |
23 |
| Blunder made |
none |
none |
8 |
1 |
| Estimated Elo1 |
2212 |
2217 |
2185 |
2205 |
| Snowie Rating2 |
0.8 |
0.7 |
1.7 |
1.1 |
| Rank |
2nd |
1st |
4th |
3rd |
Overall:
| |
eXtreme Gammon |
XG Roller |
Snowie |
GnuBG |
| |
|
| Equity lost |
1.366 |
1.162 |
2.899 |
1.785 |
| Error made |
22 |
19 |
46 |
36 |
| Blunder made |
1 |
none |
12 |
1 |
| Estimated Elo1 |
2216 |
2220 |
2190 |
2209 |
| Snowie Rating2 |
0.7 |
0.6 |
1.5 |
0.9 |
| Rank |
2nd |
1st |
4th |
3rd |
1 Elo formula is based on the formula: Elo = 2240- (Equity lost)/decisions*16500
2 Snowie Ranking is based on the formula: Rating = (Equity lost)*1000/(Number of moves)/2
(We divide by 2 because the equity lost is for both players)
|
Download
