Quote: (Chroma @ 01 Dec. 2008, 14:24 )
I *believe*, because it was anonymous, that there were appearances of data-spamming and other anomalous results that wound up rendering the whole thing fairly useless, so Pixelgeek eventually just shut it down.
That's not entirely true. The results posted were anonymous, but you had to log in to post results and the results were linked to the user ID. There were cases where 1 or 2 people posted a large number of games but we could see it on the back end.
The most notable case was a single person who posted a ton of Eldar games, which were almost all wins and ran their win ratio up to something like 75%. After contacting that user and asking, it turned out some were teaching games and that the widely recognized best tactician in their play group was primarily an Eldar player. Pulling that person's data out dropped the Eldar wins to about 65%, which was consistent with many batreps and tournament results from both the US and the UK.
It was shut down as it became increasingly garbaged up with multiple versions of each list being lumped together into one stat (and PG's ongoing friction with various GW personnel probably contributed to him pulling it as well).
====
As far as the actual topic, CS said it very well earlier.
Batreps are a tremendously useful tool, especially when illustrating a tactic that's been "theory hammered" in discussion but one side or the other does not seem to grasp. They serve best as an illustration of on-board implementation and break people's preconceived notions about how to use certain units, tactics, or combinations. That's the best thing about them - they let you potentially see into a completely different tactical/strategic mindset, which generates a much different play style and can drastically affect the relative value of a unit or formation.
I agree that much can be accomplished with number crunching and theoryhammer. I definitely think there are some things that become obvious with a bit of careful consideration. Much can be accomplished without ever rolling a die but occasionally things don't crop up until the rubber hits the road. I believe that at some point you reach an impasse and have to say "Enough with the theory. Put your dice where your mouth is," and ask people to prove their point with real games.
All that said, there are, of course, weaknesses. People go into them with pre-conceived notions, which the batrep and discussion should expose, but which can also occasionally entrench opinions based on those preconceptions. Reviewers can end up focusing on tactical or strategic mistakes and dismiss valuable commentary on feel and style or notes about portions of the game unrelated to those errors.
Batreps for controversial issues should be second-guessed and they should be picked apart. People who post batreps on playtest issues have to be willing to be raked over the coals for their in-game decisions in addition to the time and effort of putting together a detailed batrep. There should be multiple reps from different players and play groups before they are taken as broad evidence. Even Chroma's marathon playtest series eventually reach the end of their usefulness without corroboration from other sources.
