If you haven't tried his software before I urge you to do it, it's quite aweosme:
http://remi.coulom.free.fr/CLOP/
Some changes in this version as well:
2011-11-05: 0.0.9
- Stronger regularization (avoid overfitting in high dimensions)
- "Merge Replications" option in gui -> faster, better display
- Performance optimization of display and loading of large data files
- Removed "-ansi" option for Windows compilation
- Shrinking parameter ranges does not lose integer data any more
- Removed confusing columns: max-1, max-2, ...
- More explanations in the doc: biased win rate + GammaParameter
6 comments:
It's probably very good software, but I actually don't quite understand how it works. I don't even know what a negative hessian is and why it is good that the algorithm is able to deal with it. As my goal is not to produce yet another strong chess engine with little effort but to look how far "I" can push it I'm reluctant to start using the dark side of the force.
Firstly, it's a bit of the dark side I admit. But it's so wonderful to just throw stuff you thought into the evaluation and then tune it to let it reach its full potential.
The idea can still be unique, just that it can be extremely hard to know if it should be worth 10 or 70 centipawns. (so instead of playing a silly amount of test games to even make it work, you just stick it in CLOP).
Secondly, I have no idea what a hessian is either. :)
But you just have to look at two things really:
1. The Plot-tab, with time as x and the variable as y, to make sure it's actually moving to some value (and not just scattered all over the place). Usually you start seeing a pattern after 1,000 games or so.
2. The Max-tab. And more specifically the Mean-column. The values for the different parameters here is was is found to be best so far.
I really have no idea what all the other things are, and don't really care as long as it works. :)
Probably your engine is much more mature than mine. If I have a feature that requires a silly amount of games to show itself as an improvement I'm not interested in it (yet).
Feeding iCE in the clop factory might create an iCE clone that plays significantly better than the original one but it will probably be still much weaker than the other strong engines and additionally it is then not "my" iCE anymore.
Maybe I write my own tuning framework someday, maybe one that works without hessians. That is probably much more fun.
I have my own book building app and my own table base generation algorithm so it would fit in that list somehow.
Nah, you're misunderstanding. Mediocre is far from mature enough to need tuning to gain 2-3 ELO points. I'm discarding everything that's not showing like 50-60 points.
What I've been using CLOP for is actualy those huge leaps since my evaluation is so mistuned that some parts probably hurts the performance (like my passed pawn evaluation). This is of course due to having added new features incrementally without tuning them, just checking for a general improvement. Mean some feature five steps back might now be so mistuned that it doesn't help anymore (or even hurts).
But the thing is if you want to discover that your evaluation of a passed pawn on the 5th rank is vastly overtuned (for example), you'd need to first of all manually discover it by looking at the values (which is hard to start with). Then imagining up a new number and run a test, and probably discover that you got no gain or hurt performance. So you imagine up a new number and run the test again. And again. And again.
CLOP does the again and again for you. All you have to do is identify what you want to tune (which could just as well be a completely new feature).
Now CLOP isn't something magical, it's just a good algorithm for finding values in a noisy result set.
I'm using the CLOP algorithm, conveniently packaged in the CLOP GUI, together with a script that runs cutechess-cli.
You could of course write both the cutechess-cli and the CLOP GUI yourself without much effort. Question is why you would do that. Those two are just convenience tools.
What would be interesting is creating your own algorithm (other that CLOP), but that's also possible to run from CLOP GUI.
That's what I mean. I want my own method to tune values someday. It will certainly involve game playing so cutechess-cli is fine for that.
But the part that uses a set of parameters, gets a winning probability result for that set, and drives conclusions what the next set of parameters should look like until it reaches an optimum, that is the interesting because challenging one.
But also the part where you can learn the most...
I agree it would be interesting and fun to find my own algorithm for it. Mine would probably be something similar to what Joona Kiiski used to do with Stockfish. I.e. choose a guestimated value, surround it with a delta and slowly increase/decrease it for every win/loss.
Inutitive and very straight forward. I even wrote a script for it (with a few alterations of my own) before I tried CLOP.
Now it just feels like reinventing an extremely advanced wheel.
Post a Comment