Repository navigation
Allow negative humanSLChosenMovePiklLambda to prefer worse moves - #1262
Draft
ChinChangYang wants to merge 1 commit into
Draft
ChinChangYang wants to merge 1 commit into
ChinChangYang wants to merge 1 commit into
Conversation
The pikl lambda is an inverse temperature on KataGo's own utility applied to the human SL policy at final move selection, but it was clamped to non-negative values, so it could only ever shift play toward the moves KataGo likes best. Extend it to negative values so the same knob runs the other way: small positive (0.01) - always the best move KataGo searched large positive (1e9) - purely the human SL policy large negative (-1e9) - purely the human SL policy, same as +1e9 small negative (-0.01) - always the WORST move KataGo searched The formula already did this for a negative lambda, but subtracting the maximum utility makes the exponent positive there, so exp overflowed to +inf for any |lambda| below roughly 0.005 at realistic utility spreads. That produced 0.0 * inf and inf/inf NaNs which reached playSelectionValues and tripped testAssert(maxValue > 0.0), i.e. a fatal exit mid-game. So subtract the minimum utility instead when the lambda is negative, which keeps the exponent <= 0 and exp within (0,1] no matter how small the lambda gets. The subtracted utility is only a constant factor and cancels in the renormalization, so nothing else changes. Use std::signbit for the test so that -0.0 counts as negative, and map a zero utility difference to a multiplier of 1 so that -0.0 gives the limit (only the worst move keeps weight) rather than 0/0. Moves that cannot be chosen anyways do not set the reference, since otherwise every other move could underflow to zero at once - pass is one of those when humanSLChosenMoveIgnorePass is leaving passing up to KataGo. The non-negative path is left exactly as it was, including the existing quirk that a lambda of exactly 0.0 disables the blend rather than meaning "always the best move". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R53aXQZ6fprZRKsV7on3pu
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The pikl lambda is an inverse temperature on KataGo's own utility applied to the human SL policy at final move selection, but it was clamped to non-negative values, so it could only ever shift play toward the moves KataGo likes best. Extend it to negative values so the same knob runs the other way:
small positive (0.01) - always the best move KataGo searched
large positive (1e9) - purely the human SL policy
large negative (-1e9) - purely the human SL policy, same as +1e9
small negative (-0.01) - always the WORST move KataGo searched
The formula already did this for a negative lambda, but subtracting the maximum utility makes the exponent positive there, so exp overflowed to +inf for any |lambda| below roughly 0.005 at realistic utility spreads. That produced 0.0 * inf and inf/inf NaNs which reached playSelectionValues and tripped testAssert(maxValue > 0.0), i.e. a fatal exit mid-game.
So subtract the minimum utility instead when the lambda is negative, which keeps the exponent <= 0 and exp within (0,1] no matter how small the lambda gets. The subtracted utility is only a constant factor and cancels in the renormalization, so nothing else changes. Use std::signbit for the test so that -0.0 counts as negative, and map a zero utility difference to a multiplier of 1 so that -0.0 gives the limit (only the worst move keeps weight) rather than 0/0. Moves that cannot be chosen anyways do not set the reference, since otherwise every other move could underflow to zero at once - pass is one of those when humanSLChosenMoveIgnorePass is leaving passing up to KataGo.
The non-negative path is left exactly as it was, including the existing quirk that a lambda of exactly 0.0 disables the blend rather than meaning "always the best move".
Claude-Session: https://claude.ai/code/session_01R53aXQZ6fprZRKsV7on3pu