Skip to content

Allow negative humanSLChosenMovePiklLambda to prefer worse moves - #1262

Draft
ChinChangYang wants to merge 1 commit into
lightvector:masterfrom
ChinChangYang:claude/piklambda-negative-values-ns31rd
Draft

ChinChangYang wants to merge 1 commit into
lightvector:masterfrom
ChinChangYang:claude/piklambda-negative-values-ns31rd

Conversation

@ChinChangYang

Copy link
Copy Markdown
Contributor

The pikl lambda is an inverse temperature on KataGo's own utility applied to the human SL policy at final move selection, but it was clamped to non-negative values, so it could only ever shift play toward the moves KataGo likes best. Extend it to negative values so the same knob runs the other way:

small positive (0.01) - always the best move KataGo searched
large positive (1e9) - purely the human SL policy
large negative (-1e9) - purely the human SL policy, same as +1e9
small negative (-0.01) - always the WORST move KataGo searched

The formula already did this for a negative lambda, but subtracting the maximum utility makes the exponent positive there, so exp overflowed to +inf for any |lambda| below roughly 0.005 at realistic utility spreads. That produced 0.0 * inf and inf/inf NaNs which reached playSelectionValues and tripped testAssert(maxValue > 0.0), i.e. a fatal exit mid-game.

So subtract the minimum utility instead when the lambda is negative, which keeps the exponent <= 0 and exp within (0,1] no matter how small the lambda gets. The subtracted utility is only a constant factor and cancels in the renormalization, so nothing else changes. Use std::signbit for the test so that -0.0 counts as negative, and map a zero utility difference to a multiplier of 1 so that -0.0 gives the limit (only the worst move keeps weight) rather than 0/0. Moves that cannot be chosen anyways do not set the reference, since otherwise every other move could underflow to zero at once - pass is one of those when humanSLChosenMoveIgnorePass is leaving passing up to KataGo.

The non-negative path is left exactly as it was, including the existing quirk that a lambda of exactly 0.0 disables the blend rather than meaning "always the best move".

Claude-Session: https://claude.ai/code/session_01R53aXQZ6fprZRKsV7on3pu

The pikl lambda is an inverse temperature on KataGo's own utility applied
to the human SL policy at final move selection, but it was clamped to
non-negative values, so it could only ever shift play toward the moves
KataGo likes best. Extend it to negative values so the same knob runs the
other way:

  small positive (0.01)  - always the best move KataGo searched
  large positive (1e9)   - purely the human SL policy
  large negative (-1e9)  - purely the human SL policy, same as +1e9
  small negative (-0.01) - always the WORST move KataGo searched

The formula already did this for a negative lambda, but subtracting the
maximum utility makes the exponent positive there, so exp overflowed to
+inf for any |lambda| below roughly 0.005 at realistic utility spreads.
That produced 0.0 * inf and inf/inf NaNs which reached playSelectionValues
and tripped testAssert(maxValue > 0.0), i.e. a fatal exit mid-game.

So subtract the minimum utility instead when the lambda is negative, which
keeps the exponent <= 0 and exp within (0,1] no matter how small the
lambda gets. The subtracted utility is only a constant factor and cancels
in the renormalization, so nothing else changes. Use std::signbit for the
test so that -0.0 counts as negative, and map a zero utility difference to
a multiplier of 1 so that -0.0 gives the limit (only the worst move keeps
weight) rather than 0/0. Moves that cannot be chosen anyways do not set
the reference, since otherwise every other move could underflow to zero at
once - pass is one of those when humanSLChosenMoveIgnorePass is leaving
passing up to KataGo.

The non-negative path is left exactly as it was, including the existing
quirk that a lambda of exactly 0.0 disables the blend rather than meaning
"always the best move".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53aXQZ6fprZRKsV7on3pu

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants