Skip to content

Conversation

@kraktus
Copy link
Contributor

@kraktus kraktus commented Dec 1, 2025

close #1099. This is a POC, so very open to changes, and if you think it's not the right fit for python-chess I'll make a separate package without issue, but I think it's nice to have

Everything was squashed because individual commits made little sense, but can be found at master...kraktus:python-chess:binary_fen

The current public API:

@dataclass(frozen=True)
class BinaryFen:
    """
    A simple binary format that encode a position in a compact way, initially used by Stockfish and Lichess

    See https://lichess.org/@/revoof/blog/adapting-nnue-pytorchs-binary-position-format-for-lichess/cpeeAMeY for more information
    """
    occupied: chess.Bitboard
    nibbles: List[Nibble]
    halfmove_clock: Optional[int]
    plies: Optional[int]
    variant_header: int
    variant_data: Optional[Union[ThreeCheckData, CrazyhouseData]]

    def to_canonical(self) -> BinaryFen:
        """
        Multiple binary FEN can correspond to the same position:

        - When a position has multiple black kings with black to move
        - When trailing zeros are omitted from halfmove clock or plies
        - When its black to move and the ply is even

        The 'canonical' position is then the one with every king with the turn set
        And trailing zeros removed, and odd ply

        Return the canonical version of the binary FEN
        """

    @classmethod
    def parse_from_bytes(cls, data: bytes) -> BinaryFen:
        """
        Read from bytes and return a BinaryFen

        should not error even if data is invalid
        """

    @classmethod
    def parse_from_iter(cls, reader: Iterator[int]) -> BinaryFen:
        """
        Read from bytes and return a `BinaryFen`

        should not error even if data is invalid
        """

    def to_board(self) -> Tuple[chess.Board, Optional[ChessHeader]]:
        """
        Return a chess.Board of the proper variant, and std_mode if applicable

        The returned board might be illegal, check with `board.is_valid()`

        Raise `ValueError` if the BinaryFen data is invalid in a way that chess.Board cannot handle:
        - Invalid variant header
        - Invalid en passant square
        - Multiple en passant squares
        """

    @classmethod
    def decode(cls, data: bytes) -> Tuple[chess.Board, Optional[ChessHeader]]:
        """
        Read from bytes and return a chess.Board of the proper variant

        If it is standard chess position, also return the mode (standard, chess960, from_position)

        raise `ValueError` if data is invalid
        """

    @classmethod
    def parse_from_board(cls, board: chess.Board, std_mode: Optional[ChessHeader]=None) -> BinaryFen:
        """
        Given a chess.Board, return its binary FEN representation, and std_mode if applicable

        If the board is a standard chess position, `std_mode` can be provided to specify the mode (standard, chess960, from_position)
        if not provided, it will be inferred from the root position
        """

    def to_bytes(self) -> bytes:
        """
        Write the BinaryFen data as bytes
        """
    def __bytes__(self) -> bytes:
        """
        Write the BinaryFen data as bytes

        Example: bytes(my_binary_fen)
        """

    @classmethod
    def encode(cls, board: chess.Board, std_mode: Optional[ChessHeader]=None) -> bytes:
        """
        Given a chess.Board, return its binary FEN representation, and std_mode if applicable

        If the board is a standard chess position, `std_mode` can be provided to specify the mode (standard, chess960, from_position)
        if not provided, it will be inferred from the root position
        """

Open questions:

  • The code currently lives in chess/binary_fen.py (and tests in test_binary_fen.py) to reduce rebase issues, but where should it live is an open question. I suggest to keep it a separate file but merge the tests with the other test file.

  • Scalachess implementation distinguish between standard/chess960/fromPosition, with odd behavior (see test files) To keep binary fen roundtrip possibles, I have added a optional parameter std_mode: Optional[ChessHeader] with an auto mode if not set. Name of the type and parameter can be changed ofc, no strong opinion. here's the current auto mode behavior

 # TODO check if this auto mode is OK
            root = board.root()
            if root in CHESS_960_STARTING_POSITIONS:
                return cls.CHESS_960
            elif root == STANDARD_STARTING_POSITION:
                return cls.STANDARD
            else:
                return cls.FROM_POSITION
  • The most significant addition is surely the BinaryFen.to_canonical method, which aims to make an arbitrary binary FEN equal to the output from BinaryFen.parse_from_board or sclachess equivalent As with all canonical, it's a bit arbitrary but works well.

Implementation notes:

  • Internally had to re_create CrazyhousePiecePocket because variant one's equality check if it's the same object, and not content equality
  • Fuzzed it with pythonfuzz like the other formats but had trouble installing it (latest version not on pypi since aquired) so maybe would be worth changing at some point
  • No use of match instead if/elif for BC

@kraktus kraktus force-pushed the binary_fen2 branch 5 times, most recently from d993d0f to f76d2e0 Compare December 1, 2025 22:56
close niklasf#1099. This is a POC, so very open to changes, and if you think it's not the right fit for python-chess I'll make a separate package without issue, but I think it's nice to have

Everything was squashed because individual commits made little sense, but can be found at niklasf/python-chess@master...kraktus:python-chess:binary_fen

The current public API:
```py
@DataClass(frozen=True)
class BinaryFen:
    """
    A simple binary format that encode a position in a compact way, initially used by Stockfish and Lichess

    See https://lichess.org/@/revoof/blog/adapting-nnue-pytorchs-binary-position-format-for-lichess/cpeeAMeY for more information
    """
    occupied: chess.Bitboard
    nibbles: List[Nibble]
    halfmove_clock: Optional[int]
    plies: Optional[int]
    variant_header: int
    variant_data: Optional[Union[ThreeCheckData, CrazyhouseData]]

    def to_canonical(self) -> BinaryFen:
        """
        Multiple binary FEN can correspond to the same position:

        - When a position has a black king, with black to move and has an odd number of plies
        - When a position has multiple black kings with black to move
        - When trailing zeros are omitted from halfmove clock or plies
        - When its black to move and the ply is even

        The 'canonical' position is then the one with every king with the turn set
        And trailing zeros removed, and odd ply

        Return the canonical version of the binary FEN
        """

    @classmethod
    def parse_from_bytes(cls, data: bytes) -> BinaryFen:
        """
        Read from bytes and return a BinaryFen

        should not error even if data is invalid
        """

    @classmethod
    def parse_from_iter(cls, reader: Iterator[int]) -> BinaryFen:
        """
        Read from bytes and return a `BinaryFen`

        should not error even if data is invalid
        """

    def to_board(self) -> Tuple[chess.Board, Optional[ChessHeader]]:
        """
        Return a chess.Board of the proper variant, and std_mode if applicable

        The returned board might be illegal, check with `board.is_valid()`

        Raise `ValueError` if the BinaryFen data is invalid in a way that chess.Board cannot handle:
        - Invalid variant header
        - Invalid en passant square
        - Multiple en passant squares
        """

    @classmethod
    def decode(cls, data: bytes) -> Tuple[chess.Board, Optional[ChessHeader]]:
        """
        Read from bytes and return a chess.Board of the proper variant

        If it is standard chess position, also return the mode (standard, chess960, from_position)

        raise `ValueError` if data is invalid
        """

    @classmethod
    def parse_from_board(cls, board: chess.Board, std_mode: Optional[ChessHeader]=None) -> BinaryFen:
        """
        Given a chess.Board, return its binary FEN representation, and std_mode if applicable

        If the board is a standard chess position, `std_mode` can be provided to specify the mode (standard, chess960, from_position)
        if not provided, it will be inferred from the root position
        """

    def to_bytes(self) -> bytes:
        """
        Write the BinaryFen data as bytes
        """
    def __bytes__(self) -> bytes:
        """
        Write the BinaryFen data as bytes

        Example: bytes(my_binary_fen)
        """

    @classmethod
    def encode(cls, board: chess.Board, std_mode: Optional[ChessHeader]=None) -> bytes:
        """
        Given a chess.Board, return its binary FEN representation, and std_mode if applicable

        If the board is a standard chess position, `std_mode` can be provided to specify the mode (standard, chess960, from_position)
        if not provided, it will be inferred from the root position
        """
```

Open questions:

- The code currently lives in `chess/binary_fen.py` (and tests in `test_binary_fen.py`) to reduce rebase issues, but where should it live is an open question. I suggest to keep it a separate file but merge the tests with the other test file.

- Scalachess implementation distinguish between `standard`/`chess960`/`fromPosition`, with odd behavior (see test files)
To keep binary fen roundtrip possibles, I have added a optional parameter `std_mode: Optional[ChessHeader]` with an auto mode if not set. Name of the type and parameter can be changed ofc, no strong opinion.
here's the current auto mode behavior
```py
 # TODO check if this auto mode is OK
            root = board.root()
            if root in CHESS_960_STARTING_POSITIONS:
                return cls.CHESS_960
            elif root == STANDARD_STARTING_POSITION:
                return cls.STANDARD
            else:
                return cls.FROM_POSITION
```

- The most significant addition is surely the `BinaryFen.to_canonical` method, which aims to make an arbitrary binary FEN equal to the output from `BinaryFen.parse_from_board` or sclachess equivalent
As with all canonical, it's a bit arbitrary but works well.

Implementation notes:
- Internally had to re_create `CrazyhousePiecePocket` because variant one's equality check if it's the same object, and not content equality
- Fuzzed it with `pythonfuzz` like the other formats but had trouble installing it (latest version not on pypi since aquired) so maybe would be worth changing at some point
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BinaryFen in python-chess

1 participant