High [CVE-2025-5987] Libssh: invalid return code for chacha20 poly1305 with openssl backend

A flaw was found in libssh when using the ChaCha20 cipher with the OpenSSL library. If an attacker manages to exhaust the heap space, this error is not detected and may lead to libssh using a partially initialized cipher context.

This occurs because the OpenSSL error code returned aliases with the SSH_OK code, resulting in libssh not properly detecting the error returned by the OpenSSL library. This issue can lead to undefined behavior, including compromised data confidentiality and integrity or crashes.

Red Hat Product Security Team has rated this vulnerability as having a Moderate security impact. This is due to the high complexity in exploiting this flaw.

For a successful attack to take place an attacker needs to manage to exhaust the heap space to for the OpenSSL library to return the error code which wrongly match the SSH_OK return code. Red Hat severity: Moderate — CVSS 8.1 (CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H).

Weakness: CWE-393.

Affected products named by the advisory: Red Hat Enterprise Linux 10.0 Extended Update Support; Red Hat Enterprise Linux 9.2 Update Services for SAP Solutions; Red Hat Enterprise Linux 9.4 Extended Update Support; Red Hat Enterprise Linux 9.6 Extended Update Support; and 11 more.