[jboss-jira] [JBoss JIRA] (WFLY-8097) Coverity static analysis, suspicious bitwise logical expression, DigestUtil (Elytron)

[Martin Choma (JIRA)]

Martin Choma created WFLY-8097:
----------------------------------

             Summary: Coverity static analysis, suspicious bitwise logical expression, DigestUtil (Elytron)
                 Key: WFLY-8097
                 URL: https://issues.jboss.org/browse/WFLY-8097
             Project: WildFly
          Issue Type: Bug
          Components: Security
            Reporter: Martin Choma
            Assignee: Darran Lofthouse
            Priority: Critical


Coverity found suspicious logical operation https://scan7.coverity.com/reports.htm#v23632/p11778/fileInstanceId=9563899&defectInstanceId=2359232&mergedDefectId=1377462

See detailed description of possible problem in [1]

If I extend DigestUtilTest#testDecodeByteOrderedInteger with case from [1], test fails

{code}
        byte[] inputFF = CodePointIterator.ofString("000000FF").hexDecode().drain();
        assertEquals(0xFF, decodeByteOrderedInteger(inputFF, 0, 4));
{code}

If I change decodeByteOrderedInteger implementation according to [1], all tests passes.

{code}
result |= (buf[offset + i] & 0xff);
{code}

[1] http://findbugs.sourceforge.net/bugDescriptions.html#BIT_IOR_OF_SIGNED_BYTE

Setting to high priority, because correct behavior of SASL Digest mechanism could be impacted.



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)