[teiid-issues] [JBoss JIRA] Resolved: (TEIID-159) Modify our handling of BigDecimal division to use MathContext.DECIMAL128 as our rounding mode to improve scale and precision of results

[Steve Hawkins (JIRA)]

     [ https://jira.jboss.org/browse/TEIID-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Hawkins resolved TEIID-159.
---------------------------------

    Resolution: Done


Updated the floating to bigdecimal conversion to use a scale of 16 for doubles and 8 for floats.  Updated the logic in the bigdecimal x/y to use a scale of max(16, x.scale + y.precision + 1), then the result is normalized to remove trailing zeros.

> Modify our handling of BigDecimal division to use MathContext.DECIMAL128 as our rounding mode to improve scale and precision of results
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: TEIID-159
>                 URL: https://jira.jboss.org/browse/TEIID-159
>             Project: Teiid
>          Issue Type: Feature Request
>          Components: Server
>    Affects Versions: 6.0.0
>            Reporter: Jerry Helbling
>             Fix For: 7.0
>
>
> Using MathContext.DECIMAL128 should give a more usable scale and precision to the results.
> 		
> Current behavior
> 	     254403453878 / 21870000000 returns 12
>     and
> 	     254403453878.0001 / 21870000000 returns 11.6325
> 		
> Improved behavior:
> 	    254403453878 / 21870000000 returns 11.632531
>     and
> 	    254403453878.0001 / 21870000000 returns 11.632531
> 		
> We should also investigate what the impact of this change will be on users who may be relying on our current behavior.
> Reference: This enhancement was suggested by Issue Tracker Issue 175485, contributed by Greg Haber

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira