编程语言

img l1t

统计一下哪些数据库的sum函数有计算误差

发表于2004/10/13 12:08:00  2776人阅读

方法:create table aa( id char(10),v decimal(18,2));
insert into aa values('1',99999999.99);
插入8192个相同的记录
select sum(v) from aa;
结果是819199999918.08为没有误差

已经检验
mysql 4.0.18有误差
firebird 1.5没有误差

The DECIMAL and NUMERIC types are implemented as the same type by MySQL. They are used to store values for which it is important to preserve exact precision, for example with monetary data. When declaring a column of one of these types, the precision and scale can be (and usually is) specified; for example:

salary DECIMAL(5,2)

In this example, 5 is the precision and 2 is the scale. The precision represents the number of significant decimal digits that will be stored for values, and the scale represents the number of digits that will be stored following the decimal point.

MySQL stores DECIMAL and NUMERIC values as strings, rather than as binary floating-point numbers, in order to preserve the decimal precision of those values. One character is used for each digit of the value, the decimal point (if the scale is greater than 0), and the `-' sign (for negative numbers). If the scale is 0, DECIMAL and NUMERIC values contain no decimal point or fractional part.

The maximum range of DECIMAL and NUMERIC values is the same as for DOUBLE, but the actual range for a given DECIMAL or NUMERIC column can be constrained by the precision or scale for a given column. When such a column is assigned a value with more digits following the decimal point than are allowed by the specified scale, the value is converted to that scale. (The precise behavior is operating system-specific, but generally the effect is truncation to the allowable number of digits.) When a DECIMAL or NUMERIC column is assigned a value that exceeds the range implied by the specified (or default) precision and scale, MySQL stores the value representing the corresponding end point of that range.
以上文字摘自mysql manual.html,我认为它们是矛盾的。如果decimal的取值范围和double相同,它也就不可能保持额外的精度。

阅读全文
0 0

相关文章推荐

img
取 消
img