This is hard to do exactly right... The JDK code does stuff with rational arithmetic and so forth. We use a much simpler algorithm which may give an answer off in the lowest order bit. Since this is pure Java, it should still be consistent from machine to machine.
Recall that the binary representation of the double is of the form d = 0.bbbbbbbb x 2n where there are up to 53 binary digits in the binary fraction (including the assumed leading 1 bit for normalized numbers). We find a value m such that 10m d is between 253 and >263. This product will be exactly convertible to a long with no loss of precision. Getting the decimal representation for that is trivial (see formatLong). This is a decimal mantissa and we have an exponent (-m). All we have to do is manipulate the decimal point to where we want to see it. Errors can arise due to roundoff in the scaling multiplication, but should be no more than a single bit.
Namespace: nom.tam.util
Assembly: CSharpFITS_v1.1 (in CSharpFITS_v1.1.dll)
Syntax
| Visual Basic (Declaration) |
|---|
| Public Function format( _ ByVal val As Double, _ ByVal buf As Byte(), _ ByVal off As Integer, _ ByVal len As Integer _ ) As Integer |
| C# |
|---|
| public int format( double val, byte[] buf, int off, int len ) |
| C++ |
|---|
| public: int format( double val, array<char>^ buf, int off, int len ) sealed |
| J# |
|---|
| public int format( double val, byte[] buf, int off, int len ) |
| JScript |
|---|
| public
function format( val : double, buf : Byte[], off : int, len : int ) : int |
Parameters
- val
- Double to be formatted
- buf
- Buffer in which result is to be stored
- off
- Offset within buffer
- len
- Maximum length of integer
Return Value
offset of next unused character in input buffer.