Skip to content

Fixed does not fully support fraction bits >= storage bits - 1 #318

Description

@PatrickHaecker

Somehow related to #311 as it also concerns the lack of generalization, but in the other direction. If precision is really important, but range is not, we want to use all the bits to represent small numbers accepting that larger numbers, like 1, can no longer be represented.

Let b be the number of bits of the storage type and f the number of bits of the fraction. For f < b - 1 everything is fine:

using FixedPointNumbers, Unitful

julia> Fixed{Int8, 6}(1)
1.0Q1f6

julia> Fixed{Int8, 6}(1)u"m"
1.0Q1f6 m

However, for f >= b - 1 things start to break down when using units:

julia> Fixed{Int8, 7}(1)
"0.992188"

julia> Fixed{Int8, 7}(1)u"m"
ERROR: MethodError: no method matching *(::String, ::Unitful.FreeUnits{(m,), 𝐋, nothing})
The function `*` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  *(::Any, ::Any, ::Any, ::Any...)
   @ Base operators.jl:642
  *(::Missing, ::Unitful.Units)
   @ Unitful ~/.julia/packages/Unitful/8Qjvg/src/units.jl:94
  *(::LinRange, ::Unitful.Units)
   @ Unitful ~/.julia/packages/Unitful/8Qjvg/src/range.jl:143
  ...

Stacktrace:
 [1] top-level scope
   @ REPL[86]:1

The reason seems to be the additional quotes which are printed for f = 7 instead of f = 6, as can be seen above or more clearly below

julia> sprint(show, Fixed{Int8, 6}(1))
"1.0Q1f6"

julia> sprint(show, Fixed{Int8, 7}(1))
"\"0.992188\""

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions