Skip to content

model: keep all digits of Edm.Double values in literals and payloads - #332

Open
Shriraj1901 wants to merge 1 commit into
SAP:masterfrom
Shriraj1901:fix-double-literal-precision
Open

Shriraj1901 wants to merge 1 commit into
SAP:masterfrom
Shriraj1901:fix-double-literal-precision

Conversation

@Shriraj1901

Copy link
Copy Markdown

EdmFPNumTypTraits.to_literal formats every float with "{:E}", i.e. six decimals, and to_json reuses it. A value such as 3.141592653589793 was therefore sent as 3.141593E+00 in a $filter, in an entity key and in the JSON body of a create or update request, so the service silently stored or compared a different number.

Keep the current format whenever it reads back as the same float, so the output for ordinary values is unchanged (2.029 is still 2.029000E+00), and only when it would change the value use the fewest digits that round-trip.

EdmFPNumTypTraits.to_literal formats every float with "{:E}", i.e. six
decimals, and to_json reuses it. A value such as 3.141592653589793 was
therefore sent as 3.141593E+00 in a $filter, in an entity key and in
the JSON body of a create or update request, so the service silently
stored or compared a different number.

Keep the current format whenever it reads back as the same float, so
the output for ordinary values is unchanged (2.029 is still
2.029000E+00), and only when it would change the value use the fewest
digits that round-trip.
@cla-assistant

cla-assistant Bot commented Nov 18, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant