Skip to content

fix signed overflow in calculate() and MathLib constant folding - #9030

Open
Nussu06 wants to merge 1 commit into
cppcheck-opensource:mainfrom
Nussu06:constant-folding-signed-overflow
Open

Nussu06 wants to merge 1 commit into
cppcheck-opensource:mainfrom
Nussu06:constant-folding-signed-overflow

Conversation

@Nussu06

@Nussu06 Nussu06 commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

calculate() in lib/calculate.h, MathLib::add/subtract/multiply/mod and valueFlowEnumValue fold integer constants from the checked code in MathLib::bigint with plain signed arithmetic, so code like x = 0x7fffffffffffffff + 1;, A<0x8000000000000000 % -1> or enum E { A = 0x7FFFFFFFFFFFFFFE, B, C }; makes cppcheck itself hit signed integer overflow (UBSan reports in calculate.h, mathlib.cpp and valueflow.cpp, and the % -1 case can trap on x86-64). Added a calculateArithmetic helper that does +, - and * for signed integer types on the unsigned type and used it at those sites, and x % -1 now returns 0, so folded values are the same as before and only the undefined behaviour is gone. With this, testrunner is clean under ASan+UBSan with signed-integer-overflow enabled (cmake/dynamic_analyzer_options.cmake still disables it with a TODO); I added asserts to TestMathLib::calculate and TestValueFlow::valueFlowCalculations plus a new TestSymbolDatabase::enum21, and they fail under UBSan without the fix. There is no trac ticket for this yet, could you create one? I will update the title to "Fix #NNNNN (...)" once it exists.

@dmcppcheck

Copy link
Copy Markdown
Contributor

There should probably be a trac ticket for this PR. Please create it, or ask us to create it. Then the PR title should be updated.

@dmcppcheck dmcppcheck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review

I found no problems in the changed code. Under a -fsanitize=signed-integer-overflow build, all the cases the PR names no longer overflow, and the folded values stay the same as before (they wrap). One related site is still left: ValueFlowAnalyzer::writeValue in lib/vf_analyzers.cpp:374 does value->intvalue += (inc ? 1 : -1); for ++/--. So long long x = 0x7fffffffffffffff; x++; still makes cppcheck overflow a signed integer. The PR does not touch this path, and its tests don't cover it, so the claim that it gets rid of this kind of undefined behaviour is not complete. The same calculateArithmetic helper could be used there too.

Testing

  • I built cppcheck and testrunner from pr/ with the normal Makefile. ./testrunner TestMathLib TestValueFlow TestSymbolDatabase: 555 tests, 0 failed.
  • I built a second copy of pr/ with CXXFLAGS="-O1 -g -fsanitize=signed-integer-overflow,undefined -fno-sanitize-recover=signed-integer-overflow" and ran it on small inputs, one line per file:
    • 0x7fffffffffffffff + 1, 0x7fffffffffffffff * 2, LLONG_MIN % -1 in an expression and as a template argument, enum E { A = 0x7FFFFFFFFFFFFFFF, B, C };, x += 1 / y *= 3 after x = LLONG_MAX, unary minus and / -1 of LLONG_MIN, a condition x == LLONG_MAX followed by x + 1, and a loop near LLONG_MAX: no UBSan reports.
    • With --debug, the folded values are as expected. + gives -9223372036854775808, * gives -2, and the enum gives B = LLONG_MIN and C = LLONG_MIN+1. (LLONG_MIN <=> 1) still folds to -1.
  • A case that still overflows (not part of the PR's changes):
    // sp/s9.cpp
    void k() { long long x = 0x7fffffffffffffff; x++; ++x; }
    /work/pr_ubsan/cppcheck --std=c++20 --enable=all -q sp/s9.cpp
    lib/vf_analyzers.cpp:374:29: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long long int'
        #0 ValueFlowAnalyzer::writeValue(...) lib/vf_analyzers.cpp:374
        #1 ValueFlowAnalyzer::update(...) lib/vf_analyzers.cpp:837
    
  • I did not run the full testrunner under ASan+UBSan, did not test the Boost int128_t build, and did not build base under UBSan.

Automated review of 8c80299 by Claude. The pull request was built and tested locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants