I compiled LinuxCNC with
~~~~
./configure CC=”gcc -fsanitize=undefined,bool,float-cast-overflow” CXX=”g++ -fsanitize=undefined,bool,float-cast-overflow”
~~~~
built, ran the testsuite, and fixed what was broken.
For uninteresting reasons, tests/build/ui and tests/overrun fail here. With skip files created in those directories, the testsuite passes without encountering any diagnostics from -fsanitize.
Compiler is gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1). Different compilers and versions may detect slightly different stuff.
The main concerning item for me is the one in interp: before my change, we could probably with a reasonable amount of blitheness access through the pointer in question and not crash. Now, we’re pretty much guaranteed by UNIX semantics to get a nice crash. From the standpoint of bug hunting, though, this is a much better position to be in
The other detail to consider is the somewhat increased HAL shared memory area usage, because every signal now allocates 8 bytes for its data, instead of sometimes allocating as little as 1 byte. Investigation shows that memory usage typically increased by 8%, so I increased HAL shared memory area by 40KiB (13%).
The several UIs I tried also start and run without encountering -fsanitize= diagnostics.
Note: while actually fixing problems, it may be more useful to also specify -fsanitize-recover=all, since it allows the program to continue after a problem is encountered.