[LinuxCNC/linuxcnc Issue#347] LinuxCNC Status display (linuxcnctop.py) is a memory hog!

#1 – jepler 于 2017-11-06

confirmed by me on debian stretch

#2 – jepler 于 2017-11-06

typical traceback of one of the largest leaks
~~~~
$ valgrind –tool=memcheck –leak-check=full –show-leak-kinds=all linuxcnctop
[ctrl-c after waiting about 30 seconds]
[at the end of the diagnostics list]
==10487== 14,434,304 bytes in 881 blocks are possibly lost in loss record 891 of 891
==10487== at 0x4C2BBAF: malloc (vgreplacemalloc.c:299)
==10487== by 0xBB3D971: GetBlocks (tclThreadAlloc.c:1044)
==10487== by 0xBB3D971: TclpAlloc (tclThreadAlloc.c:358)
==10487== by 0xBA60607: Tcl_Alloc (tclCkalloc.c:1059)
==10487== by 0xB7753A3: TkTextIsElided (tkTextBTree.c:3502)
==10487== by 0xB77F89B: IsStartOfNotMergedLine.part.9 (tkTextDisp.c:6911)
==10487== by 0xB77BF9E: IsStartOfNotMergedLine (tkTextDisp.c:4790)
==10487== by 0xB77BF9E: TextChanged (tkTextDisp.c:4793)
==10487== by 0xB77BEFD: TkTextChanged (tkTextDisp.c:4736)
==10487== by 0xB76F76B: DeleteIndexRange (tkText.c:3185)
==10487== by 0xB6E93AA: TextReplaceCmd (tkText.c:1825)
==10487== by 0xB76BE0A: TextWidgetObjCmd (tkText.c:1501)
==10487== by 0xBA576D6: TclNRRunCallbacks (tclBasic.c:4390)
==10487== by 0xB13FEEF: ??? (in /usr/lib/python2.7/lib-dynload/_tkinter.so)
~~~~

#3 – jepler 于 2017-11-06

The following is a standalone reproducer of the problem under wish:
~~~~
proc x {} {
for {set k0 0} {$k0 < 100} {incr k0} { set k _$k0 set vranges [.t tag ranges $k] if {$vranges == {}} { .t insert end $k key "\t" {} xyzzy [list $k value] "\n" } else { .t replace $k.first $k.last "hi mom" [list $k value] } } after 1 x } text .t pack .t x ~~~~ ~~~~ $ valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all wish textleak.tcl [as above, much snippage] ==10826== 49,971,200 bytes in 3,050 blocks are possibly lost in loss record 608 of 608 ==10826== at 0x4C2BBAF: malloc (vgreplacemalloc.c:299)
==10826== by 0x52C1971: GetBlocks (tclThreadAlloc.c:1044)
==10826== by 0x52C1971: TclpAlloc (tclThreadAlloc.c:358)
==10826== by 0x51E4607: Tcl_Alloc (tclCkalloc.c:1059)
==10826== by 0x4EF93A3: TkTextIsElided (tkTextBTree.c:3502)
==10826== by 0x4F0389B: IsStartOfNotMergedLine.part.9 (tkTextDisp.c:6911)
==10826== by 0x4EFFF9E: IsStartOfNotMergedLine (tkTextDisp.c:4790)
==10826== by 0x4EFFF9E: TextChanged (tkTextDisp.c:4793)
==10826== by 0x4EFFEFD: TkTextChanged (tkTextDisp.c:4736)
==10826== by 0x4EF43C4: InsertChars (tkText.c:2683)
==10826== by 0x4EF43C4: TextInsertCmd (tkText.c:3691)
==10826== by 0x4E6D3EA: TextReplaceCmd (tkText.c:1827)
==10826== by 0x4EEFE0A: TextWidgetObjCmd (tkText.c:1501)
==10826== by 0x51DB6D6: TclNRRunCallbacks (tclBasic.c:4390)
==10826== by 0x52C373F: AfterProc (tclTimer.c:1191)
~~~~

#4 – jepler 于 2017-11-06

This bug did not manifest when using my standalone reproducer with a version of tcl/tk built from their source at tag “core86_7″ https://github.com/tcltk/

The affected version I have is is debian’s 8.6.6-1+b1.

This is the related bug upstream: https://core.tcl.tk/tk/info/28a3c366e6
That fix MAY be incomplete and if so the fix is probably https://core.tcl.tk/tk/info/62c5b7a1d7 (just because it also pertains to text replace)

It looks like Debian testing has 8.6.7, so one option you may wish to pursue if applicable is installing that version. https://packages.debian.org/buster/tk8.6

Closing this bug as the problem is with an upstream library and it can be resolved by installing a fixed version.

#5 – KurtJacobson 于 2017-11-06

Wow, that was quick detective work @jepler!
I don’t use linuxcnctop all that often, and when I do it is normally for a short period so the memory leak is not a big deal. I think I will just deal with it for now. Thank you!

P.S. Since I am in the groove, I might as well port linuxcnctop to GTK+ 3, should be easy

#6 – jepler 于 2017-11-07

I think it would be possible to mitigate this in linuxcnctop by avoiding the “replace” call when it would lead to no net change. This would probably eliminate 99% of the memory leak.
~~~~
From 1597d1e84fac815f77c187734dbd27f357abb744 Mon Sep 17 00:00:00 2001
From: Jeff Epler
Date: Tue, 7 Nov 2017 07:30:12 -0600
Subject: [PATCH] WIP mitigate leak in libtk triggered by linuxcnctop

.. by avoiding calls to “replace” whenever possible.

note, I only ran this against the fixed 8.6.7, so I don’t know how
much it actually improves things. However, linuxcnctop still seems
to work properly with the change so that’s good.
—
src/emc/usr_intf/axis/scripts/linuxcnctop.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

diff –git a/src/emc/usrintf/axis/scripts/linuxcnctop.py b/src/emc/usrintf/axis/scripts/linuxcnctop.py
index 34773634f..7868f1255 100644
— a/src/emc/usr_intf/axis/scripts/linuxcnctop.py
+++ b/src/emc/usr_intf/axis/scripts/linuxcnctop.py
@@ -168,7 +168,15 @@ def gui():
else:
vtag = “value”
if vranges:
– t.tk.call(t, “replace”, “%s.first” % k, “%s.last” % k, v, (k, vtag))
+ old = t.get(“%s.first” % k, “%s.last” % k)
+ if old == v:
+ if vtag == “changedvalue”:
+ t.tag_remove(“value”, “%s.first” % k, “%s.last” % k)
+ else:
+ t.tag_remove(“changed”, “%s.first” % k, “%s.last” % k)
+ t.tag_remove(vtag, “%s.first” % k, “%s.last” % k)
+ else:
+ t.tk.call(t, “replace”, “%s.first” % k, “%s.last” % k, v, (k, vtag))
else:
if first: first = False
else: t.insert(“end”, “\n”)
—
2.11.0

~~~~

It might also be worth checking whether a delete+add leaks memory in the affected version.

#7 – KurtJacobson 于 2017-11-07

@jepler I have applied your patch and it does seem to reduce the memory leakage some, but it seems like only by about 50%. I tested with LCNC enabled and homed, but sitting idle, which seems like it would result in essentially zero leakage as the status would not be changing and so no update would be taking place. I am going to test some more. Thanks.